Advancing the Science of Intervention Development in School Mental Health

Although there are multiple models available to guide the early development and adaptation of interventions for youth with emotional and behavioral problems, there are limited outlets for publishing these studies. This type of research is critical to sustaining the advancement of interventions for children with emotional and behavioral problems, but the studies do not often align with the traditional criteria journals have for accepting manuscripts (e.g., empirical studies mostly reliant on quantitative data). Early work often requires investigators to integrate clinical experience, previous research, and qualitative feedback from clinicians, youth, and families to develop hypotheses and protocols for evaluation. Methods can be fluid over the course of a development project and thus, often fail the systematic criteria required by some journals. This special issue provides a unique and important opportunity to share some early intervention development and adaptation research and Drs. Lawson and Owens (guest editors) deserve a great deal of credit for facilitating this outstanding collection. Intervention development and adaptation research in school mental health refers to the process of creating new interventions or modifying existing ones to better meet the needs of students in various school contexts. This work is crucial for bridging the gap between research and practice, ultimately aiming to improve mental health outcomes for diverse student populations.

The studies in this issue highlight two important reasons why early intervention development and adaptation research is critically important. The first of these is to increase the use of evidence-based practices. Given the many obstacles to widespread practice of evidence-based interventions it is important to try to develop and adapt implementation approaches that can overcome these obstacles and increase the likelihood that students can access our most effective practices (i.e., implementation adaptations). The second reason involves enhancing the benefits of interventions by developing or adapting them to meet the unique needs of students and characteristics of various settings (i.e., intervention adaptations). We know of no intervention evaluation studies in which all participants uniformly responded to an intervention. By studying the variability in responding and using those findings to adapt interventions to the unique characteristics of children or settings, we can enhance the effectiveness of care. In this commentary, we propose that if we adopt an intersectionality perspective and carefully consider moderators of intervention response to inform development work, our science can best move forward to develop school mental health interventions effective for all youth.

Implementation Adaptations

The need to improve our ability to increase the use of evidence-based practices is apparent in schools across the country. There are almost no evidence-based school mental health universal or targeted practices that are widely used with an acceptable level of fidelity. Even some of the most widely promoted frameworks, like effective use of school-wide expectations within the Positive Behavior Interventions and Supports (PBIS) or targeted interventions within a tiered support framework are far less frequently used than it may appear at first. Although many report that their schools use these models to address the emotional and behavioral needs of their students, if one actually looks closely at what is being done in the schools a large portion only modestly resemble the PBIS and MTSS frameworks. Although there are certainly examples of schools in which these models are done well, overall, implementation with appropriate fidelity appears to be the exception instead of the rule. Recent data suggest that while 69% of schools report using PBIS, only 44% implement it with high fidelity (McIntosh et al., 2021). There are similar reports noting the infrequent use of universal and targeted services in schools (Hustus et al., 2020; Spiel et al., 2014). As a result, attempts to develop or adapt services to increase implementation with fidelity and sustainment is a critically important goal for those doing intervention development and adaptation research.

One study in this special issue that adapted an intervention to facilitate use of evidence-based practices in secondary schools is based on literature showing that there is inadequate evidence-based practice (EBP) implementation for autistic adolescents in schools, despite schools being the most accessed service system by autistic youth (Locke et al., 2024). The Autism Community Toolkit: Systems to Measure and Adopt Research-based Treatments (ACT SMART) is a packaged implementation process tool designed to facilitate uptake of EBP for adolescents with autism in community agencies, with promising feasibility, utility, and effectiveness. ACT SMART was initially developed to be used outside of school settings to guide the assessment and services provided to youth with autism. The authors described using the three phases of the Discover, Design, Build, and Test (DDBT) Framework (Lyon et al., 2019) to involve community partners and modify ACT SMART to enhance the provision of evidence-based practices to middle and high school students with autism. This study describes the first iterative study of a community-partnered, iterative redesign of ACT SMART for use in educational settings. Using mixed-methods (focus groups, surveys), they gathered district and school administrators’, teachers’, paraeducators’, autistic students’, and their caregivers’ perspectives of the feasibility, usability, and appropriateness of ACT SMART. They also gathered recommendations for redesign to ensure its contextual appropriateness and usability in middle and high schools. Results indicated the perceived acceptability (satisfaction with ACT SMART), feasibility (practicability of ACT SMART), and appropriateness (perceived fit or relevance of ACT SMART) were high, but ratings of usability (extent to which ACT SMART can be used by educators to achieve intended goals) of ACT SMART for schools were lower. Key modifications include integrating the toolkit with school structures (i.e., conducting training on professional development days, aligning budget planning with the district timeline, tying the toolkit to students’ IEPs, aligning with school calendars) and content modifications (i.e., altering language, shortening assessments, incorporating toolkit engagement strategies) to improve usability in schools. This study emphasizes the importance of tools that support the selection and adoption of evidence-based practices (EBPs) in public schools.

Another study that examined the framework for selecting and implementing services into schools focused on the team process (Kuriyan et al., 2024). The investigators used a model that was designed to enhance care coordination in medical settings (TeamSTEPPS) and adapted it for use in schools. In this study, the authors describe the systems for coordinating school mental health services in schools prior to them using TeamSTEPPS. They also describe the adaptation process for tailoring TeamSTEPPS to the school setting. The long-term goal of the project is to enhance the quality and coordination of school mental health services provided to students. Similar to the ACT SMART framework, the goal of this study is to enhance implementation practices to improve care for youth.

These studies and others in this special issue address some of the many challenges to implementation that exist in schools. Given that evidence-based practices are often not used in schools, the need for approaches to improve implementation is critical. Although we are a long way from having optimally effective services that address the diversity of presenting problems of students in schools, we currently have far more effective interventions than are currently being used. Without effective implementation approaches, intervention development and adaptations will be useless as they will not reach the youth they could help. Unfortunately, many of our graduate programs in counseling, social work, and education do not train professionals to use evidence-based practices. This puts the burdens on schools to enhance the skills of their staff through professional development and the use of implementation systems that facilitate the use of evidence-based approaches.

Intervention Adaptations

Although the value of new or adapted evidence-based practices is limited by the effectiveness of implementation practices, it is still critical to develop services that meet the needs of many youth for whom we do not yet have effective practices. One important approach to guide our intervention development work is to identify moderators to intervention response. For example, researchers might discover that an intervention provided significant benefits for students in a group-by-time analysis of a randomized controlled trial (RCT), but the magnitude of benefit was moderated by whether the students wore glasses or not. Those who did not wear glasses demonstrated significantly better outcomes than those who did wear glasses. Finding that glasses moderated intervention response provides the evidence necessary to warrant intervention development or adaptation efforts to enhance benefits for students who wear glasses. Although this is a hypothetical example to illustrate the point, there are examples in the literature such as a study by Ng et al. (2020) who found that the effectiveness of a universal social-emotional learning program was moderated by students’ baseline anxiety levels. Thus, modifications to the program to address those with baseline anxiety levels that indicate an unlikely response are warranted.

Often the important moderators can be difficult to identify. For example, if someone found that their universal intervention was more effective in middle- and high-income communities than in low-income communities, it would be tempting to assume that the income level of the communities was the important moderator that needed to be targeted with adaptations. However, the income level of the community may actually be irrelevant to the effectiveness of the universal intervention. The important moderator may be characteristics of children or schools that are associated with low-income communities such as class size, staffing levels, race/ethnicity, average achievement scores, and parent education levels. Until one identifies the meaningful moderator, it is not possible to have confidence in the focus of any development or adaptation work to enhance the intervention.

The studies in this special issue are understandably not intended to identify moderators of intervention response; however, many of them refer to moderators that come from the research literature or theory in their rationale to make adaptations. Using theoretical or empirical moderators to guide intervention development or adaptation is an important and often overlooked step in this work. Some variables that moderate intervention response may indicate fairly simple adaptations are needed (e.g., access), and others may essentially indicate that investigators should start over developing the intervention for a specific population.

There are two common approaches to adapting interventions based on finding moderators of intervention response. These can include adapting the “packaging” of the intervention or the mechanisms of action of an intervention – or both. The packaging involves the language used, examples provided, and visual materials that accompany implementation of many interventions. In order to help youth feel engaged with the services, it is helpful for this packaging to include representations of the language, appearance, and values of their culture. Adaptations to modify the packaging of an intervention can be challenging. The process requires involvement from community partners to adequately modify the materials to enhance engagement. Depending on the characteristics of the population for which the intervention is being adapted, the changes could be extensive and not only include materials, but also include methods and settings for delivery. When adapting the packaging of an intervention it is important for developers to make sure that changes do not modify the mechanisms of action of the intervention, but only the packaging.

In contrast, some interventions may not be effective with some youth because the mechanisms of action do not operate the same with them as they do for others who respond well to the intervention. For example, cognitive behavioral therapy is an evidence-based intervention for youth with anxiety or depression. The approach involves extensive efforts to change the thinking or automatic thoughts of the youth. This is based on a long history of research showing that the content of automatic thoughts can causally influence mood (Teasdale, 1983). Thus, if one’s automatic thoughts drive one’s depressed mood, CBT is likely to be effective. There may be youth with depression that is only minimally related to the content of their automatic thoughts. It is unlikely that the cognitive aspects of CBT would be effective for them as their depression may be more visceral than cognitive. The cognitive/visceral distinction moderates the relationship between CBT and outcomes in this example. In this example, adapting CBT for these youth or developing an alternative intervention for them would be supported and instead of changing the packaging, the mechanism of change would need to be altered.

One study in this special issue that modified the mechanisms of change focused on Black youth with social anxiety disorder (SAD; Warner et al., 2024), a highly prevalent and impairing psychological condition in adolescents. SAD is characterized by persistent fear of social and performance situations. Typical intervention approaches for youth with SAD focus on self-statements that facilitate worry and avoidance and these approaches are well-supported. This paper discusses the development of the first culturally responsive, school-based intervention for SAD in Black adolescents, referred to as Interacting and Changing our Narratives (ICON). Following recommendations by Castro and colleagues (2010), a multiple stage process was used to adapt an empirically-based, school intervention for SAD to be responsive to the unique lived experiences of Black teenagers. Utilizing a university-community-school partnership, interested parties, such as content area experts, school personnel, caregivers, and students, were invited to participate in this process. Their recommendations guided the modifications and were clearly reflected in the newly developed intervention. Warner et al. (2024) identified a set of self-statements related to racism that appear to enhance the risk and severity of SAD with Black youth. They extended the cognitive therapy approaches typically used in interventions for SAD to include a focus on these racism-related self-statements based on the literature, collaborative experts, and feedback from students and their families. This study is an impressive report of an insightful approach to recognize why a moderating construct (race) may reduce the benefits of an evidence-based practice for specific students. In addition, they used effective mechanisms of action (i.e., cognitive restructuring) to modify the unique set of self-statements related to racism that uniquely affect a specific group of students.

Another approach to developing and adapting interventions is to identify a subgroup of students for whom a particular problem exists, but current approaches do not target some of the reasons that this group of youth experiences the problem. For example, in the paper by Capps et al. (2024), the investigators focused on the risk factor school disengagement as a pathway to substance use and other long-term problems. One group of students particularly susceptible to this path includes students with attention problems (e.g., ADHD). Interventions intended to increase school engagement often focus on facilitating relationships with adults and peers at school through various means. Although this approach also applies to youth with attention problems, another reason for their school disengagement is lack of academic success. As a result, Capps et al. (2024) integrated activities focused on enhancing relationships at school with interventions intended to enhance the academic functioning of youth with attention problems. Combining these approaches did not alter the mechanisms of action; however, they created a set of interventions that may uniquely meet the needs of a specific subgroup of students. Without improvements in academic success, typical approaches to enhance school engagement are likely to not adequately address the needs of students with attention problems. In addition, in order to enhance implementation, the investigators employed a model that included services provided by peers slightly older than the target students. Although previous research with peer-provided interventions revealed many challenges to implementation (Sibley et al., 2020), Capps and colleagues developed and evaluated a hybrid peer- and staff-delivered model that relied on peers to provide interventions but kept staff in active coordinating and supervising roles. This approach, which balances peer delivery with staff coordination, could potentially be adapted for various school-based mental health interventions. This study describes an innovative approach to developing a set of interventions that precisely target a specific group of students by addressing their unique reasons for being at risk for school disengagement.

Importance of Intersectionality

Individual characteristics that moderate responses to intervention play an important role in intervention development and adaptations of current interventions; however, too often we consider moderators of intervention response based on only one child or setting characteristic (e.g., sex, race) and learn relatively little. In recent years, there has been a growing recognition that we need to go beyond the examination of unitary constructs and consider intersectionality when trying to understand the complex dynamics of identity and experience within the context of school mental health (Smith & Trimble, 2016). Intersectionality refers to considering a wide range of intersections of racial groups, genders, socioeconomic status, and sexual orientations. The focus is on the intersection of identities associated with systematic oppression including racism and sexism. This framework acknowledges that individuals occupy multiple social positions simultaneously, and these positions intersect to shape their experiences and possibly their responses to interventions. Our intervention development efforts are most successful when they consider the unique contexts and challenges faced by diverse populations. By recognizing and valuing the combination of diverse cultural backgrounds, identities, experiences, and other characteristics of students, and by understanding the unique intersecting identities that shape their experiences (Bowleg, 2012), we can best understand unique intersections that may moderate responses to interventions and then create effective and sustainable solutions and advance the science of intervention development and evaluation.

Considering intersectionality when developing or adapting interventions can enhance our understanding of multiple aspects of interventions including access, engagement, and response. Intersectionality emphasizes the need to consider how various forms of privilege and oppression, abilities and disabilities, economic situations, and other characteristics intersect to create disparities in access to care. Taking an intersectionality approach offers a valuable perspective for studying intervention development and evaluation, but it also presents several challenges. For example, this complexity can risk reducing the significance of some of this research by limiting the impact. If one focuses on the intersection of multiple identities, then the number of people at that intersection could be relatively small, thereby limiting the potential impact of the interventions to a relatively small group of students. However, we argue that many of the identities considered when using an intersectionality approach will not moderate intervention responses, thereby reducing the risk of becoming overly narrow. Those identities that do moderate response are worth considering and can help us get closer to the experimental therapeutics approach promoted by the National Institute of Mental Health (NIMH).

We propose that in addition to considering the constructs most central to the definition of intersectionality such as racial groups, gender, socioeconomic status, sexual orientation, and other identities that have been subjected to racism and sexism, it is important to examine other child characteristics with a similar approach. When searching for characteristics that moderate intervention response we should examine a variety of other constructs that are theoretically relevant to intervention response such as cognitive ability, emotion regulation, parent education levels, and comorbidities. We have the statistical tools necessary to examine the influence of various constructs and their intersections, but we are often lacking the sample size requirements. Optimal power to address these questions may come when care is taken to recruit students into clinical trials based on being representative of specific intersections of identities and other characteristics.

Conclusions

The investigators who published in this special issue repeatedly demonstrated that one of the best practices for using our scientific tools most effectively is to include community partners in the process. Teachers, parents, school mental health professionals, and youth contributed to the research in this special issue and their contributions facilitated the success of this work. Authors described how themes generated by partners led directly to modifications to interventions. For many years, the involvement of community partners was not considered a scientific tool important to the development of effective interventions, but this approach has become much more common in the last two decades (Evans et al., 2007). This is an important advance in our field and one that improves the quality of the developed interventions.

The research in this special issue includes important advances in intervention development and evaluation research in school mental health. Understanding how we can most effectively meet the needs of students with emotional and behavioral problems in our Pre-K through 12 education system is a critical priority in our society. The challenge is tremendous as the diversity of the need is immense and there are numerous challenges to meeting it. Our responsibility is to use the tools of science to advance our interventions to increase their use in our education system and to develop them so they meet the breadth of needs present in our students. We described some priorities for improving our ability to meet this challenge and the first is the important role of identifying moderators of the relationship between intervention and response. This work yields clues for how our work to develop and enhance interventions should advance. Using these clues to focus on packaging and/or the mechanisms of actions will improve the potency and efficiency of our efforts. It is important to take an intersectionality perspective when considering and evaluating important potential moderators related to identities and histories of privilege and oppression. Similarly, it is also important to look at how traditional psychological constructs such as cognitive ability, emotion regulation, and other potential moderators of intervention response interact to affect intervention response. Future directions in this field should include developing sophisticated methods for identifying and addressing intersectional influences on intervention effectiveness, as well as exploring innovative delivery models that can overcome persistent implementation barriers in diverse school settings.