Introduction

With the ongoing growth of behavioral health integration in primary care, there is an increased need to upskill behavioral health professionals (e.g., licensed clinical social workers, licensed marriage and family therapists, licensed professional counselors, licensed psychologists) with the skills needed to be effective clinicians in the primary care setting (Serrano et al., 2018). The expansion of integrated behavioral health services created a workforce shortage of qualified professionals, as most mental health professionals are not trained for primary care service delivery (Serrano et al., 2018). The primary care behavioral health (PCBH) consultation model is a model of behavioral health integration in primary care that embeds a behavioral health professional in a primary care setting to offer full-spectrum services that support the primary care team. The behavioral health professional focuses on behavioral health issues (e.g., depression, anxiety, substance misuse), chronic disease management (e.g., Type 2 diabetes, chronic pain), and health behavior change (e.g., tobacco cessation, healthy eating; Reiter et al., 2018). Behavioral health professionals working in the PCBH model require unique retraining to attend to a large scope of patient presenting concerns across the lifespan and manage the operational processes in their role as a consultant in primary care (P. J. Robinson & Reiter, 2025). More specifically, Robinson and Reiter outline 7 core competencies domains necessary for effective training of Behavioral Consultants: brief interventions skills, pathway services skills, documentation skills, consultation skills, team performance skills, and practice management skills and administrative knowledge and skills (P. Robinson & Reiter, 2016).

Of the behavioral health disciplines, psychology has been more involved in the practice of the PCBH model, with the initial competencies for primary care practice for psychologists published in 2014 by the American Psychological Association (McDaniel et al., 2014). The core competencies for marriage and family therapists working in healthcare settings were published in 2018 in the American Association for Marriage and Family Therapy (Competencies for Family Therapists Working in Healthcare Settings., 2018). Although the National Association for Social Workers does not have formal competency for primary care or healthcare practice, the social worker integrated care competencies scale (SWICCS) was initially published in 2019 (Davis et al., 2019). To date, the American Counseling Association does not have core competencies specific to primary care practice, demonstrating the importance of continuing education, as master’s and doctoral-level behavioral health programs have not been able to meet this training need.

The development of competency driven training in PCBH is critical to prepare behavioral health professionals to work in integrated behavioral health settings (Beehler et al., 2013; Ogbeide & Bayles, 2024; Ogbeide et al., 2023). Ogbeide and Bayes (2024) conducted a Delphi study to identify supervisor competencies for pre-licensure training and cited the need for clinical supervisors to have standardized training in PCBH to competently train the next generation of integrated behavioral health clinicians. Emerging research has begun to develop competency-based models of clinical supervision with written assessments available to evaluate adherence to models of integrated behavioral health (Beehler et al., 2013; Ogbeide & Bayles, 2024; Ogbeide et al., 2023).

There is a lack of comprehensive competency-based training to prepare behavioral health professionals to enter primary care settings where the PCBH model is implemented. There are limited opportunities for behavioral health professionals to receive training with structured observation of clinical skills to treat physical and mental health conditions with short treatment courses across the lifespan. Additionally, behavioral health professionals often lack opportunities to develop clinical skills for consulting with medical clinicians, which is an essential component of the PCBH model. Ensuring behavioral health professionals have the appropriate knowledge and clinical skills to work in primary care settings is critical to ensure success of the model.

Often this retraining occurs after the points of hire due to difficulty finding candidates with the prerequisite skills to work in PCBH (Hall et al., 2015; Serrano, 2024). Post-licensure training efforts include continuing education certificates, community-based learning initiatives, training consortiums, self-study, and supplementing onboarding with in-person and distant learning modalities when available. More specifically, a majority of certificate programs are didactic focused, adopt PCBH model fidelity inconsistently, and are often delivered in a distance learning format (Serrano et al., 2018). The University of Massachusetts Medical School (UMASS) Center for Integrated Primary care (https://www.umassmed.edu/cipc/) offers 3 options for certificates in Primary Care Behavioral Health (Primary Care Behavioral Health Course Part 1; 12 h, Primary Care Behavioral Health Course Part 2; 12 h, Primary Care Behavioral Health Program; 36 h). All courses are offered online through pre-recorded content with Part 1 and Part 2 offering optional monthly offices hours. UMASS Primary Care Behavioral Health Program originally was offered live until it transitioned in September 2016 to asynchronous content. A retrospective study of pre/post-survey items of cohorts from 2011 to 2019 revealed an increase in confidence for all survey items after the course. Survey items include the following: identifying substance abuse problems, working collaboratively with MDs to help patient with substance abuse, general assessment for developmental problems, give anticipatory guidance on behavioral issues, intervene to help children with behavioral problems, describe evidence-based biopsychosocial approach to chronic illness, discuss psychotropic meds with patients, conceptualize stages of change of patients, adapt approach based on cultural factors, and conduct family meeting (Mullin et al., 2021).

The University of Michigan offers a certificate in Integrated Behavioral Health and Primary Care (IBHPC; https://ssw.umich.edu/continuing-education/certificate-courses/integrated-behavioral-health-and-primary-care) with tracks in pediatric and adult populations through a distance learning environment. Both tracks include 23 h of live interactive online instruction and 14–16 h of asynchronous self-paced online content.

To expand training for behavioral health professionals to become behavioral health consultants (BHCs) who can provide patients and communities with direct access to PCBH services, Mental Health America (MHA) of Greater Houston partnered with Rice University to develop and launch these organizations’ first-ever BHC Certificate Course (BHC-CC). The 10-month course was delivered through a series of synchronous and asynchronous learning sessions, virtual trainings, pre-recorded video sessions, classroom learning, structured observation of clinical skills, and live learning sessions. Specifically, this course was designed to reduce the training gaps in providing and receiving behavioral health consultation services in primary care (Ogbeide et al., 2022; Serrano et al., 2018). The PCBH model was the focus of this course due to the model’s generalist approach across the lifespan for addressing a variety of behavioral health and medical conditions (Reiter et al., 2018). Through this course, the goal was to train learners as BHCs that are competent in addressing health disparities, improving health equity, and promoting health in vulnerable and historically excluded populations. The purpose of this paper was to evaluate the effectiveness of this continuing education course for competency growth of behavioral health professionals in primary care.

Methods

Three cohorts of learners (i.e., behavioral health professionals) participated in the BHC-CC over a three-year period (2023–2025). The BHC-CC objectives are listed in Table 1. The first (n = 20) and second (n = 12) cohorts were in person with virtual synchronous and asynchronous content. The participants in the first two cohorts were licensed behavioral health professionals in Texas. The third cohort (n = 5) was virtual with virtual synchronous and asynchronous content and was open to anyone in the United States. Participants for the certificate program were recruited through a variety of ways: 1: flyer and social media outreach; 2: collaboration with other integrated care workgroups; 3: the BHC Project ECHO series 4: virtual information sessions; and 5: OpenRICE Sessions (i.e., initiative by the Glasscock School at Rice University to offer free and online webinars to the greater Houston community and beyond to learn about various adult learning programs offered).

Table 1 BHC-CC objectives
Full size table

The curriculum topics were developed and selected initially through feedback from a BHC-focused Project ECHO (Extension for Community Healthcare Outcomes) series that served to pilot test topics prior to the start of the certificate program (McBain et al., 2019). Additionally, stakeholder surveys were completed in the southern part of the United States where this certificate was housed to gauge level of interest from employers, specifically employers that had BHC vacancies for a significant period of time. After this, the subject matter experts (SMEs) and faculty members of this program facilitated the refinement of the topics, with a specific focus on addressing health disparities and health equity with marginalized populations. The faculty members also tested the course content before the course was made “live” for the first cohort.

The course (Table 1) consisted of five modules (Module 1: Introduction to the Behavioral Health Consultant Role and Competencies; Module 2: Teamwork in Primary Care; Module 3: Special Topics in Primary Care; Module 4: Health Equity and Health Disparities in Primary Care; Module 5: Technology in Primary Care) in which learners had 1.5 months to complete the self-paced content consistent with adult learning principles (e.g., multiple, 10–15 min lectures recording by BHC-CC faculty members within each module, articles, book chapters) for each module along with pre–post-knowledge quizzes (Arghode et al., 2017). The participants were expected to spend 1–2 h per week on the learning materials within each module. Two live skills labs (one at the half-way point and the other at the end of the course) were conducted to give learners live practice with standardized patients. These visits were observed by a BHC-CC faculty using a BHC visit observation checklist to reinforce the importance of adhering to visit structure and specific behaviors within initial and follow-up visits (Table 2).

Table 2 BHC-CC program demographic information
Full size table

Lastly, learners were able to engage in live, virtual drop-in meetings that occurred during each module for 1 h led by a BHC-CC faculty member with expertise in the PCBH model. This provided the learners with an opportunity to ask questions about the module content, the application in their specific work context, and role play with other learners or the faculty member. While specific calendar timing varied across cohorts, the standardized sequence of the five curricular modules and the corresponding evaluation points are summarized in Table 3. BHC-CC faculty members were given a teaching guide for the drop-in meetings to provide structure for the meetings and facilitate discussion for the participants.

Table 3 Standardized modular sequence and evaluation framework for the 10-Month BHC-CC Program
Full size table

Learners completed the primary care behavioral health provider adherence questionnaire (PPAQ; (Beehler et al., 2013), a psychometrically sound measure of clinician fidelity to the PCBH model, at three time points: Pre-course, mid-course, and post-course. Learners also completed a post-course satisfaction survey which included nine Likert-rated items ranging from (1) Strongly Disagree to (5) Strongly Agree. Also, 1 knowledge increase and 1 professional behavior change ranging from (1) No to (4) Significant. Demographic Information was gathered such as profession and practice type (i.e., if they are currently practice in primary care or not; Table 2).

The primary outcome was the PPAQ-E, a 34-item self-report subscale of the Primary Care Behavioral Health Provider Adherence Questionnaire (Beehler et al., 2013). It uses a 5-point Likert scale (1 = Never/Rarely to 5 = Always/Almost Always) to measure clinical adherence to the PCBH model. While the full PPAQ includes administrative domains, the PPAQ-E was used because it focuses only on the clinical skills taught in this curriculum, such as brief appointments and behavioral functional assessments. The PPAQ-E is a validated tool for measuring model fidelity, with reported Cronbach’s alpha ranges of 0.94–0.97 in similar samples (Beehler & Lilienthal, 2017; Beehler et al., 2013).

Statistical Design & Analysis

All statistical analyses were conducted in XLSTAT (Version 2024.3.0, Addinsoft, Paris, France) integrated with Microsoft Excel. Due to variation in cohort sample sizes and occasional missing responses, data from all three program cohorts were combined for primary analyses to ensure adequate statistical power and consistent interpretation. All statistical tests used a two-tailed significance threshold of α = .05. For continuous outcomes (percentage change in PPAQ-E scores), data distributions were assessed using the Shapiro–Wilk test of normality. When normality assumptions were met, one-sample, two-tailed t-tests were performed; for non-normally distributed data, the Wilcoxon signed-rank test was employed as the primary analytic approach. For categorical outcomes, including "improvement status" (defined as a numerical increase in competency scores), chi-square tests of independence and Wilks’ likelihood-ratio G2 tests were used. To account for cells with expected frequencies < 5, a Monte Carlo simulation (5000 iterations) was applied. Adjusted standardized residuals (|residual|≥ 1.96) were used to identify specific significant deviations within cohorts and modules. This dual-analysis approach and the reporting of standardized residuals for process data align with established PCBH competency evaluation frameworks (Beehler & Lilienthal, 2017; Beehler et al., 2013) and support the rigorous reporting of the 'recipe' for PCBH training outcomes (Funderburk et al., 2021).

Results

Across all analyses, the training program showed clear, statistically significant improvements in participant performance, especially over the entire assessment period from Round 1 to Round 3 (Table 4). The Shapiro–Wilk test indicated that the distribution for Round 1 to Round 3 was non-normal (W = 0.921, p = .015), which necessitated the use of the Wilcoxon signed-rank test as the primary measure of significance. Both parametric (t-test) and nonparametric (Wilcoxon signed-rank) methods confirmed that the percentage change from Round 1 to Round 3 was significant (Wilcoxon V = 531, Z = 3.53, p < 0.001; t-test t(34) = 3.78, p = .001), while changes between adjacent rounds (Rounds 1 → 2 and 2 → 3) were not. This pattern suggests that most learning gains are accumulated gradually throughout the full duration of the program rather than in shorter intervals.

Table 4 Summary of normality and one-sample T-Test results for percentage change across assessment rounds
Full size table

At the module level, chi-square analyses indicated no significant association between module type and improvement status (χ2(6) = 12.73, p = .398; Monte Carlo, 5000 iterations). Although Module 3 showed a higher-than-expected proportion of improved students (adjusted residual =  + 2.77), this was an isolated deviation that did not change the overall non-significant chi-square result. Therefore, improvement was generally consistent across modules, with no module demonstrating disproportionately high or low performance.

Cohort-by-module analyses identified several significant adjusted residuals (Wilks’ G2 = 35.33, p < 0.001 for Increase Z; G2 = 21.02, p = .021 for Increased & Maintained Z), pointing to some cohorts performing above or below expectations in specific modules. However, these effect sizes were small (Cramér’s V = .08–.13), indicating that these differences were minor and not systematic (Table 5). No cohort showed consistently better or worse performance across all modules. Instead, variations seemed confined to specific cohort–module combinations.

Table 5 Adjusted standardized residuals (Z-scores) representing significant performance deviations across modules and cohorts
Full size table

Overall, the statistical results indicate that the program yielded improvements over time, with benefits evident across modules and cohorts. While localized differences existed, they were relatively small and did not significantly impact the overall pattern of learning gains. These findings strongly support the program’s effectiveness in enhancing knowledge and skills among participants.

Discussion

Licensed behavioral health professionals require specialized PCBH professional development when transitioning to the role of BHC to enhance the effectiveness of behavioral integration in primary care (Dobmeyer et al., 2016).The outcomes of the BHC-CC demonstrate that the course may be an effective model for training licensed behavioral health professionals in the PCBH model. The professional development was successfully implemented in synchronous and asynchronous formats in a hybrid (i.e., in person and telehealth) delivery model. BHC-CC offers a flexible and promising advanced practice course that healthcare organizations could employ to ensure competence among licensed behavioral health professionals when initiating or enhancing PCBH clinical services. BHC-CC may be a viable option for organizations to include in their onboarding process for new BHCs and for veteran BHCs that have not received adequate preparation to provide clinical services utilizing the PCBH model.

The primary limitation of this study includes the use of the PPAQ-E, a self-report measure. Future research may include direct observation data with standardized measures as part of the course evaluation and comparing standardized measures to self-report measures. Additionally, the small sample size reduces broad generalizability that could be enhanced with larger samples of licensed behavioral health professionals. Lastly, we did not examine the impact of experiential learning for those concurrently working in PCBH settings, so observed improvements may reflect experiential learning rather than the effects of the training. Despite the limitations, this course serves as a promising model for continuing education programs to build the behavioral health workforce for primary care practice. Future directions in research include examining patient outcomes as a result of the BHC training and potential practice change of the BHC. Additionally, only five BHCs enrolled in the nationally available virtual course. Future directions may need to explore marketing and enrollment efforts to increase awareness and access of this course.