Abstract
Background
Many healthcare professionals are experiencing psychological distress. Electronic mental health (e-mental health) interventions are convenient and multifunctional. This review aimed to examine the effectiveness of e-mental health interventions in enhancing the well-being of healthcare professionals and to identify moderating factors.
Methods
A comprehensive and systematic retrieval of randomized controlled trial (RCT) studies was conducted across eight databases. Population, intervention, comparison, and outcome (PICO) were used to define eligibility criteria. Stress, anxiety, and depression were included as the main outcomes. The overall effect was calculated based on the random effect model, and the effect size was presented using the standardized mean difference. The characteristics of the research design, intervention object, and intervention design were further selected as potential moderating factors for subgroup analysis. Meta-regression analyses were finally performed, incorporating intervention duration and sample size as independent variables.
Results
A total of 20 studies were included in the systematic review, and 17 were included in the meta-analysis. A large effect on relieving stress and anxiety and a small-to-medium effect on reducing depression were observed. Subgroup analyses showed that features including mindfulness approaches, online courses, computer use, group interventions, and professional guidance were more favorable in the design of services. Meta-regression revealed that intervention duration only affected anxiety symptoms. Caution should be exercised, as some subgroups had fewer studies and higher heterogeneity. For the secondary outcomes, a large effect on emotional exhaustion and a small-to-medium effect on well-being were observed.
Conclusion
In general, e-mental health interventions significantly improve the psychological health of healthcare staff. Future high-quality, large-scale studies targeting healthcare professionals and specific intervention scenarios are warranted.
Similar content being viewed by others
Introduction
With the increased pace of life and unhealthy lifestyles, the prevalence of mental health conditions and substance abuse has risen rapidly in recent years [1]. Moreover, the capacities of health service systems in many countries are lagging, and people’s needs for psychological services are not met in a timely manner [2]. In the first year of the coronavirus disease 2019 (COVID-19) pandemic, the incidence of common psychiatric conditions such as depression and anxiety increased by 25% worldwide [3]. An epidemiological survey conducted in China showed that the weighted lifetime prevalence of psychiatric disorders (except dementia) among the population over 18 years of age is 16.57%; however, in early 1980, the prevalence of psychiatric disorders in the population aged over 15 years was only 12.69% [4]. Data from the United Kingdom Household Longitudinal Study (UKHLS) showed that the population prevalence of mental health issues increased from 24.3% between 2017 and 2019 to 37.8% in April 2020, with the largest increase (18.6%) among individuals aged 18–34 years [5]. Since the peak of the COVID-19 pandemic, mental health issues have remained prominent as people return to the workplace after a prolonged period of ‘working from home’. The burden of mental health issues not only increases the cost of medical treatment but also reduces human capital and productivity [6].
The nature of healthcare work is professional with a focus on social humanity. Healthcare professionals (HCPs) are involved in diagnosis and treatment and deal with complex interpersonal relationships with patients and their families. In addition, high social expectations, heavy workloads, excessive evaluations, high competition, and tension in relationships with colleagues and leadership may cause psychological strain among healthcare professionals [7]. However, healthcare institutions often pay little attention to the psychological health of staff and may not provide adequate support and resources [8]. A study from Germany showed that the COVID-19 pandemic caused symptoms of hypertonia, depression, and anxiety among healthcare professionals, and these symptoms tended to occur simultaneously [9]. A systematic review including 13 studies reported that the prevalence rates of anxiety, depression, and insomnia among healthcare professionals were 23.2%, 22.8%, and 38.9%, respectively [10]. Sevliya et al. found that during the normalization stage of the COVID-19 pandemic, mental health issues remained prominent among pediatricians, and the prevalence rates of depression, anxiety, and stress symptoms were 54.8%, 49.4%, and 33.3%, respectively [11]. An online survey conducted by the British Medical Association revealed that as doctors work longer hours, they are more likely to experience emotional distress [12]. Another study showed that burnout is more pronounced among doctors who are younger, work more than 40 h per week, and work in tertiary hospitals [13]. It has also been shown that the underreporting of psychological illness, self-stigma, and concerns about confidentiality among medical staff can significantly hinder them from seeking help, thus leading to a poorer state of health [14]. Poor mental health in healthcare professionals will not only have an impact on their quality of life but also interfere with their daily clinical judgment and may eventually lead to a decline in the quality of the medical services they provide [15].
With the development of information technology, electronic resources are playing an increasingly important role in public health, and electronic health has emerged as a new medical model. The World Health Organization defines electronic health as a service that transforms health resources and care through electronic technology [16]. Electronic mental health (e-mental health) approaches (“mental health services and information delivered or enhanced through the internet and related technologies”) are receiving increasing attention and have been indicated to play a critical role in health management [17]. During the COVID-19 pandemic, e-mental health interventions became the preferred choice due to social distancing constraints [18]. Some scholars believe that e-mental health, owing to the advantage of eliminating spatial distance constraints, can provide essential services to people living in rural areas, greatly improving the accessibility and equity of health services [19]. Moreover, e-mental health is cost-effective and reduces financial burdens [20]. Some studies also showed that e-mental health interventions reduced help-seeking barriers, such as shame, stigma, and fear of exposure, due to anonymity [21].
However, existing research on the effectiveness of e-mental health interventions is contradictory [22]. Christensen et al.concluded that elderly individuals with depression found video counseling beneficial and supportive of their mental health [23]. Similarly, another systematic review found that e-mental health services effectively ameliorated stress, depression, burnout, insomnia, and alcohol abuse, with effectiveness potentially correlating with personal characteristics such as occupation type [24]. However, not all researchers agree with these findings. Considering the diversity of programs and variability of outcomes, one study asserted that the evidence supporting internet interventions for bipolar disorder remains limited [25]. Some studies have argued that many patients are resistant to online therapy because of concerns about the confidentiality of their medical data [26]. Another study that targeted male workers at high risk of depression concluded that e-mental health courses were effective in improving sleep and reducing stress, but the lack of personal relevance and interaction was a hindering factor [27]. Bolier et al. pointed out that there was no significant difference between online and offline interventions in improving the well-being of healthcare professionals, and the latter had very low acceptance and adherence rates. Research from Poland noted that e-mental health interventions had limited effectiveness in improving burnout and depression in healthcare workers, whether immediately postintervention or at follow-up [28]. Marshall et al. noted that much of the existing research has been conducted by program companies, and more empirical evidence is needed regarding the safety and effectiveness of e-mental health services [29].
Most of the previous research has focused on the entire population, adolescents, older adults, or patients with psychological disorders, and few studies have specifically paid attention to the effects of e-mental health interventions on healthcare professionals [30]. In this systematic review and meta-analysis, the three most common psychological symptoms (stress, anxiety, and depression symptoms) were selected as outcomes to determine the effectiveness of e-mental health services. With the combination of subgroup analysis and meta-regression, we further determined the most effective intervention method. The findings can provide knowledge for psychological interventions and evidence for promoting the well-being of healthcare personnel.
Methods
In accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement, a review protocol was first developed and then registered in the PROSPERO database at the Center for Reviews and Dissemination in the United Kingdom (CRD42022360906) [31].
Search strategies
A three-step extensive search was performed according to the procedures proposed by the Cochrane Handbook for Systematic Reviews of Interventions [32]. First, we searched PubMed, Embase, Cochrane Library, PsycINFO, Web of Science, ProQuest Dissertations and Theses, China National Knowledge Infrastructure (CNKI), and Wanfang electronic databases from inception to April 13, 2022, to identify eligible trials and performed a second search on March 10, 2023. Second, a search was conducted for unpublished trials from a variety of clinical trial registries. Finally, the reference lists of similar systematic reviews and the included studies, grey literature, and target journals were hand-searched to maximize the number of potential trials.
Specific index terms and keywords were developed in line with the syntax rules of the databases to identify all relevant studies. Medical subject heading (MeSH) terms or database subject headings, including ‘health personnel’, ‘doctor’, ‘anxiety disorders’, ‘occupational stress’, ‘e-mental health’, ‘e-therapy’, ‘mental health services’, and ‘health education’, along with truncated and expanded keywords, were used to identify all available relevant studies. The detailed search strategy used for the eight databases is reported in the Supplementary material (Table S2).
There was no language filter applied during the identification of the studies to ensure that all relevant studies were identified [33]. Studies with titles or abstracts that met the selection criteria were selected for full-text evaluation by two independent reviewers, and the reasons for exclusion were documented. A third author was involved in the resolution of any disagreements [34]. All relevant abstracts from the databases were uploaded into EndNote X8. The two reviewers independently removed duplicated abstracts and then reviewed the remaining abstracts for inclusion. Original authors of relevant trials were contacted via e-mail to request missing information or to seek clarification.
Study eligibility criteria
Eligibility criteria were defined using the Population, Intervention, Comparison, and Outcome (PICO) framework. Studies qualified if they (1) involved healthcare professionals; (2) focused on e-mental health interventions; (3) included a control group with no treatment, usual care, or an active intervention; (4) measured at least one of the following outcomes: stress, depression, or anxiety; and (5) were randomized controlled trials (RCTs). Studies were excluded if they were (1) non-experimental studies; (2) discussion or review papers; (3) trial protocols; or (4) only abstracts. The specific eligibility criteria are reported in the Supplementary material (Table S3).
Study selection
Two researchers (ZZM and YJH) screened the literature independently. Initial screening was performed by reading the titles and abstracts according to the inclusion and exclusion criteria. Then, for the remaining literature, a thorough full-text reading and further screening were conducted. Controversies were resolved by consensus of the two reviewers and were finalized by a third reviewer (VXW).
Data extraction
A data extraction form for randomized trials was downloaded from the Cochrane Handbook for Systematic Reviews of Interventions, and data extraction was performed independently by two researchers [35]. The extracted information included the author, publication year, country, setting, research design, targeted population, age, sample size, intervention, follow-up, comparator, outputs, attrition rate, intention-to-treat (ITT), and missing data management. Information about e-mental health interventions, including the name, content, duration, follow-up, and outcome measures, was also extracted. Two researchers completed data extraction and then checked for differences, which were finalized by a third researcher.
Quality assessment
The Cochrane risk-of-bias tool was used for bias evaluation [32]. Two researchers independently assessed the risk of bias for the included RCTs, including seven items: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective outcome reporting, and other biases. Each item score corresponds to one of three levels: high risk, low risk, and unclear risk.
The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) framework was used to evaluate the overall quality of the literature [36]. The evaluation was conducted considering the following factors: risk of bias, inconsistency, indirectness, imprecision, and other considerations. Each factor corresponds to one of three levels: not serious, serious, and very serious. The differences were assessed by a third researcher and agreed upon before being finalized.
Statistical analysis
Data were collated and analyzed using RevMan 5.4.1 statistical software [37]. As psychological states involve continuous data measured on different scales, the standardized mean difference (SMD) was chosen as the effect indicator for analysis. All summary analyses were based on the DerSimonian and Laird random-effects model and inverse-variance method. When a random-effects model is used, the weights of each study are more similar, and the confidence interval for the result is wider. However, when the conditions of each study are different, the studies are more reflective of the real world [38]. If a study only provided the size of the sample and the data of the quartile, then the previous study was referred to estimate the mean and standard deviation [39, 40]. If a study did not provide enough data for meta-analysis, the corresponding authors were contacted by e-mail. If there was no response from the corresponding authors, descriptive statistics of the data were used.
Cochran’s Q (chi-square test) and I2 statistics were used to assess the heterogeneity of the included studies. For the significance level of the chi-square test, p < 0.05 indicated significant statistical heterogeneity. The I2 value provided information on the magnitude of heterogeneity, with values below 40% considered not important, 30% to 60% indicating moderate heterogeneity, 50 to 90% indicating severe heterogeneity, and 75 to 100% indicating significant heterogeneity [35].
Sensitivity, subgroup, and meta-regression analyses were performed when the heterogeneity was significant. Sensitivity analysis involved systematically excluding one study at a time to assess its impact on the effect size and heterogeneity [32]. Subgroup analysis was conducted to explore the sources of heterogeneity and examine the differences in the effectiveness of e-mental health interventions across different groups [41]. Based on the literature, we identified the types of subgroups [34, 42]. Predefined subgroups included the publication year, research design, occupation type, intervention context, intervention tool, intervention technology, intervention approach, supportive system, and comparators. Meta-regression was performed using Stata 16.1 software to determine whether the heterogeneity between studies was influenced by specific covariates [38]. Random-effect univariate regression models were used to assess the effect of the sample size and duration of the intervention on the effectiveness of the intervention among healthcare providers. We adopted a significance level of p < 0.05 [43].
Publication bias was assessed when ten or more studies were included [44]. To examine publication bias, contour-enhanced funnel plots were used to calculate the comparison between each effect size and standard deviation. We also used the graphical test method proposed by Egger et al. to visually evaluate the funnel plot and determine the impact of publication bias on the interpretation of the results [43].
Results
Search results
A total of 6771 articles were obtained from searches of the eight databases combined with manual searches. After eliminating duplicates, 4365 articles were further screened. After reading the titles and abstracts, 4265 articles were excluded. Eighty-nine articles were further excluded after reading the full text. Ultimately, 20 articles were included in the systematic review. Due to missing data or differences in analysis methods in some articles, we only conducted a meta-analysis on 17 articles. The literature screening process and studies are shown in Fig. 1.
Study characteristics
Of the 20 studies included, seven were from China [45,46,47,48,49,50,51], five were from the USA [52,53,54,55,56], and the rest were from Australia [57], Germany [58], Ireland [59], the Netherlands [60], Poland [28], Spain [61], Turkey [62], and Vietnam [63]. There were fifteen RCTs with two arms, four RCTs with three arms, and one RCT with four arms. The sample size ranged from 36 to 1240 participants. Eleven studies targeted nurses, one targeted doctors, and eight targeted mixed populations. The psychological interventions included mindfulness-based therapy, cognitive–behavioral therapy (CBT), stress management, emotional freedom techniques (EFTs), and mixed techniques. Fourteen studies used individual interventions, and six studies applied group interventions. The intervention duration ranged from ten days to half a year. The highest dropout rate was 82.15%. In addition to stress, anxiety, and depression symptoms, some studies also included burnout, well-being, work engagement, and self-efficacy as outcome variables. The characteristics and interventions of each study are shown in the Supplementary material (Tables S5 and S6).
Risk of bias within studies
Among the included studies, only two were classified as having a low risk of bias for each item [57, 61]. Two studies were classified as having a high risk of bias for randomization [55, 56]. Most studies did not clarify the ways in which allocation concealment or participant, researcher, or data processor blinding was implemented [28, 45,46,47,48,49,50,51,52,53,54,55,56, 58,59,60, 63]. Seven studies were classified as having a high risk of bias for incomplete data reporting due to high rates of participant dropout and a lack of explanation of the causes [28, 47, 53,54,55,56, 60]. A detailed quality assessment of the studies is shown in Supplementary material (Fig. S1).
Risk of bias across studies
According to the GRADE criteria, the overall quality of the studies related to the three outcome indicators was evaluated as low (see Supplementary Material Table S7). For stress and anxiety symptoms, the risk of inconsistency was downgraded due to the very large heterogeneity. For depression symptoms, the risk of bias was downgraded, as allocation concealment and blinding were not well implemented. A funnel plot was used to analyze publication bias regarding stress and anxiety symptoms. The funnel plot was symmetrical, and no study accepted sponsorship from stakeholders, which suggests that there was no obvious publication bias.
Results of the meta-analysis
Eleven studies including 1263 subjects were included in the meta-analysis on the outcome measure of stress. As shown in Fig. 2, stress symptoms were significantly improved in the e-mental health intervention group compared with the control group [SMD = − 1.21, 95% CI (− 1.85, − 0.56), P < 0.05]. Ten studies were included in the meta-analysis on symptoms of anxiety. As shown in Fig. 3, anxiety symptoms were significantly improved in the e-mental health intervention group compared with the control group [SMD = − 0.83, 95% CI (− 1.29, − 0.37), P < 0.05]. As shown in Fig. 4, from the meta-analysis of the remaining nine studies, depression was moderately reduced in the intervention group [SMD = − 0.30, 95% CI (− 0.49, − 0.11), P < 0.05]. Due to the significant heterogeneity [stress symptoms: I2 = 96%, P < 0.05; anxiety: I2 = 94%, P < 0.05; depression symptoms: I2 = 65%, P < 0.05], we conducted a sensitivity analysis. After removing the studies one by one, the meta-analysis was not affected, indicating that the results were stable.
Subgroup analysis
For stress symptoms, significant differences were noted in the effect of the interventions among the following subgroups: publication year, design, occupation type, intervention tool, intervention technology, intervention approach, supportive system, and control group. For anxiety symptoms, significant differences were observed among the publication year, occupation type, intervention context, intervention tool, intervention technology, intervention approach, and supportive system subgroups. For depression symptoms, there were significant differences in the effects among the region, occupation type, intervention technology, intervention approach, and supportive system subgroups. This partly explained the sources of heterogeneity. Notably, certain subgroups had a few studies or exhibited high heterogeneity, thus warranting caution in the generalizability of our results. The detailed data are shown in Table 1.
In terms of publication year, articles published after 2020 had significantly higher intervention effects than those published before 2020 (stress symptoms: SMD (≥ 2020) = − 1.94, Z = 2.90, P < 0.05 versus SMD (< 2020) = − 0.41, Z = 4.14, P < 0.05; anxiety symptoms: SMD (≥ 2020) = − 1.06, Z = 3.65, P < 0.05 versus SMD (< 2020) = − 0.07, Z = 0.58, P > 0.05; and depression symptoms: SMD (≥ 2020) = − 0.36, Z = 3.02, P < 0.05 versus SMD (< 2020) = − 0.08, Z = 0.62, P > 0.05).
For intervention groups, more studies focused on nurses compared to other healthcare professionals, and the intervention effect was better. With regard to stress and depression symptoms, the intervention effect was significantly better for nurses than for all other healthcare staff (stress symptoms: SMD (nurses) = − 2.08, Z = 3.26, P < 0.05 versus SMD (all) = − 0.22, Z = 2.81, P < 0.05; depression symptoms: SMD (nurses) = − 0.49, Z = 3.33, P < 0.05 versus SMD (all) = − 0.04, Z = 0.58, p > 0.05), and the difference between the groups was significant (stress symptoms: χ2 = 8.41, p > 0.05; depression symptoms: χ2= 7.26, p > 0.05).
Regarding stress and depression symptoms, mindfulness therapy exhibited a better effect than CBT (stress symptoms: SMD (CBT) = − 0.93, Z = 2.81, P < 0.05 versus SMD (mindfulness) = − 1.50, Z = 1.48, P < 0.05; depression symptoms: SMD (CBT) = − 0.26, Z = 1.41, p > 0.05 versus SMD (mindfulness) = − 0.47, Z = 2.53, P < 0.05). However, there was no significant difference between the groups. For anxiety symptoms, EFTs showed a better therapeutic effect, followed by CBT and mindfulness therapy, and then mixed interventions, with significant differences between the groups.
Regarding intervention technology, online course interventions were significantly better than the other two methods, namely, the app and website methods, and there were significant differences between the groups (stress symptoms: SMD (app) = − 0.24 versus SMD (online course) = − 3.66 versus SMD (website) = − 0.64, χ2 = 23.27, P < 0.05; anxiety symptoms: SMD (app) = − 0.16 versus SMD (online course) = − 2.53 versus SMD (website) = − 0.47, χ2=7.13, P < 0.05; depression symptoms: SMD (app) = − 0.16 versus SMD (online course) = − 0.89 versus SMD (website) = − 0.13, χ2=18.09, P < 0.05).
Group interventions seemed to be more effective than individual interventions (stress symptoms: SMD = − 2.83, 95% CI [− 4.95, − 0.70], Z = 2.61, P < 0.05 for group versus SMD = − 0.32, 95% CI [− 0.49, − 0.16], Z = 3.82, P < 0.05 for individual; anxiety symptoms: SMD = − 2.53, 95% CI [− 4.35, − 0.72], Z = 2.74, P < 0.05 for group versus SMD = − 0.24, 95% CI [− 0.43, − 0.06], Z = 2.54, P < 0.05 for individual; depression symptoms: SMD = 0.89, 95% CI [− 1.21, − 0.56], Z = 5.37, P < 0.05 for group versus SMD = − 0.15, 95% CI [− 0.25, − 0.05], Z = 2.89, P < 0.05 for individual). Significant differences were found within a specific subgroup (stress symptoms: χ2 = 5.31, p < 0.05; anxiety: χ2 = 6.07, p < 0.05; depression symptoms: χ2 = 18.05, p < 0.05), indicating that the intervention effect size varied significantly within this subgroup.
Professional support interventions were more effective than self-guided interventions (stress symptoms: SMD (professional support) = 2.28, 95% CI [− 3.94, − 0.61], Z = 2.68, P < 0.05 versus SMD (self-guided interventions) = − 0.36, 95% CI [− 0.55, − 0.17], Z = 3.68, P < 0.05; anxiety symptoms: SMD (professional support) = − 2.53, 95% CI [− 4.35, − 0.72], Z = 2.74, P < 0.05 versus SMD (self-guided interventions) = − 0.24, 95% CI [− 0.43, − 0.06], Z = 2.54, P < 0.05; depression symptoms: SMD (professional support) = − 0.89, 95% CI [− 1.21, − 0.56], Z = 5.37, P < 0.05 versus SMD (self-guided interventions) =− 0.15, 95% CI [− 0.25, − 0.05], Z = 2.89, P < 0.05). Significant differences were found within a specific subgroup (stress symptoms: χ2 = 5.03, p < 0.05; anxiety: χ2 = 6.07, p < 0.05; depression symptoms: χ2= 18.05, p < 0.05), indicating that the intervention effect size varied significantly within this subgroup.
Meta-regression
Meta-regression demonstrated that the intervention duration (β = 0.027, P = 0.04) had a significant effect on the relief of anxiety symptoms. For stress and depressive symptoms, the intervention duration and sample size had no significant impact. Detailed results are displayed in Table 2.
Other outcomes
Other outcomes in the studies included burnout, well-being, work engagement, self-efficacy, satisfaction, attention, somatic symptoms, sleep quality, trauma exposure, compassion, social support, work-life integration, and coping style, among which burnout and well-being were most commonly included, with five studies each. From the results shown in Table 3, e-mental health interventions significantly alleviated emotional exhaustion [SMD = − 0.91, 95% CI (− 1.33, − 0.50), P < 0.05] and improved the well-being of subjects [SMD = 0.38, 95% CI (0.01, 0.75), P < 0.05] but had no impact on depersonalization and personal accomplishment.
Discussion
To our knowledge, this review appears to be among the first to provide a systematic and comprehensive analysis of the impact of e-mental health interventions on the well-being of healthcare staff in a professional context [53]. Our results were derived from 17 RCTs, totaling 2431 subjects. Compared with the control groups, the e-mental health intervention groups showed significant effects on stress, anxiety, and depression symptoms. Subgroup analysis demonstrated a series of moderators, including intervention context and supportive system.
E-mental health interventions exhibited large effects in mitigating stress and anxiety symptoms among healthcare professionals and showed small-to-moderate effects in reducing depression symptoms, which follows the general trends reported in related studies. In another review, a significant effect (Hedge’s g = 0.51, P < 0.05) that was lower than that in our study was observed in the reduction in stress symptoms in the general population [64]. This may be due to the higher health literacy of healthcare staff, so more obvious effects were observed after the interventions. Another meta-analysis showed a significant effect (Cohen’s d = − 0.91, P < 0.05), which is comparable to our results for generalized anxiety disorders [65]. Similar to our findings, e-mental health interventions were determined to have a significant effect on depression symptoms (Hedge’s g = 0.30, P < 0.05) based on 17 studies in another meta-analysis [24]. Excessive workloads, prolonged overtime hours, competitive coworker relationships, and the occurrence of patient death have blurred work-life boundaries and negatively impacted the psychological well-being of healthcare workers [65]. Additionally, healthcare professionals usually have busy work schedules and lack time to attend face-to-face psychological sessions. E-mental health interventions provide a channel for healthcare professionals to focus on their own psychological health. Specifically, e-mental health services are characterized by eliminating time and space constraints to improve accessibility and compliance [66]. By acquiring relaxation skills and emotional support, healthcare professionals can alleviate inner discomfort and adjust their cognitive thinking, which in turn can help them communicate well with others and improve their quality of life [67]. In light of the existing research, e-mental health interventions can be complementary or alternatives to face-to-face interventions for healthcare professionals.
The results also supported the positive effects of e-mental health interventions for some secondary outcomes, indicating that the research had good systematic consistency. In terms of well-being, the intervention effect we obtained had an SMD = 0.38 (95% CI 0.01‒0.75, P < 0.05). Phillips et al. reported an effect size on well-being of Hedge’s g = 0.35 (95% CI 0.25‒0.46, P < 0.0001) based on seven studies [24]. For burnout, a significant effect of the intervention was observed in reducing emotional exhaustion (SMD = − 0.91, Z = 4.28, P < 0.05), but no significant effect was observed in reducing depersonalization and establishing a sense of achievement. Another study found that online interventions had moderate-to-large effects on job burnout [68]. These conflicting conclusions may exist because we only considered burnout as a secondary outcome and only included a small number of studies.
The subgroup analysis showed that mindfulness meditation techniques had a higher effect size than other psychological techniques, such as CBT. However, this result should be interpreted with caution, as the sample size of each subgroup was small, and the heterogeneity was high. Mindfulness refers to a state of consciousness created by directing attention to the present goal and indulging in the moment without distractions or judgment [69]. As a lifestyle-based intervention, mindfulness is cost-effective and has no side effects [70]. In addition to stress and depression, researchers have found that it can also alleviate other physiological problems, such as PTSD, pain, and eating disorders [71]. In the future, other psychological interventions, such as acceptance and commitment therapy, life-review therapy, and dialectical behavior therapy, could be integrated into e-mental health interventions to further explore their effectiveness in improving psychological health [72]. Our results also indicated that computer-based online interventions were more effective than mobile apps. However, the applicability of computer-based interventions may be potentially limited by the high heterogeneity of the corresponding subgroups, and medical personnel who attend online courses may also use their mobile phones to browse health information in their daily lives. Our explanatory subgroup analyses verified the advantage of professional support and group interventions. Previous reviews have revealed that guided psychotherapy is more effective than unguided psychotherapy owing to external pressure fostering individual accountability and positive reinforcement [73]. Baumeister and colleagues reported that internet-based mental health interventions (IMIs) together with therapeutic support had a significantly lower dropout rate (odds ratio = 2.67) and achieved a greater reduction in symptoms (Hedge’s g = − 0.27, P < 0.05) than IMIs without such support [74]. Some studies have suggested that group interventions can enhance mutual interaction and trust, facilitate emotional expression, and thus improve effectiveness [73]. However, other studies have reported that group interventions are less effective than individual delivery due to inadequate attention for each individual [75]. Therefore, it is necessary to further explore customized interventions based on the personal characteristics and job requirements of healthcare personnel to ensure program effectiveness.
According to the meta-regression, the intervention duration had a significant slight impact on anxiety symptoms (β = 0.027, p = 0.040) but no impact on the relief of stress and depression symptoms, suggesting that the design of the program should be based on clinical symptoms and that short-term interventions are more cost-effective [76]. It is also worth noting that this result could potentially be attributed to chance, as the value of 0.04 is truly at the threshold. From the meta-regression, the number of people included in the RCTs did not affect the final outcome. From the subgroup analysis, no significant difference was found with different types of control groups. In the subgroup analysis, no significant difference was found when comparing non-active control groups (no treatment, waiting list, treatment-as-usual) and active control groups (placebos such as general recommendations about mental health care, or relaxation sessions) with the intervention groups. Of course, more research on various intervention forms (e-package, social media platforms, virtual care, online platforms, online games) is needed to confirm this.
Strengths and limitations
We performed an extensive and systematic literature review, including references to related reviews and RCTs. Then, we further performed subgroup analysis and meta-regression to identify the relative effect size of various moderators, which provides guidance for future research and practical design. Notably, publication biases were not detected in the studies we included. Moreover, in addition to using the risk of bias tool to evaluate the quality of individual trials, we also used the GRADE criteria to evaluate the overall quality of evidence.
Several limitations should be considered before interpreting the findings. Firstly, the included trials exhibited clinical and statistical heterogeneity, which limited their comparability. To address this issue, we conducted a subgroup analysis to explore potential sources of heterogeneity. Moderators that emerged consistently across the four main outcomes—job, intervention technology, intervention approach, and supportive system—could explain the observed heterogeneity. Secondly, we found that the overall quality of evidence was deemed low, primarily due to issues with allocation concealment, blinding, and missing outcome data. These issues may have compromised the internal validity of our findings. Lastly, it is important to note that the outcomes of this review were primarily based on self-reported data, which could introduce recall and/or social desirability bias. Future trials may benefit from using established diagnostic criteria manuals, such as the DSM-5 (Diagnostic and Statistical Manual of Mental Disorders) or ICD-10 (International Classification of Diseases).
Implications for future research and practice
Given that the number of RCTs included was not very large, our systematic review suggested that more attention should be given to the psychological health of healthcare staff. In addition, larger sample size, multicenter, longitudinal, or follow-up studies are needed. Considering the low quality of the studies, rigorous RCTs following the CONSORT statement are recommended for future trials, especially efforts to minimize selection bias by performing adequate allocation concealment and reducing performance bias by blinding participants and researchers. Addressing emerging issues such as inequalities caused by the digital divide, the potential disclosure of privacy, and the lack of user trust is necessary. Future research could consider using objective outcomes such as biological markers or more sophisticated and intensive e-mental health approaches [77]. Based on the principles of user-centered designs and incorporating feedback from users and providers, e-mental health interventions also require codesigning and integrating cross-disciplinary collaboration among psychology, health science, and computer professionals [78].
Conclusion
Mental health interventions were found to significantly alleviate stress, anxiety, depression symptoms, and other emotional issues among healthcare professionals. Subgroup analyses showed that features including mindfulness approaches, online courses, computer use, group interventions, and professional guidance are more favorable in the design of services. Meta-regression showed that the intervention duration, as a covariate, only had an impact on preventing or reducing symptoms of anxiety, while the sample size did not affect any of the three primary symptoms. Our findings on efficacy outcomes should be interpreted cautiously due to the high heterogeneity observed in some analyses and the GRADE approach indicating a low quality of evidence. E-mental health interventions have the potential to become widely available and evidence-based interventions to address mental health issues. Future RCTs should adhere to the recommendations of the CONSORT statement, as more high-quality trials are warranted to assess the effectiveness of e-mental health interventions among healthcare professionals.
Availability of data and materials
Supplemental material for this paper is available online. Other information will be made available upon request to the author.
References
Ye C, Schröder P, Yang D, Chen M, Cui C, Zhuang L. Toward healthy and liveable cities: a new framework linking public health to urbanization. Environ Res Lett. 2022;17:064035.
Arnold-Forster A, Moses JD, Schotland SV. Obstacles to Physicians’ Emotional Health — Lessons from History. N Engl J Med. 2022;386:4–7.
World Health Organization. World mental health report: transforming mental health for all (Executive summary). Geneva: World Health Organization; 2022. Available from: https://www.who.int/publications/i/item/9789240050860
Huang Y, Wang Y, Wang H, Liu Z, Yu X, Yan J, et al. Prevalence of mental disorders in China: a cross-sectional epidemiological study. Lancet Psychiatry. 2019;6:211–24.
Daly M, Sutin AR, Robinson E. Longitudinal changes in mental health and the COVID-19 pandemic: evidence from the UK Household Longitudinal Study. Psychol Med. 2022;52:2549–58.
Wainberg ML, Scorza P, Shultz JM, Helpman L, Mootz JJ, Johnson KA, et al. Challenges and opportunities in global mental health: a research-to-practice perspective. Curr Psychiatry Rep. 2017;19:28.
Cunningham AT, Felter J, Smith KR, Sifri R, Arenson C, Patel A, et al. Burnout and commitment after 18 months of the COVID-19 pandemic: a follow-up qualitative study with primary care teams. J Am Board Fam Med. 2023;36:105–17.
Htay MNN, Marzo RR, AlRifai A, Kamberi F, El-Abasiri RA, Nyamache JM, et al. Immediate impact of COVID-19 on mental health and its associated factors among healthcare workers: A global perspective across 31 countries. J Global Health. 2020;10:020381.
Bohlken J, Schömig F, Lemke MR, Pumberger M, Riedel-Heller SG. COVID-19-Pandemie: Belastungen des medizinischen Personals: Ein kurzer aktueller Review. Psychiat Prax. 2020;47:190–7.
Pappa S, Ntella V, Giannakas T, Giannakoulis VG, Papoutsi E, Katsaounou P. Prevalence of depression, anxiety, and insomnia among healthcare workers during the COVID-19 pandemic: a systematic review and meta-analysis. Brain Behav Immun. 2020;88:901–7.
Demi̇R S, Karatepe H, Özçeli̇K T, Akkuş G, Ovali H. Psychological readiness to normalization process and burnout level of healthcare workers of pediatric clinic after three months of COVID-19 pandemic. JCBPR. 2021;10:137–45.
Bhugra D, Sauerteig S-O, Bland D, Lloyd-Kendall A, Wijesuriya J, Singh G, et al. A descriptive study of mental health and wellbeing of doctors and medical students in the UK. Int Rev Psychiatry. 2019;31:563–8.
Lo D, Wu F, Chan M, Chu R, Li D. A systematic review of burnout among doctors in China: a cultural perspective. Asia Pac Fam Med. 2018;17:3.
Forbes MP, Iyengar S, Kay M. Barriers to the psychological well-being of Australian junior doctors: a qualitative analysis. BMJ Open. 2019;9:e027558.
Kwon C-Y, Lee B, Kwon O-J, Kim M-S, Sim K-L, Choi Y-H. Emotional labor, burnout, medical error, and turnover intention among South Korean nursing staff in a university hospital setting. IJERPH. 2021;18:10111.
WHO Group Consultation on Health Telematics. A health telematics policy: in support of WHO’s Health-for-All strategy for global health development. Geneva: World Health Organization; 1997.
Christensen H, Griffiths K, Evans K. e-Mental health in Australia: implications of the internet and related technologies for policy. Canberra: Commonwealth Department of Health and Ageing; 2002.
Montagni I, Tzourio C, Cousin T, Sagara JA, Bada-Alonzi J, Horgan A. Mental health-related digital use by university students: a systematic review. Telemed e-Health. 2020;26:131–46.
Strudwick G, Impey D, Torous J, Krausz RM, Wiljer D. Advancing e-mental health in canada: report from a multistakeholder meeting. JMIR Ment Health. 2020;7:e19360.
Köhnen M, Dirmaier J, Härter M. Potenziale und Herausforderungen von E-Mental-Health-Interventionen in der Versorgung psychischer Störungen. Fortschr Neurol Psychiatr. 2019;87:160–4.
Mol M, Van Genugten C, Dozeman E, Van Schaik DJF, Draisma S, Riper H, et al. Why uptake of blended internet-based interventions for depression is challenging: a qualitative study on therapists’ perspectives. JCM. 2019;9:91.
Victor AM. Telepsychiatry and comprehensive mental health India. Perspect Psychiatr Care. 2019;55:459–63.
Christensen LF, Moller AM, Hansen JP, Nielsen CT, Gildberg FA. Patients’ and providers’ experiences with video consultations used in the treatment of older patients with unipolar depression: a systematic review. J Psychiatr Ment Health Nurs. 2020;27:258–71.
Phillips EA, Gordeev VS, Schreyögg J. Effectiveness of occupational e-mental health interventions: a systematic review and meta-analysis of randomized controlled trials. Scand J Work Environ Health. 2019;45:560–76.
Hidalgo-Mazzei D, Mateu A, Reinares M, Matic A, Vieta E, Colom F. Internet-based psychological interventions for bipolar disorder: review of the present and insights into the future. J Affect Disord. 2015;188:1–13.
Grover S, Sarkar S, Gupta R. Data Handling for E-Mental Health Professionals. Indian J Psychol Med. 2020;42:85S-91S.
Wang J, Lam RW, Ho K, Attridge M, Lashewicz BM, Patten SB, et al. Preferred features of e-mental health programs for prevention of major depression in male workers: results from a Canadian National Survey. J Med Internet Res. 2016;18:e132.
Smoktunowicz E, Lesnierowska M, Carlbring P, Andersson G, Cieslak R. Resource-Based Internet Intervention (Med-Stress) to Improve Well-Being Among Medical Professionals: Randomized Controlled Trial. J Med Internet Res. 2021;23:e21445.
Marshall JM, Dunstan DA, Bartik W. Clinical or gimmickal: The use and effectiveness of mobile mental health apps for treating anxiety and depression. Aust N Z J Psychiatry. 2020;54:20–8.
Wozney L, McGrath PJ, Gehring ND, Bennett K, Huguet A, Hartling L, et al. eMental healthcare technologies for anxiety and depression in childhood and adolescence: systematic review of studies reporting implementation outcomes. JMIR Ment Health. 2018;5:e48.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. Ann Intern Med. 2009;151:264–9.
Higgins J. Cochrane handbook for systematic reviews of interventions. Version 5.1. 0 [updated March 2011]. The Cochrane Collaboration. www cochrane-handbook org. 2011
Wu VX, Chi Y, Lee JK, Goh HS, Chen DYM, Haugan G, et al. The effect of dance interventions on cognition, neuroplasticity, physical function, depression, and quality of life for older adults with mild cognitive impairment: A systematic review and meta-analysis. Int J Nurs Stud. 2021;122:104025.
Lim SM, Shiau CWC, Cheng LJ, Lau Y. Chatbot-delivered psychotherapy for adults with depressive and anxiety symptoms: a systematic review and meta-regression. Behav Ther. 2022;53:334–47.
Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane handbook for systematic reviews of interventions. Chichester: Wiley; 2019.
Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64:383–94.
Cochrane Collaboration. Review Manager (RevMan)[Computer program]. Version 5.4. London: The Cochrane Collaboration; 2020 [cited 2021 Aug 23]. 2020
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Method. 2010;1:97–111.
Luo D, Wan X, Liu J, Tong T. Optimally estimating the sample mean from the sample size, median, mid-range, and/or mid-quartile range. Stat Methods Med Res. 2018;27:1785–805.
Wan X, Wang W, Liu J, Tong T. Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Med Res Methodol. 2014;14:135.
Sedgwick P. Meta-analyses: heterogeneity and subgroup analysis. BMJ. 2013;346:f4040–f4040.
Stratton E, Lampit A, Choi I, Calvo RA, Harvey SB, Glozier N. Effectiveness of eHealth interventions for reducing mental health conditions in employees: A systematic review and meta-analysis. Reed P, editor. PLoS ONE. 2017;12:e0189904.
Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315:629–34.
Sterne JAC, Sutton AJ, Ioannidis JPA, Terrin N, Jones DR, Lau J, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002–d4002.
Chang S, He C, Lu Y, Chen R, Qin B, Zhou Y. Effect of network structured group psychological intervention on psychological crisis of front-line nurses in COVID-19 isolation ward medical innovation of china. Med Innovat China. 2018;18:107–12.
Deng K, Yuan X, Hu L, Yang W, Chen X. Influence of Internet unstructured group counseling on negative emotions and stress of nurses in the front line of prevention and treatment of novel coronavirus pneumonia. J Nurs (China). 2020;27:64–7.
Huang Y. Application study of solution-focused approach intervention in the mobile internet background. Zhejiang: Zhejiang Sci-Tech University; 2016.
Wang C, Chen X. Study on the intervention of internet-based mindfulness based stress reduction therapy on the psychological condition of nursing staff during the outbreak. Chinese Comm Doctors. 2020;36:145–6.
Wang Q. Effects of online mindfulness--based stress reduction training on the well-being and work stress of ICU nurses. Zhengzhou: Zhengzhou University; 2021.
Wu C, Qu Q, Liao qian, Mo J, Shu L, Zhao W, et al. Impact of mobile phone APP guided mindfulness training on pregnant nurses pressure coping methods and self-efficacy. J Nurs Train. 2020;35:193–7+203.
Xie G. Intervention study of MoodGYM internet-delivered cognitive behavioral therapy on the secondary traumatic stress of medical staff. Guangzhou: Southern Medical University; 2020.
Hersch RK, Cook RF, Deitz DK, Kaplan S, Hughes D, Friesen MA, et al. Reducing nurses’ stress: A randomized controlled trial of a web-based stress management program for nurses. Appl Nurs Res. 2016;32:18–25.
Lambert KG, Aufricht WR, Mudie D, Brown LH. Does a phone-based meditation application improve mental wellness in emergency medicine personnel? Am J Emerg Med. 2020;38:2740–1.
Low V, Gebhart B, Reich C. Effects of a worksite program to improve the cardiovascular health of female health care workers. J Cardiopulm Rehabil Prev. 2015;35:342–7.
Mistretta EG, Davis MC, Temkit M, Lorenz C, Darby B, Stonnington CM. Resilience training for work-related stress among health care workers: results of a randomized clinical trial comparing in-person and smartphone-delivered interventions. J Occup Environ Med. 2018;60:559–68.
Profit J, Adair KC, Cui X, Mitchell B, Brandon D, Tawfik DS, et al. Randomized controlled trial of the “WISER” intervention to reduce healthcare worker burnout. J Perinatol. 2021;41:2225–34.
Xu H, Eley R, Kynoch K, Tuckett A. Effects of mobile mindfulness on emergency department work stress: a randomised controlled trial. Emerg Med Australas. 2022;34:176–85.
Gollwitzer PM, Mayer D, Frick C, Oettingen G. Promoting the self-regulation of stress in health care providers: an internet-based intervention. Front Psychol. 2018;9:838.
Dunne PJ, Lynch J, Prihodova L, O’Leary C, Ghoreyshi A, Basdeo SA, et al. Burnout in the emergency department: Randomized controlled trial of an attention-based training program. J Integrat Med. 2019;17:173–80.
Bolier L, Ketelaar SM, Nieuwenhuijsen K, Smeets O, Gärtner FR, Sluiter JK. Workplace mental health promotion online to enhance well-being of nurses and allied health professionals: a cluster-randomized controlled trial. Internet Interventions. 2014;1:196–204.
Fiol-DeRoque MA, Serrano-Ripoll MJ, Jiménez R, Zamanillo-Campos R, Yáñez-Juan AM, Bennasar-Veny M, et al. A mobile phone–based intervention to reduce mental health problems in health care workers during the COVID-19 pandemic (PsyCovidApp): Randomized Controlled Trial. JMIR Mhealth Uhealth. 2021;9:e27039.
Dincer B, Inangil D. The effect of Emotional Freedom Techniques on nurses’ stress, anxiety, and burnout levels during the COVID-19 pandemic: a randomized controlled trial. Explore. 2021;17:109–14.
Imamura K, Tran TTT, Nguyen HT, Sasaki N, Kuribayashi K, Sakuraya A, et al. Effect of smartphone-based stress management programs on depression and anxiety of hospital nurses in Vietnam: a three-arm randomized controlled trial. Sci Rep. 2021;11:11353.
Spijkerman MPJ, Pots WTM, Bohlmeijer ET. Effectiveness of online mindfulness-based interventions in improving mental health: a review and meta-analysis of randomised controlled trials. Clin Psychol Rev. 2016;45:102–14.
Richards D, Richardson T, Timulak L, McElvaney J. The efficacy of internet-delivered treatment for generalized anxiety disorder: A systematic review and meta-analysis. Internet Interventions. 2015;2:272–82.
Schieman S, Badawy PJ, A. Milkie M, Bierman A. Work-Life Conflict During the COVID-19 Pandemic. Socius. 2021;7:237802312098285.
Drissi N, Ouhbi S, Marques G, De La Torre Díez I, Ghogho M, Janati Idrissi MA. A Systematic literature review on e-mental health solutions to assist health care workers during COVID-19. Telemed e-Health. 2021;27:594–602.
Widberg C, Wiklund B, Klarare A. Patients’ experiences of eHealth in palliative care: an integrative review. BMC Palliat Care. 2020;19:158.
Hofer P, Waadt M, Aschwanden R, Milidou M, Acker J, Meyer A, et al. Self-help for stress and burnout without therapist contact: An online randomised controlled trial. Work Stress. 2018;32:189–208.
Zhihong Ren, Yawen Zhang, Guangrong Jiang. Effectiveness of mindfulness meditation in intervention for anxiety: a meta-analysis. Acta Psychologica Sinica. 2018;50:283–305.
Leow Y, Rashid NLBA, Klainin-Yobas P, Zhang Z, Wu XV. Effectiveness of mindfulness-based interventions on mental, cognitive outcomes and neuroplastic changes in older adults with mild cognitive impairment: a systematic review and meta-analysis. J Adv Nurs. 2023;12:4489–505.
Geiger PJ, Boggero IA, Brake CA, Caldera CA, Combs HL, Peters JR, et al. Mindfulness-based interventions for older adults: a review of the effects on physical and emotional well-being. Mindfulness. 2016;7:296–307.
Weiss L, Westerhof G, Bohlmeijer E. Can we increase psychological well-being? The effects of interventions on psychological well-being: a meta-analysis of randomized controlled trials. PLOS ONE. 2016;11:e0158092.
Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on Internet-based mental health interventions — a systematic review. Internet Interventions. 2014;1:205–15.
Shang X, Lin Z, Luo D, Bian Q, Wang M. Effect of positive psychological interventions in patients with inflammatory bowel disease:a systematic review. Chinese Nurs Res. 2018;32:3229–35.
Eberth J, Sedlmeier P. The effects of mindfulness meditation: a meta-analysis. Mindfulness. 2012;3:174–89.
Denecke K, Schmid N, Nüssli S. Implementation of cognitive behavioral therapy in e–mental health apps: literature review. J Med Internet Res. 2022;24:e27791.
Xie Q, Torous J, Goldberg SB. E-mental health for people with personality disorders: a systematic review. Curr Psychiatry Rep. 2022;24:541–52.
Acknowledgements
The authors are grateful to the reviewers for their invaluable suggestions.
Funding
This study was supported by a grant from the National Natural Science Foundation of China (grant number 72274068) and the China Scholarship Council (202106160062). The funding body does not play roles in study design, interpretation of data, and manuscript drafting.
Author information
Authors and Affiliations
Contributions
ZZ was involved in the study conception, design, data extraction, and writing of the manuscript. YH was involved in the study conception and overall review of the manuscript. SL and XF were involved in the study conception and data analysis. JY was involved in the independent review of articles during the study selection, data extraction process, and quality appraisal. LJC was involved in data analysis and critical review. SR was involved in a critical review. XVW was involved in the study conception, design, and review of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable as this is a review.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
13643_2024_2565_MOESM1_ESM.docx
Additional file 1: Supplementary Table S1. PRISMA checklist. Supplementary Table S2. Search terms of databases. Supplementary Table S3. Eligibility criteria. Supplementary Table S4. Data extraction table. Supplementary Table S5. Characteristics of included studies. Supplementary Table S6. Description of e-mental health intervention. Supplementary Table S7. Summary of Findings table. Supplementary Figure S1. Risk of bias within studies summary. Supplementary Figure S2. Funnel plot of stress. Supplementary Figure S3. Funnel plot of anxiety.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Zhang, Z., Hu, Y., Liu, S. et al. The effectiveness of e-mental health interventions on stress, anxiety, and depression among healthcare professionals: a systematic review and meta-analysis. Syst Rev 13, 144 (2024). https://doi.org/10.1186/s13643-024-02565-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13643-024-02565-6