Background

Strong research capacity is a key component to obtaining the evidence base for policies in pursuit of improved health outcomes [1]. One way on which research capacity has historically been strengthened has been through training, either via formal education degree programmes or short courses and workshops [2,3,4].

Strengthening capacity for sexual and reproductive health and rights (SRHR) research has the potential to contribute to decreasing existing inequities in the production of SRHR research as well as to supporting evidence based policy making and improved health outcomes [5]. SRHR research includes a variety of different topics that are inherently and politically charged in many environments, including abortion, family planning and contraception, gender and rights. SRHR research courses could cover commonly used methodologies in this field, including qualitative and quantitative methods or systematic reviewing specific to SRHR topics. These often-stigmatized topics pose additional challenges to learners [6,7,8].

Before the COVID-19 pandemic, most research capacity strengthening (RCS) learning activities in SRHR were conducted either face-to-face or through a blended format which combined remote learning and some in-person activities [9]. Emerging research suggests learning outcomes among healthcare professionals were achieved during the speedy shift towards online or other remote interactions during the COVID-19 pandemic [10,11,12]. Remote education is considered an approach that can potentially accelerate the transfer of skills and reduce some of the existing barriers in strengthening research capacity [3, 13]. The implementation of remote learning programmes can tame geographical barriers, increase student satisfaction, help reach a larger population, enhance collaborations, reduce costs, and give learners more control over their learning [14, 15].

Remote education fosters many of the principles of adult learning that posit that individuals learn by building on prior experiences, on their own belief systems, and on their autonomy and self-reliance, among others [16]. Remote learning platforms encourage learning by way of sharing opinions and ideas with others while simultaneously building connections with other online learners, enabling autonomous learning and allowing for the sharing of resources and experiences [17, 18].

Previous research has supported the use of remote learning to provide the opportunity to multiply access to education and facilitate contact with senior academics in other institutions [19,20,21]. Moreover, there is substantial evidence that healthcare professionals and to some extent undergraduate students participating in online education programmes can achieve similar learning outcomes as to face-to-face alternatives [20,21,22], and have strengthened sustainable research networks and communities of practice [23]. Remote education is not without challenges. For instance, educators require additional time to tailor material for different learners, there is a risk of information overload, as well as limited space for social interactions and networking. Furthermore low internet connectivity remains in many world regions and healthcare professionals may not be granted protected time for training [24,25,26,27].

Irrespective of the reported benefits and barriers of online learning, the effectiveness, potential impact and sustainability of fully remote training programmes in SRHR remains unclear [28], particularly given inherent sensitivities that surround many SRHR related issues in certain contexts, including sexuality and sexual health, abortion, contraception, and violence, among others [29]. Despite the rise in both local and global initiatives to expand RCS activities through training and education, there still remains a level of uncertainty on how to best strengthen and deliver capacity strengthening learning activities [30], and on how to ensure the goals achieved are sustained and result in improved outcomes [31].

This study aimed to conduct a systematic review to describe the effectiveness of remote education programmes to strength research capacity in SRHR, as well as to describe enablers and barriers linked to remote training from the perspective of researchers and study participants. We consider remote education as any training (exclusive of degree programmes) that is offered using the internet or other remote connectivity options, whether synchronous or asynchronous either on SRHR specific topics or research training tailored to SRHR professionals. The review contributes to the body of literature on SRHR RCS and sheds light on remote education as a potential strategy to overcome the challenges associated with sustainable training efforts, particularly among researchers in low- and middle-income countries.

Methods

We conducted and reported the systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (See Additional File 1). The review protocol was registered in International Prospective Register of Systematic Reviews database (PROSPERO, CRD42022328417). Additionally, in accordance with SAGER guidelines for reporting sex and gender information in studies, this review was designed to include studies without discrimination based on sex or gender and to analyse gender participation and differences in effectiveness by gender in the included studies [32].

Search strategy and screening

We searched the literature for articles and conference proceedings published in any language in three electronic databases (PubMed, Embase, and Scielo), from January 1990 to December 2022. The strategy was developed with the assistance of a university librarian. The search terms were performed individually and then combined across the electronic databases. We searched for studies focusing on research methods training on a wide range of sexual and reproductive health and rights (Table 1). Studies yielded by the search strategy were downloaded and imported into Covidence (Veritas Health Innovation, Melbourne, Australia) and independently screened by two reviewers in accordance with the eligibility criteria. Three researchers (VD, CI, and DE) screened imported studies based on title and abstract and full text review. Two reviewers (VB, CP) addressed disagreements. Suitable studies were included for data extraction and reasons for exclusion were documented. The search was supplemented with forward and backward chain search in references lists from the identified eligible articles using the “Connected Papers” website (https://www.connectedpapers.com/).

Table 1 Search strategy
Full size table

Eligibility criteria

Studies that met the following criteria were eligible for inclusion in the review:

  • Type of participants:

  • Adults (18 years of age or older) participating in an online/remote training/education programme on sexual and reproductive health and rights research methods.

  • Type of studies:

  • We sought to include (1) Randomized controlled trials, (2) Quasi-randomised, (3) Cluster randomized, and (4) non-randomized studies: non-randomized controlled trials, interrupted time series, controlled before-and-after studies, cohort studies, mixed-method studies, or cross-sectional/cross-sectional descriptive studies.

  • Exclusion criteria:

  • Studies that met the following criteria were excluded: (1) protocols of future studies, (2) studies that were not targeted to SRHR research education, (3) studies where the training was designed to gain a clinical skill, (4) descriptions of remote training experiences without a report on at least one measure of effectiveness, (5) studies focusing on degree programmes (e.g., online master’s or doctoral degrees).

Data extraction

A structured data extraction form was developed and piloted by the reviewers. Three reviewers (VD, DE, and CI) extracted data from eligible studies and a third reviewer (CP) assessed discrepancies. Data pertaining to study design and characteristics of educational activities were retrieved. Any discrepancies throughout the data extraction process were resolved through discussion among the research team until consensus was reached.

We gathered data on the academic institution hosting the training activity, country of origin of attendees as well as the expected learning outcomes and topic covered. We also retrieved information on how the course or module was developed and by whom, how the education programme was implemented (e.g., blended, fully online, a-synchronic or synchronic sessions), availability of mentorship, how the remote education platform and course were implemented, and duration.

We adapted concepts from the Kirkpatrick model to evaluate outcomes of education and training programmes which includes four levels for assessing these results: reaction, learning, behaviours, and results [33]. For this analysis, the following effectiveness measures were extracted if available: learner satisfaction; assessment of knowledge gained, self-reported knowledge, skills gained; belief about capabilities and intentions to apply knowledge; and lastly, impact measures (e.g., career advancement, grant applications, number of publications).

Subsequently, we searched for either reported enablers of remote learning as well as barriers and challenges. We used the definition study authors provided for enablers, which included factors that were either reported directly from the participants through open-ended exit surveys, interviews or focus groups, or as reflected upon by the study authors or programme implementers as being beneficial to achieve the desirable learning outcomes. Barriers and challenges were defined as obstacles that prevented learners from reaching the intended educational goals.

Data collation and analysis

The initial intention of this systematic review was to conduct a meta-analysis of effectiveness measures. However, given the limited data, three researchers (VD, DE, and CI) synthesized and described the findings on effectiveness and impact.

In order to identify themes relating to enablers and challenges of online education (through exit surveys, students interviews, authors’ reflections) two researchers (VD and CI) followed the traditional steps of thematic analysis: getting familiar with the data -reporting or discussion-, initial coding, searching for themes, reviewing themes and charting and compiling the data aligned with our study objectives [34].

Quality assessment

Quality assessment was undertaken to evaluate the methodological quality of the studies by two reviewers (AN, CI). The NIH quality assessment tool was used for before and after and cross-sectional studies [35].

Results

Study characteristics

Of 1,510 identified studies, six met the inclusion criteria [36,37,38,39,40,41] (Fig. 1). Two were published as abstract proceedings [36, 37]. The SRHR dimension of selected studies was either a programme tailored to professionals working on reproductive health or the host institution remit focused on sexual and reproductive health or gender studies.

Fig. 1
figure 1

Flow chart of included papers

Full size image

Study designs included three studies with before and after surveys [39,40,41] and three cross-sectional exit surveys [36,37,38] (Table 2). Three out of six studies were ranked as ‘Good Quality’ while the remaining studies were ranked as ‘Fair Quality’ (Table 2). In all remote programmes, the hosting institutions belonged to a high-income country (four US, one Canada) or an international organization based in Geneva, Switzerland. Two remote learning initiatives included only learners from the USA, while others collected data from participants from multiple countries.

Table 2 Study characteristics
Full size table

The total sample of learners was 2,058. Of the included six studies, only two studies reported the gender of participants. In one study the majority of participants were males 56.1% (N = 113/175) [38], while the other study reported a higher proportion of females 78.6% (N = 22/28) [39].

Targeted learners were researchers, government officials, public health officials and healthcare professionals. Participants in the reported studies were researchers (N = 1164), government employees (N = 61), healthcare professionals (N = 173), maternal and child health (MCH) epidemiologists (N = 347), scientists (N = 37), and medical students (N = 28) (Table 2). The type of training provided was primarily continuing education and aimed to improve research methodology tailored for either SRHR topics or professionals working on SRHR [36,37,38], scientific writing [36, 39], methods for integrating sex and gender variables in health research [41] and data analysis skills for professionals working in SRHR [40].

Course duration varied, ranging from 6 weeks up to two-years long. Programme completion rates, where reported, ranged from 65% (219/337) [38], 70% (28/40) [40] to 100% (28/28, 37/37) [36, 39]. The earliest included course began in 2001 [40], whereas some of the online programmes are still ongoing [39, 41].

Development and implementation of the programme

The methods used by various organizations and institutions to develop, implement, and evaluate online research methods training are described in Table 3. All six included programmes targeted health professionals, researchers involved in sexual and reproductive health work who had demonstrated interests or needs in enhanced research skills in SRHR. The curricula for the six programmes were designed by academic experts working for, or in partnership with, the organization in which each respective programme was based (e.g., the Duke University Clinical Research Training Program). Online lectures were provided in real-time [39], or pre-recorded to accommodate different time zones and schedules [38, 41]. Two programmes included a face-to-face component; this involved a 3-week writing sabbatical at the University of California, San Francisco [36], and in-person meetings with researchers conducting clinical trials in the United States [37]. Three initiatives specified assigning scholars with mentors [36, 38, 39]; one study mentioned the provision of detailed feedback to scholars from programme faculty [40]. Of note, none of the included studies described massive open online courses (MOOC).

Table 3 Description of implemented sexual and reproductive health and rights online research training programmes
Full size table

Main findings on effectiveness and impact

Diverse methods were used to evaluate each programme’s effectiveness and impact. One study distributed an exit survey with retrospective pre-test and post-test self-evaluations of learning outcomes using 5-point Likert scales (N = 28) [40]. Another distributed pre-test/post-test self-assessments immediately before and after the completion of each module to measure knowledge (5-point Likert scale, N = 28), beliefs and self-efficacy (4-point Likert scale, N = 28), and skill level (7-point Likert scale, N = 23) [39]. Four studies conducted follow-up surveys to measure participant satisfaction, career advancement, and application of research skills (N = 257) [38,39,40,41]; two ran focus groups or interviews (N = 59) [36, 37], and two measured impact in terms of number of publications from participants (N = 384) [36, 38] (Table 3).

Increased satisfaction (level 1)

Only one of the identified studies reported participant satisfaction. Sixty-four percent (64.3%, N = 18/28) of learners strongly agreed and 35.7% (N = 10/28) agreed with regards to satisfaction in achieving their expected learning outcomes [39] (Table 4).

Table 4 Summary of reported effectiveness and impact of remote research training in six included studiesa
Full size table

Improvements in knowledge and skills (level 2)

Five out six studies reported empirical improvements in participant research knowledge and/or skills [37,38,39,40,41] (Table 4). One programme [38] found that knowledge on research methods across six month-long modules improved by 0.75 out of 5 points on average (SD =  ± 0.90), a difference that paired samples t tests revealed to be statistically significant (N = 28, p < 0.0001) indicating knowledge gain across research methods topics (i.e., basic epidemiological and statistical concepts, qualitative and quantitative research methods, economic analysis, and geospatial mapping). Another programme also found a significant self-reported improvement in knowledge of health topics discussed during the course (N = 28, p < 0.001) [39].

Four out of six included studies measured programme effectiveness in terms of improvements in sexual and reproductive health and rights research skills. Most scholars (82%, N = 18/22) involved in one online training programme reported having acquired new and practical skills [39]. Retrospective pre-test/post-test self-assessments from another online analytic skills course showed that combined skill levels improved by 1.75 out of 7 points on average (N = 23, p < 0.05) [37]. Another study reported that for the three modules, biomedical research, data collection in humans and analysis of human data, about 95.8% (N = 520/543), 94.0% (N = 604/643), and 96.3% (N = 419/435) of participants perceived the modules as having taught them a new skill and knowledge [41].

Authors reported specific improvements in scientific writing [36, 39], scientific methodology understanding [38, 40] plagiarism understanding, data analysis, and data interpretation [36, 39, 40]. Participants’ self-reported scientific writing abilities rose from 2.6 (SD =  ± 0.7) to 3.7 (SD =  ± 0.7) out of 5 points on average after taking part in a 6-week online mentorship programme (N = 28, p < 0.0001) [40]. Specific improvements in grammar, punctuation, and use of appropriate terminology were reported, as were improvements in participants’ abilities to search for and select appropriate articles and references for scientific writing in two studies (Total N = 51) [36, 39]. In terms of scientific methodology, pre-test/post-test assessments revealed improvements in participants’ understanding of the research process and of different study methodologies (i.e., observational studies, clinical trials, systematic reviews) (N = 28), as well as improvements in qualitative and quantitative data collection and analysis methods (N = 420) [36, 38, 40]. One six-week programme also measured and captured statistically significant improvements in self-reported soft skills including punctuality and attendance, initiative, attention to detail, critical thinking, and ability to self-organize and communicate effectively (N = 28) [39].

Improvements in self-efficacy and attitudes towards remote learning (level 3)

One study reported positive changes in participants’ self-efficacy and interest in distance learning over the study period. Pre-test/post-test self-efficacy scores, calculated based on participants’ responses to questions relating to their confidence to perform research-related tasks, improved by 0.8 out of 3 points on average across modules (N = 28, p < 0.0001) [39].

Most participants in one study [37] (78%, N = 18/23) reported that their attitudes towards distance learning for continuing education had improved because of the course, though no significant changes in participants’ beliefs regarding the usefulness of specific research skills were identified (p = 0.11). Of 22 scholars surveyed, 6 (27%) felt specifically that the programme motivated them to take a more research-oriented focus in their careers. Another study [41] reported that biomedical research module participants experienced a significant increase in self-efficacy 85.0% (N = 461/543). In addition, participants in data collection in humans and analysis of human data modules in the same study reported 75.5% (N = 485/643) and 81.0% (N = 352/436) improvement in self-efficacy.

Application of skills and research career advancement (level 4)

A majority of participants from four studies reported having applied the skills gained during online training during their own research activities after completion of the course [36,37,38,39]. Most participants of one online analytic skills course reported having shared knowledge/material from the course with co-workers informally (78%, N = 18/23) or via formal presentations (22%, N = 5/23) [39]. Comparing the number of times the same professionals had used specific skills in the six months before and after completing the course, all participants reported significant average increases in selecting appropriate secondary data sources (0.61, SD =  ± 0.84), conducting web searches (0.48, SD =  ± 0.85), and collecting (0.61, SD =  ± 1.03) and analysing (0.43, SD =  ± 0.79) qualitative data [41]. Of 174 surveyed participants of another online training course on sexual and reproductive health research, 67% (N = 118) had subsequently been involved in teaching, 74% (N = 129) had been involved in the design and/or implementation of a research project, and 98% (N = 171) reported having used the skills and knowledge gained from the course [40]. Additionally, another study reported that after completing each e-learning programme, 91.7% (N = 498/543), 89.2% (N = 573/643), and 94.0% (N = 409/435) of the three modules participants indicated an intention to modify the way they account for sex and gender in research [41].

Four of the included online training courses asked participants to report career advancements following participation [36,37,38,39]. For example, one study found that 46% (N = 81/174) of surveyed participants indicated they had received a promotion within two years of completing the course and 81% (N = 142) felt the course had contributed to the advancement of their career [37].

Two studies also highlighted the link between online training and subsequent opportunities for publication [36, 40]. In one study, 47% (N = 82/174) of scholars had published a scientific article within two years of course completion [40]; in another, 74% (N = 27/37) had published a manuscript or had a manuscript in peer review [36].

Reported barriers and enablers to effective online training

Time management and workload

The most frequently cited barriers to effective online sexual and reproductive health and rights research training were related to personal organization and time management challenges, and participants’ competing professional obligations (Table 5). One study highlighted the difficulties in pacing activities and deadlines given the heterogeneity of participants’ external obligations [39]. As one example, the 15 h required to complete each of one programme’s, six data utilization modules was found to be incompatible with normal professional rhythms and may have contributed to high participant attrition [40]. Furthermore, authors emphasized the increased workload for lecturers who had to adapt to presenting remotely, as well as the added challenge for students to remain self-disciplined and self-driven with the remote format over the six-week long course.

Table 5 Summary of identified barriers and enablers to remote research training
Full size table

Despite these challenges, the flexible nature of remote training provided opportunities to overcome time and distance-related challenges. Cited ways to overcome scheduling challenges included providing participants clear indications of course content and expectations, learning objectives and organization [38, 39] setting realistic goals with mentors [40], breaking time-consuming modules into smaller segments, and generally ensuring a manageable workload [40].

Mentorship and networking

Three studies found that facilitating ongoing, one-on-one interactions between programme participants and mentors encouraged effective learning and application of skills [38,39,40]. Beyond curriculum-based interactions, student feedback from one study indicated that opportunities for mentorship through informal social interactions between facilitators and participants to the online course contributed to a sense of social inclusion, reduced power dynamics, and subsequently improved the learning process and outcome.

Opportunities to network with fellow scholars and establish future, potential collaborations [38] and to conduct hands-on application of research skills also enabled more effective learning [39]. Participants in both studies remained more engaged thanks to ongoing and regular feedback from programme faculty and mentors. Allowing participants to engage in programme design by selecting research topics relevant to their professional activities and practice also correlated with the rate of successful implementation of new research skills [37].

Enabling environments

One study ascribed identified barriers and enablers for the long-term use of the research knowledge and skills gained to the lack of an enabling research environment. For example, scientists who had participated in the International Traineeships in AIDS Prevention Studies (ITAPS) training programme found it difficult to publish research outputs due to competing time commitments and lack of support from their home institutions [36].

Discussion

This systematic review summarizes the effectiveness, barriers, and enablers of remote education in SRHR research. Our search strategy identified six studies that met the inclusion criteria. Overall, participants enrolled in well-structured remote education programmes of a duration between 6 weeks to 12 months reported increases in knowledge and skills, as well as increased self-efficacy and attitudes towards this method of learning research skills. Enablers to remote education included flexibility offered by the format, as well as opportunities for mentorship and networking and application of skills. The participants taking part in the programmes expressed satisfaction with their experience, acquired new knowledge or skills, and remained engaged with the learning process as demonstrated by relatively high retention rates. This is in line with existing research on training of healthcare professionals remotely that indicates that structured remote education can achieve similar learning outcomes, satisfaction and engagement as traditional face-to-face learning environments [12, 42, 43].

As found in our review, the success of online learning programmes related to well-structured and build-for-purpose learning environments [44], coursework designed by experts in the field, and the promotion of social interactions [25, 45, 46]. A systematic review of the factors affecting e-learning in health sciences education found that e-learning may not be suitable for all disciplines or contexts [27]. In line with the published evidence on successful online education [27, 47], the identified enablers and barriers of effective remote SRHR research training relate to adequate planning, resource allocation, and social interactions. Previous research also showed that lack of planning in the design and implementation of online programmes can impede students’ ability to manage time and coursework [47]. Moreover, while the learning benefits of facilitating interactions amongst online scholars and facilitators during the course period have been widely cited [27], our results demonstrate an additional need for improved support from home institutions for applying skills after the completion of online programmes. And finally, our results also respond to existing theories in adult learning and online education, whereby autonomy, networking, and diversity are integral to success [16, 48, 49].

Encouragingly, our findings demonstrate that the remote format is considered by participants as both acceptable and effective for research methods training on SRHR despite the potentially sensitive nature of those topics. However, the studies included in this review mostly focused on specific research methods (both qualitative and quantitative) and few reported covering issues around ethics, gender, and rights relating to SRHR research. Further, given only two of the included studies disaggregated data on sex and gender, it is unclear whether our findings hold true for people of all genders. Gender and sex disaggregated data have been signalled as critical aspects to ensuring gender equality [50,51,52,53]. Given the known additional burden that women face outside the work environment, this may hinder their possibilities for remote education [54].

Also encouraging, internet connectivity and access to devices were not cited as a barrier to effective online SRHR research training among studies included in this review, as opposed to what others have found, specific to the COVID-19 pandemic [47]. However, the shift towards online, remote education, especially resulting from the COVID-19 pandemic forces us to address the issue of ensuring access to secure and fast connectivity across the globe, which is currently unequal [55]. None of the included studies used mobile devices and the evidence regarding efficacy of mobile devices for educational purposes is mixed and will be sensitive to timing of data collection in a fast developing area [19, 20].

Overall, the enablers and barriers identified to online learning in our review echo the relevance and applicability of recommended guidance for designing and monitoring SRHR research educational programmes [56] in the online context. Such a process requires defining the programme goal, describing optimal capacity to achieve the goal, determining existing capacity gaps, devising an action plan to fill the gaps and associated indicators of change, and adapting the plan and indicators as the programme matures [56].

Strengths and limitations

To the best of our knowledge, this is the first review looking at the effectiveness of remote education in SRHR research. However, our review does have some limitations. First, the implications for policy resulting from this review are limited by the small number of studies meeting the inclusion criteria and the observational nature of the compiled evidence. Relatedly, we focused on peer-reviewed publications indexed in three large databases, but we did not explore the grey or white literature, where oftentimes reports on evaluations of training programmes are published. However, the small number of included studies in this review, did now allow for a robust meta-analysis and limits the generalisability of our findings. Nevertheless, the paucity of scientific publications on remote SRHR research training programmes is an important finding in itself, highlighting the need to make results of these efforts available to the scientific community through peer-reviewed articles. Second, half of the included studies focused on participants from high-income countries. However, participants from the remaining studies were from a wider variety of countries. Additionally, two of the six included studies were of ‘blended-learning’ format, and it was not possible to differentiate positive outcomes relating to the online versus face-to-face components. Finally, we did not find any evidence of shorter online courses or from MOOCs, meaning we cannot assure our findings can also be extended to those types of courses.

Conclusion

The available evidence is limited but demonstrates the suitability of the remote learning format for providing courses on SRHR research methods and content, when course learning objectives and expectations are outlined clearly, and ongoing mentorship provided. Online learning opportunities may also help overcome financial and geographic barriers in accessing training on SRHR research. There remains a need to document results of online learning, in particular in LMICs, to further provide proof that remote education in SRHR research can sustainably replace in-person training.