Background

The introduction of Objective Structured Clinical Examinations (OSCE) marks a significant shift in how clinical skills are assessed in the medical curriculum, driving a redefinition of clinical training. OSCEs became a mandatory component to assess practical clinical skills in several countries [1]. OSCEs are widely regarded as the gold standard for formative and summative assessments, involving multiple stations where students perform standardized tasks, often with simulated patients, to assess their clinical reasoning, decision-making, and communication skills [2, 3].

As medical curricula evolve, new teaching and learning strategies are being explored to better prepare students for these comprehensive exams. Traditional lecture-based teaching, once the dominant form of medical education, has become increasingly complemented by digital learning tools that offer more flexibility. Many students and universities now rely on alternative educational resources, such as online courses, training platforms, or review books, to prepare for exams. Furthermore, recent advancements in digital technology have introduced new and freely accessible tools, such as podcasts, to support and enhance medical education [4, 5].

Podcasts—digital audio files available for download to a computer or mobile device—are one of the fastest-growing formats for consuming educational content. They offer students a flexible, cost-effective way to supplement their learning outside traditional lecture-based settings. In medical education, podcasts have proven effective in reinforcing knowledge and skill development, providing an independent, time-efficient supplement to existing curricula [6, 7].

To our knowledge, no studies have specifically evaluated the utility of podcasts in preparing for Objective Structured Clinical Examinations (OSCE). While highly valuable, OSCEs present significant logistical challenges, particularly regarding faculty workload and the coordination of standardized patients. However, recent advancements in OSCE simulations using generative AI-powered virtual patients offer a potentially scalable and accessible alternative for assessing clinical skills [8].

This study aimed to evaluate the effectiveness of a nephrology-focused OSCE training podcast as a supplementary tool for enhancing the performance of undergraduate medical students, assessed through a generative AI-powered virtual OSCE platform.

Methods

Study design and participants

We conducted a randomized controlled study (Fig. 1). All pre-clerkship medical students at our institution in the 2023–2024 academic year who had not yet listened to the podcast were eligible. All students had completed formal nephrology coursework within the previous year.

Fig. 1
figure 1

Flow chart of the study

Full size image

Participants were recruited after two waves of study promotion between May and June 2024. All received a personal link to complete an online baseline questionnaire developed for this study (age, sex, previous internship in nephrology, previous nephrology-focused OSCE training, most important educational resource used, previously heard of NephrOdio, see Supplemental Material). Then, using computer-generated random numbers, they were randomly assigned (1:1) to the podcast group or the control group. Each participant was informed of their allocation after completing the baseline questionnaire.

All students provided informed consent to participate. The local Ethic Committee approved the “Comité d’éthique en recherche” study protocol on May 16th, 2024. The questionnaire was prepared online using the Limesurvey ® platform. All recorded data were anonymous, and data management was done in compliance with current French and European legislation regarding data privacy, notably the General Data Protection Regulation of the European Union in force since May 25, 2018, (GDPR 2016/679) and the French law on informatics and liberties dated 6 January 1978 and updated in 2018.

Intervention

Students who were assigned to the podcast group were asked to listen to all the content of NephrOdio season 2. NephrOdio is an open-access academic podcast series developed by the French College of Nephrology Professors (CUEN), freely available from several platforms. Season one was launched in March 2023, with each of the 21 episodes dedicated to one essential nephrological knowledge. Season two was released in March 2024, and each of the 16 episodes was dedicated to one essential nephrological clinical competency for OSCE preparation (Table S1). The episodes had a median duration of 7 min. Students assigned to the control group were asked not to listen to the NephrOdio podcast during the study period.

Data collection and follow-up

We utilized the DocSimulator platform (https://docsimulator.azurewebsites.net/) to simulate OSCE with Generative AI-powered virtual patients. Six nephrology-specific OSCE scenarios developed by educators were integrated into the platform, each each related to a distinct clinical competency (analysis of biologic results, therapeutic decision, diagnosis strategy, examination, announcement, and education). Every scenario was paired with a 15-point evaluation grid. GPT-4 simulated the virtual patients and assessed students’ competencies according to the predefined criteria. Students accessed DocSimulator via secure login credentials, selected a scenario, and engaged in text-based communication with the virtual patient, concluding with a personalized feedback evaluation. Evaluation notes from these simulations, linked to randomized study IDs, were then extracted and provided for statistical analysis. All simulation sessions were conducted entirely on this digital platform.

All included students completed a short survey developed for this study (see Supplemental Material) after completing all 6 OSCE stations to assess the impact of the use of the educational podcast on their perception of nephrology.

Primary and secondary outcomes

We compared the performance in OSCE exams between students who had access to the OSCE preparation podcast (Intervention group) and those who did not (Control group). All participants completed the first 3 OSCE stations at baseline using their own DocSimulator account online, whenever they wanted to within 2 weeks, followed by a four-week intervention period during which they were asked to listen (or not) to the OSCE preparation podcast content. Finaly, all completed 3 novel OSCE stations within 2 weeks. Hence, the whole study lasted 2 months between July and August 2024.

Secondary outcomes evaluate participants’ interest in the nephrology specialty, including their intention to intern or specialize, their self-perception of their knowledge and clinical skills, and their satisfaction with the NephrOdio learning path.

Statistics

We used NQuery version 4.0 to calculate the number of students needed to demonstrate a 3-point difference in OSCE score evolution before/after intervention between the two groups with a 5% α risk and a power of 90%. Including 23 students per group was necessary; hence, we planned to include 50 students in the study.

Quantitative variables are reported as mean+/-standard deviation when normally distributed (based on the Shapiro-Wilk test) or median[interquartile range]. Categorical variables are reported as numbers (percentages). The cumulated notes obtained before and after intervention were compared between the two groups using a Student t-test. The difference between the cumulated note obtained from the first 3 OSCE stations (before the intervention, maximum 45 points) and the cumulated note obtained from the last 3 OSCE stations (after the intervention, maximum 45 points) was calculated for each student and compared between the two groups using Mann-Whitney U-test. The perception of nephrology between the two groups was approached using the Chi² test or the Fisher exact test. Data were analyzed using SAS version 9.4 (SAS Institute, Cary, NC) and are reported according to the CONSORT guidelines. A P-value < 0.05 was considered statistically significant.

Results

Participants

Between May and June 2024, we randomly assigned 50 students to the podcast (n = 25) or the control (n = 25) group. All of them were recruited at the end of the fourth year of medical school. They had previously followed the academic nephrology courses within the past year at the time of enrollment. Among the included students, 34(68%) were women. Seven (14%) of them had previously listened to the first season of the NephrOdio podcast. Two (4.1%) considered academic classes the most important educational resource to learn medical knowledge. Baseline characteristics did not differ between groups (Table 1).

Table 1 Baseline characteristics of the included students
Full size table

For 39(79.6%) of the included students, the DocSimulator OSCE simulation experience felt like talking to a real patient, and 41(82%) of them declared that this platform was very useful for their preparation of OSCE exams without significant difference between the two groups (Table S2).

Primary and secondary outcomes

The students’ performance in OSCE before and after intervention is summarized in Table 2. The cumulated note at baseline did not differ between the two groups (23.8 ± 3.9 vs. 23.3 ± 5.3; p = 0.77). The cumulated note after the intervention was significantly higher in the podcast group compared to controls (27.6 ± 3.6 vs. 23.6 ± 5.0; p = 0.002). The improvement in cumulative OSCE score was greater in the podcast group compared with controls (+ 3.52[0.7,6.5] vs. -1.22[-3,5.5]; p = 0.03) (Fig. 2).

Table 2 OSCE performance of the included students before/after intervention
Full size table
Fig. 2
figure 2

Change in OSCE cumulative score. Change in OSCE cumulative score is shown for each included student before / after the interventional period for control (A, red) and podcast (B, blue) groups. **p < 0.001

Full size image

The students’ perception of nephrology at the end of the study is reported in Table 3. The use of the OSCE preparation podcast did not impact the intention to do an internship in a nephrology department or the will to become a nephrologist. However, it significantly improved the proportion of students feeling comfortable with nephrology clinical competencies (41.7% vs. 16%; p = 0.04). Among the 25 students assigned to the podcast group, 17(68%) found the OSCE preparation podcast to be very useful for their preparation of OSCE stations, and 24(96%) of them declared they will use it again in the future.

Table 3 Perception of nephrology at the end of the study period
Full size table

Discussion

This study is the first randomized controlled trial to evaluate the impact of a podcast specifically designed for OSCE preparation on students’ OSCE performance. The findings demonstrated that students who incorporated the podcast into their study regimen showed significantly greater improvement in OSCE scores compared to those relying on traditional learning methods alone, with a 17% higher improvement in scores (p = 0.002). Additionally, these students expressed significantly higher confidence in their nephrology clinical skills, suggesting that podcasts may serve as a valuable resource in medical education [9]. Moreover, this study is the first randomized controlled trial to use a generative AI-powered virtual OSCE platform and assess OSCE performance [10]. The platform, DocSimulator, demonstrated comparable baseline OSCE scores between the two groups, reporting its reliability in evaluating clinical skills within a randomized cohort.

Podcasts represent a flexible and engaging medium for delivering medical education content, allowing students to reinforce their learning outside of traditional lecture-based settings [11, 12]. As an accessible, time-efficient resource, podcasts enable learners to study at their own pace, which may enhance knowledge retention [13]. In this study, only 4.1% of students identified traditional academic classes as their preferred resource for medical learning, suggesting a shift in student preferences toward more flexible, digital learning tools like podcasts [11, 14]. This finding supports the notion that students increasingly value on-demand educational resources over rigid, lecture-based formats, particularly as digital technologies become more integrated into medical education.

Moreover, audio-based learning caters to various learning styles, including auditory learners who benefit from listening and repetition over visual or text-based materials [6]. The high proportion of students in the podcast group who found OSCE preparation podcast useful and expressed willingness to continue using it in the future further supports the development of such digital tools by other specialties. Nevertheless, it is important to emphasize the role of blended learning strategies. While digital tools such as podcasts are valuable, they should complement rather than replace traditional curricula, ensuring that students receive a comprehensive and well-rounded education [15].

The observed improvement in student confidence with nephrology-related clinical skills is particularly promising. Confidence in clinical competencies is crucial for preparing medical students to transition into clinical practice. This aligns with existing literature suggesting that targeted educational interventions can significantly enhance students’ self-efficacy in clinical settings [16]. However, the lack of change in students’ intention to pursue nephrology as a specialty or undertake a nephrology internship is a notable finding. Despite increased competency and confidence, career choices in medicine are often influenced by factors beyond education alone, such as personal interests, perceived work-life balance, and mentorship [17].

Despite these encouraging results, several limitations must be acknowledged. First, this study was conducted at a single center, and the sample size was small, which may restrict the generalizability of the findings to broader student populations or other medical specialties. Second, the study employed a text-based virtual OSCE rather than a real-time vocal OSCE. While this offers a novel and standardized approach ensuring greater consistency across participants, it may not fully replicate the nuances of live, vocal clinical examinations. This may affect how well the results translate to actual clinical performance. Third, this study relies on a generative AI-based assessment of OSCE performance. To establish external validity and fully evaluate the effectiveness of this automated approach, further studies are needed, including comparisons with human assessments [18]. Fourth, the scope of the nephrology content covered in the podcast was not exhaustive, which limits the conclusions we can draw about its utility for comprehensive nephrology education or its applicability to broader medical disciplines. Fifth, there is a risk of selection bias, as students included in the intervention group may have been more motivated to perform well. Furthermore, the reliance on self-reported confidence levels introduces the possibility of response bias. Finally, NephrOdio OSCE training podcast represents a specific format, and the study did not explore how different formats (e.g., case-based, discussion-led) might affect learning outcomes.

Overall, the study highlights the potential of podcasts like NephrOdio to serve as an effective supplementary resource in medical education, particularly in enhancing clinical competencies. However, more research is needed to determine the long-term impact of podcasts on medical learning, including their role in knowledge retention, clinical application, and career choices. Future research should also evaluate how different podcast formats (e.g., case-based, discussion-led) compared to other digital learning tools, such as video-based or interactive simulation resources, to better understand how these innovations can be integrated into broader medical education strategies.

Conclusions

In summary, this study provides evidence that OSCE preparation podcasts, such as NephrOdio, can significantly enhance medical students’ performance in nephrology-related clinical skills on AI-based simulation. As cost-effective, flexible, and accessible resources, podcasts have the potential to complement traditional learning methods in all medical specialties and cater to diverse learning styles.