Background

The outbreak of the COVID-19 pandemic triggered a rapid and unprecedented transformation in the global education landscape. Educational institutions worldwide faced the urgent task of ensuring continued high-quality learning experiences while prioritizing the safety and well-being of students, faculty, and staff [1]. In March 2020, when the pandemic hit Pakistan, medical schools were compelled for transition to online teaching methods, marking a significant departure from conventional educational delivery modes [2].

The challenges posed by the pandemic extended beyond the classroom, encompassing broader societal, technological, and pedagogical dimensions. In response, educators and institutions embraced innovation and reimagined traditional instructional methods [3]. Online and hybrid teaching emerged as practical solutions to sustain learning amidst uncertainty. In Pakistan, principals directed medical education departments to swiftly initiate faculty training for online teaching, ensuring educators were equipped to deliver sessions according to predefined plans within a week. Students experienced a blend of online and on-campus learning including lectures, small group discussions and practical demonstration through video, adjusting to synchronous and asynchronous teaching methods. Despite ongoing research in developed countries, the unique challenges faced in Pakistan, such as limited internet access, smartphone and laptop availability, and connectivity issues, underscored the need for research tailored to developing countries’ contexts [2].

Recognizing blended learning as a future educational tool post-COVID-19, evaluating its effectiveness became imperative. The study focused on evaluating the first-year integrated MBBS program of the 2019–20 fully online batch versus the 2020–21 hybrid batch in a private medical college in Lahore, employing the CIPP (Context, Input, Process, and Product) Evaluation Model for Educational Accountability” [4]. This comprehensive model facilitated both internal and external evaluations, ensuring credibility, accountability, and informed decision-making in education [3].

The study aimed to assess the curriculum’s implementation, course objectives achievement, and provide feedback for future program development or implementation. By comparing the context, input, process, and output of the first-year MBBS curriculum during the COVID-19 era, the study aimed to identify strengths, weaknesses, and areas for improvement, guiding modifications to future curricula. This endeavor reflected a proactive approach to adapting educational practices in response to unprecedented challenges, with a focus on continuous improvement and innovation.

Literature review

Program evaluation is an important tool for evaluating the quality of any educational program. A systematic review on CIPP model was done by Toosi et al., (2021) highlights the importance of this model to evaluate the managerial skills, faculty performance, students’ knowledge, facilities available, financial implications and policies for an educational program [5]. Another study was conducted at Shiraz medical school, Iran to evaluate the integrated basic sciences curriculum using CIPP model [6]. The authors concluded that this model significantly help policy makers to make decisions in the right direction. An educational framework was designed to evaluate the WFME accreditation basic standards in medical education. Logic model was used to convert the standards into evaluable items which can be used by medical schools for self-review and also can be adaptable by the accrediting bodies [7]. A study carried out in India developed competency-based tools to evaluate a community-based teaching program using Delphi technique [8]. The studies highlight the importance of program evaluation in medical education to evaluate the programs comprehensively and guide the policy makers to make informed decisions.

Significance of the study

This study will help us to identify the preferred method of teaching and learning based on evidence and highlights the gaps in the online versus hybrid methods of teaching.

Methods

The study employed a convergent mixed-method cross-sectional investigation where focus group discussions, interviews and documents review were conducted and results were compared and compiled after the completion of qualitative and quantitative analysis. Multiple data sources were used i.e. triangulation was done to fully understand the effectiveness of the program.

Setting and participants

Study was conducted at a Private Medical College established in 2010. The duration of study was one year between May 2022 and June 2023. Employing a decision-oriented CIPP model, the research included 300 MBBS first-year students from the 2019–20 and 2020–21 batches, as well as 50 faculty members who taught these students, along with administrators.

Recruitment

Participants were selected using cluster sampling technique, with students from the 2019–20 batch classified as Group A and those from the 2020–21 batch as Group B. Emails were sent to the students of both years to participate in the study and give informed consent. Faculty who have taught these years as well as principal and administrators ( Director student affairs department and Director medical education department) were also sent invitation via email to give consent and protected time for an interview. In ensuring voluntary participation, this study adopted several key strategies to prevent coercion and uphold ethical standards. Firstly, students and faculty were provided with comprehensive information about the study’s purpose, procedures, and potential risks, enabling them to make an informed decision about participation. This was reinforced by emphasizing that participation was entirely voluntary and that students had the right to withdraw at any stage without penalty. Moreover, confidentiality and anonymity were assured to safeguard privacy and encourage honest responses. Language used in all communication was carefully crafted to avoid coercion, explicitly stating the absence of negative consequences for non-participation. Ethical oversight from an institutional review board ensured adherence to ethical guidelines, with any concerns regarding coercion promptly addressed. Lastly, participants were offered access to support resources and provided with contact information for the research team, fostering an environment where their well-being was prioritized. Through these measures, the study endeavored to promote voluntarism and ethical conduct among participants, maintaining the integrity of the research process.

Individuals who did not provide consent were excluded from the study. No personally identifiable information, such as names, was collected. A committee comprising a member from Medical Education (Co-Investigator, along with the Principal Investigator as evaluators), worked closely with administrators following project approval by the Institutional Review Board of Shalamar Medical and Dental College (IORG 0010289 IRB No: 0420 Reference No: SMDC-IRB/AL/32/2022), in accordance with the Helsinki Declaration (6/EA/FKGUI/VI/2022).

Data collection

Quantitative data collection

The committee conducted an evaluation utilizing a questionnaire aligned with the standards set forth by the Pakistan Medical & Dental Council (PMDC), with 158 items in curriculum organization and management section and 42 items in infrastructure section, as outlined in their publication (https://pmdc.pk/Publication/Standards).

Qualitative data collection

This evaluation involved inspecting facilities, conducting interviews, and facilitating focus group discussions (FGDs) after obtaining informed consent from the participants. In-depth interviews were carried out using a semi-structured guide, with the questionnaire validated through a pilot study involving 10–15 MBBS students. Each participant was allotted 30 min for participation in either focus group discussions (FGDs) or interviews, scheduled based on their availability. Multiple researchers (AJ, ZB, SP, and KMC) conducted the interviews with participants, ensuring audio recordings and written documentation to minimize bias. Non-verbal cues were also observed during the study. Interviews were conducted in both English and Urdu, later translated and transcribed accordingly. A total of 10 interviews were conducted, with researchers determining saturation had been achieved. The FGD was conducted in a confidential conference room setting. Committee members reviewed data from relevant departments and medical education concerning the first year, with all data stored on password-protected computers for confidentiality.

To assess the context, surveys, and interviews were conducted, focusing on PMDC standards. For input evaluation, observations were made regarding the available human and material resources based on PMDC inspection criteria. This included reviewing documents, administering feedback questionnaires to faculty and students, and conducting pilot attempts. The process evaluation involved conducting FGDs with faculty, students, and administrators. Additionally, observations were made of recorded lectures from online classes, descriptions of the actual teaching process, continuous interaction with program operation faculty and staff, and observation of their activities. For product evaluation, data on performance in module and professional examinations were collected. This comprehensive approach allowed for a thorough assessment of the curriculum and its outcomes.

Data analysis

Quantitative data was analyzed using SPSS ver 23. For quantitative variables, Crohnbach’s alpha was used to determine reliability. Mean and standard deviation (SD) were computed, with an independent samples t-test employed to compare groups. A significance level of P < 0.05 was deemed statistically significant, guiding the interpretation of findings. Qualitative data was analyzed using NVIVO. Phenomenological framework was followed to identify themes, coding themes and subthemes. Data coding was undertaken to identify themes, with coding, themes, and subthemes agreed upon by all researchers to mitigate bias. Themes were organized according to interview questions. Nodes and sub-nodes were established to organize qualitative data, facilitating the identification of themes and sub-themes. Qualitative data collection continued until saturation was attained, ensuring comprehensive coverage of relevant insights. Coded data was reviewed and discussed by the study team to avoid any duplication and consensus was reached.

Data integration

Qualitative and quantitative data was aligned by analyzing the detailed findings along with the results of questionnaire. The study team analyzed the areas of convergence and divergence and comprehend the expansion of findings in questionnaire to detailed discussions in focus groups and interviews.

Results

Qualitative analysis involved conducting focus group discussions (FGDs) with 10 groups, each comprising 15 students. The resulting themes were as follows:

  • Perception of the usefulness of study guides: Group A students expressed mixed opinions, with 75% finding the study guides helpful and 25% considering them not useful. Conversely, Group B found them helpful overall, but some students suggested a need for better emphasis on how to effectively utilize them (Fig. 1a). Students quotations are shown in Table 1.

Fig. 1
figure 1

Word cloud. a. Most frequently used word was yes study guides effective followed by faculty, students, teaching online, study useful, PBL and assessment. b. Items clustered by word similarity (First year MBBS students). c. Attendance and Assessment Online was coded most frequently followed by SGS and PBL and advantages and disadvantages of on campus and online teaching. Least frequently coded were faculty, challenges and affective domain

Full size image
Table 1 Quotation of students on themes identified in qualitative analysis qualitative analysis was done to identify codes and specify themes. The perception of students regarding online and hybrid models was collected via FGDs (focused group discussions) and are detailed in the form of quotations against each theme
Full size table

  • Figure 2: Feedback from Students Groups A and B.

  • Utilization of study guidebooks: Group A students utilized study guides for tasks such as making short notes, summarizing studies comprehensively, revising, and determining what to study. However, some students initially encountered difficulties in using them effectively, as noted by Group B, and only managed to overcome these challenges after completing two modules. Direct quotes from students are shown in Table 1.

  • Benefits of study guidebooks: In Group A, students found study guides beneficial for enhancing knowledge, covering the syllabus comprehensively, highlighting important topics, and filtering out significant content. Additionally, they valued the learning objectives and slides provided by the teachers. Conversely, Group B students found study guides helpful for defining what needs to be studied, filtering out important topics, and guiding them on a clear path (Fig. 2). Students remarks are shown in Table 1.

Fig. 2
figure 2

Feedback from Students Groups A and B. a. Qualitative responses of FGD batch 2019–2020: First year MBBS students (Group A): These students discussed in detail the differences of SGS and PBL online followed by discussion on online teaching the most. b. Qualitative responses of FGD batch 2020-21: First year MBBS students (Group B): The highest response of the students were related to advantages and disadvantages of online teaching/learning followed by implementation of learning strategies online

Full size image

  • Perception of learning outcomes: Mixed opinions were gathered from students in Group A, with some acknowledging the study guides as well-defined and comprehensive in covering every topic, while others did not share this view. Conversely, students in Group B found the learning outcomes to be well-defined and inclusive of every topic (Fig. 1). Students views are depicted in Table 1.

  • Implementation of learning outcomes: Group A students found that learning outcomes were not truly implemented and improvements were needed, such as smaller group sessions or greater understanding of teachers regarding their significance. Group B students generally provided positive comments, stating that most of the content was covered. No specific areas of improvement were mentioned. Students comments are shown in Table 1.

  • Effectiveness of teaching sessions: Students generally found the teaching sessions effective and aligned with the learning outcomes, with some students suggesting the use of more multimedia and a wider spectrum of topics. However, group B students were concerned about the coordination between faculty members and found that the teaching sessions did not correspond with the learning outcomes (Fig. 1). Students remarks are shown in Table 1.

  • Usefulness of SGDs and PBL: Students generally found SGDs (Small Group Discussions) and PBL (Problem-Based Learning) useful for clinically oriented knowledge, improving skills, and increasing confidence (Fig. 1). However, some students found them somewhat helpful and suggested improvements such as providing topics earlier and covering a wider spectrum of topics (Fig. 2a). Group B students found SGDs and PBL useful for creating long-term memory, creating interest, and offering different perspectives. Some students suggested a need for more tutorials. Students’ perceptions are shown in Table 1.

  • Effectiveness of practical sessions: Most students in Group A found practical sessions useful for improving skills, but some students found them unnecessarily long and suggested lessening the time (Fig. 1). Group B students found practical sessions useful, but some students suggested allowing everyone to get the opportunity to use instruments by themselves. Table 1 shows students remarks about it.

  • Assessment of the affective domain: Most students found that the affective domain was mentioned in guidebooks and assessed by the faculty members. Group B students observed that the affective domain was mentioned in guidebooks but not assessed, with some students suggesting the use of log books and PBL forms for it.

  • Comparison of online teaching during COVID and in-class teaching during the non-COVID era: Students generally found in-class teaching more effective and interactive, but they appreciated that teachers provided them with slides of presentations for online teaching. Group B students found Zoom sessions useful but not webinars for online teaching during COVID.

  • Advantages and Disadvantages of Online Teaching: Recorded lectures were the most significant advantage of online teaching, as they can be accessed from home comfort and can be played again if needed. However, the lack of interaction between students and teachers and the presence of many distractions were major disadvantages. The perception of group B students was that home comfort and no need to travel were the main advantages of online teaching, while network issues and a lack of practical experience were the most significant disadvantages (Fig. 3b). Table 1 shows students remarks about it.”

  • Advantages and Disadvantages of On-Campus Teaching: Group A students found that on-campus teaching was beneficial in terms of one-to-one interaction with teachers, more interaction with peers, and hands-on experience. However, long hours, lengthy lectures, and being time-consuming were the main drawbacks. Group B students cited punctuality, routine, and interaction with teachers as advantages. In contrast, time taken for transportation and variable teacher quality were disadvantages. For students’ perceptions see Table 1.

Qualitative analysis of responses from administration and faculty

The interviews conducted with the administration, which includes the principal and the Director of Student Affairs (DSA), and the faculty exposed a number of themes related to the experience of online teaching during COVID-19.

  • Satisfaction with Online Teaching: Participants had mixed feelings regarding the usefulness and satisfaction with online teaching. The principal considered it a contextual and useful option, whereas the director of student affairs (DSA) found it ineffective due to a lack of interest and two-way communication. The DSA was of the view that forced compulsion to attend was not useful since two-way communication between teachers and students was lacking. The faculty maintained that they had initial problems related to technical aspects, but they learned to tackle these issues in a few weeks. However, the faculty had serious concerns related to the practicals, as they could not be conducted in an online setting, hence the practical application of knowledge suffered a great deal. This concern was particularly raised by the anatomy department, as the faculty felt that gross anatomy could not be taught properly. The students, however, preferred online lectures as they did not have to travel or commute, so they could concentrate more on their studies. Challenges Faced During the COVID-19 era, the administrators and faculty faced various challenges in teaching and assessment. The principal expressed concerns regarding the inability of senior faculty members to operate online modalities and utilize them appropriately. All the respondents unanimously agreed that network issues were a major hindrance to conducting online classes smoothly (Fig. 3). The faculty also stated that proctoring during assessments was ineffective and students could easily cheat; with identical answers being observed in SEQs. According to the faculty, they were only able to demonstrate the skill, but they lacked a means of determining whether or not the students had actually acquired the skill. Similarly, problem-based learning (PBL) sessions could not be conducted, leading to unsatisfactory results. The DSA noticed that even after the lockdown had lifted, fewer students were attending the classes. They blamed the lack of engagement during online lectures for this fallout in face-to-face lecture attendance, as students were finding it hard to return to the engaging routine of lectures post-COVID.

  • Faculty Training and Performance: The participants had mixed views regarding the faculty’s training and performance. The principal suggested that reverse mentoring might help in troubleshooting technology issues with senior faculty. He was of the view that, “Junior faculty is more tech savvy, so reverse mentoring helped a lot during COVID-19.” The DSA considered the faculty quick in catching up with technology, and the faculty indicated that the Medical Education department trained them well in time. He claimed that the “The Medical Education department was very supportive.” The administration claimed that students mostly had positive feedback regarding faculty’s performance because regardless of the quality of teaching, the students were happy to stay at home and take lectures (Fig. 2).

  • Differences between Online and On-Campus Teaching: The participants identified various differences between online and on-campus teaching. Physical presence, eye contact, and gestures were missing in online teaching, and non-verbal communication was lacking, leading to less effective teaching. However, the faculty believed that blended learning could be utilized post-COVID.

  • Funding for Online Teaching: The participants agreed that not much funding was required for implementing online teaching during the COVID-19 pandemic. The principal mentioned that the savings from electricity and transportation balanced the funding requirements. However, the DSA suggested that funding was required for cameras, Zoom, webinars, and laptop devices, claiming that the “the medical education department was not equipped initially, and they had to purchase webinars and Zoom hours.”

  • Satisfaction with Study Guides and Planners: Overall, the participants were satisfied with the faculty’s job in a short time to take over as compared to other institutes, but there was some variation in teaching quality as reported by the students.

  • Medical Education’s Preparedness for Online Teaching: Participants held divergent opinions regarding Medical Education’s preparedness for online teaching. The principal advocated for greater availability of teaching technologies. Conversely, the DSA emphasized the department’s focus on faculty training and suggested an increased emphasis on student training. Faculty members acknowledged effective training provided by the department but noted areas for improvement, particularly in the admission process during COVID-19 (Fig. 2b). Despite this, participants generally agreed that the admission process posed minimal challenges. The successful implementation of multiple mini-interviews (MMI) online allowed for more efficient interviewing of students. However, there was a noted absence of assessment for non-verbal communication.

  • Progress Monitoring of Online Teaching: According to the participants, progress monitoring of online teaching was carried out based on feedback from both students and faculty, while any technological issues were handled by experts.

Students feedback for faculty

Students feedback for faculty teaching during online and hybrid sessions are shown in Table 2. Students rated anatomy teaching in online sessions significantly better than hybrid sessions. These students found that learning sessions were more student-centered with supporting online classes (p < 0.05). They also reported that tutorial sessions improved their problem-solving skills. Teachers were noted to be helpful in stress management during the pandemic, with the college employing proper counselors to cater to the needs of students who required assistance.

Table 2 Students feedback for Faculty Across Basic Health Sciences subjects during Online and Hybrid Sessions
Full size table

However, faculty rating by students in physiology and biochemistry were comparable during online and hybrid teaching sessions. These students expressed satisfaction with the availability of proper infrastructure, resources in the library and IT, as well as support from faculty and staff. They were also content with the teaching and assessment environment provided. Additionally, students found co-curricular activities to be motivating.

Messages for students and faculty

The participants had various messages for students and faculty. The principal suggested that technology is here to stay, and we have to develop these skills to survive, highlighting that “technology has to stay, so better get tech-savvy”. The DSA suggested that faculty should focus on delivery, and students should try to be good doctors and better serve humanity. The faculty suggested that students and faculty should be well-prepared for online teaching, and the medical education department should conduct workshops on online teaching and learning at least twice annually (Fig. 3).

Fig. 3
figure 3

Feedback from Administration and Faculty. a. Qualitative responses of FGD by Administrators: The response from administrators were highest regarding need for improvement in the online system followed by message given to the students and more focused on funding required for it. b. Qualitative responses of FGD by Faculty Basic Health Sciences: The faculty response was mostly focused on training of faculty and network issues which had to be fixed on urgent basis. However online teaching was appreciated as it helps them to be tech savvy

Full size image

Intended outcomes

Both the 2019-20 and 2020-21 batches underwent assessment through module and professional examinations, which were conducted online as well as on campus. The admission merit of both batches was analyzed alongside their academic performance. Additionally, the examination papers prepared by the faculty for module examinations in basic sciences underwent assessment for reliability by Cronbach’s alpha (Table 3).

Table 3 Analysis of reliability of Module assessments reports based on Cronbach’s alpha
Full size table

The quality of teaching delivery by the faculty was also evaluated through the review of recorded YouTube lectures. Despite the 2020-21 batch having lower admission merit compared to the 2019-20 batch, which was taught entirely online, the former, taught in a hybrid format (partly online and partly on campus), demonstrated significantly better performance in module and professional examinations for anatomy, physiology, and biochemistry. The quality of teaching and examination papers was found to be consistent both online and on campus. However, factors such as low motivation, mental stress due to the pandemic’s effects on students and their families, ineffective proctoring mechanisms, and the absence of physical teacher presence in online classes contributed to the lower performance of the batch taught entirely online. The lack of co-curricular activities also played a role in this outcome.

Un-intentioned outcomes

The pandemic-induced shift to online teaching resulted in comprehensive teacher training for blended learning sessions and courses. This equipped educators to develop and deliver online courses as supplemental resources for students. Moreover, students gained proficiency in online teaching and assessment techniques, enabling the incorporation of low-stakes examinations on Learning Management Systems (LMS) such as Moodle. This approach not only streamlines the process but also offers flexibility for both educators and students, ultimately enhancing the teaching and learning experience.

Short-term implications

The batch that experienced solely online teaching during the pandemic came to appreciate the value of attending medical school, recognizing its role not only in providing quality education but also in fostering co-curricular activities, problem-solving skills, team building, leadership abilities, and offering counseling support when needed. Additionally, faculty members recognized the importance of being technologically proficient and the benefits of blended learning, which can encourage students to take more responsibility for their studies. There was a recognized need for strengthening the medical education department in terms of online teaching and providing regular faculty training. The 2019–20 batch achieved a passing rate of 92.58% in the university professional examination, while the 2020–21 batch scored even higher with a passing rate of 98.16%. Faculty involved in teaching and assessment noted that the professional papers in basic sciences for the 2019–20 batch were comparatively easier than those for the 2020–21 batch.

Long-term implications

The online program has proven to be an effective alternative to on-campus teaching, particularly in a blended format. Both batches showed improved performance over the next two years, achieving impressive results of 97–98% in 2021 and 2022. However, the true measure of success will be observed when these batches graduate and begin working in hospitals, providing insight into the long-term impact of the online teaching approach.

Program impact

The online program proved successful in hybrid (blended) form, albeit with certain limitations evident in the results, particularly for the total online approach.

Program effectiveness

The majority of students demonstrated good performance, particularly in hybrid learning methods, underscoring the importance of incorporating blended learning approaches that combine both asynchronous and synchronous forms.

Program sustainability

The program is integrated into the Learning Management Systems (LMS), with additional tools like webinars and Zoom purchased as needed. This expenditure does not impose a significant financial burden on the institute, making the program sustainable in its current or enhanced form.

Ease of adoption

The program is readily accessible and cost-effective. However, its sustainability and effectiveness rely on thorough training of faculty and students, coupled with adequate support from the administration and medical education department. This ensures a cost-effective and sustainable model that can be easily replicated by other institutions.

Quantitative results

Analysis of module examinations (Table 3)

Reliability of module assessment papers in subjects of anatomy, physiology and biochemistry was determined using Cronbach’s alpha during hybrid sessions and online sessions. The data shows different reliability of papers across various disciplines. The table shows reliability of assessments were low in the beginning of COVID- 19 Pandemic i.e. in 2019–2020 (totally online) but improved with passage of time in the basic health sciences subjects in 2020-21 (hybrid) except for biochemistry which shows more reliability of papers in online tests compared to hybrid.

Comparison of admission scores, internal assessment scores, and professional examination scores (table 4)

Table 4 Mean differences in Admission, Internal Assessment, and First Professional scores between SMDC MBBS students of years 2019–2020 (online) and 2020-21 (hybrid)
Full size table

The admission merit, particularly MDCAT scores, and final merit were significantly higher for the online batch (2019–2020) compared to the hybrid batch (2020–2021) P value 0.01. Internal assessments of anatomy improved significantly in the hybrid teaching batch compared to the online batch, while physiology and biochemistry remained comparable between the two batches. First professional results of anatomy and physiology showed significant improvement in the hybrid teaching batch, while biochemistry results remained comparable between the two batches.

Overall, the hybrid teaching approach resulted in improved outcomes in certain areas compared to total online teaching, particularly in internal assessments and first professional examination results.

Discussion

The study reports an in-depth mixed method to evaluate and compare the online versus hybrid model of teaching during COVID-19 utilizing the CIPP model. The context, input, process, and product were assessed during 2019–20 and 2020–21 by obtaining perspectives from students, faculty who taught them, and administrators. The context was the urgent need of transition to online teaching to maintain the continuity of education and academics during the COVID-19 pandemic. This transition occurred globally at almost every institution in developed as well as developing countries [8]. The rapid training provided to faculty and students on online teaching within a week or two was appreciated by all stakeholders. However, the students highlighted the lack of interaction between students and faculty during webinar sessions. This could be attributed to the one-way flow of information via lectures and the inability to see the students physically. Practical sessions were also only demonstrated, and students were unable to perform them. A study conducted in Shiraz, Iran, found similar findings [9]. Inadequate internet connectivity, especially in peripheral areas of the country, was the main issue encountered by the students. This led to anxiety among them during assessments. A study from India also highlighted some common downsides to remote teaching from the perspective of undergraduate medical students, including technical difficulties, ease of distraction, and some staff being poorly versed in the technologies used [10]. The major obstacles have included delivering online teaching content as well as adapting means of assessment in such unforeseen circumstances [11]. The alternative approach taken by Imperial College London was to introduce an open book examination (OBE), in which the questions were designed in such a way that students were allowed to use internet sources during the examination. The perception of 2721 medical students across 39 medical schools in the UK revealed flexibility as an advantage and internet connection as a barrier to online education [12]. OBE was implemented for internal assessments during COVID-19 in our setup but was not done for professional examinations. The students commented on the advantages of online teaching more, as traveling was not required, and they could study from home. Faculty coordination was improved, and they were trained in blended learning [13]. Similarly, students’ knowledge, attitudes, and practices were reported by Noreen et al. (2020) during COVID-19 in Pakistani medical schools, supporting our study [14]. Quantitative analysis showed that internal assessment and modular examination papers were equally reliable in all basic subjects in the first-year MBBS. However, the admission merit of group A was higher than that of group B, but the scores of internal assessments and professional examinations were higher for group B students compared to group (A) There could be multiple reasons for this. As shown in our results, the admission criteria for group A were totally based on PMDC criteria, where no marks were allocated to the medical colleges for interviews, while in group B, 20% of marks were allocated to them for interviews due to the change from PMDC to PMC. Moreover, the medical college changed its attendance and assessment criteria from 75% attendance and a 50% assessment cutoff to be eligible for professional examinations for group A to 90% attendance and a 60% assessment cutoff for the years 2020–21 for group (B) A systematic review of the academic performance of students during COVID-19 reported variable results, spanning from low to high [15]. Similarly, a study by Sulail Fatima et al. (2021) conducted in Karachi, Pakistan, reported low academic performance in module assessments conducted online versus high scores in face-to-face assessments, which supports our study [16]. Shamsa et al. (2018) evaluated the quality of school programs using the CIPP model, which revealed significant findings that were recommended to be improved [17]. Similar studies were carried out in Pakistan to evaluate the continuous development program for family physicians and the bioethics diploma program [18,19,20]. The effect of the pandemic on medical training will be analyzed after these students graduate and start practicing. The workplace-based assessment will provide a clear picture of the online teaching during COVID-19. However, it has broadened the horizons of training by integrating asynchronous and synchronous teaching models. Telemedicine and flipped classrooms are now more frequently utilized for content delivery and patient care than before, with more advantages compared to conventional archetypes. These will become more refined with the passage of time with the integration of artificial intelligence (AI) like Chat-GPT and research rabbits. This has posed challenges for faculty to identify the learning methods that can be successfully integrated into their curriculum.

Conclusions

The CIPP model program evaluation highlighted the challenges encountered by both students and faculty during total online teaching, shedding light on gaps in students’ knowledge and skills. Furthermore, it offers guidance to administrators and program directors to pinpoint areas needing improvement, facilitating the implementation of necessary changes. While students valued the hybrid model for its engaging teacher-student interaction, faculty members favored online teaching strategies and proposed the future use of blended learning. The administration recognized the faculty’s swift transition to online teaching and their commendable performance. However, they emphasized the necessity of faculty development workshops on blended learning and strengthening the medical education department. Based on our study, we recommend:

  1. 1.

    Blended learning strategies (Both synchronous and asynchronous should be used for teaching and learning as it generates sense of responsibility amongst the students, create interest and generate team work.

  2. 2.

    Workshops for blended learning techniques for faculty should be done frequently.

  3. 3.

    Medical education department should be equipped to facilitate online teaching /learning.

  4. 4.

    Feedback of the students should be taken after each module to cater their needs and improve the curriculum.