Introduction

Transition to the paradigm of competency-based education, from process- based or time-based one, requires attainment of a shared language and transforming the competencies to such constructs that are not too much subjective [1,2,3]. It has been unclear what is expected from the learner to perform in the practical context [4]. As a result, the new paradigm has imposed a need to some means and methods that can evidently evaluate the quality of performances in the work places. Such evidences must be compared to the learners’ activities in the clinical settings [5,6,7]. Moreover, currently, most of the high stake decisions like promoting to a higher level of study and entering the practice are mainly made by means of knowledge based summative assessments, particularly in graduate programs. This is more or less because of the complications regarding the high objective tests [8]. Some of the problems with such examinations like Objective Structured Clinical Exam (OSCE) are the need for advanced plans for time and cost allocation [9]. Therefore, formative work based assessments like mini clinical evaluation examination (mini-CEX) are recently more acknowledged [10, 11]. However, the psychometric investigations of these examinations have revealed different results and conclusions; some with paradoxical results even with questioning the reliability and feasibility of them [12, 13]. Therefore, it is essential to apply instruments capable to measure and evaluate the subjective and complicated constructs of competencies, along with helping medical schools become more socially accountable [14]. Such examinations must be as much authenticate as possible [15].

After suggesting EPAs by Ten Cate [4], these activities were described and their application was increasingly considered day after day; firstly in some professional medical specialties,and then, from 2013 to 2014 for general physicians in developed countries [16,17,18,19], and even other medical courses such as pharmacology [20]. It has increasingly become spread in medical education literature. EPAs are essential activities of any discipline (profession, specialty or sub specialty program), which the professionals must perform independently without direct supervision in a defined environment after acquiring appropriate competencies [3]. They are the units of the observable clinical works [21] and play a guiding role for clinical teachers for training and assessing the learners so as to make decision about the level of their entrustability and capability to accomplish the activities without close supervision. Ten Cate has stratified entrutability into five levels [6]. In the first level, the health professional learner is mainly an observer and is not allowed to get involved in the activity. The second level corresponds to the one in which (s) he may undertake the activity under close and active supervision by the clinician. In the third level, the learner performs the activity with indirect supervision and the clinician involves if needed. The learner performs the task independently without any supervision in the fourth entrustability level. And finally, in the fifth level, (s) he is eligible to supervise lower level learners. The EPAs were originally developed for graduate medical education (GME) [22,23,24]. Beeson at al. defined EPAs for emergency medicine [25]. Psychiatry, family medicine, anesthesiology and surgery were also the other programs for which the EPAs have been developed [26,27,28,29]. However, the gap perceived in continuum from undergraduate medical education (UGME) to GME, resulted in developing some EPAs for entering residency in addition to EPAs for undergraduate programs [16, 18, 30,31,32,33]. Moreover the sociocultural differences of the societies and health delivery systems in different countries and educational institutions have resulted in diversity of EPAs both in their numbers and contents. The United States has defined 13 EPAs for undergraduate medical education [34], while in Canada, despite using the American EPAs as the prototype, the number was decreased to 12 [35]. Besides to the number and content, there is another profound difference in terms of centrality and decentralization of the ways of developing EPAs. The USA and Canada centrally defined EPAs [34, 35] while Utrecht University in the Netherlands, Charit University in Germany and the Medical School of California in San Francisco developed their own EPAs [36,37,38]. As the result, EPAs are concepts that should be defined and customized for any community in order for training and assessing their health professionals, based on the needs. One of the main causes of the diversity in developing and implementing the EPAs among different communities might be stemmed from the differences in educational systems of undergraduate and graduate medical education. Such differences exist in admission, training programs, and even the rules and regulations of entering the graduates to practice. The present study, therefore, is going to determine what essential activities internal medicine residents have in developing countries like Iran. And at which entrustability level, they must be at the end of each residency year. The findings could be applied in all countries with similar situations, especially in East Mediterranean Regional Office (EMRO) countries.

Setting

This study was conducted in Internal Medicine clinical department of Isfahan University of Medical Sciences, Isfahan, Iran, in order for developing the EPAs of internal medicine specialty residency program. After getting M.D. degree, the prospect residents take an annual national entrance exam. The participants are ranked regarding their scores and then-after they select their preferred specialty program. The admitted residents spend 4 years in a mostly hospital based graduate program. Besides formative evaluations throughout the program, they must take a progress test at the end of each year. Promoting to the higher level of study is based on their progress test scores. If they get the minimum pass level, required for the progression, they are eligible for moving to the next residency year, and if not, they are not allowed and must repeat that level. At the end of the fourth year, residents take another annual national final exam for getting the board certificate.

Internal medicine department of Isfahan University of Medical Sciences, school of medicine, with 61 clinical faculty members, trains 80 specialty residents each year within four residency years. The residents take various sub-specialty and general internal medicine rotations and the rotations take 1 month in length.

Methods

Different studies followed different methodologies for developing EPAs. Ten cate et al. have suggested three steps for defining the EPAs as identification, processing, and validating [39, 40]. We extend them to six steps as follows:

  1. 1)

    Determining the characteristics of EPAs: These features were used as the main criteria for defining, processing and validating the EPAs in the following steps. Moreover, these features were applied for reaching to a shared language among the study team as well as with clinicians in order for accepting or rejecting any suggested statement.

  2. 2)

    Reviewing the literature and publications through a scoping review: This step was accomplished for fining the update EPAs developed specifically for internal medicine. We searched PUBMED, SCOPUS, and google scholar for English publications about similar experiences from 2005 to 2020. We used these keywords: “entrustable professional activities” “internal medicine” “internship“ ”residency” “undergraduate”, and “fellowship”. We also reviewed the guidelines such as Accreditation Council for Graduate Medical Education (ACGME) publications [41,42,43]. PUBMED search was performed as following: (Filters applied: Abstract, Free full text, Full text, Journal Article, in the last 5 years, Humans, English, MEDLINE): (entrustable professional activities). For SCOPUS data base we used this formula: (TITLE-ABS-KEY (“entrustable professional activities”) AND TITLE-ABS-KEY (internal AND medicine) AND TITLE-ABS-KEY (internship) OR TITLE-ABS-KEY (residency) OR TITLE-ABS-KEY (undergraduate) OR TITLE-ABS-KEY (fellowship). In Google Scholar the following string were used: entrustable professional activities“ AND (residents OR interns) And “internal Medicine”. We included only those publications that were about the specialty program of general internal medicine but not sub-specialties or undergraduate level of internal medicine like clerkship. We excluded also those cases that reported a limited part of the curriculum like discharging the patient. The screening results were crossed over between the first and second authors.

  3. 3)

    Developing initial EPAs: With consideration of the national and institutional internal medicine residency curriculum [44, 45] as well as the residents’ electronic logbook (mobile log application), these documents were also reviewed for developing initial EPAs.

  4. 4)

    Extracting the results of preceding steps: We screened the activities, EPAs and other results, yielded from step 1st to 3rd, and developed the initial list of EPAs.

  5. 5)

    Refining and confirming the final EPAs: We formed a focus group discussion with a number of internal medicine clinicians. The criteria for participation were as: being internal medicine specialist with direct contribution in general internal medicine residency training, having at least 10 year experiences in teaching and assessment of internal medicine residents, being professor or associate professor, being interested in improvement of the residency program with conducting at least one educational research project or two papers in educational journals. The head of internal medicine department nominated 20 academic members who had our inclusion criteria. Since the optimal number of focus group participants is six to twelve [46], we invited 10 clinicians for the session. Informed verbal consent to participate was obtained from all participants at the beginning of the session. They participated in refining and screening the initial list as well as the entrustability level of each residency year. The focus group also determined the information sources, eligible for evaluating the residents against the EPAs. As the final section of the meeting, the focus group cross-tabbed the EPAs with competencies. For this purpose, we used the competencies, stablished by secretariat of council of general practitioners’ education for undergraduate medical education. They include clinical skills, communication skills, patient care (diagnosis, treatment and rehabilitation), health promotion and preventive medicine, and the G.Ps’ role (in that, personal development and continuous learning, professionalism, ethics and medical law, and finally, decision making, reasoning and problem solving skills are considered).

  6. 6)

    Reaching the final number and content of the EPAs: As the final step, we sent an open ended questionnaire attached to the EPAs to those nominated academics who did not attend the focus group session. We sent them via email and asked participants to evaluate and endorse the content validity of the focus group resulted EPAs based on the characteristics yielded in the first step. They were also asked to confirm the entrustability levels and EPA-competency matrix. Eventually, we reached the final number and content of the EPAs, based on the conclusions from all resources.

Findings

Within the first step, we determined the characteristics and features of EPAs including being parts of the profession, being directly related to the patient, and being observable, repetitive, assessable, doable by one resident, doable in a defined time frame, critical, and requiring especial knowledge, attitude and skills [3, 39].In the second step and through the scoping review, we found 308 items in the initial search; and eliminating duplicates resulted in 288 titles from which, after twice reviewing the titles and abstracts independently by the first and second authors, we selected 72 articles for a full text review. Those articles that were not in the internal medicine residency programs or those studying a limited area or a sub-specialty program were excluded. We also excluded the articles reporting the mere implementation of the EPAs. Figure 1 shows the quantity of the results in each step of search flow.

Fig. 1
figure 1

flow diagram of internal medicine EPAs scoping review

Full size image

Only 4 articles studied developing EPAs in general internal medicine residency program [47,48,49,50]. Table 1 illustrates them in brief besides their resulted EPAs. The first one is the report of the initiative of the Education Redesign Committee of the Alliance for Academic Internal Medicine that reached to 16 EPAs. The next one was the result of a study in University of California in San Francisco (UCSF), and school of medicine. They devised a 30 item EPA table. The third study developed the EPAs for all 7 bed-side procedures of internal medicine residents. And the last but not the least, developed 29 EPAs throughout the four stages of internal medicine residency program (3 for entering stage, 7 for foundations, 11 for the core of the program, and finally, 8 titles for arriving at the practical work place).

Table 1 EPAs resulted in studies on internal medicine residency programs adapted from O’dawed et al. [51]
Full size table

In the third step, the national and institutional curricula were reviewed. The national curriculum has listed the internal medicine residents’ activities in their diverse roles including preventive, diagnostic-therapeutic, care giving, educational, research, and administrative roles. Moreover, these programs have included the cognitive and procedural skills for residents. In the institutional curriculum of Isfahan University of Medical Sciences, the activities, assigned to each residency year, were also indicated. And lastly, the log book, installed on the residents’ cellphones and mainly planed for procedural activities, was utilized as another source of initial EPAs.

At the end of the fourth stage, 41 activities were resulted as the initial EPAs for all four residency years. These titles had a profound variation in terms of both the content and nature. So, they needed to be more elaborated.

In the fifth stage, 10 clinical academics accepted to attend the meeting. Twenty-eight activities were developed as the EPAs for all four residency years. Table 2 shows the results. These EPAs cover all aspects of the internal medicine residency program. They consist of all roles and responsibilities of an internal medicine specialist including the therapeutic, educational, consultation, procedural and administrative roles.

Table 2 Internal medicine residency EPAs and their entrustability levelsa for each year
Full size table

At the end of the focus group meeting, the EPAs were cross-tabbed with competencies. The related competencies to each EPA are shown in Table 3. Determining the appropriate evaluators was the final topic to be discussed in the meeting. The current and the last supervisors and chief residents all agreed up on as the key informants to evaluate residents against the EPAs using a common instrument.

Table 3 EPA-competency matrix
Full size table

In the last step of the study, for content validity of the results, after follow up, all of the ten academic internists responded to our email. Their recommendations and suggestions were discussed in the research team meeting, and eventually, EPAs, entrustability levels, and EPA-competency matrix were finalized.

Discussion

In this study we developed and finalized 28 EPAs for internal medicine residents. We also determined the level of entrustment at the end of each residency year and the interrelation between the final EPAs and those competencies expected from medical doctors in Iran. We already knew that different studies developed EPAs in different ways [6]. However, collecting, refining and validating are more common steps in most of the publications [51]. Gathering the initial statements, like the ones in our study, has been based on the literature review and group discussions, followed by revising and filtering steps. In this way, Hauer et al. only collected the opinions of the chief residents for processing the initial titles [38]. Caverzagie et al. mainly investigated the experts’ ideas [47], while in other studies, the ideas of learners in different levels have been scrutinized. For example, in Australia and New Zealand, the first and second year psychiatric residents’ EPAs were developed by using the fellowship residents’ suggestions [26]. However, in other programs like pediatrics, radiology, and anesthesia, clinical faculty members and the members of the central committees of the programs were invited [28, 52, 53].

The steps, followed in this study, were to some extent similar to the five steps, proposed by Kwan et al. including topic selection, content development, developing initial titles, getting feedback from stakeholders, and filtering and finalizing the activities [54]. Though, different ways such as comparing to other programs and conducting Delphi rounds have been suggested for collecting the initial titles [48].

Our final list of 28 EPAs is in accordance with the range of 20 to 40 titles, suggested by Ten Cate [6]. Although some authors reported much fewer EPAs like Chan et al. that studied only the patient handover [55], other studies developed more activities [56]. Moreover, there are some other kinds of study that classified the EPAs. For instance, Fessler et al. used three groups as pulmonary cares, pulmonary critical cares, and combination of these two groups of activities [57]. Graafland et al. developed 66 EPAs for sub divisions of anesthesiology as anesthesiology, emergency, gastroenterology, general surgery, gynecology, psychiatry and radiology [58].

Considering the EPAs listed in Table 2, it is inferred that these EPAs are not similar with regard to the time of entrustment. For example, patient admission and discharge is an activity which is done independently and without supervision only in the fourth year of residency; while, history taking and physical examination are the ones on which a second year resident must be capable to supervise the junior learners. Besides to these, the issue of what is professional activity, what is a task, what is a competency, and what is a competency category is difficult to parse -- even ten Cate seems to waiver in his descriptions of specificity in his various papers. Somewhere between sweeping general statements that cannot be directly measured (“the resident will have good patient care skills”) and specific, measurable tasks (“the resident will be able to perform a paracentesis”) lie the professional activities are unique to physicians in that specialty, are observable and measurable, and are part of the essential professional work of the specialty and not a general medical ability [7].

One of the advantages of our study is nominating the evaluators, which is noticeable with regard to the scarcity of the implementation and evaluation studies on EPAs. Nonetheless, peter et al. working on this phase of EPA study suggested using available information resources for assessing the residents [59].

Another advantage of the current study is determining the entrustability level of different residency years. It is while Mink et al. merely defined the meaning of each entrustability level in pediatrics fellowship program [60]. Hauer et al. only explored the perceptions of the faculty members and residents about the year a resident becomes entrustable on a given EPA [48].

There is another advantage for this study which is the comprehensiveness for all aspects of the residency program including the settings and competency domains; whereas, many studies reported development of EPAs in very limited areas of internal medicine graduate study like safe patient discharge [49, 61].

As shown in the competency-EPA matrix (Table 3), most of the EPAs are such broad that are related to many competencies. It means that each EPA is necessary for meeting more than one competency. It would absolutely be indicative of the inclusiveness of each EPA. It is in accordance with the Ten Cate’s suggestion [6]. He has mentioned elsewhere that the breadth of EPAs depends on the time they are used. For summative end of course decision making purposes named as “statements of awarded responsibility (STARS)” we need broader EPAs than formative uses throughout the course. For the latter we need to define nested EPAs [39]. As a result, and since we aimed to devise an end of year EPA list, some of them are more inclusive. It is suggested as next the step to implement the resulted EPAs in a pilot study so that they could be incorporated to a broader extent such as the national exams. Moreover, our further work would be defining the nested EPAs for broad ones.

Limitation

One of the limitations of this study could be using no residents’ points of view; nonetheless, we considered the residency logbook contents.

Another limitation of our study is being a single institutional initiative. Using the viewpoints of academics and residents of other medical universities is also suggested.