Abstract
Background
A balance between autonomy and supervision can be difficult to obtain in medical education. In this study, we sought to determine whether the presence and level of supervision of ophthalmology resident outpatient clinic correlates with metrics of resident success, professionalism and stress.
Methods
A survey was emailed to all US ophthalmology program directors requesting it be forwarded to PGY4 residents. Questions included whether their program provided a resident-hosted outpatient clinic, and if so, whether residents were mandated to discuss every patient with faculty. Residents were assigned to three categories based on this question (0: no clinic, 1: mandated faculty input, 2: discretionary faculty input). Success metrics included numbers of manuscripts submitted, OKAP scores and success in obtaining fellowships. Professionalism metrics included rating comfort obtaining informed consent, breaking bad news, managing time in clinic, and confidence in providing care in various settings. Residents affirming participation in a continuity clinic also provided perceptions of the level of supervision and how the clinic affected stress.
Results
Category 1 residents perceived somewhat too much supervision, while category 2 residents felt that they had somewhat insufficient supervision. The majority of residents in either category did not feel that the continuity clinic affected their overall stress, although those who reported a change in stress usually indicated that the presence of the clinic increased stress. There were no other statistically significant differences between the responses from any category.
Conclusions
The presence of a resident-hosted continuity clinic neither adds nor detracts from the success or sense of professionalism of ophthalmology residents. However, when such a clinic is present, the degree of supervision appears to correlate inversely with resident perception of autonomy. These results suggest that the decision of a training program to offer a clinic hosted by residents offering comprehensive continuity care can be informed primarily by faculty and trainee philosophy and personal preferences without comprising education quality, clinical efficiency, residents’ perception of stress or their success in fellowship matching.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Background
Balancing autonomy and supervision is both a goal and challenge of graduate medical education [1,2,3,4,5,6,7,8]. In a previous report, we presented the landscape of supervision styles for ophthalmology trainees in the outpatient clinic setting [2]. That study, based upon a survey of program directors, suggested that there was no substantial correlation between the level of supervision in the outpatient setting and metrics such as the number of residents in the program, demographic sites of the clinics, number of faculty or contributions of resident-hosted clinics to overall resident surgery volumes. The current study has sought to determine whether the presence of a resident-hosted continuity clinic, and the degree of faculty supervision therein, might correlate with metrics of programmatic success and resident professionalism. This second study is a joint effort of the Ophthalmology Program Directors Study Group (OPDSG), currently comprised of 31 program directors (i.e., 27% of US training programs). This group formed after the first study because it was felt that the effort to identify evidence-based best practices in resident education deserved widespread support.
Methods
An 18-question online survey was emailed to all 115 ACGME-accredited ophthalmology residency program directors in April 2018, requesting they forward the survey to all third-year residents in their respective programs. Follow-up emails to all program directors as well as program coordinators were sent in May 2018. To encourage responses, a $10 charitable donation was pledged to Research to Prevent Blindness for every completed survey. All emails included an introductory letter explaining the purpose of the study, providing a link to the previous study [2] and confirming that this study was reviewed and approved by the Johns Hopkins Institutional Review Board (IRB00069769.)
This was an anonymous survey where no personal identifying information was collected. The survey was designed using forced choices, Likert scales, and in some instances, the option to provide additional free-text information. The survey uses simple Boolean logic so that responders could be directed to germane questions based upon their answers while skipping over questions that might be redundant or non-sequitur. The survey as it appeared to respondents can be accessed with this link: https://jhmi.co1.qualtrics.com/jfe/form/SV_80pRNHwoQaXTuiF. Supporting data including the anonymized respondents of those surveyed are available upon request from the corresponding author.
The survey questions and the logic-flow instructions based upon the answers to each question are presented as Additional file 1. The first set of questions were meant to assess the academic success of the residents. The OPDSG queried residents on the number of publications they submitted, their overall percentile on the Ophthalmology Knowledge Assessment Program (OKAP, an in-service exam created by the American Academy of Ophthalmology [AAO] [9]), and their success and motivation in applying for post-graduate fellowships. The second set of survey questions queried resident comfort level with situations requiring a mature level of professionalism including obtaining informed consent, relaying disappointing news to patients, and managing patients in the emergency, inpatient and outpatient setting. The third set of questions were meant to identify whether the residents’ training included participation in a resident-hosted comprehensive continuity clinic. Those responding positively were asked how that clinic was supervised in terms of whether a faculty member was required to discuss, and/or see every patient encountered. The initial survey defined five categories (listed below from lowest to highest) with respect to the degree of resident autonomy:
-
0.
No resident-hosted clinic
-
1.
Resident-hosted clinic where the supervisor must see every resident patient
-
2.
Resident-hosted clinic where the supervisor must discuss every patient with the resident but not necessarily see the patient
-
3.
Resident-hosted clinic where the supervisor was not required to discuss every patient with the resident.
-
4.
Resident-hosted clinic where the supervisor was not onsite but available by telephone for indirect supervision.
However, the number of responses for categories 1 and 4 were considered too low to obtain meaningful statistics so category 1 was combined with category 2 and category 3 was combined with category 4. The new categories are listed below.
-
0.
No resident-hosted clinic
-
1.
Resident-hosted clinic where the supervisor must at least discuss (if not also see) every patient encountered
-
2.
Resident-hosted clinic where the supervisor discussed (and possibly saw) a patient at the resident’s discretion.
The final set of survey questions were directed to residents identifying with (initial) categories 1–4, (i.e., new categories 1 and 2) exploring how they perceived the degree of supervision they were provided and whether the clinic experience affected their levels of stress.
The finalized categories of resident clinics were assigned an ordinal score of 0–2 to explore whether statistical correlations existed between the presence of a resident-hosted clinic (and if so, the level of supervision) and the other metrics gleaned in the survey.
For the sample size calculation, we determine that we required at least 97 complete response to obtain a marginal error of 5% and confident interval of 95%. We employed the ANOVA test for the questions exploring numbers of submissions for publication, OKAP score, rank of matched fellowship program, comfort breaking bad news, comfort obtaining informed consent, numbers of patients seen daily in clinic and confidence managing patients in the outpatient setting, emergency department and inpatient setting. The chi squared test was employed for the questions pertaining to whether a resident applied and matched to a post-graduate fellowship and whether they felt that the outpatient continuity clinic affected their level of stress during training. Finally, the Fisher Exact test was employed for the questions concerning difficulty managing time in the clinic and whether the outpatient clinic increased or decreased the level of stress experienced during training.
Results
There were 116 completed responses: 39, 29 and 48 in categories 0, 1 and 2, respectively. Estimating that there are approximately 490 senior ophthalmology residents in the United States, this represents a response rate of 24%. The average number of submissions for publications, OKAP overall percentile, rate of application for fellowship, motivation therein, and success and rank of program if matched are provided in Table 1. There was no statistically significant correlation between these metrics and the category of resident outpatient clinic.
Table 2 provides the data for the resident scoring concerning their comfort breaking bad news and obtaining informed consent, their difficulty managing time in the outpatient clinic and the number of clinic patients they see on a typical day, and their confidence managing patients in the clinic and providing consultations in the emergency department and inpatient wards. There was no statistically significant correlation between these metrics and the category of resident outpatient clinic.
For those residents in clinic category 1 and 2 (i.e., in programs providing a resident-hosted outpatient comprehensive continuity clinic), the average levels of perceived supervision and effect on overall stress during training is presented in Table 3. No resident felt that there was far too much or far too little supervision. Residents with category 1 clinics felt there was moderately more than enough supervision while residents in category 2 sensed there was somewhat insufficient supervision and this difference was statistically significant (p < .0001). Of the residents who felt that the continuity clinic affected their stress during training, the great majority in both categories felt that the experience increased stress. Notably, while it appears that residents in category 2 felt their clinics contributed to stress more often than those in category 1, this difference did not reach statistical significance (p = .11). Concerning the increased stress reported, it should be noted that of their 12 free-text comments, 7 were negative while 5 were positive. Positive comments included “good stress promotes learning”, “relationships with known patients alleviated stress”, “stress is appropriate as I care about my own patients’ outcomes”, “increased stress is not necessarily a bad thing” and stress “was helpful preparing for real practice”.
Discussion
As in our previous study [2], we did not find significant correlations between the presence of a resident-hosted continuity clinic or, when such a clinic was present, the level of supervision offered therein, and the various metrics of resident success explored. These metrics included the number of resident manuscripts submitted, OKAP scores, and application and success rates of obtaining fellowships. One might suggest that a greater number of completed surveys might have permitted the wider granularity in the categories of training environments described in our previous study, thus revealing significant differences. In addition, since residents volunteered to participate, the authors could not control for negative or positive biases of respondents. Furthermore, one could suggest employing different metrics of resident success than the ones chosen. Presently, there are no established or generally accepted metrics of this type; there may be utility for a governing body such as the Accreditation Council for Graduate Medical Education to consider creating such a list.
Aside from metrics of success, we also explored metrics related to professionalism and resident perceptions thereof, such as comfort obtaining informed consent, relaying disappointing news to patients, and managing patients in the emergency, inpatient and outpatient setting. Patient volumes were similar for all clinic styles, suggesting there is no apparent effect on clinical efficiency or the quantity of outpatient educational opportunities inherent in every encounter. Furthermore, fewer than half the residents serving in their own continuity clinics felt that this teaching environment affected overall stress. In addition, even though perceived stress was increased almost uniformly when there was a reported effect, this stress was not always considered to be negative. This was borne out by the responses to the free-text questions which supported the idea that the residents felt that the clinic was a good learning opportunity and that they appreciated having ownership of their patients.
Further research in this area should include surveying young post-graduates, querying them on metrics such as their professional setting (e.g., academic, research, industry, private practice), application- and pass-rate as well as number of attempts for the American Board of Ophthalmology Written Qualifying- and Oral-Examinations, and perception of how well their outpatient experiences prepared them for practice. The authors conclude that residency training programs can make decisions about offering resident-hosted continuity clinics with equanimity.
This key result of the present study could have important implications for residents as well as training programs. For example, increased autonomy has been correlated with lower levels of resident burn-out [10,11,12], a significant and increasingly recognized concern of training [13,14,15,16]. It may be that residents prone to burnout would prefer a training program with opportunities to gain a sense of greater autonomy such as that outpatient clinic where they take ownership of patients and have discretion concerning supervisor input. Alternatively, residents who are more stress- or risk-averse might prefer an outpatient setting with closer supervision; it has been reported that a resident’s perception of insufficient oversight was correlated with other negative perceptions of clinical training, such as increased stress and risk of medical errors [5].
Whereas residents must be focused on learning, residency programs must take other aspects of clinical operations into account. For example, a program might require supervisors to corroborate (and bill for) every resident encounter to ensure the economic viability of the resident clinic and ensure positive margins for supervisors in the resident clinic. The present study suggests that this arrangement would not reduce resident success or their sense of professionalism. Indeed, such an arrangement might even reduce medical errors [17,18,19], although there is debate on this issue [20]. On the other hand, in situations where 100% sign-off of charts has no financial advantage, then the perceived diminution to the residents’ sense of autonomy may rise in prioritization. Furthermore, one could suggest that if programs employed preceptors in the clinic with a variety of supervision styles, there is no evidence that this would diminish the educational or professional training.
In conclusion, the present study suggests that the decision of a training program to offer a clinic hosted by residents offering comprehensive continuity care can be informed primarily by faculty and trainee philosophy and personal preferences without comprising education quality, clinical efficiency, residents’ perception of stress or their success in fellowship matching.
Conclusions
The presence of a resident-hosted continuity clinic neither adds nor detracts from an ophthalmology resident’s success (as measured by number of resident manuscripts submitted, OKAP scores, and application and success rates of obtaining fellowships) or sense of professionalism (as measured by self-reported comfort in obtaining informed consent, relaying disappointing news to patients, and managing patients in the emergency, inpatient and outpatient setting). However, when such a clinic is present, the degree of supervision appears to correlate inversely with resident perception of autonomy. These results suggest that the decision of a training program to offer a clinic hosted by residents offering comprehensive continuity care can be informed primarily by faculty and trainee philosophy and personal preferences without comprising education quality, clinical efficiency, residents’ perception of stress or their success in fellowship matching.
Availability of data and materials
Anonymized data is available from the corresponding author upon request.
Abbreviations
- AAO:
-
American Academy of Ophthalmology
- ACGME:
-
Accreditation Council for Graduate Medical Education
- OKAP:
-
Ophthalmology Knowledge Assessment Program
- OPDSG:
-
Ophthalmology Program Directors’ Study Group
- PGY4:
-
Post Graduate Year 4
References
Happel JP, Ritter JB, Neubauer BE. Optimizing the balance between supervision and autonomy in training. JAMA Intern Med. 2018. https://doi.org/10.1001/jamainternmed.2018.1250 published Online First: Epub Date.
Singman EL, Srikumaran D, Green L, Tian J, McDonnell P. Supervision and autonomy of ophthalmology residents in the outpatient Clinic in the United States: a survey of ACGME-accredited programs. BMC Med Educ. 2017;17(1):105. https://doi.org/10.1186/s12909-017-0941-0 published Online First: Epub Date.
Kempenich JW, Willis RE, Rakosi R, Wiersch J, Schenarts PJ. How do perceptions of autonomy differ in general surgery training between faculty, senior Residents, hospital administrators, and the general public? A multi-institutional study. J Surg Educ. 2015;72(6):e193–201. https://doi.org/10.1016/j.jsurg.2015.06.002 published Online First: Epub Date.
Farnan JM, Johnson JK, Meltzer DO, et al. Strategies for effective on-call supervision for internal medicine residents: the superb/safety model. J Grad Med Educ. 2010;2(1):46–52. https://doi.org/10.4300/JGME-D-09-00015.1 published Online First: Epub Date.
Baldwin DC Jr, Daugherty SR, Ryan PM. How residents view their clinical supervision: a reanalysis of classic national survey data. J Grad Med Educ. 2010;2(1):37–45. https://doi.org/10.4300/JGME-D-09-00081.1 published Online First: Epub Date.
Farnan JM, Johnson JK, Meltzer DO, Humphrey HJ, Arora VM. On-call supervision and resident autonomy: from micromanager to absentee attending. Am J Med. 2009;122(8):784–8. https://doi.org/10.1016/j.amjmed.2009.04.011 published Online First: Epub Date.
ACGME. ACGME Program requirements for graduate medical education in ophthalmology. Secondary ACGME program requirements for graduate medical education in ophthalmology 2016. https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/240_ophthalmology_2017-07-01.pdf. Accessed 27 May 2019.
Caniano DA, Hamstra SJ. Program strengths and opportunities for improvement identified by Residents during ACGME site visits in 5 surgical specialties. J Grad Med Educ. 2016;8(2):208–13. https://doi.org/10.4300/JGME-D-15-00322.1 published Online First: Epub Date.
AAO American Academy of Ophthalmology. OKAP Exam; https://www.aao.org/okap-exam; accessioned 6/27/18. Secondary OKAP Exam; https://www.aao.org/okap-exam; accessioned 6/27/18. 2018. https://www.aao.org/okap-exam.
Zis P, Anagnostopoulos F, Sykioti P. Burnout in medical residents: a study based on the job demands-resources model. ScientificWorldJournal. 2014;2014:673279. https://doi.org/10.1155/2014/673279 published Online First: Epub Date.
Rutherford K, Oda J. Family medicine residency training and burnout: a qualitative study. Can Med Educ J. 2014;5(1):e13–23.
Kimo Takayesu J, Ramoska EA, Clark TR, et al. Factors associated with burnout during emergency medicine residency. Acad Emerg Med. 2014;21(9):1031–5. https://doi.org/10.1111/acem.12464 published Online First: Epub Date.
Rothenberger DA. Physician burnout and well-being: a systematic review and framework for action. Dis Colon Rectum. 2017;60(6):567–76. https://doi.org/10.1097/DCR.0000000000000844 published Online First: Epub Date.
Levin KH, Shanafelt TD, Keran CM, et al. Burnout, career satisfaction, and well-being among US neurology residents and fellows in 2016. Neurology. 2017;89(5):492–501. https://doi.org/10.1212/WNL.0000000000004135 published Online First: Epub Date.
Holmes EG, Connolly A, Putnam KT, et al. Taking Care of our own: a multispecialty study of resident and program director perspectives on contributors to burnout and potential interventions. Acad Psychiatry. 2017;41(2):159–66. https://doi.org/10.1007/s40596-016-0590-3 published Online First: Epub Date.
Guenette JP, Smith SE. Burnout: prevalence and associated factors among radiology Residents in New England with comparison against United States resident physicians in other specialties. AJR Am J Roentgenol. 2017;209(1):136–41. https://doi.org/10.2214/AJR.16.17541 published Online First: Epub Date.
Singh H, Thomas EJ, Petersen LA, Studdert DM. Medical errors involving trainees: a study of closed malpractice claims from 5 insurers. Arch Intern Med. 2007;167(19):2030–6. https://doi.org/10.1001/archinte.167.19.2030 published Online First: Epub Date.
Baldwin DC, Jr., Daugherty SR, Ryan PM, Yaghmour NA, Philibert I. Residents' Ratings of their clinical supervision and their self-reported medical errors: analysis of data from 2009. J Grad Med Educ 2018;10(2):235–241 doi:https://doi.org/10.4300/JGME-D-18-00200.1[published Online First: Epub Date]|.
Naveh E, Katz-Navon T, Stern Z. Resident physicians' clinical training and error rate: the roles of autonomy, consultation, and familiarity with the literature. Adv Health Sci Educ Theory Pract. 2015;20(1):59–71. https://doi.org/10.1007/s10459-014-9508-6 published Online First: Epub Date.
Finn KM, Metlay JP, Chang Y, et al. Effect of increased inpatient attending physician supervision on medical errors, patient safety, and resident education: a randomized clinical trial. JAMA Intern Med. 2018. https://doi.org/10.1001/jamainternmed.2018.1244 published Online First: Epub Date.
Acknowledgements
The writing committee of the OPDSG would like to thank all the members of the OPDSG for their input during the formulation of the survey questions and their comments on the manuscript. The OPDSG would also like to thank the program coordinators for their assistance in distributing the survey to their residents. The members of the OPDSG are listed below:
Name | Title | Program |
---|---|---|
Andrew J. Barkmeier, MD | Program Director | Mayo Clinic |
Andrew J. Hendershot, MD | Program Director | Wexner Med. Ctr. Eye & Ear Inst. |
Andrew Thliveris, PhD MD | Program Director | Univ. Wisconsin-Madison |
Daniel B. Moore, MD | Program Director | Univ. Kentucky |
Darrell WuDunn, MD PhD | Program Director | Eugene & Marilyn Glick Eye Inst., Indiana Univ. Sch. Of Medicine |
David J. Goldman, MD MBA | Program Director | Henry Ford Hosp. |
Divya Srikumaran, MD | Vice Chairman for Medical Education | Wilmer Eye Inst., Johns Hopkins Univ. |
Evan L. Waxman, MD PhD | Program Director | Univ. Pittsburgh Sch. Of Med. |
Gary Domeracki, MD | Program Director Emeritus | Temple Univ. Hospital |
Gary L. Legault, MD | Program Director | San Antonio Uniformed Health Svc. Consortium - Brooke Army Med. Ctr. |
Jeff Pettey, MD | Program Director | John Moran Eye Center, Univ. Utah |
John J. Chen, MD PhD | Assoc. Program Director | Mayo Clinic |
Joshua H. Olson, MD | Associate Program Director | Univ. Minnesota |
Jules Winokur, MD | Program Director | Northwell Health |
Laura K. Green, MD | Program Director | Lifebridge Health Krieger Eye Inst. |
Marcus Colyer, MD | Program Director | National Capital Consortium (Walter Reed) |
Martin Mayers, MD | Chairman & Program Director | Bronx-Lebanon Hospital Med. Ctr. |
Michael J. Wilkinson, MD | Program Director | Penn State Hershey Med. Ctr |
Michael S. Lee, MD | Program Director | Univ. of Minnesota |
Michael V. Boland, MD PhD | Program Director | Wilmer Eye Inst., Johns Hopkins Univ. |
Misha Syed, MD | Program Director | Univ. Texas Medical Branch, Galveston |
Mitchell Drucker, MD | Program Director | Univ. South Florida Coll. Of Med. |
Mitchell P. Weikert, MD | Program Director | Cullen Eye Inst., Baylor Coll. Of Medicine |
Parisa Taravati, MD | Program Director | Univ. Washington |
Peter Veldman, MD | Program Director | Univ. Chicago Medicine |
Pratap Challa, MD | Program Director | Duke Univ. |
Preston H. Blomquist, MD | Program Director | Univ. Texas Southwestern Med. Ctr. |
R. Michael Siatkowski, MD | Program Director | Dean McGee Eye Inst., Univ. Oklahoma |
Robert Granadier, MD | Program Director | Beaumont Hospital, Royal Oak |
Shane Havens, MD | Program Director | Truhlsen Eye Inst., Univ. Nebraska Med. Ctr. |
Susan M. Culican, MD PhD | Program Director | Washington Univ. |
Tara A. Uhler, MD | Program Director | Wills Eye Hospital, Sidney Kimmel Medical College at Thomas Jefferson University |
Thomas J. Whittaker, MD JD | Program Director | Univ. Kansas |
Funding
This study was self-funded.
Author information
Authors and Affiliations
Consortia
Contributions
Development of concept: ES, MVB, LKG, DS. Creation of survey: ES, MVB, LKG, DS, members of OPDSG. Data analysis: ES, JT. Writing of manuscript: ES, all members of OPDSG. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
This study was approved by the Johns Hopkins Institutional Review Board. The consent form for participation was included in the email distributed to program directions and ensured that participation was voluntary.
Consent for publication
All members of the OPDSG are considered authors and all have provided written consent for publication by email.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional file
Additional file 1:
The survey in the format that the residents experienced. (DOCX 18 kb)
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Singman, E.L., Boland, M.V., Tian, J. et al. Supervision and autonomy of ophthalmology residents in the outpatient clinic in the United States II: a survey of senior residents. BMC Med Educ 19, 202 (2019). https://doi.org/10.1186/s12909-019-1620-0
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12909-019-1620-0