Abstract
Introduction
Prediction of proficiency of laparoscopic skills is essential to establish personalized training programs. Objective assessment of laparoscopic skills has been validated in a laparoscopic box trainer with force, motion and time recognition. The aim of this study is to investigate whether acquiring proficiency of laparoscopic skills can be predicted based on performance in such a training box.
Methods
Surgical residents in their first year of training performed six different tasks in the Lapron box trainer. Force, motion and time data, three objective measures of tissue manipulation and instrument handling, were collected and analyzed for the six different tasks. Linear regression tests were used to predict the learning curve and the number of repetitions required to reach proficiency.
Results
A total of 6010 practice sessions performed by 42 trainees from 13 Dutch hospitals were assessed and included for analysis. Proficiency level was determined as a mean result of seven experts performing 42 trials. Learning curve graphs and prediction models for each task were calculated. A significant relationship between force, motion and time during six different tasks and prediction of proficiency was present in 17 out of 18 analyses.
Conclusion
The learning curve of proficiency of laparoscopic skills can accurately be predicted after three repetitions of six tasks in a training box with force, path length and time recognition. This will facilitate personalized training programs in laparoscopic surgery.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Surgeons require a specific set of advanced technical skills to safely perform minimally invasive surgery (MIS). These technical skills include hand–eye coordination, depth perception and handling long instruments with reduced tactile feedback [1,2,3,4]. The surgical residency program has followed an apprenticeship model of ‘‘see one, do one, teach one’’ for more than a century [5]. However, after the introduction of the relatively complex MIS, this approach of training in the operating room resulted in increased patient injuries and associated health care costs [6,7,8]. During the last decades, reports on improvement of patient safety by simulation training have been published [9,10,11]. This has started a paradigm shift of the way surgeons are trained, following a new model of “see one, simulate many, do one’’ [5, 12].
Traditionally, laparoscopic skills have been assessed subjectively using forms such as OSATS, GOALS, and OPRS [13,14,15]. However, it is important to include objective assessment in skills training to provide supervisors with a consistent tool to assess the skills of surgical residents [1]. A combination of performance parameters has been classified, representing tissue manipulation and instrument handling skills, which enables objective assessment of laparoscopic skills [16]. Our research group reported earlier successful implementation of objective performance parameters in basic laparoscopic skills training of first-year surgical residents, enabling objective assessment of learning curves [17].
Objective assessment of the learning curve is essential to determine when proficiency levels have been acquired [17,18,19,20]. It is demonstrated that baseline performances of psychomotor ability uniquely predict the learning curve during laparoscopic skills training with virtual reality simulators [21, 22]. Predicting learning curves at an early stage of training allow creating individually adjusted skills training programs in the near future [21, 23]. The aim of this study was to analyze and predict the learning curve of basic laparoscopic technical skills.
Methods
Participants
First-year surgical resident who completed a basic laparoscopy course between April 2020 and June 2021 were included for prospective data-analysis. Surgical residents from multiple teaching hospitals in the Netherlands were included. The Basic Laparoscopy Course is part of the surgical residency program and participating in the study was voluntary (and without consequences). The study was exempt from Ethical Board review.
Protocol and the basic laparoscopy course
The basic laparoscopy course consisted of a 3-week at-home laparoscopic box training course, followed by a hands-on training day at the Amsterdam Skills Centre, consisting of performing a laparoscopic appendectomy and cholecystectomy on fix for life cadaver models [17]. Trainees received a laparoscopic box trainer and were instructed to train a minimum of five sessions a week, performing six different validated laparoscopic tasks [13, 24,25,26] (Supplemental File A). Measurements were compared to predefined proficiency levels, which were equal to mean parameter outcomes of 7 surgeons [13]. The scoring system consists of a scale of 1–10 with 8 being the proficiency level (pre-set competency based on experts). The score consists of the average of the force, motion and time which are each scored individually.
At the end of this course, the trainees performed the six tasks once again as a post-course assessment. Overall progression was measured by comparing baseline and post-course assessment. Objective force, motion and time parameters were measured, representing tissue manipulation and instrument handling skills [16, 27].
System and materials
The Lapron box trainers (Amsterdam Skills Centre, Amsterdam, The Netherlands) [28] were utilized during the basic laparoscopy course. The box trainers were equipped with the ForceSense objective measuring system (MediShield B.V., Delft, the Netherlands) [29], which uploaded all measurements and recordings to an online database. Six previously validated laparoscopic tasks were included: Post and Sleeve, Loops and Wire, Flap task, Wire chaser, Pattern cut and Zigzag loop [17] (Supplemental File A). Furthermore, the Lapron box trainer was equipped with two curved Maryland grasping forceps, one laparoscopic scissor and a laparoscopic axial needle holder (Aesculap, B. Braun, Melsungen, Germany). All statistical analyses were performed using the 26th version of IBM SPSS Statistics. Graphs were created using GraphPad (Prism 9.0.0, San Diego, California USA).
Statistical analyses
Learning curves that show maximum force (N), path length (mm) and time (s) were created for the six tasks, displaying the group mean and proficiency levels. The path length was defined as the total distance travelled by the laparoscopic instruments and the maximum force was defined as the maximum absolute force applied on the laparoscopic tasks [13].
Linear regression tests were performed to predict the learning curve at an early stage of training using IBM SPSS statistics 28 (SPSS Inc., Chicago, Illinois USA). Baseline performances of the parameters maximum force (N), path length (mm) and time (s), were included as independent variables. These baseline performances were defined as the average scores of the first three measurements. The number of sessions that were needed to reach the proficiency level were included as dependent variables. Linear regression tests were performed separately for each parameter of the six tasks. All sessions that were not successfully completed due to unforeseen circumstances and tasks that were performed less than three times were excluded from analysis. Trainees that did not reach the proficiency level for one of three parameters were excluded from analysis of this specific parameter. Post hoc power analyses were performed using GPower (Supplemental File A, Table A1).
Results
Learning curve analysis
A total of 6010 trials, performed by 42 trainees from 13 Dutch hospitals were included for analysis. Figure 1 shows the proficiency graphs of the parameters: maximum force (N), path length (mm) and time (s) for the Post and Sleeve laparoscopic task. Proficiency level graphs of all six tasks are provided in Supplemental file B.
For the Post and Sleeve, the benchmark of maximum force was reached at the 4th session, while the benchmark of the path length was reached at the 32nd session and the benchmark of time at the 21st session of the training. Supplementary Figures B2-B6 show an improvement of the mean and standard deviation over the initial training sessions, after which it gradually levels out in the plateau phase.
Table 1 shows the proficiency level of each parameter and the number of trainees that reached the proficiency level. For all tasks, the proficiency level of the maximum force was the first to be acquired, except for the Wire chaser, in which the benchmark for the maximum force was reached at the 38th session. The benchmarks of the parameters time and path length were reached at the same time for the Flap task and the Pattern cut. While for the other tasks, the proficiency level for time reached before the proficiency level of the path length. Moreover, 19 out of 42 reached proficiency for the path length of the Loops and wire, 21 out of 42 trainees reached the proficiency level of the maximum force for the Wire chaser, and 31 out of 42 trainees reached the proficiency level of the path length for the Zigzag loop. These three parameters were reached by the lowest number of trainees. The remaining 15 parameters were reached by more than three quarters of the trainees.
Learning curve prediction
The results of the Linear regression analyses are provided in Table 2. For 17 of 18 parameters, the baseline performance had a statistically significant relationship with the number of sessions needed to reach the benchmark. Within the path length of the Loops and wire, this relation was insignificant. Fifteen out of 18 dependent variables were not normally distributed and therefore were either log-transformed or square-root transformed, as shown in Table 2. The relation between the number of sessions needed to reach the benchmark and the baseline performance was quadratic for two out of 18 parameters. For these parameters, curvilinear regression tests were performed in which squared independent variables were included for analysis.
Table 3 shows the linear regression equations for the estimation of the number of sessions needed to reach the benchmark. Transformed models are either Log – Linear (Log10) or Square-root – Linear. Post hoc power-analysis revealed high power (> 0.8) for 16 out of 18 linear regression tests. The power of the path length within the Loops and wire was 0.138, while the power of time within the Zig-zag loop was 0.610. See Supplementary file A for results of the post hoc power-analyses (Supplementary Table A1).
Discussion
This study showed that it was possible to predict the learning curve of laparoscopic technical skill in a basic laparoscopy course at an early stage of training. By performing a laparoscopic task three times, it is possible to calculate how many repetitions are needed to acquire the benchmark for force, motion and time parameters. For example, according to the calculations in Fig. 2, 19 repetitions for reaching the time benchmark, 31 repetitions for reaching the path length benchmark and two repetitions to not exceed maximum force are advised when a trainee completed the Post and Sleeve task three times with the following average outcome: time 238 s, path length 12,836 mm and maximum force 2.94 N.
This allows identifying trainees who require more time and feedback for their laparoscopic training. Furthermore, the possibility arises to find trainees require less training time for basic laparoscopic skills, and hence, can advance earlier to more complex laparoscopy. Lastly, using the current methods it is possible to recreate this learning curve prediction model for other laparoscopic (and robotic) training tasks and curricula.
Proficiency graphs displaying the group learning curve made it possible to analyze the learning curve and determine if and when the proficiency level is reached for each parameter of the six tasks. For all except one tasks, the benchmark of maximum force was the first to be reached. The benchmarks of the parameters time and path length were either reached at once, or the benchmark of time was reached before the benchmark of the path length. This indicates that trainees need the most time to improve their path length. This is supported by the analysis of the number of trainees that reach the proficiency level, in which the benchmark of the path length was reached the least often. This is consistent with our prior conducted research [17]. At the start of the training, the majority of novices is focused on safe tissue manipulation and on completion of the task. Resulting in a longer completion time and more instrument movements. Furthermore, efficient handling of instruments is more inherent to experts and their proficiency levels in this metric are relatively high. Using the above mentioned prediction model, more feedback and guidance can now be given in an early phase for path length parameters.
Furthermore, differences between tasks and parameters can be examined. It is found that the proficiency levels for the Zig-zag loop are reached at a later stage of training, while the three benchmarks for the Pattern cut are reached within three sessions, suggesting that the Pattern cut is easier to perform, compared to other tasks.
The learning curve of all residents improved rapidly during the first sessions, after which it gradually leveled out in the plateau phase, which is as expected [30]. This suggests that it is possible to predict the learning curve at an early stage of training. Stefanidis et al. (2017) stated that baseline performances, which were defined as average scores of the first three measurements, might be of value in the prediction of skill acquisition in laparoscopic skills training with virtual reality simulators [31]. This was consistent with our analysis, the mean of the first three measurements were defined as baseline performances and included as predictor variables in the prediction of the learning curve in the basic laparoscopy course.
Improving a personalized curriculum could be achieved by showing the trainees performance level and the proficiency levels during training. This enables comparing the trainees performance with the proficiency level, the group mean and quartiles 1 to 3. Displaying these proficiency graphs during training can enhance the individual feedback that is received directly by trainees. [32]. As an example of personalized training, the Amsterdam UMC and the 13 affiliated teaching hospitals have a Minimally Invasive Surgery Curriculum in which personalized training is implemented. Since 2018, the Basic Laparoscopy Course is mandatory for junior residents. The course participants receive box training with objective feedback and are examined on fix4life human cadavers in the Amsterdam Skills Centre. After obtaining the certificate, the residents perform laparoscopic procedures in the OR.
A limitation of this study is that trainees that did not reach the proficiency level were excluded from analysis within the learning curve prediction. This could have the effect that the prediction model is more optimistic in prediction. This was because for statistical reasons making a prediction model only the trainees that achieved proficiency could be reliably used for prediction. However, the prediction model is still able to distinguish between underperformers and overperformers in an early phase. This implies that overperformers would not need to use expensive training facilities for an extended period. And underperformers will be identified in an early phase for additional personalized training. A personalized prediction model can be applied universally for all trainees.
In conclusion, measurement of objective force, motion and time parameters can predict the time of reaching proficiency allowing tailored and personalized training.
References
Zendejas B, Brydges R, Hamstra SJ, Cook DA (2013) State of the evidence on simulation-based training for laparoscopic surgery: a systematic review. Ann Surg 257(4):586–593
Munz Y, Kumar BD, Moorthy K, Bann S, Darzi A (2004) Laparoscopic virtual reality and box trainers: is one superior to the other? Surg Endosc Other Interv Tech 18(3):485–494
Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G et al (2004) Proving the value of simulation in laparoscopic surgery. Ann Surg 240(3):518–528
Schreuder HWR, Van Den Berg CB, Hazebroek EJ, Verheijen RHM, Schijven MP (2011) Laparoscopic skills training using inexpensive box trainers: which exercises to choose when constructing a validated training course. BJOG An Int J Obstet Gynaecol 118(13):1576–1584
Stefanidis D, Sevdalis N, Paige J, Zevin B, Aggarwal R, Grantcharov T et al (2015) Simulation in surgery: what’s needed next? Ann Surg 261(5):846–853
Gaba DM (2007) The future vision of simulation in healthcare. Simul Healthc 2(2):126–135
Bonrath EM, Gordon LE, Grantcharov TP (2015) Characterising “near miss” events in complex laparoscopic surgery through video analysis. BMJ Qual Saf 24(8):516–521
Babineau TJ, Becker J, Gibbons G, Sentovich S, Hess D, Robertson S, et al. (2004) The Cost of Operative Training for Surgical Residents. Arch Surg [Internet]. Apr 1 [cited 2021 Nov 30];139(4):366–70. Available from: https://jamanetwork-com.vu-nl.idm.oclc.org/journals/jamasurgery/fullarticle/396495
Sevdalis N, Hull L, Birnbach DJ (2012) Improving patient safety in the operating theatre and perioperative care: obstacles, interventions, and priorities for accelerating progress. Br J Anaesth 109(SUPPL1):i3-16
Meling TR, Meling TR. (2021) The impact of surgical simulation on patient outcomes: a systematic review and meta-analysis. Neurosurg Rev [Internet]. Apr 1 [cited 2022 Jan 17];44(2):843–54. Available from: https://springerlink.fh-diploma.de/article/https://doi.org/10.1007/s10143-020-01314-2
Scott DJ. (2006) Patient safety, competency, and the future of surgical simulation. Simul Healthc [Internet]. [cited 2022 Jan 17];1(3):164–70. Available from: https://pubmed.ncbi.nlm.nih.gov/19088586/
Reznick RK, MacRae H (2006) Teaching surgical skills — changes in the wind. N Engl J Med 355(25):2664–2669
Hardon SF, Horeman T, Bonjer HJ, Meijerink WJHJ (2018) Force-based learning curve tracking in fundamental laparoscopic skills training. Surg Endosc 32(8):3609–3621
Kramp KH, Van Det MJ, Hoff C, Lamme B, Veeger NJGM, Pierie JPEN. (2015) Validity and reliability of global operative assessment of laparoscopic skills (GOALS) in novice trainees performing a laparoscopic cholecystectomy. J Surg Educ [Internet]. Mar 1 [cited 2021 Nov 23];72(2):351–8. Available from: https://pubmed.ncbi.nlm.nih.gov/25441259/
Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D, et al. (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg [Internet]. Jul [cited 2021 Nov 23];190(1):107–13. Available from: https://pubmed.ncbi.nlm.nih.gov/15972181/
Horeman T, Dankelman J, Jansen FW, Van Den Dobbelsteen JJ. (2014) Assessment of laparoscopic skills based on force and motion parameters. IEEE Trans Biomed Eng [Internet]. Mar [cited 2021 Feb 13];61(3):805–13. Available from: https://pubmed.ncbi.nlm.nih.gov/24216633/
Hardon SF, van Gastel LA, Horeman T, Daams F. (2021) Assessment of technical skills based on learning curve analyses in laparoscopic surgery training. Surgery [Internet]. Sep 1 [cited 2022 Jun 28];170(3):831–40. Available from: https://pubmed.ncbi.nlm.nih.gov/34090675/
Louridas M, Szasz P, De Montbrun S, Harris KA, Grantcharov TP (2016) Can we predict technical aptitude?: a syste matic review. Ann Surg 263(4):673–691
Louridas M, Szasz P, Fecso AB, Zywiel MG, Lak P, Bener AB et al (2017) Practice does not always make perfect: need for selection curricula in modern surgical training. Surg Endosc 31(9):3718–3727
Grantcharov TP, Funch-Jensen P (2009) Can everyone achieve proficiency with the laparoscopic technique? learning curve patterns in technical skills acquisition. Am J Surg 197(4):447–449
Groenier M, Groenier KH, Miedema HAT, Broeders IAMJ (2015) Perceptual speed and psychomotor ability predict laparoscopic skill acquisition on a simulator. J Surg Educ 72(6):1224–1232
McClusky DA, Ritter EM, Lederman AB, Gallagher AG, Smith CD, Park AE et al (2005) Correlation between perceptual, visuo-spatial, and psychomotor aptitude to duration of training required to reach performance goals on the MIST-VR surgical simulator. Am Surg 71(1):13–21
Ritter EM, McClusky DA, Gallagher AG, Enochsson L, Smith CD (2006) Perceptual, visuospatial, and psychomotor abilities correlate with duration of training required on a virtual-reality flexible endoscopy simulator. Am J Surg 192(3):379–384
Peters JH, Fried GM, Swanstrom LL, Soper NJ, Sillin LF, Schirmer B et al (2004) Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery 135(1):21–27
Horeman T, Van Delft F, Blikkendaal MD, Dankelman J, Van Den Dobbelsteen JJ, Jansen FW. (2014) Learning from visual force feedback in box trainers: Tissue manipulation in laparoscopic surgery. Surg Endosc [Internet]. [cited 2021 Feb 8];28(6):1961–70. Available from: https://pubmed.ncbi.nlm.nih.gov/24519025/
Derossis AM, Fried GM, Abrahamowicz M, Sigman HH, Barkun JS, Meakins JL (1998) Development of a model for training and evaluation of laparoscopic skills. Am J Surg 175(6):482–487
Horeman T, Rodrigues SP, Willem Jansen F, Dankelman J, Van Den Dobbelsteen JJ. (2012) Force parameters for skills assessment in laparoscopy. IEEE Trans Haptics [Internet]. [cited 2021 Feb 13];5(4):312–22. Available from: https://pubmed.ncbi.nlm.nih.gov/26964129/
The Lapron Box Trainer by ASC & ForceSense. https://lapron.net/ [Internet]. [cited 2021 Nov 23]; Available from: https://lapron.net/
Horeman T. (2014) Force Based Assessment of Tissue Handling Skills.. 181 p.
Hopper AN, Jamison MH, Lewis WG. (2007) Learning curves in surgical practice [Internet]. Vol. 83, Postgraduate Medical Journal. Postgrad Med J; [cited 2020 Nov 4]. p. 777–9. Available from: https://pubmed.ncbi.nlm.nih.gov/18057179/
Stefanidis D, Korndorffer JR, Black FW, Dunne JB, Sierra R, Touchard CL et al (2006) Psychomotor testing predicts rate of skill acquisition for proficiency-based laparoscopic skills training. Surgery 140(2):252–262
Ahlborg L, Weurlander M, Hedman L, Nisel H, Lindqvist PG, Felländer-Tsai L et al (2015) Individualized feedback during simulated laparoscopic training:a mixed methods study. Int J Med Educ 6:93–100
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Disclosures
Drs. A. Masie Rahimi, drs. Sem F. Hardon, Ezgi Uluç, prof. dr. H. Jaap Bonjer and dr. Freek Daams have no conflict of interest or financial ties to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Rahimi, A.M., Hardon, S.F., Uluç, E. et al. Prediction of laparoscopic skills: objective learning curve analysis. Surg Endosc 37, 282–289 (2023). https://doi.org/10.1007/s00464-022-09473-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00464-022-09473-7