Abstract
Background
The collection of patient-reported outcome measures (PROMs) has many benefits for clinical practice. However, there are many barriers that prevent them from becoming a part of routine clinical care. The aim of this feasibility study was to pilot the use of a digital platform to facilitate the routine collection of pre- and post-operative electronic PROMs (ePROMs) in participants undergoing elective laparoscopic cholecystectomy and to validate the use of existing patient-reported outcomes for our population.
Methods
Participants scheduled for elective laparoscopic cholecystectomy were asked to complete digital versions of the Otago gallstones Condition-Specific Questionnaire (CSQ), and the RAND 36-item health survey (SF36). An assessment of methodological quality of ePROM questionnaires was also performed.
Results
Preoperative ePROMs were completed by 200 participants undergoing laparoscopic cholecystectomy. Post-operatively attrition was high (completion at 30 days, 3 months, and 6months: n = 61, 54, and 38, respectively) due to difficulties accessing our ePROMs portal. Of those able to complete, a significant improvement in quality of life was seen across all health domains post-operatively when compared with baseline preoperative values for both disease-specific and generic PROMs. Methodological quality was assessed as good to excellent in both digital questionnaires.
Conclusion
The collection of ePROMs is possible with current technological advances. Although it may be an acceptable, and convenient process for patients, and a useful measure of quality-of-life trends for clinicians, further developmental work is necessary to improve accessibility for patients, improve compliance, and reduce reporting bias from high attrition rates.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Gallstones are a common problem, affecting 10–15% of adults over the age of 18 years, with around 60,000 cholecystectomies being performed in England alone [1]. Symptomatic gallstone disease can pose a significant health and societal burden, impacting physical well-being, and an individual’s ability to work [2]. Despite the clear benefits of surgery and the potential economic savings resulting from preventing further morbidity from gallstones, laparoscopic cholecystectomy is not without risks, some of these being associated with a significant reduction in quality of life [3, 4].
Patient-reported outcomes (PROs) are used as a subjective measure for a patient’s health-related quality of life (HRQoL), at a set point in time, and from the patient’s perspective [5]. The use of validated questionnaires or patient-reported outcome measures (PROMs) has been mandatory practice in the UK National Health Service’s (NHS) national PROMs programme since 2009 [6, 7]. HRQoL data collected for patients undergoing hip and knee replacement surgery, groin hernia repair, and varicose vein surgery have been used to enable providers, commissioners, and stakeholders to make informed changes to service delivery which can then be used to improve clinical outcomes following intervention or surgery [6, 8]. The collection of these data often involves postal surveys which are time-consuming and costly [9, 10]. However, recent studies, including a meta-analysis, have demonstrated no major differences or limitations in using electronically collected PROMs (ePROMs) compared with more traditional paper PROMs [10,11,12,13].
Given current deficiencies in the NHS budget, and a recent government drive to a paperless NHS, ePROMs offer a functional solution for the routine use of PROMs and incorporation into Digital Health Records [14, 15].
The aim of this feasibility study was to pilot the use of a digital platform to facilitate the routine collection of pre- and post-operative ePROMs in participants undergoing elective laparoscopic cholecystectomy to calculate a change in HRQoL. Secondary objectives were to validate the use of existing PROs for our population by appraising their methodological quality.
Methods
PRO selection
Several PROs relevant to patients undergoing laparoscopic cholecystectomy have previously been identified and appraised [16,17,18]. However, due to the commercial nature of many trademarked PROs and the cost of licensing, we were limited in the selection available for this study. We took guidance from existing literature on the reporting of HRQoL studies and selected both disease-specific and generic PROs and planned for PRO validation within the study [19, 20].
Due to the inclusion of two PROs, and to prevent question fatigue, the 6-item Otago gallstones condition-specific questionnaire (CSQ) [21] was identified as the disease-specific PRO, and the RAND 36-Item health survey - version 1.0 (SF36) [22], as the generic PRO. Each of the items in both the CSQ and SF36 has one of 5 potential answers based on a Likert scale. Each answer has an allocated score: for CSQ 0 = best health, and 100 = worst health [21], and for SF36 0 = worst health, and 100 = best health [22]. Further domain scores can also be calculated for each PRO by combining and averaging specific items.
Variables
Participant and hospital variables collected included details on age, sex, Charlson comorbidity index [23], body mass index (BMI), and total length of hospital stay. Data were also collected on 30-day outcomes, specifically post-operative complications, re-attendances and readmission to hospital, a return to theatre, and mortality.
For comparison with our local population, data were also collected on all other patients undergoing laparoscopic cholecystectomy in the study time who opted not to participate in our PROMs study.
Outcomes
The main outcome measures included generic and disease-specific quality of life. PROs were offered to participants preoperatively, and post-operatively at 30 days, 3 months, and 6 months. Domain scores were calculated for each completed PRO, with post-operative results compared directly with baseline (preoperative) results.
The validity of PROs as ePROs was assessed by appraising their methodological quality within the study [16, 24]. Response rates, reliability, internal consistency, floor and ceiling effects, and responsiveness were all assessed within the study sample. Cross-correlation between CSQ and SF36 was also performed.
Statistical analysis
Data were analysed using GraphPad Prism® version 8.3.0. (GraphPad Software LLC, San Diego, CA, USA). Nonparametric tests were used to compare differences between groups as results were not normally distributed. Fisher’s exact or Chi-square tests were used for categorical variables and the Mann–Whitney U test for continuous variables. Spearman’s r was taken as a measure of correlation and reliability between scales, and Cronbach’s alpha for internal consistency assessment. Differences were considered significant at p < 0.05.
Ethics and reporting
Study participants were recruited to participate in this study on referral to hospital for management of symptomatic gallstones. On receiving study information either postally or in person, each participant indicated their consent to participate digitally. The study was performed as part of a larger PhD project sponsored by Nottingham University Hospitals NHS Trust, through a collaboration with EIDO Healthcare Limited and the Royal College of Surgeons of England. The study proposal was appraised by the confidentiality advisory group: 16/CAG/0045, with public and patient involvement, and research ethics committee approval: 16/SW/0088. It was registered with ClinicalTrials.gov: NCT02810860. The study was conducted and reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines for observational studies [25].
Results
Participant recruitment and study demographics
A total of 255 eligible patients who were invited to participate in the study and use our digital interface completed the preoperative PROMs surveys between 6 September 2016 and 16 April 2018. Only 200 of these went on to undergo a laparoscopic cholecystectomy. These participants completed their preoperative survey a median of 86.5 days (IQR: 51–141.3) before laparoscopic cholecystectomy. The remaining participants were managed non-operatively because of patient choice, incorrect initial diagnosis, or being medically unfit for surgery. Only a small proportion of the 200 participants who completed their preoperative PROMs questionnaire completed the post-operative surveys at 30 days, 3 months, and 6 months (30.5, 27.0, 19.0%, respectively) (Fig. 1).
None of the 255 participants who completed the preoperative survey asked to withdraw from the study. Most participants, who opted not to complete a post-operative survey, maintained contact with the research team via a support email and described the complexity of accessing our ePROMs interface as the main reason for non-participation.
Participant demographics are described in Table 1. Participants in the study group were statistically younger than our local population. There were no other significant differences between the two groups.
30-Day outcomes
Thirty-day outcomes were collected for all participants (Table 2). Most attendances for complications [61.5% (16 of 26) 30-day complications] were managed in an ambulatory setting and did not require overnight admission. Admission for treatment other than surgery or radiological drainage included intravenous hydration or medications (antibiotics, anti-emetics, analgesia), wound management, or a period of observation.
Patient-reported outcome measures
Quality-of-life domain scores were calculated from item scores for both CSQ (Table 3) and SF36 (Table 4). Compared with baseline preoperative values, statistically significant improvements in quality of life were seen in most domains in both CSQ and SF36.
The change in HRQoL before and after surgery is mapped in Fig. 2. These radar plots demonstrate that all participants experienced an improvement in HRQoL, in all domains, between preoperative and post-operative time points as demonstrated by no crossover between blue (solid) and green (dashed) plot lines.
Completion rates
The digital nature of our ePROMs platform meant that surveys were either completed in their entirety or not all and there were no missing items within individual returned surveys.
Reliability
Test–retest correlations were performed between post-operative groups using Spearman’s coefficient of correlation on PROMs taken at 30 days and 3 months, and at 3 and 6 months post-operatively, and showed moderate-to-strong reliability (Table 5).
Internal consistency
Homogeneity within PROs or internal consistency was good or excellent (Table 6).
Floor and ceiling effects
To measure the ability of the PROs to identify population variation, floor and ceiling effects were assessed in both HRQoL instruments. Whilst floor and ceiling effects were seen in both CSQ and SF36 domain measurements, there were few instances exceeding > 15% (Table 7).
Responsiveness and effect sizes
The ability to capture statistically significant changes in HRQoL over time was assessed by calculating effect sizes (Table 8). Effect sizes were classified as < 0.3, 0.5, or > 0.8, as small, medium, or large. Scores for CSQ were converted to a positive scale much like SF36 for the purposes of analysis.
Cross-correlation between CSQ and SF36
Assessment of cross-correlation between the disease-specific and generic PROs showed moderate-to-strong reliability across the domains of ‘Physical Functioning’, ‘Systemic Functioning’, ‘Emotional Functioning’, ‘Social Functioning’, and ‘Overall health’ (Supplementary Figs. 1–5 and Supplementary Table 1).
Discussion
This novel feasibility study demonstrates the potential benefits of using a digital platform to collect real-time quality-of-life information utilising ePROMs. Despite initial problems in implementing our digital platform, and difficulties with user access, the process was deemed acceptable by most participants. However, these difficulties did lead to a significant attrition rate in the completion of post-operative questionnaires and resultant reporting bias.
Participant recruitment
Participants described the remote process of initial contact and invitation to the study positively and were pleased to have a study helpline for the digital platform. Whilst a significant number of participants subsequently dropped out of the study, discussion with those who contacted the helpline described this as mainly due to difficulties with Website onboarding due to the multiple security measures which were requested by our local information governance team (email, secure password, two type-sensitive memorable words). Of the 80 participants who contacted the user helpline, 64 described difficulties with either site registration or log-in (requiring support on between 1 and 6 occasions).
Participants subsequently recruited and retained in the study were found to be younger and have fewer comorbidities than those from a similar cohort that underwent laparoscopic cholecystectomy during the same time frame but out-with the study. This generational difference in our study group was in keeping with other studies utilising ePROMs [13].
All other study demographics demonstrated no significant differences between the two groups, indicating that the study group was representative of our local population.
Disease-specific PROMs
Statistically significant improvements in disease-specific HRQoL were seen across all survey items, and all quality-of-life domains following laparoscopic cholecystectomy when compared with baseline preoperative values. Marked changes with over twofold improvements in quality-of-life scores were evident at 30 days post-operatively in all domains.
Continued improvement was seen in emotional functioning and social functioning at 6 months post-operatively, whereas physical functioning, systemic functioning, and overall health either peaked or plateaued at 3 months post-operatively. Post-operative HRQoL did not fall to preoperative levels in any of the domains demonstrating that the continued benefit laparoscopic cholecystectomy had on participants.
Generic PROMs
Post-operative improvements were seen in all domains for generic HRQoL measures after laparoscopic cholecystectomy. This finding was in keeping with other non-UK-based studies on HRQoL [26, 27]. Continued significant improvement in quality of life was seen at 30 days, 3 months, and 6 months for fatigue, social functioning, and pain, whereas significant sustained improvement was seen at 3 and 6 months for physical functioning, and role limitations due to physical health. Emotional well-being and role functioning due to emotional problems showed peak improvement at 3 months post-operatively, whereas general health measurement peaked at 30 days. As demonstrated with disease-specific PROMs and other historic studies, this confirms the continued benefit laparoscopic cholecystectomy had on participants [26, 27].
Validation assessments
Despite a poor overall return rate of surveys post-operatively, all returned surveys were 100% complete. This all or nothing process is more frequent with digital surveys where the ability to skip questions can be removed, or users can be prevented from progressing. Other than high response rates, the additional benefits are that missing or spurious data points do not need to be accounted for, and there are lower chances of imputation error [8]. This level of acceptability and ease of ePROM dissemination are in keeping with existing evidence which has shown similar benefits when compared to more traditional PROM dissemination methods [10, 28, 29].
Assessments to validate CSQ and SF36 for use as ePROs in our study population demonstrated strong reliability scores for CSQ across all domains, with good internal consistency scores. Comparatively, SF36 demonstrated strong reliability scores with good-to-excellent measures of internal consistency. These results were similar to those described by the original CSQ research group [21], and the Oxford PROM Group review [17], suggesting that the PROs were accurate in measuring what they set out to and that they were stable in their measurements across the different time points.
The low percentages for floor and ceiling effects in both CSQ and SF36 surveys, and the medium-to-large effect sizes, also confirmed sufficient scale variance and responsiveness suggesting suitability to continue using them as ePROs and measure the wide variability of change in HRQoL.
The moderate-to-strong cross-correlation between surveys in this study demonstrates that future platforms may be justified in offering patients a single HRQoL survey to reduce survey fatigue.
Strengths and weaknesses
The benefit of using a digital platform meant that the PROs were easily disseminated to participants for completion at the correct time points, with an easy system in place to facilitate reminders for non-responders. The digital modality also meant that participants had access to complete HRQoL surveys in real time at any time and place, and from any digital medium which allowed Internet access. This proved especially convenient for participants who had already returned to work.
Results from completed ePROMs were also immediately available with no concerns about managing missing results, imputation errors, or researcher bias which may occur when collecting PROMs by more traditional means [8].
Although the results from this study suggest significant improvements in quality of life following laparoscopic cholecystectomy in our patient population, the study is significantly underpowered to be able to draw any reliable conclusions given the limited number of patients who completed both pre- and post-operative questionnaires. It is not possible to assume that all participants who suffered complications completed post-operative surveys and, therefore, this reporting bias could have impacted the overall results.
Despite initial problems with study set-up and participant recruitment, a large proportion of participants still engaged actively with the study process with positive interactions and acceptability demonstrating a place for digital mediums in delivering ePROMs. Although the use of ePROMs removes many of the logistic and cost issues associated with more traditional paper or telephone versions, access to ePROMs systems needs to be more inclusive and usable for both patients and clinicians [5, 30, 31]. Whilst further developmental costs may be necessary, this is fundamental to improve survey response rates and reduce recruitment bias.
Future work and recommendations
This feasibility study has helped to provide the groundwork and real-world data necessary to plan for a powered randomised controlled trial across more than one region. Further studies should also aim to actively involve patient advisory groups in study planning to develop study protocols and help test any digital medium prior to study commencement. The inclusion of focus groups within new studies would also be beneficial in recording patient attitudes, in particular reasons for non-participation.
References
Association of Upper Gastrointestinal Surgeons of Great Britain and Ireland/Royal College of Surgeons of England (2016) Commissioning guide: gallstone disease. Available at https://www.rcseng.ac.uk/-/media/files/rcs/standards-and-research/commissioning/gallstone-disease-commissioning-guide-for-republication.pdf. Accessed 6 Nov 2021
Aerts R, Penninckx F (2003) The burden of gallstone disease in Europe. Aliment Pharmacol Ther 18(Suppl 3):49–53
Barbier L, Souche R, Slim K et al (2014) Long-term consequences of bile duct injury after cholecystectomy. J Visc Surg 151:269–279
Hariharan D, Psaltis E, Scholefield JH et al (2017) Quality of life and medico-legal implications following iatrogenic bile duct injuries. World J Surg 41:90–99
Dawson J, Doll H, Fitzpatrick R, Jenkinson C, Carr AJ (2010) The routine use of patient reported outcome measures in healthcare settings. BMJ 340:c186
Department of Health (2008) Guidance on the routine collection of patient reported outcome measures (PROMs). Available at https://www.mcgill.ca/can-pro-network/files/can-pro-network/nhs_guidance_on_routine_collection_of_proms.pdf. Accessed 6 Nov 2021
Hutchings A, Neuburger J, Gross Frie K, Black N, van der Meulen J (2012) Factors associated with non-response in routine use of patient reported outcome measures after elective surgery in England. Health Qual Life Outcomes 10:34
Black N (2013) Patient reported outcome measures could help transform healthcare. BMJ 346:f167
Browne J, Jamieson L, Lewsey J et al (2007) Patient reported outcome measures (PROMs) in elective surgery. Report to the Department of Health. Health Services Research Unit, London School of Hygiene & Tropical Medicine: London
Roberts N, Bradley B, Williams D (2014) Use of SMS and tablet computer improves the electronic collection of elective orthopaedic patient reported outcome measures. Ann R Coll Surg Engl 96:348–351
Gwaltney CJ, Shields AL, Shiffman S (2008) Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health 11:322–333
Muehlhausen W, Doll H, Quadri N et al (2015) Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013. Health Qual Life Outcomes 13:167
Nguyen J, Marx R, Hidaka C, Wilson S, Lyman S (2017) Validation of electronic administration of knee surveys among ACL-injured patients. Knee Surg Sports Traumatol Arthrosc 25:3116–3122
The NHS Long Term Plan (2019) Available at https://www.longtermplan.nhs.uk/wp-content/uploads/2019/08/nhs-long-term-plan-version-1.2.pdf. Accessed 6 Nov 2021
Fullfact (2019) Spending on the NHS in England. Available at https://fullfact.org/health/spending-english-nhs/. Accessed 6 Nov 2021
Daliya P, Gemmill EH, Lobo DN et al (2019) A systematic review of patient reported outcome measures (PROMs) and quality of life reporting in patients undergoing laparoscopic cholecystectomy. Hepatobiliary Surg Nutr 8:228–245
Casañas i Comabella C, Gibbons E, Fitzpatrick R (2011) A structured review of patient-reported outcome measures for patients undergoing cholecystectomy. Report to the Department of Health, 2011. Department of Public Health, University of Oxford, Patient-Reported Outcome Measurement Group: Oxford. Available at https://www.yumpu.com/en/document/view/33554463/2011-review-patient-reported-outcomes-measurement-university. Accessed 6 Nov 2021
Mueck KM, Cherla DV, Taylor A et al (2018) Randomized controlled trials evaluating patient-reported outcomes of studies after cholecystectomy: a systematic review. J Am Coll Surg 226(183–193):e185
Calvert M, Blazeby J, Altman DG, Revicki DA, Moher D, Brundage MD, CONSORT PRO Group (2013) Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. JAMA 309:814–822
Calvert M, Kyte D, Mercieca-Bebber R, Slade A, Chan A-W, King MT, the SPIRIT-PRO Group (2018) Guidelines for inclusion of patient-reported outcomes in clinical trial protocols: the SPIRIT-PRO extension. JAMA 319:483–494
Chen TY, Landmann MG, Potter JC et al (2006) Questionnaire to aid priority and outcomes assessment in gallstone disease. ANZ J Surg 76:569–574
Hays RD, Sherbourne CD, Mazel R (1995) User’s manual for the medical outcomes study (MOS) core measures of health-related quality of life. Rand Corporation, Santa Monica, CA. Available at https://www.rand.org/pubs/monograph_reports/MR162.html. Accessed 6 Nov 2021
Charlson ME, Pompei P, Ales KL et al (1987) A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 40:373–383
Mokkink LB, Terwee CB, Patrick DL et al (2010) The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 19:539–549
von Elm E, Altman DG, Egger M et al (2007) The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 370:1453–1457
Palsson SH, Rasmussen I, Lundstrom P et al (2011) Registration of health-related quality of life in a cohort of patients undergoing cholecystectomy. ISRN Gastroenterol 2011:507389
Shi HY, Lee HH, Tsai MH et al (2011) Long-term outcomes of laparoscopic cholecystectomy: a prospective piecewise linear regression analysis. Surg Endosc 25:2132–2140
Deshpande PR, Rajan S, Sudeepthi BL et al (2011) Patient-reported outcomes: a new era in clinical research. Perspect Clin Res 2:137–144
Rydbeck D, Anesten B, Barje T et al (2015) Health-related quality-of-life in a cohort undergoing cholecystectomy. Ann Med Surg 4:22–25
Chang CH (2007) Patient-reported outcomes measurement and management with innovative methodologies and technologies. Qual Life Res 16(Suppl 1):157–166
Raine R, Fitzpatrick R, Barratt H et al (2016) Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. NIHR Journals Library: Southampton (UK)
Funding
This work was supported by the Medical Research Council [Grant Number MR/K00414X/1]; and Arthritis Research UK [Grant Number 19891]. Prita Daliya was a recipient of a Research Fellowship funded by the Royal College of Surgeons of England and EIDO Healthcare Limited.
Author information
Authors and Affiliations
Contributions
PD, DNL, and SLP took part in study design. PD took part in data collection and was involved in data analysis. PD, DNL, and SLP participated in data interpretation. PD, DNL, and SLP took part in writing of manuscript, critical review of manuscript, and final approval.
Corresponding author
Ethics declarations
Conflict of interest
SLP is a company director for EIDO Healthcare Limited. None of the other authors has a conflict of interest to declare.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Daliya, P., Lobo, D.N. & Parsons, S.L. Utilising Electronic PROMs to Measure a Change in Health Following Elective Laparoscopic Cholecystectomy: A Feasibility Study. World J Surg 46, 2155–2165 (2022). https://doi.org/10.1007/s00268-022-06588-9
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00268-022-06588-9