Abstract
Objective
Since the implementation of value-based healthcare, there has been a growing emphasis on utilizing patient-reported experience measures (PREMs) to enhance the quality of care. However, the current PREMs are primarily generic and static, whereas healthcare is constantly evolving and encompasses a wide variety of aspects that impact care quality. To continuously improve care requires a dynamic PREM. The aim of this study was to propose an item bank for the establishment of a dynamic and care-specific patient-reported evaluation.
Methods
In co-creation with patients, a mixed methods study was conducted involving: (1) an explorative review of the literature, (2) a focus group analysis with (ex-)patients, (3) qualitative analyses to formulate themes, and (4) a quantitative selection of items by patients and experts through prioritization.
Results
Eight existing PREMs were evaluated. After removing duplicates, 141 items were identified. Through qualitative analyses of the focus group in which the patient journey was discussed, eight themes were formulated: “Organization of healthcare,” “Competence of healthcare professionals,” “Communication,” “Information & services,” “Patient empowerment,” “Continuity & informal care,” “Environment,” and “Technology.” Seven patients and eleven professionals were asked to prioritize what they considered the most important items. From this, an item bank with 76 items was proposed.
Conclusion
In collaboration with patients and healthcare professionals, we have proposed a PREM-item bank to evaluate the experiences of patients’ receiving cancer care in an outpatient clinic. This item bank is the first step to dynamically assess the quality of cancer care provided in an outpatient setting.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
In the Netherlands, approximately 120,000 patients are diagnosed with cancer every year [1]. These patients often undergo a long and intensive healthcare trajectory from the moment of diagnosis through to long-term follow-up. Patients’ opinions about the care provided are an important quality measure as they can offer different perspectives and provide insights into aspects of healthcare that professionals are unaware of. Consequently, there is increasing attention given to the use of patient-reported experience measures (PREMs) in seeking to improve quality as part of value-based healthcare (VBHC) [2,3,4]. In the Netherlands, most tertiary hospitals measure patients’ experiences with the Picker Institute’s patient experience questionnaire [5]. This experience measure is based on “Picker’s principles of patient-centered care” and determines generic experiences of the care process using topics such as experience with “waiting time” and “friendliness of personnel” [5]. These generic PREMs are convenient for general quality improvements at the hospital level or for integrated care. Since the delivered care and logistic processes will differ between departments within a hospital (for example in “the use of volunteers on the diagnostic day” or “the use of prediction models to inform patients in decision making”), patients’ experiences with such care-specific processes are useful for improving quality at the department level [3].
Although there are potential advantages, the effectiveness of routine PREM assessments in clinical practice in enhancing healthcare services is uncertain as the evidence is inconclusive [6,7,8,9]. These inconsistent findings may be due to heterogeneity in the PREMs used. Healthcare is dynamic and contains a wide range of aspects that contribute to quality of care. To further enhance the healthcare process within a department, it is important to respond to opportunities for improvement identified through the PREM data. At a certain moment, when the items in the PREM are fully satisfied, it is no longer possible to further improve on these items and it is no longer useful to assess them. A more dynamic approach would offer the opportunity to switch to including other items covering another or new aspects of the healthcare process in seeking continuous improvement.
Based on this idea, we have developed a PREM-item bank for improving cancer-specific VBHC. This PREM-item bank complements the generic PREM (Picker) by providing opportunities to add specific or detailed questions about care experiences. This PREM item bank can be dynamically used to measure patients’ experiences with their care at the outpatient clinic.
Methods
In a co-creation process with patients, we took the following steps to create the PREM-item bank: (1) a literature search to identify existing PREMs, (2) the identification of themes through focus group discussions in an approach based on patient journeys, (3) combining themes from the literature and themes from the qualitative analysis, (4) mapping by healthcare professionals of items within themes, (5) prioritizing the items seen as most important by patients and by healthcare professionals, and (6) rewriting items in a language understandable for most people (level B1) (Fig. 1).
1. Explorative literature search
An explorative literature search was performed using the PubMed, Embase, and Medline databases in August 2019 to obtain details of available generic and cancer-specific PREMs. For the search strategy, the words “patient reported experience measure” OR “experience measure” AND “cancer” were used. Articles were included when they described a PREM validated in the Dutch language that contained items assessing experience with the healthcare process covering both emotional and cognitive evaluation of the delivered care. We included generic and cancer-specific PREMs. Exclusion criteria were studies describing patient-reported outcomes measures (PROMs) and other questionnaires that did not include any patient-reported experience measures.
2a. Patient journey: focus group
Together with patients and healthcare professionals, we gathered relevant themes for the PREM-item bank. We asked (ex-) patients (n = 8) and healthcare professionals (n = 3) to participate in a focus group meeting during which a “patient journey of outpatient clinic experiences” was evaluated (Figs. 2 and 3). From our previous experience with focus groups and qualitative research, we know that a group size of 6–10 participants is ideal for ensuring a trusted environment and that everyone has a chance to speak, while also obtaining diverse input from different individuals. Creating a patient journey is a method to obtain the patient’s perspective on all (possible) activities and contact moments between patients and healthcare professionals. We focused on two moments in the outpatient clinic in this full patient journey: the diagnostic phase and the follow-up phase, as these specific contact periods contribute to how the total provided cancer care is experienced. The diagnostic phase was defined as the time between the moment of referral through to the consultation in which treatment options are discussed. Contact moments were grouped within four phases: referral, registration procedure, contact with healthcare professionals on intake to the hospital, and consultation about treatment options. The follow-up phase started with the first regular follow-up visit after treatment and ended at the last follow-up visit, five years after treatment. Contact moments were arranged in three phases: before the follow-up visit, during the registration procedure, and later contact with several healthcare professionals. Each possible contact moments was visualized and discussed separately. Patients were asked if they thought there were missing aspects in the proposed patient journey. Following this, all the steps were evaluated using the following questions: “What were positive experiences with the provided care?”, “What were negative experiences with the provided care?”, “What did you miss in the provided care?”, “Do you have suggestions to improve our care?”, and “Who or what was indispensable in the provided care?”. Data saturation was reached when no new themes were opted.
2b. Qualitative data analysis
An inductive approach for coding the data was followed by three researchers (MO, KD, and HW) who independently performed thematic analysis on the data. Consensus on the themes present was achieved through discussion, with verification by a fourth researcher (KH).
3. Combining themes from the literature and from the qualitative analysis
Three researchers (MO, KD, and HW) compared the themes identified from the literature with the themes from the focus group and formed eight new themes that covered different aspects of the intake or diagnostic phase and of the follow-up phase.
4. Mapping items within themes
After identifying suitable PREMs in step 1, all the individual items included in the PREMs were listed. Duplicates were removed. Items provided by the focus group that were not covered by the list of existing PREM questions were added. Following this, all the items were independently mapped on to the formulated themes by two senior researchers (MO and KD) and one junior researcher (HW). Consensus was achieved through later discussion.
5. Prioritizing
In order to select the most important items for each theme, a prioritization exercise was completed by both patients (n = 7) and 11 healthcare professionals (e.g., head and neck surgeons, psychologists, and oncology nurses). For themes with a fairly limited number of items (1, 2, 5, 7, and 8), a top-five list was compiled, while for more extensive themes (3, 4, and 6), a top-ten was generated. Items were weighted based on their ordering (i.e., first place = 5 and second place = 4). Final prioritization was based on the square root of the frequency of occurrence of an item multiplied by the weight of that item: \(\sqrt{\left(\textrm{frequency}\ \textrm{of}\ \textrm{appearance}\ \textrm{x}\ \textrm{weight}\right)}\). The outcome was adjusted for group size such that both groups (patients and experts) had an equal influence on the outcome. Both healthcare professionals and patients were asked whether items were relevant, understandable, and whether important items were missing.
6. Qualitative analysis open comments
After the prioritization, three researchers (MO, KD, and HW) reviewed the open comments in which participants could formulate missing items. After discussion, the researchers reached a consensus on the items to be added when they had been mentioned multiple times and fit within the themes formulated from the previously conducted focus groups.
7. Rewriting items in Dutch language level B1 (intermediate level)
Next, all the items were rewritten using B1 language, i.e., put in a form that patients with limited health literacy could understand. Following the method applied with the Picker questionnaire, questions were partly repeated in the answer options to make them easier to understand [5]. The reformulated items were again sent to the participating patients to check for understandability.
Results
1. Explorative literature search
Two previously validated generic PREMs (Consumer Quality Index (CQI) [10], Picker questionnaire [11]) plus five cancer-specific PREMs (FACE-Q head and neck [12], PREM VBHC head and neck [13], EORTC-INFO25 [14], and EORTC-COMU26 [15]) were included in the review. In addition, a PREM concerning the diagnostic day at the outpatient clinic constructed by the Department of Head and Neck Oncology of Erasmus Medical Center was included [16]. After removing duplications, 141 items were identified from the above instruments (Table 1). Additionally, six themes were derived from the included PREMs: (1) organization, (2) treatment by healthcare providers, (3) expertise of healthcare providers, (4) patient empowerment, (5) end of treatment process, and (6) outpatient clinic.
2. Qualitative analyses of patient journey: focus group
In the focus group, the patient journey was discussed with head and neck cancer patients and healthcare professionals. Two key points in the full patient journey were visualized (the diagnostic stage and the follow-up phase), and additional contact moments were added. Six themes were derived from this qualitative analysis: (1) “Communication,” (2) “Information,” (3) “Shared decision-making,” (4) “Provided care,” (5) “Environment,” and (6) “Technology.” The topics considered important by the focus-group participants in the first phase are included in Fig. 2 and in the second phase in Fig. 3.
3. Combining themes from literature and themes from qualitative analysis
After comparing our themes resulting from the focus group with those derived from the literature, we proposed eight final themes: (1) organization, (2) competence of healthcare professionals, (3) communication, (4) information and services, (5) patient empowerment, (6) continuity of provided care, (7) environment, and (8) technology (Table 2). Further, during the qualitative analysis, 19 additional items were formulated that were not covered in the 141 items that were identified in the existing PREMs.
4 & 5. Mapping and prioritizing
Subsequently, 160 items were mapped within the eight themes. Following this, 7 (ex-)patients and 11 healthcare professionals participated in prioritizing the items within each theme. The top-five items selected by the two groups, from each of the smaller themes (1, 2, 5, 7, and 8), and the top-ten from the larger themes (3, 4, and 6) were included in the item-bank (n = 57). In most cases, the prioritizations of the (ex-)patients and healthcare professionals were similar: 72% of the selected items were in the top five/ten of both groups, and 25% of the selected items were in the top five/ten of one group and in the top 11/16 respectively of the other group (Δ < 6 places). Only two items were selected in which the prioritization of (ex-)patients and healthcare professionals differed widely: “Were you satisfied with the communication between you and the specialist nurse?” (theme 3) was put in fifth place by the (ex-)patients but only 24th by the healthcare professionals. Further, “Did you receive information about the effects of smoking and alcohol on your illness?” (theme 4) was placed 11th and 22nd, respectively.
6. Qualitative analysis open comments
Eventually, the open comments were evaluated and discussed. An additional 19 items were added to the PREM-item bank after consensus of all three researchers. In total, 76 items were included in the final PREM-item bank (Table 2).
Discussion
To the best of our knowledge, there is currently no patient-reported experience measure (PREM) item bank applicable to cancer care at an outpatient clinic. In this mixed-method study, we proposed a PREM-item bank that can be used to dynamically evaluate the provided care at a cancer outpatient clinic. Through the discussion of a patient journey with patients and professionals, which assessed the key contact moments of care during the intake and diagnostic phase and the follow-up phase, eight themes were derived. Together with patients and healthcare professionals, the most important items from existing PREMs were chosen and missing relevant items were added. Eventually, an item bank with 76 items was formed (Table 2).
Since the introduction of value-based healthcare (VBHC) in 2006, there has been increasing interest in the patient’s perspective on healthcare [17, 18]. Both the perspective of patients on their own functioning (PROMs) and their experiences with the provided care (PREMs) are seen as important instruments for quality improvement [3]. As the World Health Organization (WHO) notes, person-centered care is a key component of providing high-quality healthcare [19]. Furthermore, patient-reported experiences are associated with higher patient safety and clinical effectiveness [7, 20].
To date, most PREMs that are used are generic so that results can be compared between departments and hospitals (e.g., the patient experience monitor (PEM) [5]). Although the PEM is used in all academic university hospitals in the Netherlands, it can be difficult to pinpoint concrete actions to improve the care quality in specific departments based on the results of this generic PREM. Some of the evaluated items are outside the control of a department, such as how parking is experienced. The review by Gleeson et al. similarly noted the problem of a lack of specificity in national surveys, and that staff often thought that such data were not applicable or relevant to their daily work [8].
Furthermore, there are problems with the PEM in that there appears to be a ceiling effect: the overall experience is often rated as good (score of 8 out of 10) in all participating university hospitals [21]. This makes it difficult to further improve healthcare based on these already good scores. While we acknowledge the importance of this generic measurement for benchmarking experiences with the provided care, our PREM-item bank can complement this generic measurement with a department-specific measurement, in which the care delivered by a specific department can be evaluated. Each department has its own care pathways and may provide care in ways that are not seen in other departments (e.g., the use of volunteers on the first day).
In particular, the use of VBHC during consultations and remote care is relatively a new aspect of healthcare, and the ways these are provided often differ between departments. For patients to actively take part in VBHC, they have to contribute by completing questionnaires or using a web application. Since we can only request this of patients, it is particularly important that we continuously assess their experiences. On one hand, we want to identify areas where we can further improve the care we provide while, on the other hand, we want to know what patients already value. From an earlier study in which we used a department-specific PREM, we learned that patients did not see completing these PROMs as a burden, and, further, that filling in and discussing PROMs during consultations enhanced patient empowerment [13]. These appreciated aspects can be used to convince other stakeholders that VBHC leads to quality improvements from the patient’s perspective [22]. Although quality improvements based on PREM data have often been described, only a few studies report a quantitative impact as a result of using PREM-data [3]. It is our hypothesis that with the introduction of this dynamic manner of collecting department-specific PREM-data, identifying potential quality improvements will be more straightforward and applicable for a specific care pathway. When items are given the highest possible rating or it is not possible to further improve an aspect, PREM items could be changed in order to focus on another aspect of care. In this way, patients’ experiences will be able to optimally contribute to improving the quality of various aspects of healthcare. In the future, we intend to evaluate the impact of using aggregated PREM-data on patient-reported outcomes and experiences.
Strengths and limitations
A strength of this study is its mixed-method design. Furthermore, inputs from both patients and healthcare professionals were taken into account when forming the PREM-item bank. Both groups prioritized similar items that they considered important to include in the PREM-item bank. This item bank could be used at any oncology department worldwide, and additional items can be added to enable the constantly evolving healthcare to be structurally evaluated. The questions were phrased in an easily understandable way so that patients with low health literacy could participate. A limitation is that our study only included head and neck cancer patients and healthcare professionals who were working in this area. Nevertheless, to make this item bank applicable to all cancer departments, we included both generic and cancer-specific PREMs based on our explorative literature search. Another limitation is that the literature search was explorative, limited to known PREMs, meaning that some relevant PREMs might have been overlooked. Furthermore, only PREMs validated in the Dutch language were included. In this phase, the item bank is not psychometrically validated, and we added non-validated items from our focus group analysis. However, this is the first step and validation will be performed in the upcoming paper.
Future perspectives
In order to evaluate the VBHC offered in various oncology departments, we intend, in co-creation with other departments and patients, to develop a PREM-VBHC that will be structurally embedded in the provided care. We aim for the associated questionnaire to be limited to approximately 15 fixed items and 5 dynamic items. In this way, oncology departments can be compared, while department-specific care can be dynamically evaluated. The PREM-VBHC will be part of our electronic patient-reported outcome system so it will not increase the workload of healthcare professionals during their daily practice, which has been reported as a barrier to the use of PREMs [22]. The PREM-VBHC results will not be used during consultations as this could affect patients’ honesty [22]. Rather, the aggregated data will be discussed during regular meetings with all involved healthcare professionals following a plan-do-study-act cycle. Ideally, the aggregated PREM data will be complemented with aggregated PROM results. Although the use of aggregated PROMs is still in its infancy, their combination with PREMs for quality improvement seems promising [23]. The PREM-item bank can additionally be used as a basis when a specific topic needs to be addressed (e.g., acute care).
Conclusions
In collaboration with patients and healthcare professionals, we have developed a bank of patient-reported experience measure (PREM) items to evaluate patients’ experiences while receiving cancer care in an outpatient clinic. This PREM-item bank is the first step to dynamically assess the quality of cancer care provided in an outpatient setting. The questions were formulated in straightforward language so that patients with limited health literacy could understand them. We will use the PREM-item bank to develop a PREM that is purposefully designed to evaluate the value-based healthcare that is offered in various oncology departments in our institute and could be used elsewhere.
Data availability
Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.
Abbreviations
- PREM:
-
Patient-reported experience measure
- PROM:
-
Patient-reported outcome measure
- VBHC:
-
Value-based healthcare
- CQI:
-
Consumer Quality Index
- PEM:
-
Picker experience measure
- EORTC-INFO25:
-
EORTC information module
- EORTC-COMU26:
-
EORTC communication module
- WHO:
-
World Health Organization
References
Dutch Cancer Registration (2021) Integraal kankercentrum Nederland (IKNL). https://iknl.nl/nkr-cijfers
de Boer D et al (2022) Assessing the outcomes and experiences of care from the perspective of people living with chronic conditions, to support countries in developing people-centred policies and practices: study protocol of the International Survey of People Living with Chronic Conditions (PaRIS survey). BMJ Open 12(9):e061424
Jamieson Gilmore K et al (2023) The uses of patient reported experience measures in health systems: a systematic narrative review. Health Policy 128:1–10
Austin E et al (2020) Capturing the patient voice: implementing patient-reported outcomes across the health system. Qual Life Res 29(2):347–355
Bastemeijer CM et al (2020) Patient experience monitor (PEM): the development of new short-form picker experience questionnaires for hospital patients with a wide range of literacy levels. Patient Relat Outcome Meas 11:221–230
Davies E, Cleary PD (2005) Hearing the patient's voice? Factors affecting the use of patient survey data in quality improvement. Qual Saf Health Care 14(6):428–432
Doyle C, Lennox L, Bell D (2013) A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open 3(1):e001570
Gleeson H et al (2016) Systematic review of approaches to using patient experience data for quality improvement in healthcare settings. BMJ Open 6(8):e011907
Groene O (2011) Patient centredness and quality improvement efforts in hospitals: rationale, measurement, implementation. Int J Qual Health Care 23(5):531–537
Smirnova A et al (2017) Closing the patient experience chasm: a two-level validation of the Consumer Quality Index Inpatient Hospital Care. Health Expect 20(5):1041–1048
Jenkinson C, Coulter A, Bruster S (2002) The Picker Patient Experience Questionnaire: development and validation using data from in-patient surveys in five countries. Int J Qual Health Care 14(5):353–358
Cracchiolo JR et al (2019) Leveraging patient-reported outcomes data to inform oncology clinical decision making: introducing the FACE-Q Head and Neck Cancer Module. Cancer 125(6):863–872
Dronkers EAC et al (2020) Keys to successful implementation of routine symptom monitoring in head and neck oncology with “Healthcare Monitor” and patients’ perspectives of quality of care. Head Neck 42(12):3590–3600
Arraras JI et al (2010) An international validation study of the EORTC QLQ-INFO25 questionnaire: an instrument to assess the information given to cancer patients. Eur J Cancer 46(15):2726–2738
Arraras JI et al (2017) EORTC QLQ-COMU26: a questionnaire for the assessment of communication between patients and professionals. Phase III of the module development in ten countries. Support Care Cancer 25(5):1485–1494
Kerrebijn JD Internal evaluation outpatient clinic head and neck oncology. Erasmus MC Rotterdam, The Netherlands
Porter ME (2010) What is value in health care? N Engl J Med 363(26):2477–2481
Groenewoud AS, Westert G, Kremer JAM (2019) Value based competition in health care’s ethical drawbacks and the need for a values-driven approach. BMC Health Serv Res 19(1):256
World Health Organization (2007) People-centred health care: a policy framework
Haugum M et al (2014) The use of data from national and other large-scale user experience surveys in local quality work: a systematic review. Int J Qual Health Care 26(6):592–605
Kwaliteitcijfers en behandelresultaten van alle ziekenhuizen (2023) https://www.ziekenhuischeck.nl/kwaliteit/patienttevredenheid/
Shunmuga Sundaram C et al (2022) Patient and healthcare provider perceptions on using patient-reported experience measures (PREMs) in routine clinical care: a systematic review of qualitative studies. J Patient Rep Outcomes 6(1):122
Dorr MC et al (2022) Quality improvements of healthcare trajectories by learning from aggregated patient-reported outcomes: a mixed-methods systematic literature review. Health Res Policy Syst 20(1):90
Acknowledgements
We would like to thank Hester van Willigen and Peter Koemans for their valuable contributions to this study.
Funding
Zorginstituut Nederland (2018017074).
Author information
Authors and Affiliations
Contributions
K.H., K.D., and M.O. contributed to the study conception and design. Material preparation, data collection, and analysis were performed by M.O. and K.D. The first draft of the manuscript was written by K.H. and K.D.; A.S., R.B., and M.O commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and informed consent
This study is in accordance with the Declaration of Helsinki. This study was approved by the Medical Ethical Committee of the Erasmus Medical Center.
Consent to participate
Written informed consent was obtained from all participants.
Competing interest
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
ESM 1
(DOCX 47 kb)
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
van Hof, K.S., Dulfer, K., Sewnaik, A. et al. The first steps in the development of a cancer-specific patient-reported experience measure item bank (PREM-item bank): towards dynamic evaluation of experiences. Support Care Cancer 32, 100 (2024). https://doi.org/10.1007/s00520-023-08266-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00520-023-08266-5