Abstract
As the pharmaceutical industry advances towards more patient-focused product development, it is well recognized that meaningful patient engagement is required for the authentic patient voice to inform research and regulatory decisions. However, for this to happen systematically and consistently across the industry, there is still a need to evaluate and communicate the value of patient engagement to all stakeholders. Evaluating engagement also informs process improvement, elevating the value further. We describe the development of a conceptual, yet practical, framework for measuring the impact of engagement to achieve this. The framework depicts how metrics can be used to capture and assess the inputs, outputs, and value of patient engagement across the medicines lifecycle. Although conceived in the context of systems and processes within one company, Novartis, the framework was co-created with patient advisors and designed to be both patient-relevant and adaptable for any pharmaceutical organization. The adoption and evolution of the framework will help to demonstrate the value—to patients, healthcare systems, and businesses—of integrating patient engagement into core activities across the medicines lifecycle. We encourage the pharmaceutical industry to apply impact measurement to build a robust evidence base, through measuring, publishing, and communicating the value of patient engagement.
Plain Language Summary
Drug companies often talk to patients about making new medicines. This is called patient engagement. It helps to ensure that new medicines are right for the patients who will take them. Patient engagement is useful, but it does not always happen. Sometimes, companies make important decisions without involving patients. All drug companies need to understand why it is important to listen and learn what patients need. We made a practical way to measure this value. Our team included people from Novartis (a drug company), and patient groups. We developed a structure called the ‘framework’. It measures:
-
How we involve patients (‘Inputs’)
-
What we learn from patients (‘Outputs’)
-
How this benefits patients, healthcare systems, and companies (‘Value’)
It can measure all steps from the start to the end of drug development. We invite other companies to use a similar approach and share what they measure. Doing this will show the value of patient engagement, to make it more standard practice.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
In this paper we describe the co-creation, by a working group of industry stakeholders and a council of patient expert advisors, of a framework to measure and demonstrate the value of integrating the experiences of patients and their representatives in all stages of pharmaceutical development from conception to market. |
This detailed framework enables tracking of patient engagement activities, insights gathered, resulting decisions and actions, and quantitative and qualitative measures of downstream value over time, in a structure designed to be adaptable to support long-term usage and contextual relevance across the industry. |
By sharing this framework, we aim to help others adopt the process of measurement to support and improve the quality and value of patient engagement, ensuring patient-relevant treatment development. By taking measurable decisions and actions informed by the authentic voice of those living with a condition, pharmaceutical research and development organizations, regulators and policy makers can ensure new treatments meet patients’ needs, to improve their quality of life and deliver the outcomes that matter to them. |
1 Background: Demonstrating the Value and Impact of the Patient Voice
A well-performing and cost-efficient healthcare system needs to accomplish multiple goals. First and foremost is to improve outcomes that matter for people living with a health condition, while delivering benefits to various stakeholders at a cost that is acceptable to society. This must be balanced with sufficiently incentivizing life science industry stakeholders to invest in research and development (R&D) of novel, effective treatments [1].
To achieve these goals, it is fundamental that patients, as the ultimate beneficiaries of healthcare, are equal partners in health research and co-managers of their care processes. Collaboration between industry and patients has benefits for all stakeholders, including enhanced clinical study recruitment and retention, more attention on reducing burden on patients and families, and a stronger focus on health equity [2,3,4,5]. Trade-offs include the resources and time required of patients to contribute, and that patient insights cannot always be acted upon due to other influencing factors [3]. Consideration and evaluation of methodologies used, and recording the context of when insights cannot be acted on are therefore important processes. Integration of the patient voice is increasingly expected by industry regulators such as the European Medicines Agency and the United States Food and Drug Administration (US FDA) to improve the quality of and trust in regulatory processes and new, approved medicines [6, 7]. In recent years, the US FDA has issued guidance for collecting and applying patient experience data [7]. Moreover, there is an ethical imperative to involve patients [8]; the World Health Organization (WHO) recently (and for the first time, unanimously) made a resolution to strengthen, systematize, and sustain social participation in health and wellbeing [9, 10].
There are many opportunities for the authentic voice of patients and caregivers to inform R&D and regulatory decisions. However, for this to happen systematically across the pharmaceutical industry, there is still a need to demonstrate the value of patient engagement for all stakeholders. Communicating the proven value will help establish patient engagement as the unequivocal backbone of the medicines lifecycle (Suppl. Fig. 1, see electronic supplementary material [ESM]).
Important groundwork toward achieving this ambition has been laid by initiatives such as the Innovative Medicines Initiative (IMI) PARADIGM and global Patient Focused Medicines Development (PFMD) projects, and the US Patient-Centered Outcomes Research Institute (PCORI). The PARADIGM consortium’s roadmap for action and toolbox, including a framework of impact measures, provides a strategic foundation and practical guidance for conducting sustainable patient engagement by R&D organizations [11, 12]. The PFMD collaboration made further important advances including developing a Metrics Selector tool to help identify metrics to monitor and evaluate patient engagement initiatives [13]. PCORI has a multi-year initiative dedicated to creation and testing of measures of patient engagement [14]. Yet there remains more to be done. Building on the existing foundation, this work was initiated to develop a pilot approach for practical implementation within 1–2 years; address known gaps, such as business and access planning, and societal-level impact reporting; and investigate the methodologies required to collect the metrics data. We also sought to add clarity on the categories of metrics given the heterogeneity that exists. Tools are needed to enable measurement of the metrics, and specific guidance is needed on how metrics should be used and which tools to apply when at different lifecycle stages. Moreover, metrics must include both quantitative (for corporate and comparative reporting) and qualitative measures that capture rich context, and measurement tools must be practically applicable and refined as needed to ensure consistency with patient needs, while also being practical within the structure of different organizations [15].
Thus, we sought to build on this existing foundation to co-create a measurement framework to demonstrate the value—to patients, healthcare systems, and businesses—of patient engagement across the medicines lifecycle, relevant across various industry functions and related systems such as clinical development, commercial, and access, as well as for stakeholders such as regulators, health technology assessment bodies, health systems, patients, and society. To ensure relevance, the framework needed to be designed to include metrics that are quantitative, as well as qualitative metrics for inclusion of humanistic stories and context. The objective was to develop a framework to measure the impact of patient engagement at all crucial phases, from early planning and needs assessments, through early development, clinical trials, and market approval, to post-approval support. This conceptual framework, although tested in the context of systems and processes within Novartis, a pharmaceutical R&D company, was co-developed with patients and designed to be adaptable for any organization, including pharmaceutical companies and their suppliers, patient organizations, and others across the health ecosystem. It was our aim to promptly develop a framework that could leverage and further inform the ongoing work of PCORI, PARADIGM and PFMD, while being fit to be tested and iterated in pilot projects in the next 2 years.
2 Patient-Engagement Reporting Environment
2.1 An Environmental Scan Using Mixed Methods and Varied Sources
Prior to framework development, a scan of the patient engagement environment (with respect to use and measurement of patient engagement insights to develop treatment and care) was performed to identify any lexicon, frameworks, tools, methods, and metrics used to measure and report patient engagement and its impact, as well as examples of evaluated impact (ESM Suppl. Fig. 2). The environmental scan was performed by Novartis with external consultants from Dot I/O Health with expertise in patient-centered data science-driven system design. Given the existence of earlier, extensive literature reviews that informed our scan [15, 16], this included an exploratory review of peer-reviewed academic literature published in the English language since those reviews (2020–2023) to identify new information. The broad search terms used identified 5140 original studies, which were manually screened down to 98 for analysis based on exclusion of ‘shared decision-making’ (i.e., patient engagement in the context of individual care decisions rather than treatment or care development) and inclusion of frameworks, tools, metrics, relevant patient engagement activities, or if patient authorship could be identified.
In addition, publicly available documents from 12 global pharmaceutical R&D companies were analyzed to identify if and how corporations reported on patient-relevant or patient engagement-related topics and metrics. The companies were selected from a global ranking list of major companies. The materials reviewed included environmental, social, and governance (ESG) reports; ESG/sustainability-related presentation decks; annual reports; and priority assessment documents. Relevant information on corporate websites was also reviewed, including sustainability and responsibility sections, ESG portals or sections, and investor-relations sections. A total of 6534 pages of publicly available documents were analyzed for the number and type of patient-related terms used; stage(s) of the medicines lifecycle where patient insights or involvement were mentioned; the type of patient engagement activities mentioned; patient-relevant priority topics; and any declared key performance indicators (KPIs) or ESG targets. Internal, proprietary information from Novartis regarding business strategy, patient engagement tools and structures, and guidance for existing standard processes was also reviewed.
Further, a specialist Insights Automation Platform, AMPLYFI, was used to apply artificial intelligence and machine learning to collect and analyze 16,000 open-source online articles, publications, and discussions related to patient engagement. These general, news-like web content and academic articles were found using criteria that included title-level mention of a patient engagement/patient co-creation related term and a healthcare/treatment related term (from a list of terms identified within the peer-reviewed literature and corporate literature scans). This analysis used AMPLYFI machine learning tools followed by human review focused on understanding the most common topics, organizations and institutions, and positive and negative sentiments shared online. The findings from the environmental scan were consolidated and categorized by Dot I/O Health consultants, shared and discussed with the internal working group and advisor council, and are summarized in brief in the following sections.
2.2 Lack of Standardized Instruments and Validated Tools to Assess Patient Engagement and its Impact Across the Medicines Lifecycle
With regard to examples of evaluated impact, the exploratory literature review highlighted the value of involving patients in designing educational tools [17], patient portals [18, 19], and frameworks for patient safety [20, 21], as well as the contribution of patient advisory councils to developing patient-powered research agendas [22, 23]. It also showed that collaboration with patients may also help to develop tools to improve mutual understanding of patient centeredness [24], and that regular feedback and open dialogue with patients will improve patient contributors' sense of empowerment and understanding of the research cycle [25, 26]. There are tools that have been developed for specific patient engagement activities, or at specific stages in the medicines lifecycle (e.g. Patient Engagement In Research Scale [PEIRS], Research Engagement Survey Tool [REST]) [27, 28]. However, there is a need for more tools to measure metrics other than patient experience of patient engagement. Tools are also available that could help to measure the impact of patient engagement. For example, if an engagement activity is intended to leverage patient input to co-create informational materials for patients, the Patient Education Materials Assessment Tool (PEMAT) [29] and/or Patient Activation Measure (PAM) [30] could be used to help assess comparative or longitudinal improvement in the materials attributable to the engagement, and/or patient activation in their own care, though not measuring engagement per se. We found no single standardized instrument able to assess all aspects of patient engagement across medicines lifecycle, and no validated tools for measuring the societal-level impact of patient engagement in the context of medicines development. Our specific insights for the development of the framework from the exploratory literature review were that terminology is inconsistent and unclear, and that we need to work with a range of measurement tools, and likely over time create, adapt, and/or validate more where needed in the framework.
2.3 Lack of Standardized Metrics to Track Across the Whole Medicines Lifecycle
The analysis of publicly available corporate information revealed that of 93 patient-relevant declared corporate priorities (referred to as ‘materiality’ topics or focus areas), eight (9%) were directly related to patient engagement. In ESG reporting, the majority of actions were implicitly described as being done ‘for’ or ‘to’ patients, not ‘with’ patients. It was noted that patient-relevant ESG reporting is typically focused on drug development and clinical trials (e.g., measures of patient enrollment in studies, trial result summaries, diversity in clinical trials, and number of pipeline assets), with later stages of the medicines lifecycle (e.g., regulatory submission and post-marketing) receiving less attention. R&D organizations’ reporting of patient-related activities (e.g., numbers of patients reached through affordability and access/product donation programs, education, and screening programs) was mostly (highest number of KPIs reported on) related to access and affordability, which are not engagement activities. Our specific insights for the development of the framework from the analysis of corporate information were that metrics for measuring patient engagement were largely absent from corporate publications and there were no reports of the outcomes of patient engagement and its impact. In addition, the very beginning and end of the medicines lifecycle receive less focus than the clinical development stage when it comes to patient-engagement activities.
The web analysis indicated that online content related to patient engagement and impact measurement most often related to clinical trials, adherence, and digital technology, with an increasing focus over time on technological innovation. Examples include articles about artificial intelligence (AI)-powered platform use of patient population insights to drive increased patient participation as study subjects in relevant clinical trials, and a dynamic questionnaire for measuring patient-reported outcomes which adapts based on the patient's previous responses, resulting in fewer questions overall [31, 32]. Others reported the integration of AI to create personalized mobile apps and patient support programs that use patient-generated data and analytics to allow personalization at a patient level based on individual response and action [33,34,35]. We also noted differences in volumes for some topics between academic article types and general, news-like articles (which were mostly corporate news). One specific insight for the development of the framework from the web analysis was that measured patient engagement activity types needed to include emerging technology for facilitating engagement and/or measuring impact. In addition, the contrast between volume of academic (low) and corporate (high) articles on post-launch activities suggested that inclusion of, for example, the use of patient engagement to inform patient support program development and improvement should be done in a way to support peer-reviewed publication of the results to contribute to academic literature.
The review of Novartis' internal information identified company-specific structures and processes that were in place to support patient engagement across the medicines lifecycle, including personnel, online tools, guidance documents, learning management system, patient organization and insight repositories, and activity tracking. This information was subsequently used to support the development of the measurement framework aligned with existing resources and structures.
2.4 Lack of Shared Language in Patient Engagement and Reporting
The review of peer-reviewed literature, corporate documents, and online content all showed that a common language is currently lacking for patient engagement in the context of collaborating with patients to design health-related interventions that are relevant to their needs [36]. The term ‘patient engagement’ may differentially be used to describe patient involvement in research activities and/or information dissemination and knowledge sharing [37, 38]. In other contexts, ‘patient engagement’ is also used to describe active participation or empowerment of patients in their personal healthcare decisions. In our work, we define patient engagement as “the effective and active collaboration of patients, patient advocates, patient representatives and/or caregivers in processes and decisions within the medicines lifecycle, along with all other relevant stakeholders when appropriate.” This reflects the WHO description of patient engagement as a process of developing relationships that enable collaboration, and also US FDA guidance, which uses patient engagement to mean activities in which patient stakeholders share their experiences, perspectives, needs, and priorities—both with the goal of achieving positive health impact [7, 39]. Other definitions of terms used in the proposed framework are provided in Table 1.
3 Our Ways of Working: Partnership Between Corporate and Patient Experts For Collaborative Development
Following the review of the existing environment, Novartis assembled a co-creation team comprising an internal cross-functional working group, a council of four patient expert advisors (authors of this paper EMP, EMO, FW, TCAL), and external consultants from Dot I/O Health (ESM Suppl. Fig. 2). The external parties were independent experts selected by Novartis’ Patient Engagement Center of Excellence, using word-of-mouth referrals to identify individuals with an active interest in impact measurement, familiar with the medicines lifecycle and patient engagement and patient experience data collection, and who understand how patient advocacy can add value across stakeholder interactions, including healthcare systems, clinical research, health technology assessments, regulatory processes, and policy development. The advisors on the council have experience across health outcomes research, patient and caregiver advocacy, collecting and applying patient experience data in the context of drug development, healthcare communications, living with chronic diseases, primary caregiver for family member, and/or leadership roles at patient associations.
At the initiation of the project, the internal working group, external consultants, and advisor council co-developed and agreed on a project charter outlining the scope, expectations, objectives, and vision. Core to the success of the project was two-way communication and respect. Communication was open, honest, and transparent. Across the course of 8 months of collaboration, including two rounds of individual meetings and four group meetings, the advisors were regularly consulted and informed, including in reviewing draft reports from interviews and council meetings to verify their most important insights were correctly recorded and understood, and Engagement Feedback Questionnaires (EFQ) were issued after council meetings and at the end of the year to evaluate advisor experience of the council (ESM Suppl. Table 1). The internal working group provided input on what was practically achievable and relevant to the functions consulted.
4 Development: Co-Creation and Evolution of the Patient Engagement Impact Measurement Framework
The team, after review of the environmental scan, agreed there was a need for a specific framework for tracking over time and across the lifecycle, connecting work done to the short- and long-term value created by the organization and patient community. This would need to be accompanied by a well-defined and easy-to-comprehend lexicon, and be achievable with clear metrics, and tools for measuring and reporting.
To meet the needs identified by the environmental scan and prioritized by the team, the internal working group and advisor council co-created a framework to demonstrate the impact of patient engagement through consistent, longitudinal measurement. Starting with a ‘straw man’ presented by Dot I/O Health based on behavior change program models [40], Novartis’ business priorities to measure quality and impact, and concepts identified within the environmental scan, there followed multiple iterations, each evolving as the result of working group and advisor council meetings, using pre-reads, follow-up reports, and individual feedback sessions. The draft framework was developed to align with existing internal workflows, resources, and decision frameworks. Although we used the experiences of Novartis as a known basis for the draft framework, the objective was to develop a structure and identify metrics that could be applicable, with appropriate modifications, to suit the particular needs of different R&D organizations.
To assess the quality and value of patient engagement, it was recommended by the advisor council to evolve the initially proposed framework by adopting concepts developed by Donabedian [41]. Originally, this model was conceived to measure quality of care as delivered by healthcare providers. However, the fundamental assessment approach could be adapted and applied to all aspects of the medicines lifecycle. The model was founded on the recognition of three components: structure, process, and outcome, as described in Table 2. These three components are interdependent: a good structure increases the likelihood of a good process, and good structure and processes increase the likelihood of good outcomes. Yet, the interaction between the categories can be bidirectional, and there is no simple separation between cause and effect [42].
Our draft Patient Engagement Impact Measurement (PEIM) framework builds on that model, with expanded characterization and definition of input, output, and value-demonstration measurements (Fig. 1). We determined that to demonstrate value, the outcomes and impact must be measured in ways that can channel into clear actions, leading to improvement in a continual, iterative, and cyclical process (as represented by a circular format). Additionally, it was considered of particular importance to represent collaboration with the patient community throughout as the center of the framework, as shown in Fig. 1.
5 Results: The Sequential Pillars of Patient Engagement Impact Measurement
Following a widely recognized logic model structure, the six pillars of the PEIM framework are described hereafter and in Fig. 2.
Input describes patient engagement, including dialogue with patients as partners and collection of patient experience data. Efficient input requires optimal structures and processes, which are best developed in close consultation with the patient community. Structurally, industry stakeholders need to build team capacity and ensure that relevant tools for patient engagement and its measurement are available. Structures to manage patient organization mapping are available in many organizations. Standardized patient Engagement Feedback Questionnaires (EFQ), which evaluate patient perceptions of the quality and value of engagement, should be developed in collaboration with patients to avoid making them onerous to complete. For any structures, adequate processes of uptake and application need to be established. Many pharmaceutical R&D organizations have processes, such as sourcing and engaging an advisory board, to obtain information about patient engagement and to capture patient insights efficiently. Patient organization mapping further helps to identify the relevant partners for specific activities. Consistent implementation across an organization may be a challenge, however. Capability building with a strong focus on individual training is a fundamental undertaking if patient-centered thinking and practice is to become integrated throughout the fabric of a pharmaceutical organization.
The output of a successful patient-engagement process comprises patient-informed learnings, decisions, and actions. Tools must capture what was learned from patients, and what was done with the learnings (actions—by which functions in the company). If an undesired action was taken, or a relevant action was unable to be taken, this must be recorded too, with the reasons/barriers documented. This could lead to future actions to remove barriers, such as through collaboration with, or action by, patient organizations, experts, or advocacy groups. However, how to measure the output represents a challenge. By their nature, valuable learnings from patient engagement are often descriptive and contextual, not easily quantified. Engagement Feedback Questionnaires are valuable tools to assess patient satisfaction with the process; shorter versions can be used after every interaction, whereas longer, more comprehensive questionnaires are suitable at key project milestones, the end of a project, or after advisory board meetings. Such tools may be standardized for use beyond a single research organization. Other tools to measure and/or track output may be designed to suit the internal needs and structure of individual companies.
Value can be divided into outcomes and impact, with outcomes defined as the direct results of the actions taken and impact representing the consequences of the outcomes, typically at the societal level. Potential positive outcomes may include an improved understanding of patient experiences by decision makers; product label claims that reflect outcomes most relevant to patients, or acceptance of patient-reported outcome measure (PROM) tools and data by regulatory agencies and other stakeholders. Such outcomes may contribute to beneficial impacts such as greater relevance to patients of future clinical trials; rapid and equitable access to therapy, or strengthening healthcare systems.
Figure 2 illustrates how the three components of the previously described Donabedian model for healthcare quality assessment may be mapped to the six pillars of the PEIM framework. Of note is the addition of the ‘Outputs’ pillars (learnings and actions), which are critical to ensuring that learnings from patient engagement are memorialized and that decisions and actions related to those insights are recorded.
5.1 Practical Application of the Patient Engagement Impact Measurement Framework
Relevant metrics and their definitions need to be identified before the start of a project. As the framework is cyclical (Fig. 1), the measurements need to be adequate to inform actions and refinements at the input stage of a new iteration. The choice of metrics is important; metrics need to be pragmatic and relevant to the short term as well as the long term. They also need to reflect many different perspectives, not only those of patients and commercial organizations but of diverse stakeholders with responsibilities for medicines development and access, including regulatory agencies, payers, and clinical guidelines committees. Examples of metric themes (categories) and potential tools for measuring inputs, outputs, and value are shown in Fig. 3. These are non-exhaustive and represent how the framework may be applied in a way that is suitable for an individual organization.
To illustrate the descriptions of the framework pillars, we present a list of example metrics applicable to the PEIM framework components for specific patient-engagement activities at two stages of the lifecycle for which the environmental scan indicated current gaps. Namely, the acquisition of patient experience data to inform medicines development (Table 3), and the creation of programs/materials for patients, including patient support programs (Table 4). These examples were developed in the context of systems and processes within Novartis, but are adaptable for any organization.
A project-specific example of the framework in action is given in Table 5, where patient engagement informed a clinical trial protocol.
It should be reiterated that the learnings generated from patient engagement at any stage of the lifecycle may be realized as both short-term and long-term outcomes and impacts. Because this realization of value may not transpire for many years, it is important to maintain longitudinal connectivity and tracking of value, rather than thinking about patient engagement activities as single isolated moments in time only affecting that one activity.
6 Considerations: Application of the Patient Engagement Impact Measurement Framework
As the pharmaceutical industry advances toward more patient-centric development, the role of the patient-engagement function is to elevate the patient voice. To motivate more consistent and systematic patient engagement across the industry, there remains a need for evidence showing the value of patient engagement for all stakeholders. To effectively measure the quality and value of patient engagement and patient experience integrated into core activities, we concluded that a detailed framework of fit-for-purpose tools needed to be developed, including adequate metrics [11, 43,44,45,46]. Consistent metric gathering will enhance the integration of patients’ experiences to inform and support pharmaceutical decision making.
In this paper we describe the co-creation, by a working group of industry stakeholders and a council of patient expert advisors, of a framework to measure and demonstrate the value of patient engagement at all key steps of the medicines lifecycle, from early planning and needs assessments through early development, pivotal trials and market approvals, to programs that optimize access and strengthen healthcare systems. To our knowledge, this is the first such framework for measuring the value of patient engagement that includes metrics and industry-relevant measurement tools applicable across the medicines lifecycle and using all components of a health-system-relevant logic model (as described by Donabedian).
Many R&D organizations employ internal tools to keep track of engagements with patients and patient organizations. However, the medicines lifecycle comprises specialized tasks that are not always connected. A key insight from the internal environmental scan and discussions with the advisor council was that to assess the quality and understand outcomes and impact of patient engagement, it is necessary to track projects across the whole logic framework (inputs, outputs, outcomes, impact), across stakeholders and functions (including patient organizations), as well as over time. No single indicator or method was identified that covered all these assessment needs. Thus, a coherent set of targeted metrics would need to be developed and standardized to ensure that impact measurement is effective and relevant within the varied stages of the medicines lifecycle.
As quality improvements to patient engagement are made from continual assessment and adjustment, the integration and evaluation process must be iterative, with learnings feeding back into actions and refinement, maintaining the continuous patient community partnership at the core. The circular framework (Fig. 1) can be considered to grow in magnitude with each iterative cycle. The inputs, designed to lead to outcomes that inform a new round of actions when consistently applied at each step of the lifecycle, increase the opportunity for impact over time. The framework is designed to be ‘hands-on’ as well as comprehensive, and helps define the necessary tools and metrics across the different stages of the medicines lifecycle.
To make the model work practically, fit-for-purpose tools and systems must be connected across R&D areas and organizational roles, as well as over time to support longitudinal analyses. This highlights the need for more tool and system development, as many existing tools are designed for single patient-engagement methods or moments in medicines lifecycle. A common set of compatible metrics will help to enable continuing collaboration with patient communities as well as cross-functional collaboration inside R&D organizations. Such metrics will improve as more data on patient experience become available. At present, such information resides predominantly inside commercial R&D organizations and may be protected by confidentiality agreements or privacy clauses. With greater effort to put patient experience data in the public domain, whether as formal papers in the scientific literature or in other media, the necessary tools and metrics can be refined and be drawn upon by all stakeholders in medicines development.
In planning patient-experience research, it is important to be aware of potential sources of bias to ensure the learnings are objective and relevant to the objectives and research questions to be answered. Importantly, patients or caregivers recruited for engagement activities should be representative of the target population [47]. They should also have the capacity and capability to participate in high-quality engagement [48]. Patient organizations can help to identify representative individuals, and some also train patients to engage effectively with regulatory bodies and industry.
The proposed framework has several strengths. The qualitative and quantitative metrics were developed to align with patients’ perceptions and needs as well as those of other healthcare stakeholders. Further, it drives gathering of information about the context of engagement and subsequent decisions, thereby enhancing their relevance and potential for ongoing quality improvement. The framework enables us to view the entire medicines lifecycle through one comprehensive lens, with a consistent approach across all steps and decision points.
However, as noted above, the framework is a draft and leaves much work to be done. One challenge is that our environmental scan, while very broad, was limited to identifying approaches and metrics that are publicly available—and was made particularly challenging by the high number of similar and related terms used, with common terms used to mean different things by different organizations. Despite this, we benefited from advisors with deep experience in the topics that helped ensure creation of a relevant framework. As with patient experience data, it would be enormously valuable for commercial stakeholders to make at least some of their current impact measurement metrics publicly available. There will likely never be a single solution to all stakeholder needs, but if a harmonized framework with common, validated tools can be adopted, this could speed up processes and accelerate patient-centric research across the R&D community and hopefully lead to faster regulatory approvals.
In summary, the draft framework developed by our interdisciplinary effort between representatives from the patient community and a pharmaceutical R&D company provides a way forward to consistently measure and demonstrate the value of patient engagement across the medicines lifecycle. Experiences with applying the framework will be described in future publications.
7 Conclusion: A Call to Join Us in Cross-Industry Progress with Patients
The PEIM framework was designed to work alongside and build upon existing frameworks and contribute to the overall practice of patient engagement measurement and elevation. By publishing this work we hope to encourage more measurement (and tool development and testing) across the industry. Other organizations now have this new option to utilize, adapt, and build upon, to enhance the discipline. A summary of this framework has been presented at annual meetings of ISPOR and Patients as Partners, the ISPOR Patient Summit, and the Measuring Patient Engagement Summit. We are pleased to report we have received interest from other pharmaceutical companies, patient organizations, and supplier companies (e.g., agencies and contract research organizations) to learn from, adapt, and apply it.
By measuring the impact of patient engagement, we can demonstrate the value of our work, and decide on the most effective moments and practices to put our engagement efforts towards, helping patient partners focus their resources and time. As industry makes great strides in working 'with’ patients, not only ‘for’ patients, wider adoption of impact measurement, such as through this co-created framework, will demonstrably improve provision and access to treatments with effects that really matter to patients. As such, we call on colleagues across the pharmaceutical industry to join us in applying impact measurement to measure, publish, and communicate the value of patient engagement and its impact.
References
World Health Organization. Fair pricing forum 2021 meeting report. https://www.who.int/publications/i/item/9789240038585. Accessed 21 March 2024.
Frank L, et al. Conceptual and practical foundations of patient engagement in research at the patient-centered outcomes research institute. Qual Life Res Int J Qual Life Asp Treat Care Rehabil. 2015;24(5):1033–41. https://doi.org/10.1007/s11136-014-0893-3.
Forsythe LP, et al. Patient engagement in research: early findings from the patient-centered outcomes research institute. Health Aff (Millwood). 2019;38(3):359–67. https://doi.org/10.1377/hlthaff.2018.05067.
Zickmund SL, Frosch DL, Carman KL. Patient and veteran engagement in health research: the emergence of a field of study. J Gen Intern Med. 2022;37(1):3–5. https://doi.org/10.1007/s11606-022-07393-9.
Merker VL, Hyde JK, Herbst A, Solch AK, Mohr DC, Gaj L, Dvorin K, Dryden EM. Evaluating the impacts of patient engagement on health services research teams: lessons from the veteran consulting network. J Gen Intern Med. 2022;37(1):33–41. https://doi.org/10.1007/s11606-021-06987-z.
European Medicines Agency. Engagement framework: European medicines agency and patients, consumers and their organisations. https://www.ema.europa.eu/en/partners-networks/patients-and-consumers. Accessed 21 March 2024.
US FDA. FDA patient-focused drug development guidance series for enhancing the incorporation of the patient’s voice in medical product development and regulatory decision making. https://www.fda.gov/drugs/development-approval-process-drugs/fda-patient-focused-drug-development-guidance-series-enhancing-incorporation-patients-voice-medical. Accessed 21 March 2024.
World Health Organization. Declaration of Alma-Ata. https://www.who.int/docs/default-source/documents/almaata-declaration-en.pdf. Accessed 1 August 2024.
Boivin A, Mothci D, Dumez V, Shore F, Bok A. World leaders unite to embed social participation in health systems. BMJ. 2024;386: q1460. https://doi.org/10.1136/bmj.q1460.
W. H. Organization. Social participation for universal health coverage, health and well-being. https://apps.who.int/gb/ebwha/pdf_files/WHA77/A77_R2-en.pdf. Accessed 01 August 2024.
Cavaller-Bellaubi M, Faulkner SD, Teixeira B, Boudes M, Molero E, Brooke N, McKeaveney L, Southerton J, Vicente MJ, Bertelsen N, García-Burgos J, Pirard V, Reid K, Ferrer E. Sustaining meaningful patient engagement across the lifecycle of medicines: a roadmap for action. Ther Innov Regul Sci. 2021;55:936–53. https://doi.org/10.1007/s43441-021-00282-z.
PARADIGM Consortium. PARADIGM patient engagement toolbox. https://imi-paradigm.eu/petoolbox/. Accessed 21 March 2024.
Barron D. From PE metrics framework to actionable tool—what’s after PARADIGM? Patient Engagement for Medicines Development (PFMD). https://patientfocusedmedicine.org/from-pe-metrics-framework-to-actionable-tool-whats-after-paradigm/. Accessed 21 March 2024.
Patient-Centered Outcomes Research Institute (PCORI). Measuring what matters for advancing the science and practice of engagement. https://www.pcori.org/resources/measuring-what-matters-advancing-science-and-practice-engagement. Accessed 21 March 2024.
Vat LE, et al. Evaluation of patient engagement in medicine development: a multi-stakeholder framework with metrics. Health Expect. 2021;24(2):491–506. https://doi.org/10.1111/hex.13191.
Harrington RL, et al. Defining patient engagement in research: results of a systematic review and analysis: report of the ISPOR patient-centered special interest group. Value Health Rev. 2020;23(6):677–88. https://doi.org/10.1016/j.jval.2020.01.019.
Walsh CM, Jones NL, McCreath GA, Connan V, Pires L, Abuloghod L, Buchanan F, Macarthur C. Codevelopment and usability testing of patient engagement 101: a patient-oriented research curriculum in child health e-learning module for health care professionals, researchers and trainees. CMAJ Open. 2022;10(4):E872–81. https://doi.org/10.9778/cmajo.20210336.
McAlearney AS, et al. Effect of in-person vs video training and access to all functions vs a limited subset of functions on portal use among inpatients: a randomized clinical trial. JAMA Netw Open. 2022;5(9):e2231321–e2231321. https://doi.org/10.1001/jamanetworkopen.2022.31321.
Park J, Liang M, Alpert JM, Brown RF, Zhong X. The causal relationship between portal usage and self-efficacious health information-seeking behaviors: Secondary analysis of the health information national trends survey data. J Med Internet Res. 2021;23(1): e17782. https://doi.org/10.2196/17782.
Chegini Z, Shariful Islam SM. Expert perspectives on the active role of patients in their safety: toward a framework using Delphi methodology. Nurs Forum. 2021;56(3):490–9. https://doi.org/10.1111/nuf.12567.
Wang W, Zhang H, Lin B, Zhang Z. Feasibility of a patient engagement and medication safety management program for older adults suffering cardiovascular disease in community settings. Medicine. 2021;100(21): e26125. https://doi.org/10.1097/MD.0000000000026125.
Charlot M, Carolan K, Gawuga C, Freeman E, Sprague Martinez L. Patient powered research: an approach to building capacity for a hardly reached patient population to engage in cancer research. Res Involv Engagem. 2021;7(1):74. https://doi.org/10.1186/s40900-021-00317-7.
Mohammed K, et al. Creating a patient-centered health care delivery system: a systematic review of health care quality from the patient perspective. Am J Med Qual. 2016;31(1):12–21. https://doi.org/10.1177/1062860614545124.
Babione J, et al. Alignment of patient-centredness definitions with real-life patient and clinician experiences: a qualitative study. Health Expect. 2023;26(1):419–28. https://doi.org/10.1111/hex.13674.
Carroll P, et al. Applying patient and public involvement in preclinical research: a co-created scoping review. Health Expect. 2022;25(6):2680–99. https://doi.org/10.1111/hex.13615.
Easley J, et al. Patient engagement in health research: perspectives from patient participants. Curr Oncol. 2023;30(3):2770–80. https://doi.org/10.3390/curroncol30030210.
Hamilton CB, et al. Development and pre-testing of the patient engagement in research scale (PEIRS) to assess the quality of engagement from a patient perspective. PLoS ONE. 2018;13(11): e0206588. https://doi.org/10.1371/journal.pone.0206588.
Goodman MS, Ackermann N, Haskell-Craig Z, Jackson S, Bowen DJ, Sanders Thompson VL. Construct validation of the Research Engagement Survey Tool (REST). Res Involv Engagem. 2022;8(1):26. https://doi.org/10.1186/s40900-022-00360-y.
Shoemaker SJ, Wolf MS, Brach C. Development of the patient education materials assessment tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns. 2014;96(3):395–403. https://doi.org/10.1016/j.pec.2014.05.027.
Hibbard JH, Stockard J, Mahoney ER, Tusler M. Development of the patient activation measure (PAM): conceptualizing and measuring activation in patients and consumers. Health Serv Res. 2004;39(4 Pt 1):1005–26. https://doi.org/10.1111/j.1475-6773.2004.00269.x.
Leal Health. Leal Health launches first-of-its-kind, patient-centric program-level analytics platform to help pharmaceutical companies improve patient participation in clinical trials. PR Newswire. https://www.prnewswire.com/news-releases/leal-health-launches-first-of-its-kind-patient-centric-program-level-analytics-platform-to-help-pharmaceutical-companies-improve-patient-participation-in-clinical-trials-301767426.html. Accessed 21 March 2024.
Signant Health. Signant's novel electronic PROMIS CAT assessment enhances patient-centric drug development. PR Newswire. https://www.prnewswire.com/news-releases/signants-novel-electronic-promis-cat-assessment-enhances-patient-centric-drug-development-301739724.html. Accessed 21 March 2024.
Medisafe. Medisafe launches “Rapid Release” for expedited development of configurable, scalable patient engagement programs. Accesswire. https://www.accesswire.com/786089/medisafe-launches-rapid-release-for-expedited-development-of-configurable-scalable-patient-engagement-programs. Accessed 21 March 2024.
MedTech Breakthrough. Orthofix STIM on track recognized for patient engagement innovation in 2023 MedTech Breakthrough Awards Program. PR Newswire. https://www.prweb.com/releases/orthofix-stim-ontrack-recognized-for-patient-engagement-innovation-in-2023-medtech-breakthrough-awards-program-825868387.html. Accessed 21 March 2024.
Nye Health. Nye Health launches digital patient support programme capability. https://www.digitalhealth.net/2023/11/nye-health-launches-digital-patient-support-programme-capability/. Accessed 21 March 2024.
Hickmann E, Richter P, Schlieter H. All together now—patient engagement, patient empowerment, and associated terms in personal healthcare. BMC Health Serv Res. 2022;22(1):1116. https://doi.org/10.1186/s12913-022-08501-5.
Heckert A, et al. Researchers, patients, and other stakeholders’ perspectives on challenges to and strategies for engagement. Res Involv Engagem. 2020;6(1):60. https://doi.org/10.1186/s40900-020-00227-0.
Frank L, Morton SC, Guise JM, Jull J, Concannon TW, Tugwell P. Engaging patients and other non-researchers in health research: defining research engagement. J Gen Intern Med. 2020;35(1):307–14. https://doi.org/10.1007/s11606-019-05436-2.
World Health Organization. Community engagement: a health promotion guide for universal health coverage in the hands of the people. https://www.who.int/publications/i/item/9789240010529. Accessed 21 March 2024.
W. H. Organization. The MEL manual. Measurement, evaluation, and learning. https://cdn.who.int/media/docs/default-source/dco/who-mel-manual-2021.pdf?sfvrsn=74307e9f_1. Accessed 01 August 2024.
Donabedian A. The quality of care: how can it be assessed? JAMA. 1988;260(12):1743–8. https://doi.org/10.1001/jama.1988.03410120089033.
Donabedian A. Evaluating the quality of medical care. Milbank Q. 2005;83(4):691–729. https://doi.org/10.1111/j.1468-0009.2005.00397.x.
Faulkner SD, et al. Understanding multi-stakeholder needs, preferences and expectations to define effective practices and processes of patient engagement in medicine development: a mixed-methods study. Health Expect. 2021;24(2):601–16. https://doi.org/10.1111/hex.13207.
Feldman D, et al. Co-creation of practical “how-to guides” for patient engagement in key phases of medicines development—from theory to implementation. Res Involv Engagem. 2021;7(1):57. https://doi.org/10.1186/s40900-021-00294-x.
Sittig DF, et al. A lifecycle framework illustrates eight stages necessary for realizing the benefits of patient-centered clinical decision support. J Am Med Inf Assoc. 2023;30(9):1583–9. https://doi.org/10.1093/jamia/ocad122.
Gorbenko O, et al. Co-creating with patients an impact framework across the medicine’s life cycle: a qualitative study exploring patients’ experiences of involvement in and perceptions of impact measures. Res Involve Engagem. 2022;8(1):1. https://doi.org/10.1186/s40900-022-00334-0.
US FDA. Patient-focused drug development: collecting comprehensive and representative input. Guidance for industry, food and drug administration staff, and other stakeholders. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-collecting-comprehensive-and-representative-input. Accessed 21 March 2024.
Patient Focused Medicines Development (PFMD). Patient engagement quality guidance. https://patientfocusedmedicine.org/peqg/patient-engagement-quality-guidance.pdf. Accessed 21 March 2024.
Council for International Organizations of Medical Sciences (CIOMS). Working Group XI, Patient involvement in the development, regulation and safe use of medicines. Geneva, Switzerland; 2022.
US FDA. Patient-focused drug development glossary. https://www.fda.gov/drugs/development-approval-process-drugs/patient-focused-drug-development-glossary. Accessed 21 March 2024.
McCue M, et al. Mobile app to enhance patient activation and patient-provider communication in major depressive disorder management: collaborative, randomized controlled pilot study. JMIR Form Res. 2022;6(10): e34923. https://doi.org/10.2196/34923.
Miller VM, et al. Increasing patient activation through diabetes self-management education: outcomes of DESMOND in regional Western Australia. Patient Educ Couns. 2020;103(4):848–53. https://doi.org/10.1016/j.pec.2019.10.013.
Acknowledgements
The authors wish to thank Zoe Healey, PhD, and Nigel Breakwell, both of Dot I/O Health, London, UK, for their consultancy funded by Novartis Pharma AG, during the environmental scan and the conception and development of the impact measurement framework. The Insights Automation Platform used in the environmental scan was from AMPLYFI, Cardiff, UK. Medical writing support, in the form of developing an outline, incorporating author comments, figure development and editing was provided by Pelle Stolt (MagliaRotta, Basel, Switzerland), and Zoe Healey, PhD and Danielle Russell, PhD (Dot I/O Health, London, UK), supported by Novartis Pharma AG in accordance with Good Publication Practice guidelines. The authors thank Zack Pemberton-Whiteley (Novartis Pharma AG) for valuable contributions as a reviewer during the development of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
Consultancy in the development of the framework, and medical writing support were funded by Novartis Pharma AG. Open Access publication of this paper was sponsored by Novartis Pharma AG.
Conflicts of Interest
BK is an employee and shareholder of Novartis Pharma AG. EMP has performed paid advisory roles and consultancy on meaningful patient engagement for Novartis. EMO is employed by Applied Patient Experience, LLC, which has contracts with a variety of nonprofit organizations, pharmaceutical companies, and academic institutions, including paid advisory roles and consultancy on meaningful patient engagement for Novartis. FW is an employee of Avalere Health and participated in this work in an independent capacity. FW has performed paid advisory roles and consultancy on meaningful patient engagement for Novartis. TCAL has performed paid advisory roles and consultancy on meaningful patient engagement for Novartis. MB is an employee and shareholder of Novartis Pharma AG. No authors received financial compensation for their involvement in authoring this manuscript.
Availability of Data and Material
Due to the nature of the research, supporting data are not available.
Ethics Approval
Not applicable.
Consent to Participate
Not applicable.
Consent to Publish
Not applicable (no identifiable information).
Code Availability
Not applicable.
Author Contributions
All authors were closely involved in the iterative development and critical review of the presented framework. BK also provided direction for the background environment analysis. All authors contributed to the development, revision, and critical review of the manuscript. All authors read and approved the final version. The responsibility for opinions and conclusions presented herein lies with the authors.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/.
About this article
Cite this article
Klein, B., Perfetto, E.M., Oehrlein, E.M. et al. Measuring and Demonstrating the Value of Patient Engagement Across the Medicines Lifecycle: A Patient Engagement Impact Measurement Framework. Patient (2024). https://doi.org/10.1007/s40271-024-00713-7
Accepted:
Published:
DOI: https://doi.org/10.1007/s40271-024-00713-7