Background

Antimicrobial resistance (AMR) has been highlighted by the World Health Organization (WHO) as one of the top 10 global public health threats facing humanity [1]. In the European Union, antimicrobial resistant infections are estimated to be responsible for at least 25,000 deaths annually [2]. Globally, these infections claim around 700,000 lives each year [3]. Inappropriate and suboptimal use of antimicrobials in healthcare are key contributors to AMR [4], which can lead to an increase in and spread of resistant bacteria and increase risk of poor outcomes from bacterial infections due to a reduced number of effective antimicrobial therapeutics. It is therefore imperative to stem inappropriate antimicrobial use [5].

The European Centre for Disease Prevention and Control revealed that the United Kingdom (UK) had the third highest hospital consumption of systemic antibiotics per capita in Europe [6], with hospital inpatient antibiotic consumption increasing by 6.3% between 2016 and 19 [7]. The English Surveillance Programme for Antimicrobial Utilisation and Resistance Oversight (ESPAUR) found increases in the rate of bloodstream infections caused by Escherichia coli and Klebsiella pneumoniae between 2016 and 2019 [7], as well as a slight increase in the proportion of bloodstream infections resistant to piperacillin/tazobactam between 2016 and 2020. This increased resistance places further pressure on clinicians to use ‘last resort’ antibiotics such as carbapenems. The exceptional impact of the first waves of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic [7] compounded these challenges, the 10.6% increase in hospital inpatient antibiotic consumption in 2019–2020 potentially leading to increased inappropriate use.

In response to this growing threat, Public Health England (now UK Health Security Agency) championed guidance encouraging clinicians to “Start Smart–Then Focus” in relation to the initiation and maintenance of antibiotics [8, 9]. Moreover, the National Health Service (NHS) England (NHSE) Antimicrobial Resistance and Antimicrobial Stewardship Commissioning for Quality and Innovation (CQUIN) aims to promote a “reduction in antibiotic consumption per 1000 admissions” [10].

As NHSE rapidly moves towards increasing digitisation of hospitals [10], electronic prescribing (ePrescribing) systems are crucial to antimicrobial stewardship (AMS) relating to prescribing [11]. Furthermore, guideline-based clinical decision support (CDS) systems can help, and the effects of CDS rules can be enhanced through techniques that support clinicians and hospitals to prioritise AMS through, for example, facilitating timely review of antibiotics [12]. A review exploring the appropriate use of antibiotics through hospital ePrescribing systems [12]. and related conceptual work [13] indicate ePrescribing systems—integrated with behavioural and organisational support—have a major role in improving AMS. We have carefully conceptualised a complex ePrescribing-based Anti-Microbial Stewardship (ePAMS+) intervention that aligns with the UK government Five Year Antimicrobial Resistance Strategy [14] and ESPAUR [7].

The overall aim of our mixed-methods research programme is to plan, develop and optimise the ePAMS+ complex intervention and to assess its clinical and cost-effectiveness within a hybrid cluster-randomised stepped-wedge clinical trial. Prior to finalising the protocol of the full-scale trial assessing intervention effectiveness, we plan a feasibility trial involving testing of data extraction and the implementation and acceptability of the ePAMS+ intervention to inform the trial design.

Aims

Qualitative and Fidelity Index research aims:

  1. 1.

    Explore user acceptability of the content of the ePAMS+ technical component and identify any barriers to use

  2. 2.

    Assess whether ePAMS+ Antibiotic Order Plans are used as intended in clinical practice and if not, identify barriers

  3. 3.

    Assess acceptability to healthcare professionals of the content of the ePAMS+ intervention plan (ePAMS+ organisational component) and training materials (ePAMS+ educational component)

  4. 4.

    Understand how ePAMS+ may be best delivered across multiple care settings and site information systems

  5. 5.

    Determine whether the procedures for implementing ePAMS+ are acceptable and feasible

  6. 6.

    Develop a Fidelity Index to quantify the extent to which core principles of ePAMS+ are enacted in antibiotic prescribing practice and test its usability

  7. 7.

    Confirm hypothesised mechanisms of action, refine programme theory and identify appropriate process analysis measures of mechanisms of action for a future full-scale trial.

Feasibility trial and quantitative analysis aims:

  1. 1.

    Assess whether ePAMS+ can be successfully integrated into hospital settings to enable changes in prescribing behaviour

  2. 2.

    Develop processes of collecting outcome data from ePrescribing systems prior to and following introduction of ePAMS+.

  3. 3.

    Determine between-patient variability in total antibiotic consumption to enable planning of the full-scale cluster-randomised hybrid stepped wedge clinical trial

Methods

Patient and public involvement

Guided by the recommendations on how to optimise patient and public involvement (PPI) from the PPI group on a preceding programme grant, our overall aim for PPI is to ensure that it is embedded within every stage of the research programme of which this feasibility trial forms a part. We consider PPI to be an equal partner in the research programme. Our two highly experienced PPI collaborators (AC, JB), have each worked for over 10 years in support of multiple research programmes. They guided the development of our funding application, reviewed and commented on the plans and contributed to the study design, and advised on the lay summary of the research. They lead all aspects of the PPI work, including attending and contributing actively to all group meetings; coordinating PPI involvement on the Independent Programme Steering Committee (IPSC); managing PPI across the four work packages of the research programme; and taking budgetary responsibility for PPI activities. AC also brings experience of points to consider when anonymising personal data extracts from routinely collected data, particularly relevant given the structure of the feasibility trial and the planned larger-scale effectiveness trial.

Members of the independent programme steering committee for grant RP-PG-0617-20009, including an independent patient and public representative and the two PPI collaborators, will oversee the trial conduct.

Design

This initial phase aims to assess the feasibility of embedding the ePAMS+ intervention into existing technological systems and organisational practices and extracting trial outcome measures. The feasibility trial will be conducted between October and December 2022 within selected hospital departments at two NHS acute hospital provider organisations in England. It has three main elements:

  1. 1.

    Through focus groups, interviews and observation of practice, qualitative research will explore how the ePAMS+ intervention is received and how it may need to be adapted for other contexts. It will also identify likely mechanisms of action to be examined further in the follow-on process evaluation of the planned full-scale stepped-wedge trial evaluating the ePAMS+ intervention. We will seek to understand barriers and facilitators to implementation, including usability and acceptance issues.

  2. 2.

    Quantitative analysis will develop methods of deriving key summaries of antibiotic use and will estimate the variability in these measures, using routine administrative data extracted from the Cerner ePrescribing and Medicines Administration (EPMA) system at each site.

  3. 3.

    We will develop a Fidelity Index prototype to assess the enactment of the ePAMS+ intervention. This will involve examining how closely practice matches its underlying principles. We intend to automate the Fidelity Index by developing rating scales to be applied automatically at critical decision-points for ePrescribing in individual patients using data from the EPMA system.

ePAMS+ intervention (Fig. 1)

Our intervention builds on the work of ARK (Antibiotic Reduction and Konservation), which produced an Antibiotic Review Kit that increased the proportion of antibiotic prescriptions reviewed within 72 h from 91 to 99% and the proportion of antibiotic prescriptions stopped within 72 h from 9 to 35% [15].

Fig. 1
figure 1

ePAMS+ intervention components alongside the national ‘Start Smart-Then Focus’ Guidelines

Full size image

ePAMS+ takes valuable lessons from the behavioural and organisational intervention work of ARK and extends it in three crucial ways:

  1. 1.

    Whereas ARK focused on stopping antibiotic prescribing at review, ePAMS+ aims to improve the decision-making process for all viable options, including starting and stopping treatment, optimisation of the dose regimen, switching the route of administration, changing the antibiotic, and continuing treatment for an appropriate duration;

  2. 2.

    ARK was implemented only in acute admission contexts whereas ePAMS+ applies to all hospital in-patient antibiotic prescribing; and

  3. 3.

    ARK was a behavioural and organisational intervention, whereas in addition to these aspects ePAMS+ also implements a CDS tool that exploits existing ePrescribing system functionality in order to automate, sustain and integrate effective support for appropriate antibiotic prescribing into all hospital prescribing pathways across multiple sites.

To inform the novel elements of ePAMS+, we have liaised extensively with policymakers, professional and patient representatives, vendors and international experts to conceptualise a prototype complex intervention which has the potential to support healthcare professionals and clinical teams at all key stages of antibiotic medicines management [13]. We have identified the core requirements of ePAMS+ and how it can interface with Cerner Millennium, a commonly used commercial and integrated EPMA system in the UK. The principles of ePAMS+ have been designed to be adaptable for implementation in other EPMA systems.

Additional file 1 summarises the ePAMS+ intervention using the Template for Intervention Description and Replication (TIDieR) checklist [16]. It aligns (Fig. 1) with best clinical practice and the national ‘Start Smart–Then Focus’ guidelines [9]. These guidelines state that antibiotics should be started promptly for patients if there is a suggestion of bacterial infection; reviewed regularly within 48–72 h of initial prescription to see if antibiotics are still needed; and stopped or switched or optimised as appropriate, once all test results to inform decision-making are received. The ePAMS+ intervention consists of the following tools embedded within the Cerner EPMA system:

  • Antibiotic order plans (Fig. 2A, B) to help prescribe antibiotics and schedule a series of review points (Fig. 2C) where changes in prescription may be required

  • Decision aid to help communicate the original prescriber’s level of certainty about the need for antibiotics in order to facilitate a later decision to cease prescription where appropriate (based on the ARK intervention classification [15] of possible risk of infection, probable infection or finalised diagnosis of infection)

  • Decision aid (Fig. 2D) includes fields to record proposed site of infection (body system) and working diagnosis (indication)

  • Information pages to help adopters benefit from using ePrescribing tools

  • Antibiotic ward task list to identify patients on antibiotics that may need review

  • Prompting rules for prescribers to promote antibiotic review

  • Links to microbiology and/or pathology results within the review screen.

Fig. 2
figure 2

Screenshots illustrating technical components of the ePAMS+ intervention. A ePAMS+ order plans. B ePAMS amoxicillin order plan. C Antibiotic review. D Decision aid includes fields to record proposed site of infection (body system) and working diagnosis (indication)

Full size image

Within each participating hospital, an ePAMS+ Champion will form a local Implementation Team (see Table 1 for details and approximate implementation timeline) to promote ePAMS+ through grand rounds, departmental/specialty team meetings, clinical governance meetings and training sessions for junior doctors/nurses/pharmacists. Prescribers, pharmacists and nurses working within study hospital wards will be encouraged to complete the ePAMS+ online eLearning training module.

Table 1 Implementation timeline
Full size table

Setting and number of sites and wards

Two NHS Trusts which use the Cerner EPMA system have been selected as feasibility sites. Within each site, as many wards as is feasible will be purposively selected to ensure the ePAMS+ intervention feasibility is evaluated across a wide range of clinical settings.

Qualitative component

Qualitative assessments (Table 2) will collect data in one multidisciplinary focus group at each site, approximately 10 h of observation of clinical practice and approximately 10 interviewees (five at each site) each participating in up to three interviews. We will purposively sample a range of stakeholders. Participants will include both patients and staff including junior or senior doctors from a range of wards and specialties, nurses, pharmacists, IT and informatics staff, managers, other relevant healthcare professionals and system vendors. Staff participants may be involved in focus groups and interviews. Patients may participate in focus groups, the rationale being that this will enable us to obtain a multi-stakeholder perspective. To mitigate a potential limitation of this approach, where frank discussion with patients may be inhibited in the presence of healthcare staff, we plan to undertake one to one patient interviews in the future larger scale effectiveness trial. Each focus group will include approximately 10–12 participants, incorporating up to three patients and up to nine staff participants. We anticipate that these numbers will lead to data saturation, giving us insights into potential intervention modifications to achieve maximum effectiveness and ensure acceptability to a range of stakeholders.

Table 2 Qualitative assessments
Full size table

Qualitative: patient participants

Patients will be eligible if they are able to provide informed consent, are aged ≥ 18 years, received antibiotic treatment in the last six months while in hospital, and are fluent in English. They will be excluded if they are temporarily unavailable (e.g., sleeping or receiving treatment) or if ward staff consider them too unwell to be interviewed. Potential participants will be identified by members of the direct care team according to the inclusion/exclusion criteria. An information poster for patients will also be displayed in wards.

Qualitative: staff participants

Staff will be eligible if they are able to provide informed consent, are aged 18 years and older, have experience in dealing with prescribing or administration of antibiotics or ePrescribing systems and are either junior or senior doctors, nurses, pharmacists, information technology (IT) staff, managers, other relevant healthcare professionals, or vendors.

Staff will be approached via two pathways, either on recommendation of a senior clinician on their ward or through recruitment leaflets displayed in wards. Individual participants at each site will either be approached in person (where feasible) or by telephone or email to enquire whether they are interested in participating in the study and if so whether they would prefer a face-to-face or online interview.

All potential patient and staff participants will receive written information on the project from the research team, outlining what participation will involve. They will be given at least 24 h to consider their decision to participate and can withdraw at any point. Upon receiving the completed consent form, a researcher will contact participants in order to arrange a suitable time for an interview or focus group.

Qualitative: data collection

Focus groups

Focus groups will take place remotely and will last no more than 60 min. Each focus group will cover experiences and opinions of ePAMS+ from a variety of perspectives, and explore its potential wider usability. Focus groups will be audio-recorded (if all participants agree) and transferred on encrypted equipment. If, however, audio-recording is not consented to by all participants, researchers will take detailed notes from the focus group session. Additional file 2 contains the focus group topic guide.

Interviews

Interviews will be conducted face-to-face or online via Microsoft Teams or Zoom depending on local requirements and will be recorded. All audio-recordings will be transcribed by an external transcription company contracted to the University of Edinburgh.

Interviews will be one-to-one. Staff will be asked about the way they work, how they use IT systems, what they think about AMS and how it can be promoted, as well as their impression of the ePAMS+ tool and their experiences of using it. Additional file 3 presents the healthcare professional interview topic guide. Interviews will take up to an hour each, but could be significantly shorter depending on participant preference. Participants may choose to take part in up to three interviews over the course of the project.

Observations

Staff who participate in observations will be shadowed by a researcher during their normal working day. The length of observations could range from 30 min to up to 4 h, depending on participant preferences. During the observation, the researcher will take notes about their impressions of how the participant uses ePAMS+. Observations will be non-participant in nature.

Qualitative: analysis

Qualitative data collection and analysis will be iterative, allowing emerging themes to be explored further and disconfirming evidence to be sought. Thematic analysis will allow us to access a diverse range of interviewees/perspectives, facilities and contexts. Detailed within-case analysis will be followed by analysis across cases to identify over-arching themes, similarities and differences between cases, and potential implications. Results of the analysis will inform development and implementation of the intervention.

Thematic analysis of focus group, interview and observation data will investigate how the intervention was received and how it may need to be adapted for other contexts and to identify likely mechanisms of action to be examined in the process evaluation in the future full-scale stepped-wedge trial evaluating the ePAMS+ intervention. Issues regarding effective design (usability; fit with existing workflows) and implementation (training; user acceptance) of the ePAMS+ intervention will be explored.

We will employ deductive and inductive approaches [17, 18] in the thematic analysis. The deductive element will consist of developing a coding structure, based on an evaluation framework we have developed in related work [19]. This includes considerations surrounding technology (for example, existing health information infrastructures), work practices (such as the way system users accommodate the new functionality), organisation factors (for example, how the organisation introduces the new system) and wider macro-environmental considerations (including surrounding political and economic drivers).

Tensions, trade-offs, and differences in stakeholder opinions and experiences over time will be explored in greatest detail. We will hold designated analysis workshops combining researchers involved in the feasibility trial qualitative and quantitative data analyses, alongside the wider research team including public contributors.

Fidelity Index

A Fidelity Index will be developed to capture the extent to which prescribers apply ePAMS+ ‘core principles’ (e.g. using the decision-aid for review and revise or using the patient leaflet for shared-decision making), in their practice. Assessing fidelity helps increase confidence that changes in the dependent variable are attributable to the independent variable and that behavioural interventions are implemented as described in the protocol [20, 21].

Through Cerner EPMA system data extracts, we will

Explore the critical decision-making points for prescribers

We will map behavioural elements of ePAMS+ to the data, to identify which items must be tagged for automation. One such element would be the diagnostic confidence decision aid, in which the prescriber rates their certainty about the presence of infection: none; possible risk from infection; probable diagnosis of infection; prophylaxis. The corresponding data item would be whether diagnostic confidence had been recorded in the EMPA at initial prescription. These behavioural elements will be the ‘critical decision-making points’ for prescribers that reflect the application of the ePAMS+ intervention core principles in their practice. This is critical for outcomes evaluation in the future full-scale trial, as these items would serve as ‘intermediate outcomes’ to help explain the relationship between the outcome and the intervention.

Understand the data structure for automating the fidelity coding

We will identify the critical decision-making points for prescribing within Cerner EPMA (relevant to ePAMS+), develop codes for automatic categorisation of their level of implementation and consider key locations within an EPMA system where these can be embedded.

Develop individual and composite scales for capturing practice

This part of the Fidelity Index measures the ‘actual’ implementation of ePAMS+ intervention as opposed to the ‘intended’. This will involve quantifying each ‘critical decision-making point’ (for example, whether antibiotic review was conducted within 48–72 h of initial prescription) into a 3-point rating scale that reliably discriminates between ‘fully’, ‘partially’ and ‘not implemented’. Although successfully used previously [22,23,24] these categories might not apply to ePAMS+. The feasibility work will help confirm these categories or explore alternatives such as codifying into ‘present’, ‘absent but should be present’ and ‘not applicable’ [25]. The scores from the rating scales will combine in a cumulative score for intervention fidelity (per case, per prescriber) for linking with outcome measures.

After developing the Fidelity Index, the specifications for its automation within Cerner EPMA systems and the methods of deriving summary measures of antimicrobial use, these will be pre-tested in the feasibility trial.

Quantitative component

Quantitative: sample size

We aim to study at least 100 admissions per ward, which would translate to over 2000 admissions per site in the event that 20 wards were included in a site. These numbers would enable precise estimation of between-patient variability, by ward and overall, in antibiotic use and allow exploration of the feasibility of data extraction across a wide range of clinical presentations. Furthermore, inclusion of a diverse range of wards and cases will support the development of the Fidelity Index quantifying the extent to which practice has adhered to ePAMS+ core principles.

Quantitative: patient participants

Individuals eligible for inclusion in the study will be aged ≥ 16 years, will have been admitted to hospital as a medical inpatient and will have an antibiotic order plan initiated or an existing antibiotic prescription flagged within the EPMA.

As ePAMS+ is a service-level intervention, all eligible admissions to participating wards in study sites will be included in quantitative analyses. Although patient informed consent from eligible admissions is not required or sought as part of this study, the implementation pack contains a patient information leaflet to help clinical staff explain the process of antibiotic use and review to patients. There is no mechanism to allow patients within participating wards from opting out of the collection and use of routine de-identified administrative data.

Quantitative: data collection

Under the ePAMS+ intervention, prescription of antibiotics automatically flags within the EPMA system to trigger decision aids and task lists for appropriate antibiotic management. Information on how order plans, prescribing interventions and review processes are managed within the Cerner EPMA system using the ePAMS+ tool is available in the supplementary information on the intervention summary (Additional file 1).

Data on EPMA interactions will be automatically logged within the data audit system of the Cerner EPMA system already in use at trial sites. A standardised data query to the system will be run by the local NHS Trust information services team. These queries will be run regularly within the data audit system (prior to activation of the ePAMS+ intervention and at intervals after implementation). We will extract two types of data: outcomes for quantitative analysis purposes, such as data contributing to calculation of total antibiotic use; and process measures (Table 3) to help understand how the ePAMS+ system is being used.

Table 3 Feasibility trial quantitative secondary outcomes and process measures
Full size table

Personal data will be processed as follows. Prior to data extraction from the Cerner EPMA system, a unique non-identifiable alias will be created for each record. The data extracted will not include any direct identifiers, but will include participant age at time of extract, diagnosis and ward of treatment, in addition to details of antibiotics prescribed and administered. Data extracted from participating NHS Trusts will be transferred via secure file transfer protocol (Serv-U FTP) to the National Safe Haven maintained by Public Health Scotland. Data controller/data controller information sharing agreements will be established between each site and University of Edinburgh (Sponsor).

All data will be held in a project-specific area in the National Safe Haven maintained by Public Health Scotland with access limited to named project researchers via a unique username and multi-factor authentication. All use will be subject to a user agreement covering responsibilities, access requirements, data security and processes for release of analytical output. The National Safe Haven will review all outputs to ensure these would not disclose the identity of any participant.

Quantitative: training and learning data

Site staff ePAMS+ training information will be captured on the Learning Management System to assess completion of training (i.e. professional discipline, date/time of module completion, time spent on learning and pre and post-test scores). The data will be anonymised by the ePAMS+ Champion for each site, prior to analysis by the research team.

Quantitative: outcomes

The primary feasibility trial outcomes will be (1) to determine the ability to derive total antibiotic consumption, measured as the number of defined daily doses (DDD) per admission, and to estimate the variability in this outcome; (2) to assess the feasibility of standardised queries to capture data from local Cerner EPMA system configurations and completeness of the data extracted; and (3) to assess the feasibility of measuring defined outcomes such as mortality at 30 days post-admission. DDD per admission and mortality at 30 days post-admission will be co-primary outcomes in the future full-scale trial. Table 3 outlines the secondary outcomes and process measures for which feasibility of extraction will be assessed.

As data will be extracted at intervals and the process of extraction is a key feasibility objective, there will not be scope to monitor the occurrence of adverse events in real time. All patients within participating sites will be managed according to best clinical practice and in line with local and national clinical guidelines.

Quantitative: statistical analysis

Descriptive summary statistics will be reported on antibiotic consumption, overall and by site. Separate summaries will also be provided for intravenous, oral, broad spectrum and narrow spectrum antibiotics.

Between-patient variability in total antibiotic consumption, measured as the number of DDD per admission, will be quantified using a normal linear model, including site and ward as factors, to estimate the components of variance. Log-transformation will be performed if necessary to satisfy the model assumptions. Factors for seasonal effects and implementation of the ePAMS+ intervention will also be considered (Additional file 4).

Other quantitative outcomes (Table 3) will be assessed according to two criteria. First, we will determine whether it is possible to derive each outcome using the information available in the EPMA data extract. Secondly, we will summarise the measures descriptively, overall and by site and by ward, with a particular focus on the rate of missing data.

Discussion

Strengths and limitations

This mixed-methods study, incorporating qualitative and quantitative elements, will assess feasibility of a trial evaluating the ePrescribing-based Anti-Microbial Stewardship (ePAMS+) intervention. In doing so it will inform refinements of ePAMS+ intervention and its future full-scale evaluation. The development of a Fidelity Index will also enable adherence to the ePAMS+ intervention to be assessed objectively.

The enrolment of just two study sites may limit generalisability, although inclusion of several ward types will ensure the trial covers a breadth of clinical contexts. Implementation of ePAMS+ within the Cerner ePrescribing and Medicines Administration system means feasibility in other systems will still need to be established in future extensions of this research.

Dissemination

Results will be shared with study sites and with participants in the qualitative research. Findings will be published in peer-reviewed journals and presented at academic conferences. Published results will not contain any personal data and will be in a form where individuals are not identified and re-identification is unlikely to take place.

Progression to larger-scale trial

It is essential to determine the feasibility of implementing the ePAMS+ intervention and measuring relevant outcomes, prior to evaluating its clinical and cost-effectiveness in a full scale hybrid cluster-randomised stepped-wedge clinical trial. At the end of the feasibility trial the investigators will meet to review and integrate the quantitative and qualitative findings.

Using the qualitative thematic analysis of key feasibility aspects, they will decide whether the results merit progression to the planned larger-scale effectiveness trial. Specifically, they will consider (1) user acceptability of the technical component of the ePAMS+ intervention; (2) use of the ePAMS+ Antibiotic Order Plans as intended; (3) user acceptability of the organisational and training components of the ePAMS+ intervention; and (4) confirmation of the acceptability and feasibility of the processes for implementing ePAMS+.They will also use the thematic analysis outputs to guide any specific modifications required to the intervention, documenting these by updating the relevant parts of the intervention specification outlined in the TIDieR checklist entries.

Important quantitative feasibility aspects the investigators will consider are (5) development of a Fidelity Index to quantify the extent to which ePAMS+ has been enacted; (6) confirmation that relevant outcome data may be obtained from ePrescribing systems; (7) measurement of between-patient variability to refine sample size calculation for the larger-scale effectiveness trial. Under items (5) and (6), for each outcome and for each data item required to develop the Fidelity Index, a binary success criterion will record whether it was possible to extract the outcome or data item. Where data were successfully extracted for an outcome measure, but the missing data rate was 10% or more, this will raise a further indicator of concern regarding feasibility. We will then undertake an in-depth review of the causes of missingness and their potential impact on interpretability of the outcome, before deciding on its suitability for inclusion in the larger-scale trial protocol. The final feasibility consideration, under item (7), will be whether the observed between-patient variability in the co-primary outcome of total antibiotic use leads to a sample size required for the future trial that would be realistic, given the number of available Cerner sites and the expected numbers of hospital admissions over the duration of the trial.