Abstract
Accurate and reliable guidelines for referral of children from resource-limited primary care settings are lacking. We identified three practicable paediatric severity scores (the Liverpool quick Sequential Organ Failure Assessment (LqSOFA), the quick Pediatric Logistic Organ Dysfunction-2, and the modified Systemic Inflammatory Response Syndrome) and externally validated their performance in young children presenting with acute respiratory infections (ARIs) to a primary care clinic located within a refugee camp on the Thailand-Myanmar border. This secondary analysis of data from a longitudinal birth cohort study consisted of 3010 ARI presentations in children aged ≤ 24 months. The primary outcome was receipt of supplemental oxygen. We externally validated the discrimination, calibration, and net-benefit of the scores, and quantified gains in performance that might be expected if they were deployed as simple clinical prediction models, and updated to include nutritional status and respiratory distress. 104/3,010 (3.5%) presentations met the primary outcome. The LqSOFA score demonstrated the best discrimination (AUC 0.84; 95% CI 0.79–0.89) and achieved a sensitivity and specificity > 0.80. Converting the scores into clinical prediction models improved performance, resulting in ~ 20% fewer unnecessary referrals and ~ 30–50% fewer children incorrectly managed in the community. The LqSOFA score is a promising triage tool for young children presenting with ARIs in resource-limited primary care settings. Where feasible, deploying the score as a simple clinical prediction model might enable more accurate and nuanced risk stratification, increasing applicability across a wider range of contexts.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Introduction
Acute respiratory infections (ARIs) are the leading reason for unscheduled childhood medical consultations worldwide1,2. Primary care workers function as gatekeepers to the formal health system, aiming to distinguish the minority of ARIs requiring onward referral from those suitable for community-based care3.
In rural regions of many low- and middle-income countries (LMICs) poorly functioning infrastructure, as well as geographic, climatic, socioeconomic, and cultural factors, can complicate referral mechanisms. Particularly in humanitarian and conflict settings referral can entail risks for both patients and providers4. Consequently, there can be substantial inter- and intra-health system variation in referral thresholds.
Existing tools to support community healthcare providers in their assessment of unwell children, such as the World Health Organization’s Integrated Management of Childhood Illnesses (IMCI) and Integrated Community Case Management (iCCM) guidelines, recommend certain ‘Danger Signs’ to guide referrals5,6. However, these lack sensitivity and specificity, and suffer from considerable interobserver variability7,8. A systematic review of paediatric triage tools concluded that none would be reliable in resource-constrained settings and that lack of follow-up data on children managed in the community rendered the validity of existing tools questionable9.
In this study we identified paediatric severity scores suitable for use in resource-limited primary care settings and externally validated their ability to guide referral of young children presenting with ARIs10. We characterised the improvement in performance that might be expected if the scores were deployed as simple clinical prediction models and updated to include variables relevant to children presenting with ARIs in rural LMIC settings.
Methods
Study population
Data were collected during a prospective birth cohort study at a medical clinic for refugees and internally displaced people on the Thailand-Myanmar border10. Between September 2007 and September 2008 pregnant women receiving antenatal care at the clinic were invited to participate. Children of consenting women were reviewed at birth and followed-up each month (routine visit) and during any intercurrent illness (illness visit) until 24 months of age. The local circumstances (inability of the population to move freely out of the camp and lack of other medical providers) contributed to low attrition rates and capture of the majority of acute illnesses for which care was sought.
All ARI illness visits were included in this secondary analysis. An ARI was defined as (A) a presentation with rhinorrhoea, nasal congestion, cough, respiratory distress (chest indrawing, nasal flaring, grunting, tracheal tug, and/or head bobbing), stridor, and/or abnormal lung auscultation (crepitations and/or wheeze), and (B) a compatible contemporaneous syndromic diagnosis (rhinitis, croup, bronchiolitis, influenza-like illness, pneumonia, viral infection, and/or wheeze) for children sent home directly from the clinic.
Identification and shortlisting of scores
Drawing on the results of two recent systematic reviews, we longlisted 16 severity scores that might risk stratify young children presenting from the community with ARIs (Supplementary Table 1)11,12. After considering reliability, validity, and feasibility for implementation we excluded eight scores that required specialist equipment and/or laboratory tests unlikely to be practical for the assessment of young children in busy LMIC primary care settings13,14,15,16,17,18,19,20. Four others were excluded as ≥ 25% of the constituent variables were unavailable in the primary dataset (Supplementary Table 2)21,22,23,24. Two of the remaining scores (the quick Sequential Organ Failure Assessment [qSOFA] and the quick Pediatric Logistic Organ Dysfunction-2 [qPELOD-2]) contained blood pressure25,26. Hypotension is a late sign in paediatric sepsis and not suitable for early recognition of impending serious illness at the community level27. Furthermore, accurate use and maintenance of sphygmomanometers and stethoscopes may not be feasible in resource-limited settings28. Recently, Romaine et al. replaced systolic blood pressure (SBP) with alternate signs of circulatory compromise (heart rate and capillary refill time) to develop the Liverpool-qSOFA (LqSOFA) score, and demonstrated superior performance compared to qSOFA in febrile children presenting from the community29. Hence, we elected to evaluate the LqSOFA score in preference to qSOFA and to evaluate an adapted qPELOD-2 score (replacing SBP with capillary refill time and assessing mental status using the simpler Alert Voice Pain Unresponsive [AVPU] scale rather than the Glasgow Coma Scale [GCS]). The three scores shortlisted for evaluation were the LqSOFA, qPELOD-2, and modified Systemic Inflammatory Response Syndrome (mSIRS) scores (Table 1)26,29,30.
Selection of variables for model updating
To update and improve model performance additional predictors relevant for children presenting with ARIs in LMIC primary care settings were considered for inclusion. Nutritional status (weight-for-age z-score [WAZ]) and presence of respiratory distress were selected a priori, after considering resource constraints, reliability, validity, biological plausibility, availability of data in the primary dataset, and sample size (Supplementary Table 3)28.
Data collection
All data were measured by study staff and entered onto structured case report forms. With the exception of anthropometric data, all clinical data were collected at the time of presentation. Core (rectal) temperature was measured for neonates and infants and adjusted to axillary temperature by subtracting 0.5 °C6. Mental status was assessed using the AVPU scale. Capillary refill time was measured following the release of gentle pressure on the child’s sternum. For children admitted to the clinic, weight was measured at the time of presentation (seca scale; precision ± 5 g for neonates or ± 50 g after birth). In addition, all children had their mid-upper arm circumference (MUAC), weight, and height measured at each monthly routine visit. For the purposes of these analyses, age-adjusted z-scores (R package: z scorer)31 were calculated using the closest anthropometric data to the illness visit within the following window periods: height ≤ 28 days; MUAC ≤ 28 days without intervening admission; weight ≤ 14 days without intervening admission. Median time between the index illness visit and each anthropometric measurement is reported. Participants were followed-up each day during admission to the clinic and at monthly routine visits conducted as part of the longitudinal birth cohort study.
Primary outcome
The primary outcome was receipt of supplemental oxygen at any time during the illness visit. Study staff were unaware which baseline variables were to be used as candidate predictors at the time of ascertaining outcome status. Clinic treatment protocols specified that peripheral oxygen saturation (SpO2) must be checked prior to initiation of supplemental oxygen, with therapy only indicated if SpO2 was < 90%. Elevation within the camps ranged from 200 to 1000 m and adjustment of SpO2 readings for altitude was not required32. All staff had either undergone formal nurse training in Myanmar before being displaced to the camp or had undergone a 6-month training programme in the camp (run by Médecins Sans Frontières). All had more than a year’s clinical experience in the camp and were trained on the clinic treatment protocols prior to study commencement.
Missing data
616 presentations were missing data on one or more candidate predictors (616/3010; 20.5%) with capillary refill time containing the highest proportion of missingness (442/3,010; 14.7%; Supplementary Table 4). Under a missing-at-random assumption (Supplementary Fig. 1; Supplementary Table 5), we used multiple imputation with chained equations (MICE) to deal with missing data (R package: mice)33. Analyses were done in each of 100 imputed datasets and results pooled. Variables included in the imputation model are reported in Supplementary Table 6.
Statistical methods
We assessed discrimination and calibration of each score by quantifying the area under the receiver operating characteristic curve (AUC) and plotting the observed proportion of participants that met the primary outcome at each level of a score. We examined predicted classifications at each of the scores’ cut-offs.
Prior to model building we explored the relationship between continuous predictors and the primary outcome using locally-weighted scatterplot smoothing (LOWESS) to identify non-linear patterns34. Accordingly, temperature was modelled using restricted cubic splines (R package: rms)35 with three knots placed at locations based on percentiles (5th and 95th) and recognised physiological thresholds (36 °C)36,37. We used logistic regression to derive the models and tested for important interactions using likelihood ratio tests (LRT). Although a number of children presented more than once during the study period, mixed-effects models accounting for repeat presenters failed to converge due to a substantial proportion of children presenting only once (22%; 169/756), hence random-effects were not modelled. Sensitivity analyses restricting the analysis to one ARI presentation per child indicated that this is unlikely to have had an impact on the findings (Supplementary Table 7). All predictors were prespecified and no predictor selection was performed during model development. Internal validation was performed using 100 bootstrap samples with replacement and optimism-adjusted discrimination and calibration reported (R package: rms)35.
Finally, the models were updated by including respiratory distress and WAZ as additional candidate predictors. Penalised (lasso) logistic regression was used for model updating, variable selection, and shrinkage to minimise overfitting (R package: glmnet)38. A sensitivity analysis confirmed that median imputation grouped by outcome status produced similar results to MICE and hence to avoid conflicts in variable selection across multiply imputed datasets we used this approach to address missing data for model updating (Supplementary Table 8). We assessed discrimination and calibration of the updated models, examined predicted classifications at clinically-relevant referral thresholds, and compared their clinical utility (net-benefit) to the best-performing points-based severity score using decision curve analysis (R package: dcurves)39. A sensitivity analysis was performed excluding children who were hypoxaemic at the time of presentation.
All analyses were done in R, version 4.0.240.
Sample size
No formal sample size calculation for external validation of the existing severity scores was performed. All available data were used to maximise power and generalisability. Of the 3010 eligible ARI presentations, 104 met the primary outcome, ensuring sufficient outcome events for a robust external validation41. For derivation and updating of the clinical prediction models we followed the methods of Riley et al. and assumed a conservative R2 Nagelkerke of 0.1542. At an outcome prevalence of 3.5% (104/3010) we estimated that up to 13 candidate predictors (events per parameter [EPP] = 8) could be used to build the prediction models whilst minimising the risk of overfitting (R package: pmsampsize)43.
Ethics and reporting
Ethical approvals were provided by the Mahidol University Ethics Committee (TMEC 21-023) and Oxford Tropical Research Ethics Committee (OxTREC 511-21). Informed consent was obtained from the legal guardians of all participants. The study is reported in accordance with the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines (Supplementary Table 9)44.
Results
From September 2007 to September 2008, 999 pregnant women were enrolled, with 965 children born into the cohort. Amongst 4061 acute illness presentations, 3064 were for ARIs. Fifty-four ARI presentations were excluded as information on oxygen therapy was not available in the study database, leaving 3010 presentations from 756 individual children for the primary analysis (Supplementary Fig. 2).
Baseline characteristics of the cohort are summarised (Table 2; Supplementary Table 10). The majority of children were managed in the community (72.3%; 2175/3010). Median length of stay for the 835 admissions was 3 days (IQR 2–4 days). One hundred and four (3.5%; 104/3010) presentations received supplemental oxygen during their illness visit (met the primary outcome), with those with signs of respiratory distress, age-adjusted tachycardia and/or tachypnoea, lower baseline SpO2, prolonged capillary refill times, altered mental status, and lower WAZ more likely to require supplemental oxygen (p < 0.001 to 0.014; Table 2). There was one death: a child who was admitted and received supplemental oxygen.
LqSOFA and qPELOD-2 scores outperform the mSIRS score for risk stratification of ARIs
Discrimination and calibration of the LqSOFA (AUC = 0.84; 95% confidence interval [CI] = 0.79 to 0.89) and qPELOD-2 (AUC = 0.79; 95% CI = 0.74 to 0.84) scores were considerably better than the mSIRS score (AUC = 0.57; 95% CI = 0.51 to 0.63; Fig. 1; Supplementary Table 11; Supplementary Fig. 3). At a cut-off of ≥ 1 the LqSOFA score demonstrated a sensitivity of 0.80 (95% CI = 0.72 to 0.89) and specificity of 0.86 (95% CI = 0.85 to 0.88); neither the mSIRS nor qPELOD-2 scores achieved a sensitivity and specificity > 0.70 at any cut-off (Table 3).
Improved performance of clinical severity scores when deployed as clinical prediction models
Relationships between continuous predictors and the primary outcome are illustrated (Supplementary Fig. 4). There was no evidence of interaction between heart rate (LRT = 2.09; p = 0.35) or respiratory rate (LRT = 0.77; p = 0.68) and age. Optimism-adjusted discrimination of the three models ranged from 0.81 to 0.90, with the LqSOFA model appearing most promising (AUC = 0.90; 95% CI = 0.86 to 0.94; Fig. 2; Supplementary Fig. 5). Calibration of the qPELOD-2 model was good. The LqSOFA and mSIRS models overestimated risk at higher predicted probabilities.
Discrimination of all three updated models containing respiratory distress and WAZ improved (AUCs = 0.93 to 0.95). Notably, improvements were more substantial for the qPELOD-2 and mSIRS models, compared to the LqSOFA model, which already had comparably high discrimination prior to inclusion of the additional variables. Calibration of the updated LqSOFA and qPELOD-2 models was good, whereas the updated mSIRS model underestimated risk at higher predicted probabilities (Fig. 3). The full models are reported in Supplementary Table 12.
Promising clinical utility of the LqSOFA and qPELOD-2 models to guide referrals from primary care
We recognised that the relative value of correct and incorrect referrals is highly context-dependent, reflecting resource availability, practicalities of referral, and capacity for follow-up. Decision curve analyses accounting for differing circumstances suggest that the updated models could provide greater utility (net-benefit) compared to the best points-based score (the LqSOFA score), with the LqSOFA and qPELOD-2 models appearing most promising over a wide range of plausible referral thresholds (Fig. 4).
The ability of each updated model to guide referrals at thresholds ranging from 1 to 40% is shown (Table 4). A referral threshold of 5% reflects a strategy whereby any child with a predicted probability of requiring oxygen ≥ 5% is referred. At this cut off, the models would suggest referral in ~ 15% of all presentations, correctly identifying ~ 86 to 87% of children requiring referral, at a cost of also recommending referral in ~ 12 to 13% of children not requiring referral; i.e., a number needed to refer (NNR; the number of children referred to identify one child who would require oxygen) of five. In contrast, at a similar threshold the LqSOFA score using a cut-off ≥ 1 would suggest referral in a similar proportion of presentations but result in a ~ 25% increase in incorrect referrals (a NNR of six) and a ~ 25-30% increase in the number of children incorrectly identified as safe for community-based management (a ratio of correct to incorrect cases managed in the community of 171 to 193:1 vs. 131:1).
Sensitivity analysis
The WHO recommend that pulse oximetry should be universally available at first-level health facilities6,45. Although many barriers exist to realising this laudable goal, to account for the fact that in such contexts a severity score would not be required to guide referral for children who are already hypoxaemic at the time of presentation, we performed a sensitivity analysis excluding attendances with SpO2 < 90% at presentation. Discrimination remained comparable but clinical utility of the models reduced slightly, with higher NNRs at the lowest referral thresholds (Supplementary Tables 13 and 14).
Discussion
We report the external validation of three pre-existing severity scores amongst young children presenting with ARIs to a medical clinic on the Thailand-Myanmar border. Unlike other studies which investigated the scores’ prognostic accuracy in hospital settings17,25, we evaluated their performance at the community level and demonstrate that the LqSOFA and qPELOD-2 scores could support early recognition of children requiring referral or closer follow-up in primary care settings with limited resources. In keeping with previous literature, we found that the mSIRS score was poorly discriminative, not well calibrated, and led to substantial misclassification17.
An LqSOFA score ≥ 1 yielded a sensitivity and specificity > 80%. Encouragingly, this is remarkably consistent with the performance reported in the original LqSOFA development study and may reflect similarities in the use-case (febrile children presenting from the community) and severity of the cohorts (outcome prevalence 1.1% vs. 3.5%; admission rate 12.1% vs. 27.7%), albeit despite obvious demographic differences29. In contrast to qPELOD-2, LqSOFA contains age-adjusted tachypnoea, which may have improved performance in children with respiratory illnesses. Furthermore, the performance of LqSOFA (or qSOFA) has been shown to improve outside of the PICU, when predicting more proximal outcomes (e.g., critical care admission rather than mortality), and if the AVPU scale (vs. GCS) is used to assess mental status46. These all apply to our cohort.
We demonstrated improvement in performance when the severity scores were deployed as clinical prediction models and when nutritional status and respiratory distress were included as additional predictors. Whilst discrimination of all three updated models was good, the AUC is a summary measure of model performance and does not necessarily reflect clinical utility47,48,49. Decision curve analyses illustrate the superiority of the LqSOFA and qPELOD-2 models compared with the mSIRS model across a range of clinically-relevant referral thresholds.
With growing access to smartphones there may be contexts where the increased accuracy afforded by a clinical prediction model outweighs the simplicity and practicality of points-based scoring systems. At a 5% referral threshold, the updated LqSOFA model identified a similar proportion of presentations for referral as the LqSOFA score at a cut-off of ≥ 1 (14.1% vs. 16.1%), however use of the model would have resulted in ~ 20% fewer incorrect referrals and a ~ 30% decrease in the number of presentations incorrectly recommended for community-based management. In addition to greater accuracy, prediction models permit more nuanced evaluation of risk; referral thresholds can be adjusted to the needs of an individual patient and/or health system and this flexibility may be particularly impactful in the heterogeneous environments commonplace in many LMIC primary care contexts. For example, in locations where community follow-up is feasible (e.g., via a telephone call or return clinic visit) and/or referral carries great cost (to the patient or system), a higher referral threshold (lower NNR) may be acceptable, compared with settings where safety-netting is impractical and/or access to secondary care is less challenging.
We followed the latest guidelines in prediction model building and used bootstrap internal validation, penalised regression, placed knots at predefined locations, and limited the number of candidate predictors to avoid overfitting the models42,44,50,51. Nevertheless, they require validation on new data to assess generalisability and provide a fairer comparison with the pre-existing points-based scores. We have published our full models to encourage independent validation.
As others have highlighted, a limitation of many studies evaluating community-based triage tools in resource-limited settings is the lack of follow-up data for patients categorised as low risk9; 72.3% (2175/3010) of our cohort were sent away from the clinic without admission. As acute illness visits were nested within the longitudinal birth cohort, we were able to confirm that 1.4% (30/2083) of presentations sent away from the clinic without admission received supplemental oxygen within the next 28 days, although it is unknown whether this related to the index ARI or a new illness. A sensitivity analysis conservatively classifying these 30 presentations as meeting the primary outcome (i.e., assuming the oxygen therapy related to the index ARI) resulted in a decrease in the sensitivity of all three models (Supplementary Tables 15 and 16). Prospective research with dedicated outpatient follow-up is ongoing to investigate this issue further52.
We selected supplemental oxygen therapy as the primary outcome as this reflects a clinically-meaningful endpoint for ARIs and a pragmatic referral threshold for many resource-limited primary care settings. Although confirmed hypoxaemia would have been a more robust endpoint, oxygen saturations prior to starting oxygen therapy were not documented in the study database. However, oxygen was a scarce resource during the study (cylinders were transported in each week from ~ 60 km away) and oxygen therapy was protocolised (only indicated if SpO2 < 90%); hence outcome misclassification is less likely with the primary outcome reflecting a reliable surrogate for hypoxaemia rather than a subjective decision by a health worker to provide supplemental oxygen.
For those who met the primary outcome, the time of oxygen initiation was not available in the primary dataset. Although no patient had met the outcome when baseline predictors were measured, some may have done so shortly after. Nevertheless, the sensitivity analysis excluding presentations with baseline SpO2 < 90% (the qualifying criterion for supplemental oxygen) produced similar results.
Finally, due to the proportion of children who presented only once, we were unable to include a random-effect term in the models. Although this may have biased findings towards those children who presented more frequently, the sensitivity analyses restricted to a single presentation per child indicate that this is unlikely to be the case.
We externally validated three severity scores that could guide assessment of young children presenting with ARIs in resource-limited primary care settings (particularly where pulse oximetry is not readily available) to identify those in need of referral or closer follow-up. Performance of the LqSOFA score was encouraging and comparable to that in the original derivation setting29. Converting the LqSOFA score into a clinical prediction model and including additional variables relevant to resource-constrained LMIC settings improved accuracy and might permit application across a wider range of contexts with differing referral thresholds.
Data availability
De-identified, individual participant data from this study will be available to researchers whose proposed purpose of use is approved by the data access committees at the Mahidol-Oxford Tropical Medicine Research Unit. Inquiries or requests for the data may be sent to datasharing@tropmedres.ac.
References
Bigio, J. et al. Most common reasons for primary care visits in low- and middle-income countries: A systematic review. PLoS Glob. Public Health 2(5), e0000196 (2022).
Finley, C. R. et al. What are the most common conditions in primary care?. Can. Fam. Phys. 64(11), 832–840 (2018).
Buntinx, F., Mant, D., Van den Bruel, A., Donner-Banzhof, N. & Dinant, G. J. Dealing with low-incidence serious diseases in general practice. Br. J. Gen. Pract. 61(582), 43–46 (2011).
Debarre, A. Hard to Reach: Providing Healthcare in Armed Conflict (International Peace Institute, New York, 2018).
World Health Organization. Integrated Community Case Management. Geneva, Switzerland (2012).
World Health Organization. Integrated Management of Childhood Illnesses. Geneva, Switzerland (2014).
Keitel, K., Kilowoko, M., Kyungu, E., Genton, B. & D’Acremont, V. Performance of prediction rules and guidelines in detecting serious bacterial infections among Tanzanian febrile children. BMC Infect. Dis. 19(1), 769 (2019).
Izudi, J., Anyigu, S. & Ndungutse, D. Adherence to integrated management of childhood illnesses guideline in treating south Sudanese children with cough or difficulty in breathing. Int. J. Pediatr. 2017: 5173416.
Hansoti, B. et al. Reliability and validity of pediatric triage tools evaluated in Low resource settings: A systematic review. BMC Pediatr. 17(1), 37 (2017).
Turner, C. et al. High rates of pneumonia in children under two years of age in a South East Asian refugee population. PLoS ONE 8(1), e54026 (2013).
Chandna, A. et al. Predictors of disease severity in children presenting from the community with febrile illnesses: A systematic review of prognostic studies. BMJ Glob. Health 6(1), e003451 (2021).
Deardorff, K. V., McCollum, E. D. & Ginsburg, A. S. Pneumonia risk stratification scores for children in low-resource settings: A systematic literature review. Pediatr. Infect. Dis. J. 37(8), 743–748 (2018).
Olson, D. et al. Development of a severity of illness scoring system (inpatient triage, assessment and treatment) for resource-constrained hospitals in developing countries. Trop. Med. Int. Health 18(7), 871–878 (2013).
Hooli, S. et al. Predicting hospitalised paediatric pneumonia mortality risk: An external validation of RISC and mRISC, and local tool development (RISC-Malawi) from Malawi. PLoS ONE 11(12), e0168126 (2016).
Reed, C. et al. Development of the Respiratory Index of Severity in Children (RISC) score among young children with respiratory infections in South Africa. PLoS ONE 7(1), e27793 (2012).
Pollack, M. M. et al. The pediatric risk of mortality score: Update 2015. Pediatr. Crit. Care Med. 17(1), 2–9 (2016).
van Nassau, S. C. et al. Translating Sepsis-3 criteria in children: prognostic accuracy of age-adjusted quick SOFA score in children visiting the emergency department with suspected bacterial infection. Front. Pediatr. 6, 266 (2018).
Goldstein, B., Giroir, B. & Randolph, A. International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics. Pediatr. Crit. Care Med. 6(1), 2–8 (2005).
Egdell, P., Finlay, L. & Pedley, D. K. The PAWS score: Validation of an early warning scoring system for the initial assessment of children in the emergency department. Emerg. Med. J. 25(11), 745–749 (2008).
Parshuram, C. S., Hutchison, J. & Middaugh, K. Development and initial validation of the Bedside Paediatric Early Warning System score. Crit. Care 13(4), R135 (2009).
George, E. C. et al. Predicting mortality in sick African children: The FEAST paediatric emergency triage (PET) score. BMC Med. 13, 174 (2015).
Emukule, G. O. et al. Predicting mortality among hospitalized children with respiratory illness in Western Kenya, 2009–2012. PLoS ONE 9(3), e92968 (2014).
Berkley, J. A. et al. Prognostic indicators of early and late death in children admitted to district hospital in Kenya: Cohort study. BMJ 326, 361 (2003).
Helbok, R. et al. The Lambarene Organ Dysfunction Score (LODS) is a simple clinical predictor of fatal malaria in African children. J. Infect. Dis. 200(12), 1834–1841 (2009).
Schlapbach, L. J., Straney, L., Bellomo, R., MacLaren, G. & Pilcher, D. Prognostic accuracy of age-adapted SOFA, SIRS, PELOD-2, and qSOFA for in-hospital mortality among children with suspected infection admitted to the intensive care unit. Intensive Care Med. 44(2), 179–188 (2018).
Leclerc, F. et al. Can the pediatric logistic organ dysfunction-2 score on day 1 be used in clinical criteria for sepsis in children?. Pediatr. Crit. Care Med. 18(8), 758–763 (2017).
Brierley, J. et al. Clinical practice parameters for hemodynamic support of pediatric and neonatal septic shock: 2007 update from the American College of Critical Care Medicine. Crit. Care Med. 37(2), 666–688 (2009).
Fung, J. S. T. et al. Determining predictors of sepsis at triage among children under 5 years of age in resource-limited settings: A modified Delphi process. PLoS ONE 14(1), e0211274 (2019).
Romaine, S. T. et al. Accuracy of a modified qSOFA score for predicting critical care admission in febrile children. Pediatrics 146(4), e20200782 (2020).
Beane, A. et al. Comparison of quick sequential organ failure assessment and modified systemic inflammatory response syndrome criteria in a lower middle income setting. J. Acute Med. 7(4), 141–148 (2017).
Myatt, M. & Guevarra, E. zscorer: Child anthropometry z-score calculator. R package version 0.3.1. (2019).
Rojas-Camayo, J. et al. Reference values for oxygen saturation from sea level to the highest human habitation in the Andes in acclimatised persons. Thorax 73(8), 776–778 (2017).
van Buuren, S. & Groothuis-Oudshoorn, K. mice: Multivariate imputation by chained equations in R. J. Stat. Softw. 45(3), 1–67 (2011).
Loader, C. Smoothing: Local regression techniques. In Handbook of Computational Statistics: Concepts and Methods (eds Gentle, J. E. et al.) 571–596 (Springer, Berlin, 2012).
Harrell, F. E., Jr. rms: Regression modeling strategies. R package version 6.2-0. (2021).
UK National Institute for Health and Care Excellence. Algorithm for managing suspected sepsis in children aged under 5 years outside an acute hospital setting. United Kingdom (2017).
World Health Organization. Pocket Book of Hospital Care for Children: Guidelines for the Management of Common Childhood Illnesses. Geneva, Switzerland (2013).
Friedman, J., Hastie, T. & Tibshirani, R. Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 33(1), 1–22 (2010).
Sjoberg, D.D. dcurves: Decision Curve Analysis for Model Evaluation. R package version 0.3.0. (2022).
R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing (2020).
Vergouwe, Y., Steyerberg, E. W., Eijkemans, M. J. & Habbema, J. D. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J. Clin. Epidemiol. 58(5), 475–483 (2005).
Riley, R. D. et al. Calculating the sample size required for developing a clinical prediction model. BMJ 368, m441 (2020).
Ensor, J., Martin, E.C. & Riley, R.D. pmsampsize: Calculates the Minimum Sample Size Required for Developing a Multivariable Prediction Model. R package version 1.1.1. R (2021).
Collins, G. S., Reitsma, J. B., Altman, D. G. & Moons, K. G. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. Ann. Intern. Med. 162(1), 55–63 (2015).
World Health Organization. Interagency List of Priority Medical Devices for Essential Interventions for Reproductive, Maternal, Newborn and Child Health. Geneva, Switzerland (2016).
Eun, S. et al. Age-adjusted quick sequential organ failure assessment score for predicting mortality and disease severity in children with infection: A systematic review and meta-analysis. Sci. Rep. 11(1), 21699 (2021).
Fackler, J. C., Rehman, M. & Winslow, R. L. Please welcome the new team member: The algorithm. Pediatr. Crit. Care Med. 20(12), 1200–1201 (2019).
Vickers, A. J., Van Calster, B. & Steyerberg, E. W. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ 352, i6 (2016).
de Hond, A. A. H., Steyerberg, E. W. & van Calster, B. Interpreting area under the receiver operating characteristic curve. Lancet Digit. Health 4(12), e853–e855 (2022).
Steyerberg, E. W. Clinical Prediction Models (Springer, Berlin, 2019).
Harrell, F. E. Jr. Regression Modeling Strategies (Springer, Berlin, 2006).
Chandna, A. et al. Prediction of disease severity in young children presenting with acute febrile illness in resource-limited settings: A protocol for a prospective observational study. BMJ Open 11(1), e045826 (2021).
Acknowledgements
This research was funded by the UK Wellcome Trust (219644/Z/19/Z). RPS acknowledges part support from the NIHR Applied Research Collaboration Oxford and Thames Valley, the NIHR Oxford Medtech and In-Vitro Diagnostics Co-operative and the Oxford Martin School. CK is supported by a Wellcome Trust/Royal Society Sir Henry Dale Fellowship (211182/Z/18/Z). For the purpose of open access, the author has applied a CC BY public copyright license to any Author Accepted Manuscript version arising from this submission.
Author information
Authors and Affiliations
Contributions
A.C., C.T., and P.T. conceptualized the study. A.C. shortlisted the existing severity scores. C.T. and P.T. collected the primary data. A.C. curated and cleaned the data. A.C. conducted the analysis under the supervision of L.M., C.K., and R.P.S. A.C. wrote the original draft of the manuscript. A.C., L.M., C.K., D.L., F.N., Y.L., R.P.S., C.T., and P.T. reviewed, edited, and approved the manuscript. A.C. verified the underlying data.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Chandna, A., Mwandigha, L., Koshiaris, C. et al. External validation of clinical severity scores to guide referral of paediatric acute respiratory infections in resource-limited primary care settings. Sci Rep 13, 19026 (2023). https://doi.org/10.1038/s41598-023-45746-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-023-45746-4
- Springer Nature Limited