Introduction

Urinary tract infection (UTI), encompassing a spectrum of infectious syndromes that affect the urinary tract anywhere from the urethra to the kidneys, is one of the most common community- or hospital-acquired infections [1, 2]. Approximately 60% of women are estimated to experience at least one UTI during their lifetime [3]. Furthermore, UTIs have a high recurrence rate ranging from 30 to 50% in women and 12% in men. UTI treatment and management cause a large burden on social healthcare, although the morbidity rate is low [3].

The major microorganisms causing UTIs are bacteria (i.e., Escherichia coli, Klebsiella pneumoniae, and Proteus mirabilis), and some fungi and viruses can also cause UTIs, especially in patients with indwelling catheters, diabetes, recent antibiotic use, or immunosuppression [2, 4, 5]. Precise and rapid pathogen identification is key in UTI treatment. The combination of urinary symptoms and urine culture is the “gold standard” for UTI diagnosis; however, the frustration from culture, including low sensitivity, long testing period, and inconsistent diagnostic threshold, greatly promotes the development of novel diagnostic strategies [2, 6, 7]. A culture-independent tool, metagenomic next-generation sequencing (mNGS), is an unbiased pathogen detection method that can characterize all DNA or RNA sequences and analyze the entire microbiome, human host genome, and transcriptome in clinical samples. Its advantages, such as high throughput, wide coverage, high accuracy, and efficiency, make it attractive for clinical use [8, 9]. Previous studies have systematically evaluated the clinical and diagnostic value of mNGS in different samples from various clinical scenarios, including blood, bronchoalveolar lavage fluid, cerebrospinal fluid, and synovial fluid [10,11,12,13]. In UTIs, mNGS circumvents some of the limitations of conventional urine culture, such as relatively narrow detectable microorganism spectrum and long detection period [14, 15]. mNGS has preliminarily demonstrated potential for diagnosing UTIs with promising efficacy, especially for identifying pathogens that are not cultivable with conventional urine culture [7, 16]. However, an integrated conclusion of mNGS in UTIs has not been reached. Herein, we performed this systematic review and meta-analysis to explore the clinical and diagnostic value of mNGS for detecting UTIs.

Methods

This is a systematic review and meta-analysis following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guidelines [17]. The protocol has been registered in the International Prospective Register of Systematic Reviews (PROSPERO: CRD42024526048. More details can be accessed via https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=526048).

Search strategy and selection criteria

Two researchers (SH and HL) independently performed comprehensive searches in PubMed, Web of Science, Embase, and the Cochrane Library from inception to March 11, 2024. The complete search terms used were as follows: (mNGS OR metagenomic next-generation sequencing OR metagenomic next-generation sequencing OR metagenomic sequencing OR shotgun metagenomic OR next-generation sequencing OR next-generation sequencing OR NGS) AND (urinary tract infection OR UTI OR urinary tract infections OR UTIs OR infection of urinary tract OR infections of urinary tract). The language of the included studies was limited to English.

The inclusion criteria were as follows: (i) were case-control studies or retrospective and prospective cohort studies that reported the clinical value of mNGS for detecting UTIs, including both its diagnostic performance and impact on prognosis; (ii) included a study population consisting of at least 10 patients with UTIs; and (iii) used reference standards such as conventional microbiological tests or clinical diagnosis based on clinical signs/symptoms and laboratory examinations.

The exclusion criteria were as follows: (i) a case report or case series consisting of fewer than 10 participants, a narrative review, a systematic review, and conference abstracts that duplicate original research findings; (ii) studies that did not report direct outcomes; and (iii) studies that did not state the reference standards.

Study outcomes

The primary outcomes included the positive detection rate and diagnostic performance of mNGS. The secondary outcome was the effectiveness of treatment (e.g., antibiotic adjustment and length of hospitalization).

Data extraction

The following data from the individual studies were extracted: (a) basic information, including the first author, publication year, area, type of study, and sample size; (b) the methodological quality, the reference standard for diagnosis of UTI, and the criteria for a positive mNGS result; (c) mNGS sequencing technology, mNGS sequencing method, and mNGS sequencing conditions (sequencing platform, DNA/RNA extraction, and bioinformatics analysis); (d) the positive detection rate (PDR) and diagnostic accuracy measurements (true positive [TP], false positive [FP], false negative [FN], true negative [TN]); and (d) the value of mNGS in treatment. Two researchers (HS and SG) independently extracted the data, and disagreements between the two reviewers were resolved by consensus with the third author (ZH).

Study quality evaluation and appraisal of evidence uncertainty

Three researchers (HS, GS, and JZ) independently performed a quality assessment of all included studies, and disagreements were resolved by discussion. The quality of the studies was measured by using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS)-2 tool [18] (via Review Manager version 5.3, The Nordic Cochrane Centre, The Cochrane Collaboration and Copenhagen, 2014). Then, the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) system was used to measure the certainty of evidence (CoE) (GRADEpro GDT software, https://www.gradepro.org) [19].

Statistical analysis

Statistical analysis was performed using Stata version 16.0, Review Manager version 5.3, and MetaDiSc version 1.4. For dichotomous variables, odds ratios (ORs) were calculated with the Review Manager. The diagnostic sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR) were calculated by a random-effects model or a fixed-effects model by Stata and MetaDiSc, and forest plots were also drawn with two software programs. The heterogeneity among the studies was assessed using the chi-square test and the I2 statistic. Summary receiver operating characteristic (sROC) curves were also plotted for studies reporting sensitivity and specificity, and the results were calculated. To analyze the potential factors that may affect heterogeneity, subgroup analyses, leave-one-out analyses, and meta-regressions were performed according to the continent, study direction, study type, reference standard, and basic condition of the patients. A two-tailed p-value of < 0.05 was considered to indicate statistical significance. P < 0.10 or I2 > 50% was considered statistically significant for heterogeneity. Significant heterogeneity was pooled using a random-effects model.

Results

Search results and study selection

A total of 1182 records were collected from online databases, and 864 were screened after 324 duplicates were removed. A total of 801 records were excluded based on title and abstract. Then, 57 records were further excluded by full-text review. Finally, 17 studies were selected for the meta-analysis [20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36]. The study selection was performed following the PRISMA flow diagram (Fig. 1).

Fig. 1
figure 1

Flow chart of study retrieval

Full size image

Characteristics of the studies

Among the 17 included studies, nine were prospective, and eight were retrospective. The sample size ranged from 10 to 1200. Culture was the reference standard in 12 studies [20,21,22,23,24,25,26, 30, 31, 33,34,35], clinical diagnosis was the reference standard in three studies [27, 29, 36], and two studies used both methods [28, 32]. Urine samples were used in 17 studies, and blood samples were tested in one study [26]. For the main outcomes, the PDR was reported by 17 studies, the sensitivity was established by 13 studies [21, 22, 24,25,26,27,28,29, 31, 32, 34,35,36], and the specificity was provided by 10 studies [21, 24, 25, 27,28,29, 31, 32, 35, 36]. Duan et al. and Jia et al. reported ARA rates [23, 28]. More information is shown in Table 1.

Table 1 Characteristics of eligible studies in the meta-analysis
Full size table

Study quality assessment

The study quality was estimated by the QUADAS-2 tool, and the detailed information is shown in Fig. 2. Most of the studies had a high risk of bias and low applicability concerns. The description of index tests and reference standards contributes most to risk bias, and patient selection and reference standards also impact clinical applicability concerns.

Fig. 2
figure 2

Quality assessment of the included studies by the QUADAS-2 tool

Full size image

Positive pathogen detection rate of mNGS versus culture

A total of 17 studies involving 3367 samples measured the PDR of mNGS versus (vs.) culture. The overall PDR of mNGS in urine was 55.46% (1812/3267), which was significantly greater than that of culture (1275/3267, 39.03%) (Fig. 3 and Fig. S1). The meta-analysis revealed that mNGS had greater PDR than did culture (OR = 2.87, 95% confidence interval [CI]: 1.72–4.81, p < 0.001, I2 = 90%). The funnel plot did not show obvious publication bias (p = 0.57) (Fig. S2). GRADE score indicated that the CoE was low (Fig. S3). Considering the pathogen types, three studies calculated the PDR of bacteria and fungi, and the PDR of mNGS was greater than that of culture for detecting bacterial infection (OR = 1.73, 95% CI: 1.48–2.02, p < 0.001, I2 = 88%), but the PDR of fungal infection was not significantly greater for mNGS than for culture (OR = 3.49, 95% CI: 0.71–17.15, p = 0.12, I2 = 59%) (Fig. S4A and B) [23, 28, 29].

Fig. 3
figure 3

Comparison of pathogen detection positive rate between mNGS and culture in urine

Full size image

Diagnostic performance of mNGS

The sensitivity of mNGS for diagnosing UTIs ranged from 0.81 to 1.00, and the pooled sensitivity was 0.89 (95% CI: 0.86–0.91, I2 = 39.65%, p = 0.06) (Fig. 4A). The specificity ranged from 0.14 to 1.00, and the pooled specificity was 0.75 (95% CI: 0.51–0.90, I2 = 88.64%, p < 0.001) (Fig. 4B). The area under the curve of the sROC curve (AUROC) was 0.89 (95% CI: 0.86–0.92) (Fig. 5). Deeks’ funnel plot asymmetry test did not reveal significant publication bias (Fig. 6, p = 0.90). GRADE score indicated that the CoE was very low (Fig. S5). With respect to the reference standard, the pooled sensitivity was 0.89 (95% CI: 0.85–0.92, I2 = 45.06%, p = 0.12), and the pooled specificity was 0.73 (95% CI: 0.34–0.94, I2 = 86.11%, p < 0.001) when the clinical diagnosis was used as the reference (Fig. S6). The pooled sensitivity was 0.88 (95% CI: 0.83–0.91, I2 = 0.00%, p = 0.52), and the pooled specificity was 0.76 (95% CI: 0.44–0.93, I2 = 94.95%, p < 0.001) when culture was used as the reference (Fig. S7). For both references, the publication bias was not significant (p = 0.52 [clinical diagnosis], p = 0.12 [culture]) (Fig. S8 and S9). The AUROCs of mNGS according to clinical diagnosis and culture are shown in Fig. S10. The PLR, NLR, and DOR of mNGS were also elaborated (Fig. S11-S14). Taken together, the main results of the diagnostic performance of mNGS for UTIs are listed in Table 2.

Fig. 4
figure 4

Forest plot of pooled sensitivity (A) and specificity (B) of mNGS in UTIs

Full size image
Fig. 5
figure 5

The sROC of mNGS diagnosis in UTIs

Full size image
Fig. 6
figure 6

Publication bias of diagnostic performance assessed by Deek’s funnel test

Full size image
Table 2 The summary of the diagnostic performance of mNGS in UTI
Full size table

Subgroup analysis

A subgroup analysis was performed on the predefined subgroups to investigate the source of heterogeneity. The studies were divided into five subgroups based on continent, study direction, type of study, reference standard, and the basic condition of the patients (ill or immunocompromised). Among these factors, continent (Asia: 0.91 vs. Non-Asia: 0.87), study direction (retrospective: 0.94 vs. prospective: 0.87), study type (cohort: 0.88 vs. case-control: 0.91), reference standard (culture: 0.88 vs. clinical diagnosis: 0.89), and the basic condition of patients (yes: 0.86 vs. no: 0.89) are all responsible for the heterogeneity of sensitivity (p < 0.001 for all), and the heterogeneity of specificity was influenced only by the basic condition of patients (yes: 1.00 vs. no: 0.72, p < 0.001), and no obvious heterogeneity was observed in the other four subgroups (p > 0.05 for all). More detailed results are shown in Table 3.

Table 3 Subgroup analysis for the diagnostic performance of mNGS in UTI
Full size table

Sensitivity analysis

The leave-one-out analysis revealed that five studies were the main contributors to heterogeneity in sensitivity; when these studies were excluded separately, the value of heterogeneity changed by more than 10% [24, 28, 29, 34, 35]. Similarly, Jia et al. and Zhao et al. also reported obvious heterogeneity in specificity [28, 36]. Detailed information is displayed in Table S1.

Other clinical outcomes

Antibiotic adjustment results were reported by two studies. Duan et al. indicated that the targeted adjustment rate of antibiotic treatment was 76.9% in culture-negative cases (10/13) according to mNGS, and the rate was 33.3% in culture-positive cases (2/6), indicating no significant difference (p = 0.129) [23]. Jia et al. reported an adjustment rate of 92% (23/25) for patients with a positive clinical diagnosis, while the remaining two patients continued current treatment because the pathogens were already covered by the original antibiotic regimens [28].

Discussion

This systematic review and meta-analysis revealed that mNGS had a significantly greater PDR than culture (OR = 2.87). The pooled sensitivity and pooled specificity of mNGS were 0.89 and 0.75, respectively, and the AUROC of 0.89 indicates a favorable diagnostic efficacy. However, the effectiveness of this treatment has been poorly investigated. The meta-regression revealed high heterogeneity, but the sensitivity analysis identified only a small source of heterogeneity.

The use of mNGS in pathogen detection is not a new concept. Since the first successful diagnosis of neuroleptospirosis by mNGS in 2014, mNGS has become a useful tool for pathogen diagnosis, and it has been proven to have excellent performance, with an AUROC of 0.88 [37, 38]. However, pooled analyses of the efficacy of mNGS in treating UTIs are limited.

This is the first systematic review and meta-analysis to evaluate the clinical and diagnostic value of mNGS in UTIs. Similar to previous studies of mNGS in other clinical samples, mNGS in UTIs exhibits satisfactory diagnostic efficacy with an AUROC greater than 0.8 [10,11,12,13]. Liu et al. performed the largest meta-analysis to update the diagnostic efficacy of mNGS in infectious disease and the results were similar to those of our study, and the minor differences may be caused by the internal heterogeneity between diverse samples, the number of studies, and the study population. Considering the basic mechanism, the universality of mNGS in different infectious diseases is unsurprising. Our historical report summarizes the current meta-analyses of mNGS, and this present study appraises the value of mNGS in UTIs, which further demonstrates the application of mNGS in common samples [39].

Unlike previous meta-analyses of mNGS, we additionally compared the PDR of mNGS to culture and set the PDR as a primary outcome. It is not surprising that mNGS has a greater PDR than culture because mNGS enables a broad range of pathogens to be identified from culture or directly from clinical samples based on uniquely identifiable DNA and/or RNA sequences instead of depending on special etiology [8]. Culture is influenced by normal bacteria in the body, especially after antibiotic use. UTI patients are often initially treated with antibiotics, which reduces the PDR in culture. mNGS can improve the PDR and is less affected by external factors to achieve a more accurate diagnosis [8, 40]. Wang et al. investigated the performance of mNGS in the nephrology department and showed that mNGS was more efficient than culture for detecting pathogens in UTIs and has the potential to identify pathogens that cannot be characterized by culture, such as complex infections with specific microorganisms or subclinical infections due to early antibiotic use [33]. Furthermore, the risk factors for UTIs, including diabetes, iatrogenic immunosuppression, and an indwelling catheter, imply the importance of UTI screening in special populations that are exposed to risk factors, such as older patients, patients with diabetes, patients after surgery, patients in intensive care units (ICUs), and patients with immunosuppression (i.e., AIDS patients, organ transplant recipients, and patients with autoimmune diseases treated with immunosuppressants) [4, 41,42,43,44]. The high PDR makes mNGS capable of UTI screening; however, the economic burden must be carefully considered. Although there was no difference in the PDR for fungal detection between mNGS and culture, this finding is consistent with the preliminary conclusion. Given that our meta-analysis included only three studies, bias may have been caused by Jia et al. [28]. Considering the results of fungal PDR in other meta-analyses and the etiology of UTIs [12, 45], the inconsistency in fungal detection does not impact the robustness of mNGS efficacy or its clinical application. Furthermore, only one study compared the PDR in blood sample, and the difference did not reach statistical significance [26]. Taken together, mNGS in urine, the preferred sample in UTIs is reliable, and its application in blood is potential but needs to be verified on a larger sample.

The diagnostic performance was also satisfactory. Similar to previous meta-analyses of different samples, mNGS showed excellent diagnostic efficacy [10,11,12,13]. Undoubtedly, such similarities in multiple clinical samples enhance the extrapolation of mNGS for pathogen diagnosis. The studies reported a sensitivity ≥ 0.80, which guaranteed high pooled sensitivity. However, the specificity reported in each study varied widely. The data from Zhao et al. and Jia et al. significantly decreased the pooled specificity and the final AUROC [28, 36]. Fortunately, the small sample size of these studies limits their reliability, preventing our confidence in the high specificity of mNGS. Overall, the lack of international consensus on the results of mNGS suggests that mNGS tests should be used to evaluate the pathogenicity, epidemiology, and bioinformatics data of microorganisms carefully and should be judged based on the combination of the clinical characteristics of the patients [10].

Subgroup meta-regression and sensitivity analysis were performed to explore the heterogeneity in diagnostic performance. Interestingly, all five grouping factors were recognized as potential sources of heterogeneity. Since culture has a high negative rate, biased results can contribute to heterogeneity. Normally, the use of culture as the reference standard usually leads to lower specificity [12]. However, the specificity is lower when the clinical diagnosis is used as the reference standard, which is inconsistent with the primary viewpoint. This difference could be attributed to the extremely low specificity reported by Zhao et al. [36]. Therefore, the study direction produces heterogeneity in sensitivity. Prospective studies can balance the proportions of uninfected and infected populations according to the inclusion criteria, whereas retrospective studies include mostly infected patients. Additionally, heterogeneity also existed based on study type, although the result were not convincing since there was only one case-control study. The study design is a vital parameter for determining heterogeneity, and the combination of differently designed studies in one meta-analysis may lead to higher estimates of diagnostic accuracy [46]. The basic condition of patients is the sole factor causing significant heterogeneity in both sensitivity and specificity, and this phenomenon has also been found in other studies [10, 12]. In our analysis, although the sensitivity of mNGS in patients in poor condition was lower than that in normal patients, the limited number of studies reduced the reliability of this finding. Guo et al. reported that the clinical presentation in these patients is often atypical, and it is difficult to determine the infection origin and select the most direct specimen for mNGS, which can decrease the sensitivity of mNGS [47]. Moreover, the sensitivity of mNGS in severe pulmonary infection patients in the ICU or patients with weakened immunity was significantly greater than that in non-severe patients. The best clinical utility of mNGS may be for immunocompromised patients, in whom the spectrum of potential pathogens is greater. This is a main signal of the usefulness and attractiveness of mNGS [8, 10]. All the evidence supports that mNGS remains relatively advantageous for patients with poor basic conditions, and more data are needed to prove this opinion. On the continent, differences in mNGS technology availability, clinician proficiency, demographic differences, and pathogen disparities may influence heterogeneity, which needs further validation by international multicenter research [48]. The leave-one-out analysis revealed a small proportion of the heterogeneity, and a large amount of unexplainable heterogeneity still existed. This may be caused by the limited number of samples and the inconsistent diagnostic threshold of mNGS. More studies are necessary to confirm this hypothesis.

Currently, many case reports and original studies have demonstrated the value of mNGS for pathogen diagnosis, but some improvements, such as determining the test threshold, eliminating interference from external nucleic acid fragments, identifying pathogen virulence and drug sensitivity, and optimizing economic benefits, should be considered [11, 49]. The high cost of mNGS and the high incidence of UTIs in low- and middle-income regions are significant barriers to the use of mNGS as a routine diagnostic modality for UTIs [50, 51]. Although the cost of mNGS has dropped sharply since 2014, the average cost ranges from $1,000–2,500 per sample [52]. Another study mentioned that the cost of mNGS is significantly greater than that of any other traditional test [53]. Although the cost-effectiveness of mNGS is sometimes attractive, mNGS is still a complementary approach after the failure of traditional tests, and such a combination is reliable [48]. Some efforts have been made to decrease the cost, including optimizing sequencing procedures, accelerating technological innovation, and encouraging more high-quality sequencing platforms to participate in market competition, and widespread popularization is expected [54].

There are several strengths in the present study. First, this is the first meta-analysis that estimates the clinical and diagnostic value of mNGS in UTIs to fill this gap, further indicating its application value in different clinical backgrounds and reinforcing the universality of mNGS in pathogen diagnosis. Then, the PDR was calculated as a primary outcome to determine the diagnostic performance of mNGS rather than sensitivity or specificity alone. In addition, we suggest that the high PDR makes mNGS potential for use in UTI screening in high-risk populations. Moreover, the different cohort data based on culture and clinical diagnosis from Jia et al. and Wang et al. were separately extracted and included in the meta-analysis, maximizing the sample size and guaranteeing the quality of the evidence. Finally, the CoE assessment was performed by the GRADE score to enhance the reliability of this meta-analysis and offer more information for use in clinical practice.

Some limitations should be acknowledged. Although meta-regression and sensitivity analysis were employed to reveal heterogeneity, some heterogeneity could not be well explained. The included studies were not prospective randomized controlled trials (RCTs). Third, the findings concerning mNGS in terms of treatment effectiveness are insufficient, and conclusions for clinical practice have not been reached.

This study elucidates the clinical and diagnostic value of mNGS in UTIs and proves that mNGS is meaningful in UTIs and has similar efficacy in different clinical backgrounds, increasing the confidence for the wider use of mNGS in the future. Owing to the power of mNGS, early pathogen detection, the accurate and precise treatment could be provided for patients to obtain better prognosis, which is inspiring in the era of precision medicine. The role of mNGS in infectious diseases is pivotal and we envisage that mNGS will become a key tool in the field of infectious disease diagnosis in the next decade.

Conclusion

mNGS has a much greater PDR than conventional culture and excellent diagnostic performance for UTIs. More large-size RCTs are vital to support our findings and provide more data about the treatment value of mNGS. The application of mNGS technology still has several limitations, such as cost and the determination of a positive test threshold, but we still believe that the improvement of mNGS will facilitate its utilization in treating UTIs.