Abstract
Discriminating between malignant pleural effusion (MPE) and benign pleural effusion (BPE) remains difficult. Thus, novel and efficient biomarkers are required for the diagnosis of pleural effusion (PE). The aim of this study was to validate calprotectin as a diagnostic biomarker of PE in clinical settings. A total of 425 patients were recruited, and the pleural fluid samples collected had BPE in 223 cases (53.7%) or MPE in 137 patients (33%). The samples were all analysed following the same previously validated clinical laboratory protocols and methodology. Calprotectin levels ranged from 772.48 to 3,163.8 ng/mL (median: 1,939 ng/mL) in MPE, and 3,216–24,000 ng/mL in BPE (median: 9,209 ng/mL; p < 0.01), with an area under the curve of 0.848 [95% CI: 0.810–0.886]. For a cut-off value of ≤ 6,233.2 ng/mL, we found 96% sensitivity and 60% specificity, with a negative and positive predictive value, and negative and positive likelihood ratios of 96%, 57%, 0.06, and 2.4, respectively. Multivariate analysis showed that low calprotectin levels was a better discriminator of PE than any other variable [OR 28.76 (p < 0.0001)]. Our results confirm that calprotectin is a new and useful diagnostic biomarker in patients with PE of uncertain aetiology which has potential applications in clinical practice because it may be a good complement to cytological methods.
Similar content being viewed by others
Introduction
The diagnosis of pleural effusion (PE) is a clinical challenge because it can be produced by over 60 diseases1,2. Nevertheless, in clinical practice the priority is to establish whether the PE is malignant or not. A diagnosis of malignant PE (MPE) implies the presence of advanced-stage tumours and is therefore associated with a poor prognosis1,3 which requires urgent diagnosis.
Thoracocentesis is the first and most simple procedure for the diagnosis of PE2,3,4. Unfortunately, while its specificity for establishing malignancy is 100%, the diagnostic sensitivity of pleural fluid (PF) cytological analysis is low. Although the odds of establishing an MPE diagnosis by immunohistochemistry are improved by applying a panel of different antibodies, its diagnostic sensitivity is still only approximately 60% for metastatic PE and less than 30% for mesothelioma5,6.
When the cytology results are negative, more invasive methods such as a pleural biopsy or thoracoscopy are necessary2,4,7. In this context, more groups are searching for PF biomarkers for malignancy with the aim of avoiding these invasive procedures8,9,10. Recent meta-analyses have evaluated the ability of new biomarkers such as survivin, vascular endothelial growth factor, fibulina-3, or combinations of tumour markers11,12,13,14 to improve the diagnosis of PE. However, the results achieved so far are inconclusive and novel non-invasive biomarkers for the precise diagnosis of malignancy in PE are still required.
In two previous studies, one conducted in a single centre and the other in a research laboratory15,16, we concluded that measuring calprotectin levels in PE could predict malignancy with a high degree of accuracy. The limitations of these studies were that calprotectin was measured using a non-automated commercial kit and the patients came from a single hospital16. Thus, the aim of this multicentre prospective study was to validate calprotectin levels as a diagnostic biomarker of MPE in patients from three different hospitals during routine clinical practice.
Methods
Study design and patient selection
This study was conducted at three centres in Spain: the University Hospital Complex in Vigo (coordinating centre; Centre 1), the University Hospital Complex in Santiago de Compostela (Centre 2), and the University Hospital Complex in Ourense (Centre 3). Patients with PE admitted in the participating hospitals from January 2014 to June 2016 were consecutively enrolled into this study. The same protocol was used at all three participating centres. The inclusion criteria were: (1) patients aged over 18 years; (2) presence of transudate or exudate PE with a specific diagnosis; (3) patient provided their signed written and informed consent to participation. The exclusion criteria were: (1) previous pleurodesis diagnosis; (2) current treatment with intrapleural or systemic anti-neoplastic agents; (3) the presence of pus in the pleural space (empyema); or (4) PE of an unknown aetiology (when the procedure for PE diagnosis was not completed and its cause was uncertain).
The sample size was calculated using the results obtained in our previous study14. Considering that calprotectin determination had a sensitivity of 97% and a specificity of 89% (for a cut-off value of ≤ 545 ng/mL), and assuming 20–40% were MPEs, with a probability of sensitivity accuracy estimation error of 3%, the required sample size was 394 patients.
Pleural effusion diagnosis
The PE diagnosis protocol followed the recommendations set out in the guidelines published by different medical societies2,4 and was employed in previous studies by our group15,16. The patients underwent a baseline assessment, which included taking their medical history, completing a complete physical examination, and undertaking a chest radiography. In addition, a diagnostic thoracocentesis for biochemical, microbiological, and cytological studies was performed in all the patients. If a diagnosis was not obtained after this procedure, an ultrasound-guided pleural needle-biopsy was performed in individuals with exudative PE.
Subsequently, and at the pulmonologist’s discretion, patients in whom the cause of PE had not yet been identified were submitted for medical or surgical thoracoscopy or were followed-up with clinical and radiological examinations for at least 1 year to confirm the resolution of their symptoms and confirm that PE had not recurred. Light’s criteria were used to differentiate between PF exudates and transudates17. The aetiology of PE was determined based on accepted criteria, as described by different medical societies2,4. Patients with exudates were classified into five main groups according to the cause of the PE as follows.
MPE was defined by the presence of malignant cells in PF cytology or pleural biopsy. Tuberculous PE was defined by the presence of pleural caseating granulomas; a lymphocytic-predominant exudate and high adenosine deaminase (ADA) levels inasuitable non-clinical epidemiological context; or after detection of Mycobacterium tuberculosis in patient sputum, pleural fluid (PF), or pleural biopsy specimens (either by microscopy or cell culture).
Parapneumonic PE was defined as PE associated with bacterial pneumonia, a lung abscess, or bronchiectasis.
Non-malignant PE was defined as nonspecific pleuritis observed during thoracoscopy, thoracotomy, or autopsy (idiopathic PE); or the absence of symptoms or non-recurrence of PE during a 1-year follow-up period (reactive PE).
Miscellaneous PE was diagnosed based on predefined criteria from medical guidelines and defined patients who had not been included in any of the previous groups.
Demographic, clinical, radiological, and biochemical variables
Demographic data such as gender, smoking status, previous cancer diagnosis, age, and other clinical variables including chest pain, dyspnoea, cough, fever, and weight loss were recorded. The radiological size of the PE in the chest radiographs was classified according to the following criteria: as ‘PE occupying more than two thirds of the chest’ when the PE produced opacification of the entire hemithorax or when the fluid reached the arch of the aorta, and otherwise as ‘PE occupying less than two thirds of the chest’. The PF biochemical parameters, differential cell counts, pH, proteins, levels of lactate dehydrogenase (LDH), glucose, and ADA, were determined in the Department of Clinical Chemistry at each participating hospital using automated assays according to the manufacturer’s instructions.
Collection and processing of pleural fluid samples
PF samples were taken by thoracocentesis or during pleural biopsy before starting any treatment. Five mL of PF was obtained and introduced into a pipe with or without an anticoagulant, depending on the standard processing protocol used at each centre. The PF was then centrifuged at 800 g for 15 minutes and was immediately frozen in 0.5 mL aliquots at −80 °C. All the samples were included in the CHUVI Biobank (RETIC-FIS-ISCIII RD09/0076/00011).
Calprotectin determination
The samples were analysed in each participating centre following a common protocol and independently of the clinical diagnosis. Calprotectin levels were determined using a sandwich ELISA (fCALTM-EK-CAL) from Bühlmann Laboratories AG, according to the manufacturer’s guidelines, with some variations to adapt it to the determination of calprotectin in PF. Samples were diluted 1/100. This methodology was previously described and evaluated to confirm its validity and robustness18. The assay was performed with the DSXTM Automated System (DYNEX Technologies, Worthing, West Sussex-UK) according to the manufacturer’s instructions. The absorbance was read at 450 nm and calprotectin concentrations were calculated using the known concentration calibrators provided in the kit and a 4-parameter logistic regression model (4PL).
Ethical approval and consent to participate
Patient data and PF samples were obtained in full compliance with the clinical and ethical practices of the Spanish Government and the Helsinki Declaration. The study protocol was approved by the Galicia Ethics Committee (2014/053). All the patients received written and oral information prior to their inclusion in the study and provided written informed consent before its commencement. Participant anonymity was maintained in all cases.
Statistical methods
Kolmogorov–Smirnov and Levine tests were applied to continuous variables to verify their normal distribution and homogeneity of their variances, respectively. When the normal distribution hypothesis was rejected, we conducted our analyses with non-parametric tests: Mann–Whitney U tests were used to evaluate the differences between independent samples and Kruskal–Wallis tests for multiple comparisons. The results are given as the median (25th–75th percentiles) for quantitative variables and as percentages and absolute frequencies for qualitative variables.
The accuracy of calprotectin levels to discriminate MPE from BPE was evaluated using receiver operating characteristic (ROC) curves. Univariate logistic regression was performed to test the association between calprotectin levels and the presence of MPE; this relationship was also examined for other variables. The unadjusted odds ratio (OR), calculated as an estimate of the relative risk, and the corresponding 95% confidence intervals (CI) were reported. Significant predictors in the univariate analysis (p < 0.1) were entered into a multivariate logistic regression model to assess the independent predictive value of calprotectin levels, considering a p-value of <0.05 as statistically significant. The data were analysed using SPSS statistical software (version 21, IBM Corp., Armonk, NY).
Results
This study included 415 consecutive cases of PE referred to the three participating hospitals; 223 (53.7%) were BPE, 137 (33%) were MPE, and there were 55 (13.3%) transudates. The aetiologies of these PEs are shown in Table 1. Out of the 313 (75.4%) cases of PE with negative cytology results, 35 (11.2%) were MPEs; 112 (27%) patients underwent closed pleural biopsy: 57 (50.9%) were BPEs, 52 (46.4%) MPEs, and 3 (2.7%) transudates.
Thoracoscopy was performed in 30 (7.2%) cases and 17 (56.6%) of these patients were diagnosed as having MPE. The other 13 cases (43.3%) were diagnosed with non-specific fibrinous pleuritis. The demographic characteristics and symptoms of the study population are shown in Table 2 for the 415 patients included in the study, and these results are broken down by centre in Supplementary Table S1.
The biochemical parameters of these PE cases are shown in Table 3 and are separated by centre in Supplementary Table S2. The patient symptoms and biochemical parameters of PE were similar in all three participating centres, thus indicating a high level of homogeneity in the study population. The calprotectin concentrations in the different diagnostic groups are presented in Table 4; the calprotectin levels in MPE ranged from 772.48 to 3,163.8 ng/mL (median: 1,939 ng/mL) and were clearly lower than in the BPE samples (3,216 to 24,000 ng/mL; median: 9,209 ng/mL). In transudates, the calprotectin levels ranged from 400.0 to 548.5 ng/mL (median: 400.0 ng/mL).
Calprotectin levels were significantly higher in the PF from patients with BPE than from samples obtained from patients with MPE (p < 0.01) or transudate (p < 0.001). The calprotectin concentration also differed between sub-groups of patients with BPE; patients with tuberculous PE had median concentration of 24,000 ng/mL (6,850 to 24,000 ng/mL) while those with parapneumonic PE had a median concentration of 24,000 ng/mL (7,481 to 24,000 ng/mL).
Patients in the non-MPE group had a median concentration of 3,216 ng/mL (1,718.3 to 11,597.5 ng/mL), whereas the miscellaneous group had a median concentration of 5,202.4 ng/mL (3,142.8 to16,304 ng/mL). We did not find any significant differences in the calprotectin concentrations between groups with different MPE aetiologies. The calprotectin concentrations in PE samples from the three different centres are provided in Supplementary Table S3. Figure 1 shows the distribution of the calprotectin concentrations in pleural fluid by based on the PE cause.
We evaluated the usefulness of calprotectin to differentiate MPE from BPE. ROC curve analysis for a combined total of 360 samples from the three centres showed an AUC value of 0.848 [95% CI, 0.810–0.886]. For a cut-off value of ≤6,233.2 ng/mL, it had a sensitivity of 96% and a specificity of 60%, with negative and positive predictive values and negative and positive likelihood ratios (NLR and PLR) of 96%, 57%, 0.06, and 2.4, respectively. The diagnostic accuracy results for calprotectin levels in PE for predicting MPE, classified by centre and for the entire population are shown in Supplementary Table S4. In the group of patients with MPE with negative cytology results (35), the AUC was 0.855 [95% CI, 0.803–0.907]. For the same cut-off value of ≤ 6,233.2 ng/mL, the sensitivity and specificity were 97% and 60%, respectively.
The relationship between the malignant origin of PE and other variables, including calprotectin, was also analysed in a univariate analysis (Table 5). These variables were entered into a multivariate analysis for MPE identification and showed that low calprotectin levels in PF had higher discriminatory properties than any other variable [OR 28.76 (p < 0.0001); (Table 5)]. Finally, we proposed an algorithm for the diagnostic management of suspected MPE which includes calprotectin as an additional test (Fig. 2).
Discussion
This study confirms the value of calprotectin as a new diagnostic biomarker for PE which might be a useful complement for inclusion in a diagnostic algorithm and would therefore avoid the need for more invasive procedures in some cases. Our results confirmed those obtained in our previous studies15,16. In this present study, the determination of calprotectin was adjusted to the workflow of the clinical laboratories at the three participating centres. Calprotectin levels were measured in conventional robotised equipment using a sandwich ELISA kit from Bühlmann Laboratories, which was extensively tested in our laboratory before initiating this present study18.
We found that the calprotectin levels in BPE samples were higher than those in the MPE samples, and showed a good accuracy for predicting MPE. The pooled sample ROC curve analysis produced an AUC value of 0.848 and with a selected cut-off, an elevated sensitivity for MPE diagnosis exceeding that of cytological analysis and thus, one which would help to identify patients with a suspected neoplasia. Moreover, calprotectin had a very good NLR, which is especially important when considering new diagnostic biomarkers.
Univariate and multivariate analysis also demonstrated a strong association between low levels of calprotectin and malignancy. Multivariate analysis revealed significant associations between MPE and other variables, however, the contribution of these factors was minimal compared to that of the calprotectin levels. In our study, the patients with a radiological PE size greater than two thirds had higher risk of MPE, which is in line with other evidence that large PEs are usually associated with a higher probability of MPE19.
In addition, women had a higher probability of MPE, although this may be because the cases with MPE in female patients were often associated with ovarian or breast cancer. Patients with a previous neoplasia or with symptoms such as weight loss, absence of fever, and low ADA levels in the PF were more often associated with MPE than with BPE in this study, and this agrees with previous reports of associations in patients with MPE2,3.
Transudates showed very low calprotectin levels, probably because of the non-inflammatory and/or non-infective condition of these patients. In our study, none of the patients with MPE presented the criteria required for a transudate diagnosis. Nonetheless, several studies have documented cases of MPEs (up to 10%) that are biochemically classified as transudates19,20, and so calprotectin should be used after confirming the exudative origin of PE.
Of note, the levels of calprotectin reported by Centre 3were lower than that from the other two centres. One explanation may lie in the fact that fewer patients had inflammatory diseases (tuberculous or parapneumonic conditions) in this centre (52% in Centre 1 and 61% in Centre 2, versus 38% in Centre 3). Another explanation might be that, unlike Centres 1 and 2, Centre 3 did not use anticoagulants during the sample collection procedures. Nonetheless, we did not find any differences between the CI of the ROC curves for the three centres, thus demonstrating the diagnostic accuracy of calprotectin in a routine clinical scenario.
Calprotectin has been studied as a biomarker for several different diseases, and its main current use in the assessment of inflammatory bowel diseases by analysing faeces21. However, calprotectin has been studied as a diagnostic marker in urine for the diagnosis and staging of bladder cancer22. We were only able to identify one study that used calprotectin as a marker in PE to analyse the efficacy and diagnostic accuracy of a combination of calprotectin and CXCL1223. These authors showed a low sensitivity (25.6%) but a specificity of 95.4%, although it is worth noting that, compared to this present study, they included fewer patients and only examined the results from one centre.
In the algorithm for the diagnostic management of suspected MPE we propose here, calprotectin levels could be determined simultaneously alongside other biochemical parameters. Patients with calprotectin levels <5,500 ng/mL would be submitted for a pleural biopsy. Patients with levels between 5,500 ng/mL and 8,000 ng/mL and a positive cytology result would be diagnosed with a malignancy while those who are cytologically negative will receive individualised follow-ups, although a biopsy will be especially recommended for female patients, those with a history of neoplasia, a PE radiological size exceeding two thirds of the chest, low ADA levels, or weight loss in the absence of fever. Finally, patients with calprotectin levels over 8,000 ng/mL would be referred for a biopsy if their ADA levels are also high4.
When we applied this algorithm in our settings, there were no false negatives, and only one patient had high PF calprotectin levels (6,888.95 ng/mL) and was subsequently diagnosed with a neoplasm (mesothelioma). However, it is important to consider that this disease is a rare form of cancer. CT has also been reported to have a high specificity for the diagnosis of MPE24 and so it is likely that we would have found similar findings if we had also analysed CT.
Thus, based on our findings, we propose that the PF calprotectin concentration should be measured in any patients with suspected MPD, and then, based on the standard clinical, radiological, and biochemical characteristics, cytohistological techniques will be performed. This would improve diagnostic accuracy and allow a new predictive scale to be created in order to improve the indication for more invasive tests. Therefore, future multicentre studies should aim to create, test and potentially validate this algorithm.
One limitation of this study was that it only represented one region in Spain. Thus, these findings must be confirmed in other populations with different ethnic and racial profiles. The development of a technology that could allow the immediate determination of calprotectin alongside the standard biochemical and cytological parameters25, especially in a rapid and cost-effective form (i.e. a monotest), would also be of great interest. Finally, each centre could choose to determine its own calprotectin cut-off points in order to overcome potential differences in sample processing.
In summary, measuring calprotectin in patients with PE with an uncertain aetiology would improve diagnostic accuracy and help clinicians in the PE diagnosis decision-making process because it helps to exclude the possibility of MPE in certain cases, thus avoiding invasive procedures.
References
Thomas, J. M. & Musani, A. I. Malignant pleural effusions: a review. Clin. Chest. Med. 34, 459–471 (2013).
Roberts, M. E., Neville, E., Berrisford, R. G., Antunes, G. & Ali, N. J. BTS Pleural Disease Guideline Group Management of a malignant pleural effusion: British Thoracic Society Pleural Disease Guideline 2010. Thorax. 65(Suppl2), ii32–ii40 (2010)
Wu, S. G. et al. Survival of lung adenocarcinoma patients with malignant pleural effusion. Eur. Respir. J. 41, 1409–1418 (2013).
Villena Garrido, V. et al. Recommendations of diagnosis and treatment ofpleural effusion. Update. Arch. Bronconeumol. 50, 235–249 (2014).
Assawasaksakul, T., Boonsarngsuk, V. & Incharoen, P. A comparative study of conventional cytology and cell block method in the diagnosis of pleural effusion. J. Thorac. Dis. 9, 3161–3167 (2017).
Arnold, D. T. et al. Investigating unilateral pleural effusion: the role of cytology. Eur. Respir. J. 52, 1801254 (2018).
Koegelenberg, C. F. & Diacon, A. L. Image-guided pleural biopsy. Curr. Opin. Pulm. Med. 19, 368–373 (2013).
Karpathiou, G., Stefanou, D. & Froudarakis, M. E. Pleural neoplastic pathology. Respir. Med. 109, 931–943 (2015).
Froudarakis, M. E. Pleural diseases in the molecular era-time for more answers: Introduction. Respiration. 83, 2–4 (2012).
Sriram, K. B. et al. Diagnostic molecular biomarkers for malignant pleural effusions. Future Oncol. 7, 737–75 (2011).
Tian, P. et al. Diagnostic value of survivin for malignant pleural effusion: a clinical study and meta-analysis. Int. J. Clin. Exp. Pathol. 7, 5880–5887 (2014).
Fafliora, E., Hatzoglou, C., Gourgloulianis, K. I. & Zarogiannis, S. G. Systematic review and meta-analysis of vascular endothelial growth factor as a biomarker for malignant pleural effusions. Physiolo Rep. 4, e12978 (2016).
Ren, R. et al. Diagnostic value of fibulin-3 for malignant pleural mesothelioma: a systematic review and meta-analysis. Onco Target. 7, 84851–84859 (2016).
Yang, Y., Liu, Y. L. & Shi, H. Z. Diagnostic accuracy of combination of tumor markers for malignant pleural effusion: an updated meta-analysis. Respiration. 94, 62–69 (2017).
Rodríguez-Piñeiro, A. M., Blanco-Prieto, S., Sánchez-Otero, N., Rodríguez-Berrocal, F. J. & Páez de la Cadena, M. On the identification of biomarkers for non-small cell lungcancer in serum and pleural effusion. J. Proteomics. 73, 1511–1522 (2010).
Sánchez-Otero, N. et al. Calprotectin: A novel biomarker for the diagnosis of pleural effusion. Brit J. Cancer. 107, 876–1882 (2012).
Light, R. W., Macgregor, M. I., Luchsinger, P. C. & Ball, W. C. Jr. Pleural effusions: the diagnostic separation of transudates and exudates. Ann. Intern. Med. 77, 507–513 (1972).
Vázquez-Iglesias, L. et al. Evaluation of an automated commercial ELISA method for calprotectin determination in pleural fluid. Clin. Chem. Lab. Med. 55, e172–e174 (2017).
Heffner, J. E. Diagnosis and management of malignant pleural effusions. Respirology. 13, 5–20 (2008).
Ferreiro, L. et al. Predictive models of malignant transudative pleural effusions. J. Thorac. Dis. 9, 106–116 (2017).
Daj, C., Jiang, M. & Sun, M. J. Fecal calprotectin in monitoring the disease activity in colonic inflammatory bowel disease. Dig. Dis. Sci. 62, 3625–3626 (2017).
Yaser, O., Akcay, T., Obek, C. & Turegun, F. A. Significance of S100A8, S100A9 and calprotectin levels in bladder cancer. Scand. J. Clin. Lab. Invest. 77, 437–441 (2017).
Luo, J. et al. A novel combination of calprotectin and CXCL12 for predicting malignacy in patients with exudative pleural effusion. Medicine. 94, e2015 (2015).
Hallifax, R. J. et al. Role of CT in assessing pleural malignancy prior to thoracoscopy. Thorax. 70, 192–193 (2015).
Wang, Y. Development of cancer diagnostics—from biomarkers to clinical tests. Transl. Cancer Res. 4, 270–279 (2015).
Acknowledgements
We would like to thank Publibeca SEPAR 2016, Cecilia Mouronte-Roibás, and Ana Priegue Carrera at the Pulmonary Department, Hospital Álvaro Cunqueiro, EOXI Vigo, Spain, and the PneumoVigoI+I Research Group, Heath Research Institute, Galicia Sur, Spain. This project was funded by grant PI13/01538 (FIS-FEDER), from the Xunta de Galicia (CN/2014), AEER (2015), and SOGAPAR (2014). Lorena Vázquez was supported by funding from the Fundación Biomédica Galicia Sur and with SEPAR integrated research programs (PII Neumología Intervencionista 2015).
Author information
Authors and Affiliations
Contributions
All the authors made substantial contributions to this manuscript. P. Casado-Rey, M.A. Andrade-Olivié, L. García-Nimo, M.E. San-Jose performed data acquisition, analysis, and interpretation calprotectin levels, J. Abal-Arca, L. Ferreiro-Fernández, L. Valdés Cuadrado and F.J. Rodriguez-Berrocal performed demographic, clinical, radiological, and biochemical data acquisition of study cases, M. Botana-Rial, L. Vázquez-Iglesias, y M. Paez de la Cadena wrote the main manuscript text, and A. Fernández-Villar developed the conception and design, and reviewed this manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Botana-Rial, M., Vázquez-Iglesias, L., Casado-Rey, P. et al. Validation of Calprotectin As a Novel Biomarker For The Diagnosis of Pleural Effusion: a Multicentre Trial. Sci Rep 10, 5679 (2020). https://doi.org/10.1038/s41598-020-62388-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-020-62388-y
- Springer Nature Limited