Abstract
The precise prediction of acute kidney injury (AKI) after nephrectomy for renal cell carcinoma (RCC) is an important issue because of its relationship with subsequent kidney dysfunction and high mortality. Herein we addressed whether machine learning (ML) algorithms could predict postoperative AKI risk better than conventional logistic regression (LR) models. A total of 4104 RCC patients who had undergone unilateral nephrectomy from January 2003 to December 2017 were reviewed. ML models such as support vector machine, random forest, extreme gradient boosting, and light gradient boosting machine (LightGBM) were developed, and their performance based on the area under the receiver operating characteristic curve, accuracy, and F1 score was compared with that of the LR-based scoring model. Postoperative AKI developed in 1167 patients (28.4%). All the ML models had higher performance index values than the LR-based scoring model. Among them, the LightGBM model had the highest value of 0.810 (0.783–0.837). The decision curve analysis demonstrated a greater net benefit of the ML models than the LR-based scoring model over all the ranges of threshold probabilities. The application of ML algorithms improves the predictability of AKI after nephrectomy for RCC, and these models perform better than conventional LR-based models.
Similar content being viewed by others
Introduction
Renal cell carcinoma (RCC) represents approximately 3% of cancers, and is the 3rd most common type of cancer in the genitourinary tract1. During the last two decades, there has been an annual increase of 2% in its incidence worldwide2. In particular, small RCCs with T1 stage account for more than half of the newly diagnosed cases3. The early detection of small RCCs can improve overall survival of patients by curative nephrectomy4. Along with this trend, the American and European guidelines recommend partial nephrectomy (PN) rather than radical nephrectomy (RN) for localized tumors in stage T1 as a curative approach2,5. Despite an increasing tendency in performing PN, RN is also carried out, particularly in patients with chronic kidney disease, because of the high complication rate, long operation time, and potential morbidities of PN compared to RN6,7,8. The worsening of postoperative renal function continues to be a great issue in patients who undergo nephrectomy for RCC because of their superior survival and large remnant functioning tissues.
The loss of normal kidney tissues after PN or RN may result in an inevitable decline in kidney function despite the compensation of remnants9,10,11. Compensatory hypertrophy and hyperfiltration of the remaining kidney occurs within hours after nephrectomy, and a subsequent decrease in glomerular filtration rates is transient or subclinical12. However, 2–54% of patients experience postoperative acute kidney injury (AKI), which is attributable to several factors, such as elderly age, male sex, preoperative chronic kidney disease, diabetes mellitus, and RN13,14,15,16,17,18,19,20. AKI after nephrectomy for RCC leaves sequelae in the remaining kidneys, which is a strong risk factor for irreversible kidney dysfunction18,19,20. Furthermore, there is increasing concern that the transition to chronic kidney disease after nephrectomy is associated with both all-cause21,22 and cancer-specific mortality23.
Although previous studies have focused on postoperative kidney function after nephrectomy in the short- or intermediate-to-long term13,14,16,17,18,19, few models for predicting postoperative AKI have been developed. Moreover, these studies included patients who underwent certain types of surgery (e.g. laparoscopic or robot-assisted laparoscopic) rather than all kinds of operations15,20. Preparing for AKI beforehand may not be easy because several conditions in addition to operative settings have interactive and complex effects on the risk. The heterogeneous features of patients may also make it difficult to accomplish precise prediction. A previous logistic regression (LR) model (e.g., the simple postoperative AKI risk [SPARK] index) has suitable performance in predicting the risk of postoperative AKI in noncardiac surgery, but its performance has not been validated in the urologic surgery24. To overcome these limitations, we aimed to apply several machine learning models in predicting AKI after nephrectomy for RCC, and compared their performance with that of conventional LR models.
Methods
Patient and study design
A total of 4659 patients who were diagnosed with RCC and thus had undergone unilateral PN or RN between January 2003 and December 2017 were retrospectively reviewed. Patients were excluded if they met any of the following criteria: less than 18 years old (n = 11); metastatic RCCs (clinical T stage = 4; N stage > 0; and M stage > 0) (n = 331); previous history of nephrectomy (n = 3); kidney transplant recipients (n = 13); staged nephrectomy due to bilateral RCCs (n = 6); congenital single kidney before surgery (n = 4); presence of postoperative complications requiring re-operation (n = 3); and incomplete laboratory information (n = 184). Accordingly, 4,104 patients were analyzed in the present study. The study was approved by the institutional review boards of Seoul National University Hospital (H-1904-005-1021) and Seoul National University Bundang Hospital (B-1905-538-404) and was conducted in accordance with the principle of the Declaration of Helsinki. The requirement to obtain informed consent from the patients was waived by the above two IRBs.
Study variables
Patient demographics such as clinical and laboratory data were recorded. Preoperative and intraoperative data (such as age, sex, body mass index, smoking status, hypertension, diabetes mellitus, histories of myocardial infarction, stroke, peripheral vascular disease, chronic hepatitis B and C, and other cancers, medications of angiotensin-converting enzyme inhibitors and angiotensin receptor blockers, type of operation, total and ischemic time of operation, estimated amounts of blood loss, intraoperative transfusion) and tumor-specific data (such as tumor size and clinical T stage) were extracted from electronic medical records. Blood laboratory data, such as preoperative serum creatinine, blood urea nitrogen, albumin, and hemoglobin, were obtained. For serum creatinine, postoperative values were also obtained. The estimated glomerular filtration rate (eGFR) was calculated using the Chronic Kidney Disease Epidemiology Collaboration equation25. Proteinuria was defined as ≥ 1+ on a dipstick test.
The primary outcome was postoperative AKI, defined as an increase in serum creatinine level to ≥ 0.3 mg/dL within 48 h or ≥ 1.5 times baseline within 7 days after operation according to the Kidney Disease Improving Global Outcomes guideline26. If the serum creatinine decreased within the non-AKI range and was at least 0.3 mg/dL below the peak level, the cases were defined as recovered AKI27.
Statistical analysis
All analyses were implemented using R software (version 3.6.3; R Foundation for Statistical Computing). Comparisons of baseline characteristics were performed with the Wilcoxon rank-sum test for continuous variables and the chi-square test for categorical variables. The patients were randomly assigned to training (70%) and testing (30%) datasets. Using the training dataset, we developed machine learning models such as support vector machine (SVM), random forest, extreme gradient boosting (XGBoost), and light gradient boosting machine (LightGBM) to predict the risk of AKI. As a reference model, we used multivariable LR analysis (herein termed the LR-scoring model). Variables with a P value of < 0.2 in the univariate model were adjusted with a stepwise fashion. The logistic coefficients were used as clinical scores by proportionally assigning points and rounding to the nearest integer. For another reference, we used the SPARK index which had been validated in patients undergoing noncardiac operations24. SVM constructs a hyperplane in a high-dimensional space, which can be used for classification. Random forest is an ensemble of decision trees created by using bootstrap samples of the training dataset and random selection in tree induction28. For the random forest model, we used a grid search strategy to identify the best combination of hyperparameters with the caret package. XGBoost is an ensemble approach with a gradient descent–boosted decision tree algorithm29. We selected a low learning rate (0.0001), interaction depth of 5, and a maximum of 3000 iterations. LightGBM is an improvement framework based on the gradient descent–boosted decision tree algorithm and is more powerful than the previous XGBoost with a fast training speed and less memory occupation30. To minimize potential overfitting in the above machine learning models, we used tenfold cross-validation and out-of-bag estimation during development.
The model performance was assessed with the area under the receiver operating characteristic curve (AUROC), accuracy, and F1 score in the testing dataset. To calculate the performance of the SPARK index, we used the best threshold point of the curve. The DeLong test was used to compare AUROCs31. The net benefit over a specified range of threshold probabilities in outcome was evaluated using decision curve analysis32,33. The Hosmer–Lemeshow test was used to assess calibration. Two-sided P values less than 0.05 were considered significant.
Results
Baseline characteristics of the patients
The mean age of the patients was 56 ± 13 years and 2855 (69.6%) were male. 443 patients (10.6%) had diabetes mellitus. The proportion of patients who underwent PN was 66.5%. The median ischemic time during PN was 21 min (interquartile range 16–28 min). Postoperative AKI developed in 1167 patients (28.4%) after nephrectomy (423 after PN [15.5%] and 744 after RN [54.1%]; 817 [28.4%] in the training dataset and 350 [28.4%] in the testing dataset). 41.6% of patients with postoperative AKI had fully recovered renal function at discharge. Other baseline characteristics are shown in Table 1. These baseline characteristics did not differ between the training and testing datasets.
Model performance in predicting AKI
When adjustment with a stepwise fashion was applied, several factors, such as male sex, diabetes mellitus, hypertension, RN, large tumor size, long operation time, intraoperative transfusion, and low eGFR were selected as risk factors for AKI in the LR-scoring model (Table S1). The corresponding clinical scores in this LR model are presented in Fig. S1.
We set up two LR-based models, the SPARK index and the LR-scoring model as a reference for comparison with the machine learning models. Among the models developed, the LightGBM model had the highest AUROC value (0.810 [0.783–0.837]), whereas the SPARK index showed the lowest AUROC value (0.626 [0.607–0.644]) (Table 2). All the machine learning models had higher AUROC values than the SPARK index. The LightGBM model had a higher AUROC value than the LR-scoring model with marginal significance. Corresponding curves supported these results (Fig. 1). When other performance indices, such as accuracy and F1 score, were examined, the XGBoost model had the best performance, and the LR-based models, including the SPARK index and the LR-scoring model, had the poorest performance. In decision curve analysis (Fig. 2), the net benefit was greater for machine learning models than for the SPARK index over all the ranges of threshold probabilities. The LightGBM, XGBoost and SVM models had the highest net benefits among the models. The LR-scoring model had a negative benefit in > 0.6 of the threshold probabilities. The LightGBM, XGBoost, random forest, and LR-scoring models were well calibrated (all P > 0.05), but the other models were not (all P < 0.05) (Fig. 3). Based on these results, the LightGBM model was chosen as the best model for predicting postoperative AKI.
Variable ranking analysis
To estimate the contribution degree of each variable in predicting the risk of AKI, variable ranking analysis was performed (Fig. 4). Relative values ranged from 0 to 1, which indicated the proportional contribution of variables in predicting AKI. Accordingly, type of operation, sex, tumor size, operation time, and baseline eGFR were highly ranked as the top predictors.
Discussion
It has become more important to precisely predict AKI in patients undergoing nephrectomy for RCC because surviving patients with AKI will suffer from subsequent chronic kidney disease and other worse outcomes. The present study first applied machine learning algorithms to accomplish the precise prediction of postoperative AKI, and the performance and calibration of these models were better than those of the LR-based reference models. Based on ranking analysis, certain variables were noted to contribute more to the predictive performance of the models. These results indicate that the precise prediction of postoperative AKI is achievable by machine learning despite the complex and interactive relationships of several variables.
A meta-analysis of 71 studies suggested that machine learning algorithms did not improve discriminative power over traditional LR-based models in predicting various clinical outcomes such as diabetes mellitus, infection, heart failure, and cancer34. Nevertheless, one study reported the superiority of machine learning models to the LR model in predicting AKI after minimally invasive laparoscopic or robot-assisted laparoscopic nephrectomy for RCC15. The present study dealing with all operation types supports this result with better model performance. Particularly, the performance improvement by the LightGBM model can be acceptable to alert clinicians of the risk of postoperative AKI.
Decision curve analysis takes into account the weights of different misclassification types with a direct clinical interpretation of the net benefit (i.e., the trade-off between undertreatment and overtreatment in the model)32,33. It is useful to compare models where the default strategies predict all-or-none outcomes such as AKI. All the machine learning models had greater net benefit over the range of threshold probabilities than the SPARK index. The LR-scoring model had a negative value of net benefit in a high range of threshold probabilities. These results provide clues on how machine learning models will be applicable to clinical practice.
The ranking analysis showed that certain variables such as nephrectomy type, patient characteristics (e.g., age and sex), and laboratory findings (e.g., eGFR and hemoglobin), contributed to the model performance. These results support the findings of previous large cohort studies focusing on postoperative AKI14,15,16,17,18,19. Only one or two variables may not be enough to accomplish a perfect prediction. Accordingly, modeling with at least the top variables obtained from the ranking analysis is needed if another model in an independent population should be developed.
Although the results were informative, some limitations should be discussed. The study design was retrospective in nature which may have potential selection bias. The study identified the most important variables with respect to predicting mortality, but we could not obtain certain degrees of risk, such as the relative risk, which is a common limitation of machine learning algorithms. The study results may not be applicable to some specific populations such as patients with metastasis or kidney transplant recipients. Concerns could be raised regarding other issues such as the absence of external validation and the effects of unidentified factors.
The application of machine learning algorithms improves the predictability of AKI after nephrectomy for RCC, and these models performed better than conventional LR-based models. If machine learning-based prediction models are successfully applied in clinical practice, the overall patient outcomes will improve by implementing earlier management. Future studies will explore whether machine learning is also applicable to predicting other outcomes after nephrectomy with validating results in independent cohorts.
References
Global Burden of Disease Cancer Consortium. Global, regional, and national cancer incidence, mortality, years of life lost, years lived with disability, and disability-adjusted life-years for 29 cancer groups, 1990 to 2017: A systematic analysis for the Global Burden of Disease Study. JAMA Oncol. https://doi.org/10.1001/jamaoncol.2019.2996 (2019).
Ljungberg, B. et al. European Association of Urology Guidelines on renal cell carcinoma: The 2019 update. Eur. Urol. 75, 799–810. https://doi.org/10.1016/j.eururo.2019.02.011 (2019).
Hollingsworth, J. M., Miller, D. C., Daignault, S. & Hollenbeck, B. K. Rising incidence of small renal masses: A need to reassess treatment effect. J. Natl. Cancer Inst. 98, 1331–1334. https://doi.org/10.1093/jnci/djj362 (2006).
Kane, C. J., Mallin, K., Ritchey, J., Cooperberg, M. R. & Carroll, P. R. Renal cell cancer stage migration: Analysis of the National Cancer Data Base. Cancer 113, 78–83. https://doi.org/10.1002/cncr.23518 (2008).
Campbell, S. C. et al. Guideline for management of the clinical T1 renal mass. J. Urol. 182, 1271–1279. https://doi.org/10.1016/j.juro.2009.07.004 (2009).
Bjurlin, M. A. et al. National trends in the utilization of partial nephrectomy before and after the establishment of AUA guidelines for the management of renal masses. Urology 82, 1283–1289. https://doi.org/10.1016/j.urology.2013.07.068 (2013).
Patel, S. G. et al. National trends in the use of partial nephrectomy: A rising tide that has not lifted all boats. J. Urol. 187, 816–821. https://doi.org/10.1016/j.juro.2011.10.173 (2012).
Schiffmann, J., Bianchi, M., Sun, M. & Becker, A. Trends in surgical management of T1 renal cell carcinoma. Curr. Urol. Rep. 15, 383. https://doi.org/10.1007/s11934-013-0383-0 (2014).
Aguilar Palacios, D. et al. Compensatory changes in parenchymal mass and function after radical nephrectomy. J. Urol. 204, 42–49. https://doi.org/10.1097/JU.0000000000000797 (2020).
Choi, D. K. et al. Compensatory structural and functional adaptation after radical nephrectomy for renal cell carcinoma according to preoperative stage of chronic kidney disease. J. Urol. 194, 910–915. https://doi.org/10.1016/j.juro.2015.04.093 (2015).
Takagi, T. et al. Compensatory hypertrophy after partial and radical nephrectomy in adults. J. Urol. 192, 1612–1618. https://doi.org/10.1016/j.juro.2014.06.018 (2014).
Rojas-Canales, D. M., Li, J. Y., Makuei, L. & Gleadle, J. M. Compensatory renal hypertrophy following nephrectomy: When and how?. Nephrology (Carlton) 24, 1225–1232. https://doi.org/10.1111/nep.13578 (2019).
Bhindi, B. et al. Predicting renal function outcomes after partial and radical nephrectomy. Eur. Urol. 75, 766–772. https://doi.org/10.1016/j.eururo.2018.11.021 (2019).
Zhang, Z. et al. Acute kidney injury after partial nephrectomy: Role of parenchymal mass reduction and ischemia and impact on subsequent functional recovery. Eur. Urol. 69, 745–752. https://doi.org/10.1016/j.eururo.2015.10.023 (2016).
Kim, N. Y. et al. Development of a risk scoring system for predicting acute kidney injury after minimally invasive partial and radical nephrectomy: A retrospective study. Surg. Endosc. https://doi.org/10.1007/s00464-020-07545-0 (2020).
Schmid, M. et al. Predictors of 30-day acute kidney injury following radical and partial nephrectomy for renal cell carcinoma. Urol. Oncol. 32, 1259–1266. https://doi.org/10.1016/j.urolonc.2014.05.002 (2014).
Schmid, M. et al. Trends of acute kidney injury after radical or partial nephrectomy for renal cell carcinoma. Urol. Oncol. 34, e291–e293. https://doi.org/10.1016/j.urolonc.2016.02.018 (2016).
Garofalo, C. et al. Effect of post-nephrectomy acute kidney injury on renal outcome: A retrospective long-term study. World J. Urol. 36, 59–63. https://doi.org/10.1007/s00345-017-2104-7 (2018).
Cho, A. et al. Post-operative acute kidney injury in patients with renal cell carcinoma is a potent risk factor for new-onset chronic kidney disease after radical nephrectomy. Nephrol. Dial. Transplant. 26, 3496–3501. https://doi.org/10.1093/ndt/gfr094 (2011).
Martini, A. et al. A nomogram to predict significant estimated glomerular filtration rate reduction after robotic partial nephrectomy. Eur. Urol. 74, 833–839. https://doi.org/10.1016/j.eururo.2018.08.037 (2018).
Lane, B. R. et al. Survival and functional stability in chronic kidney disease due to surgical removal of nephrons: Importance of the new baseline glomerular filtration rate. Eur. Urol. 68, 996–1003. https://doi.org/10.1016/j.eururo.2015.04.043 (2015).
Streja, E. et al. Radical versus partial nephrectomy, chronic kidney disease progression and mortality in US veterans. Nephrol. Dial. Transplant. 33, 95–101. https://doi.org/10.1093/ndt/gfw358 (2018).
Antonelli, A. et al. Below safety limits, every unit of glomerular filtration rate counts: Assessing the relationship between renal function and cancer-specific mortality in renal cell carcinoma. Eur. Urol. 74, 661–667. https://doi.org/10.1016/j.eururo.2018.07.029 (2018).
Park, S. et al. Simple postoperative AKI risk (SPARK) classification before noncardiac surgery: A prediction index development study with external validation. J. Am. Soc. Nephrol. 30, 170–181. https://doi.org/10.1681/ASN.2018070757 (2019).
Levey, A. S. et al. A new equation to estimate glomerular filtration rate. Ann. Intern. Med. 150, 604–612. https://doi.org/10.7326/0003-4819-150-9-200905050-00006 (2009).
Khwaja, A. KDIGO clinical practice guidelines for acute kidney injury. Nephron Clin. Pract. 120, c179–c184. https://doi.org/10.1159/000339789 (2012).
Xu, X. et al. Epidemiology and clinical correlates of AKI in Chinese hospitalized adults. Clin. J. Am. Soc. Nephrol. 10, 1510–1518. https://doi.org/10.2215/CJN.02140215 (2015).
Breiman, L. Random forests. Mach. Learn. 45, 5–32. https://doi.org/10.1023/A:1010933404324 (2001).
Huang, J. C. et al. Predictive modeling of blood pressure during hemodialysis: A comparison of linear model, random forest, support vector regression, XGBoost, LASSO regression and ensemble method. Comput. Methods Progr. Biomed. 195, 105536. https://doi.org/10.1016/j.cmpb.2020.105536 (2020).
Wang, Y. & Wang, T. Application of improved LightGBM model in blood glucose prediction. Appl. Sci. Basel. https://doi.org/10.3390/app10093227 (2020).
DeLong, E. R., DeLong, D. M. & Clarke-Pearson, D. L. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 44, 837–845 (1988).
Fitzgerald, M., Saville, B. R. & Lewis, R. J. Decision curve analysis. JAMA 313, 409–410. https://doi.org/10.1001/jama.2015.37 (2015).
Vickers, A. J. & Elkin, E. B. Decision curve analysis: A novel method for evaluating prediction models. Med. Decis. Making 26, 565–574. https://doi.org/10.1177/0272989X06295361 (2006).
Christodoulou, E. et al. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J. Clin. Epidemiol. 110, 12–22. https://doi.org/10.1016/j.jclinepi.2019.02.004 (2019).
Acknowledgements
This work was supported by a grand from the National Research Foundation, Republic of Korea (No. 2019R1A2C1085411). Also, this work was supported by a grand No. 14-2019-017 from the Seoul National University Bundang Hospital Research Fund. The authors gratefully acknowledge the assistance of Hojin Ju for the data collection.
Author information
Authors and Affiliations
Contributions
S.S.H., S.K. and C.K. designed the study. Y.L. collected the data, analyzed and interpreted the results, and drafted the manuscript. M.W.K., K.H.S., J.K. and J.J.M. analyzed and interpreted the data. Y.C.K., D.K.K., K.O. and K.W.J. conceived the study and assisted in the analyses. C.K. and Y.S.K. collected the data. S.S.H. conceived the study, interpreted the data, and reviewed the manuscript. All authors read and approved the final manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Lee, Y., Ryu, J., Kang, M.W. et al. Machine learning-based prediction of acute kidney injury after nephrectomy in patients with renal cell carcinoma. Sci Rep 11, 15704 (2021). https://doi.org/10.1038/s41598-021-95019-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-021-95019-1
- Springer Nature Limited
This article is cited by
-
Machine learning models to predict systemic inflammatory response syndrome after percutaneous nephrolithotomy
BMC Urology (2024)
-
Detection and classification of diabetic retinopathy based on ensemble learning
Advances in Computational Intelligence (2024)
-
Development of artificial neural networks for early prediction of intestinal perforation in preterm infants
Scientific Reports (2022)