Abstract
To predict 3-Level version of European Quality of Life-5 Dimensions (EQ-5D-3L) questionnaire utility from the chronic obstructive pulmonary disease (COPD) assessment test (CAT), the study attempts to collect EQ-5D-3L and CAT data from COPD patients. Response mapping under a backward elimination procedure was used for EQ-5D score predictions from CAT. A multinomial logistic regression (MLR) model was used to identify the association between the score and the covariates. Afterwards, the predicted scores were transformed into the utility. The developed formula was compared with ordinary least squares (OLS) regression models and models using Mean Rank Method (MRM). The MLR models performed as well as other models according to mean absolute error (MAE) and root mean squared error (RMSE) evaluations. Besides, the overestimation for low utility patients (utility ≤ 0.6) and underestimation for near health (utility > 0.9) in the OLS method was improved through the means of the MLR model based on bubble chart analysis. In conclusion, response mapping with the MLR model led to performance comparable to the OLS and MRM models for predicting EQ-5D utility from CAT data. Additionally, the bubble charts analysis revealed that the model constructed in this study and MRM could be a better predictive model.
Similar content being viewed by others
Introduction
Quality of life-related to chronic obstructive pulmonary disease (COPD) is impaired, and it deteriorates significantly with increases in severity in COPD patients1. In addition to causing disabilities, the disease generates high healthcare costs and heavy socioeconomic burdens2. In 2013, the COPD prevalence in people aged over 40 years was 6.1% in Taiwan3, and it was the seventh leading cause of death in 2018. The estimated loss of life expectancy in patients at moderate and severe stages of the disease was 6.2 and 9.4 years, respectively4.
COPD-specific health-related quality of life instruments, including the CAT (COPD Assessment Test), CCQ (Clinical COPD Questionnaire), and SGRQ (St George's Respiratory Questionnaire), were designed to reliably assess the impact of the disease on patients. The CCQ and CAT have the advantage of being simpler to administer5. Furthermore, the CAT has been introduced as a tool for differentiating the severity of COPD in patients6. The usage of this questionnaire has been also extended to many clinical studies and practice guidelines7. The short and self-administered eight-item questionnaire consists of coughing, phlegm, chest tightness, breathlessness, home activities, leaving home, sleep problems, and energy. Each item has six levels ranked on a scale ranging from 0 to 5. Therefore, the total CAT scores will range from 0 to 40.
The CAT cannot be used directly for quality adjustments in the measurement of quality-adjusted life-years in a cost-utility analysis8. Therefore, studies using mapping algorithms to estimate European Quality of Life-5 Dimensions (EQ-5D) utilities from the CAT were investigated9,10. However, overestimated utilities were reported in COPD patients at advanced stages of the disease using the previous mapping algorithms from CAT scores, developed by Hoyle et al. in the UK9. A similar situation occurred by using the mapping algorithms, created by Lim et al. in South Korea and even greater underestimations of utilities were found for mild COPD cases10.. The aim of this study was thus to explore a suitable mapping algorithm for COPD patients in Taiwan.
Methods
The Institutional Review Board of National Cheng Kung University Hospital (NCKUH) approved this study before commencement (IRB number: B-ER-98-289 and B-ER-111-254). Informed consent was obtained from all subjects, and all methods were carried out in accordance with the relevant guidelines and regulations of the research ethics committee.
In this study, 323 patients were enrolled who were diagnosed with COPD in the outpatient Pulmonary Medicine Clinic at National Cheng Kung University Medical Center from April 2017 to December 2020. All patients were enrolled in the pay-for-performance program of COPD and had been receiving regular medical treatment for COPD for more than three months. These COPD cases were defined according to the GOLD diagnosis guideline and criteria7. All pulmonary function tests were performed according to a joint consensus of the American Thoracic Society and the European Respiratory Society11. Patients who were unwilling to participate, unable to receive the pay-for-performance program (for example, bed-ridden), or had advanced lung cancer and pulmonary fibrosis were excluded.
Participants were classified using the GOLD 2017 classification and were divided into four stages (mild to very severe), which corresponded to the GOLD 2017 grades 1 to 4, based on the post-bronchodilator forced expiratory volume in one second (FEV1): grade 1 or mild stage (FEV1 ≥ 80%), grade 2 or moderate stage (50% ≤ FEV1 < 80%), grade 3 or severe stage (30% ≤ FEV1 < 50%) and grade 4 or very severe stage (FEV1 < 30%)7. In this study, the participants with FEV1 < 50% were incorporated into the “severe” category to obtain a sufficient number in the sample for estimation.
The quality of life of the COPD patients was consistently monitored with the EQ-5D-3L and the CAT in order to develop an algorithm for estimating EQ-5D equivalent utilities from the CAT. The Validated Taiwanese version of EQ-5D-3L and Chinese version of CAT questionnaires were used in this study12,13.
The questionnaires were administered by the case manager of the pay-for-performance program at the outpatient department.
Model development
The COPD patient dataset was randomly split into a training group of 160 patients and a validation group of 163 patients. While the predictive model was built, the coefficients of the final model were derived based on the full sample (all 323 patients) to get the most accurate estimates. In this study, we considered two OLS-based procedures to build predictive models for COPD patients. The first was the model recommended by Hoyle et al.9. They regressed the EQ-5D utility on 8 CAT scores and chose 4 CAT scores (chest tightness, activities, confidence, and energy) with p-values smaller than 0.05 to build the final model. We created modified versions of the models by Hoyle et al. and Lim et al. that better fit the Taiwanese population. A backward elimination procedure was applied to obtain a final parsimonious model with a type I error rate of 0.05 when statistical hypothesis testing was performed.
Response mapping is another feasible approach that can be used for utility prediction14,15. While OLS is aimed toward predict EQ-5D utility, response mapping is aimed toward predict five EQ-5D scores, each taking values of 1, 2, or 3. The five predicted scores are then transformed to the utility. The transformation formula varies across different underlying populations. In this study, the formula, which is based on Taiwanese population obtained from Lee et al. was applied16. The formula was: EQ-5D-3L utility = 1–0.185–0.123*Mobility at level 2–0.272*Mobility at level 3–0.167*Self-care at level 2–0.276*Self-care at level 3–0.085*Usual activities at level 2–0.208*Usual activities at level 3–0.121*Pain/discomfort at level 2–0.261*Pain/discomfort at level 3–0.154*Anxiety/depression at level 2–0.282*Anxiety/depression at level 3–0.190*Any dimension on level 3.
Furthermore, because the EQ-5D score takes discrete values, a multinomial logistic regression (MLR) is a relevant model to use to identify the association between the score and covariates, 8 CAT scores, age, and sex. In our dataset, most of patients filled out both the EQ-5D and CAT questionnaires multiple times during the follow-up period. In other words, the experiment consisted of repeated measurements. Therefore, a generalized estimating equation (GEE) was applied with an independent working correlation for estimation and hypothesis testing17. For each EQ-5D score prediction, the final multinomial logistic regression was chosen so that the resulting QIC was minimized18. The GEE models were performed using the SAS GEE procedure.
We also applied the Mean Rank Method (MRM), developed by Wee, et al.19, as the other method for developing a predictive model of mapping EQ-5D utilities from CAT. The MRM considers nonparametric matching among EQ-5D and CAT scores, preventing potentially erroneous model assumptions and providing less interpretation information.
Validation
Among all applied methods, OLS, MLR, and MRM, the training data was used to build predictive models, whereas the validation data was used to evaluate the performance of these models via both the root mean squared error (RMSE) and mean absolute error (MAE). In addition, the models of Hoyle et al. and Lim et al. were modified by re-estimating their coefficients using the training group and validated by the validation group for comparison.
In this study, as expected, the model with the best predicting ability should have the smallest RMSE and MAE. Additionally, to visualize the potential prediction biases, we suggest the bubble chart drawn with R version 4.2.1statistical software, R Core Team (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/ in this paper. The best model should locate a majority of bubbles on the diagonal line of the bubble chart. As for statistical comparisons among groups, continuous variables were analyzed by t-test, and the categorical variables were analyzed by chi-square. All statistical analyses were conducted by using SAS version 9.4 statistical software (SAS Institute Inc., Cary, NC, USA).
Ethics approval
The Institutional Review Board of National Cheng Kung University Hospital (NCKUH) approved this study before commencement (IRB number: B-ER-98-289 and B-ER-111-254).
Results
In this study, 323 patients were recruited, and 2327 repeated measures were done. The mean number of visits for each patient was seven. The range of visit frequency was from one to sixteen times. Overall, the mean EQ-5D-3L utility and CAT total scores data were 0.917 and 9.88, respectively. Between the training and the validation group, the EQ-5D-3L utility and CAT total scores were similar at baseline. No differences were revealed over the distribution of patient characteristics (supplementary Table 1).
A negative correlation (-0.69) between EQ-5D-3L utility and the total CAT score was observed (Fig. S1). The distribution of the EQ-5D-3L utility and CAT scores from all eligible patients are presented in Fig. S2. The largest cluster was located at EQ-5D utility = 1: n = 1624 (69.79% of observations). The other clusters were 0.5 ≤ EQ-5D < 1: n = 638 (27.42%) and 0 ≤ EQ-5D < 0.5: n = 65 (2.79%).
Model development
Two MLR models were produced from training group using either the total CAT score or selected CAT items, the coefficients of the final model were derived based on all 323 patients to get the most accurate estimates (supplementary Table 2). The formula for developed models in this study, including the models developed by Hoyle et al. and Lim et al. and their modified versions are presented in Table 1. Estimated utility scores were presented with mean, maximum, and minimum scores for each model with subsets of the EQ-5D utility values, CAT scores, or FEV1 stages. The RMSE and MAE were calculated for each subset to examine the distribution of errors across the different disease severities (Table 2). An overestimation of the mean EQ-5D was presented among the patients categorized as having poor health (0 ≤ EQ-5D < 0.25 and 0.25 ≤ EQ-5D < 0.5) in both models.
Comparing with the models developed by Hoyle et al., Lim et al. and Wee et al., non-inferior predictive effectiveness over the total and selected CAT items was found based on the MAE and RMSE results (Table 3). According to the real-predictive bubble charts, the method developed for the purposes of this study revealed comparable predictive capability to the other models.. Based on these bubble charts, all of the models had good predictive accuracy for the COPD patients with a better health status (Fig. 1). Figure 1a showed the real-predictive bubble chart using the formula developed in this study. The selected CAT items model led to obtaining a more precise prediction than the total CAT score model. The accuracy of the predictive model was better in the case of patients with higher EQ-5D utility. Figure 1b showed the real-predictive bubble chart using the formula recommended by Hoyle et al. applied to Taiwan datasets, and Fig. 1c showed the predictive EQ-5D utility with the model of modified Hoyle et.al., the model was developed by using the CAT score in this study and equations based on OLS (ordinary least square) method from Hoyle et.al. The model with the M6_OLS equation showed overestimation for lower utility (utility ≤ 0.5) patients and underestimation for near health (utility > 0.9) patients. The model with the M3_OLS equation had better predictive power than the M6_OLS equation for Taiwan datasets, but overestimated lower EQ-5D (utility ≤ 0.5) patients. Figure 1d showed the real-predictive bubble chart presenting the predictive EQ-5D utility with the formula recommended by Lim et al. for Taiwan datasets, and Fig. 1e showed the model of modified Lim et.al.. Both models showed that better predictive effectiveness was reported with the CAT items equation for patients with higher EQ-5D utilities. However, poor predictive power with overestimation was found for patients with lower EQ-5D utilities (utility ≤ 0.6) in the models with the CAT total scores and CAT items equations. Figure 1f showed the real-predictive bubble chart presenting the predictive EQ-5D utility with the mean rank method, recommended by Wee et al. for Taiwan datasets. The accuracy of the predictive model was similar to the developed model in this study (Fig. 1a). The overestimation for low utility and underestimation for near health patients in models developed by Hoyle et al. and Lim et al. was improved in the presented model and model developed by Wee et al.
Discussion
In this study, backward elimination with response mapping was carried out to develop the formula by which to transform the CAT scores into an EQ-5D score. Subsequently, estimating the utility weights of the EQ-5D was implemented with time trade-off method based on the EQ-5D scores. The performance of selected CAT items model was better in terms of estimation effectiveness than CAT total scores model.
The RMSE and MAE are standard measurements for choosing the best model among several models. Therefore, both the RMSE and MAE were used to evaluate their predicting ability for developed models in this study. Compared with the models in Hoyle et al.9 and Lim et al.10, comparable predictive effectiveness over selected CAT items was found based on the RMSE and MAE results.
However, previous models severely overestimate the EQ-5D utilities when the true utilities are relatively low9,10. By contrast, the model proposed in this study was more appropriate for the patient with COPD in Taiwan. From the bubble charts, overestimated EQ-5D utilities for the actual low utility value (< 0.5) were more distinct using the previous models within the datasets.
In the case of the model recommended by Holey et al., overestimated EQ-5D utility was reported for the COPD patients with poor health (utility < 0.7) or the extremely severe cases (CAT: 31 ~ 40). Also, underestimation of EQ-5D utility was noted in COPD patients with near health status (utility > 0.9). In the model proposed by Lim et al., there was also overestimation in predicting EQ-5D utility reported in patients at the very severe stage of COPD, and underestimation of the EQ-5D utility was reported in COPD patients at the mild stage.
In literature, OLS based approaches are popular for mapping disease-related measurements onto the EQ-5D utility9,10. The OLS algorithm using the CAT profile, the M3_OLS, was recommended by Hoyle et al. In a comparison with the MLR algorithm in this study, a higher RMSE was reported for the MLR model after adjusting the coefficient of the OLS algorithm using the data from this study. However, a lower MAE was noted when using the MLR model for the datasets in the present study. Another mapping algorithm for EQ-5D-3L utility prediction of COPD patients, the OLS1 and OLS3 models recommended by Lim et al., were also applied to the datasets in the present study. An even higher RMSE, but a lower MAE, were reported using the MLR model even after adjusting the coefficient of the OLS algorithm with the data from this study.. When ranking the mapping algorithms based on the MAEs and RMSEs, the model developed for this study was found to be comparable with developed models by Holey et al. and Lim et al., However, when comparing our model with models from Holey et al. and Lim et al. by the real-predictive bubble charts, our model had better predictive effectiveness among patients with poor utility and near health in this study t.
This model recommended in the present study exhibited better predictive power than other models in terms of mapping EQ-5Q utility from the CAT for COPD patients in Taiwan. However, overestimated EQ-5D utilities were still observed for patients with poorer health status (actual utility < 0.5), because of small sample size. Therefore, the large prediction bias may have been due to the small sample size of very severe COPD patients or due to the heterogeneity arising because of COPD severity. This phenomenon calls attention to the fact that choosing the best model by merely considering one or two indices may result in an unexpected result.
The time trade-off values of coefficients in estimating quality weight of EQ-5D health states differ from country to country20,21. This difference might be due to the sociodemographic background of the respondents and methodological differences in studies21. And then, the difference could contribute not only one appropriate model for developing mapping algorithms. For example, the respondents in South Korea put more weight on mobility and self-care domains than the other three dimensions22. For UK respondents, the pain/discomfort domain was considered to be more important than the other four dimensions23. As for Taiwanese, the respondents devote their mind to the anxiety/depression domain more than others16.
Apart from the MAE and RMSE results, according the real-predictive bubble charts, the method developed from this study reveals comparable predictive capability to the models developed by Hoyle et a. in UK and Lim et al. in South Korea. The models from the UK, South Korea, and Taiwan group all presented well accuracy of prediction over the COPD patients with better health status. By contrast, poorer predictive performance was revealed under the models of UK and South Korea in the COPD patients with poorer utility and near health than the present models. We have tried to apply MRM, developed by Wee et al. as the other method for developing a predictive model of mapping EQ-5D utilities from CAT in this study. The predictive ability of the MRM model was better than the models of UK and South Korea and was similar to the present models base on real-predictive bubble charts. Therefore, other methods even developed not for mapping EQ-5D utilities from CAT in original, should be tried in the future to get the best predictive model for different populations.
Conclusions
Response mapping with MLR model and model using MRM method has comparable performance with OLS model for predicting EQ-5D utility from CAT in Taiwan. In addition, the overestimation for low utility patients and underestimation for near health in previous developed OLS models was improved in the presented models and model using MRM method. However, it is better to administer both CAT and EQ-5D-3L if the cost-utility analysis is planned for clinical trial or study; the mapping should be the last resort as it can only give an approximate utility value.
Data availability
Full data set are not available publicly currently for protecting patient privacy. However, the data can be obtained through a reasonable request to the corresponding author.
References
Jones, P. W. et al. Health-related quality of life in patients by COPD severity within primary care in Europe. Respir. Med. 105, 57–66 (2011).
GBD. Diseases and injuries collaborators. Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet 396, 1204–1222. https://doi.org/10.1016/S0140-6736(20)30925-9(2020) (2019).
Cheng, S. L. et al. COPD in Taiwan: A national epidemiology survey. Int. J. Chron. Obstruct. Pulmon. Dis. 10, 2459–2467 (2015).
Chen, C. Z. et al. Life expectancy (LE) and loss-of-LE for patients with chronic obstructive pulmonary disease. Respir. Med. 172, 106132 (2020).
Ringbaek, T., Martinez, G. & Lange, P. A comparison of the assessment of quality of life with CAT, CCQ, and SGRQ in COPD patients participating in pulmonary rehabilitation. COPD 9, 12–15 (2012).
Jones, P. W. et al. Development and first validation of the COPD assessment test. Eur. Respir. J. 34, 648–654 (2009).
Singh, D. et al. Global strategy for the diagnosis, management, and prevention of chronic obstructive lung disease: The GOLD science committee report 2019. Eur. Respir. J. 53, 1900164 (2019).
Adamiak, G. Methods for the economic evaluation of health care programmes, (3rd ed). J. Epidemiol. Community Health. 60:822–823 (2006).
Hoyle, C. K., Tabberer, M. & Brooks, J. Mapping the COPD assessment test onto EQ-5D. Value Health: The J. Int. Soc. Pharm. Outcomes Res. 19, 469–477 (2016).
Lim, J. et al. Mapping analysis to estimate EQ-5D utility values using the COPD assessment test in Korea. Health Qual. Life Outcomes 17, 97 (2019).
Miller, M. R. et al. & ATS/ERS Task Force. Standardization of spirometry. The Eur. Respir. J. 26, 319–338 (2005).
Chang, T. J. et al. Taiwanese version of the EQ-5D: Validation in a representative sample of the Taiwanese population. J. Formos. Med. Assoc. 106, 1023–1031 (2007).
Kwon, N. et al. Validity of the COPD assessment test translated into local languages for Asian patients. Chest 143(703–710), 14 (2013).
Gray, A. M., Rivero-Arias, O. & Clarke, P. M. Estimating the association between SF-12 responses and EQ-5D utility values by response mapping. Med. Decis. Making: An Int. J. Soc. Med. Decis. Making. 26, 18–29 (2006).
Rivero-Arias, O. et al. Mapping the modified Rankin scale (mRS) measurement into the generic EuroQol (EQ-5D) health outcome. Med. Decis. Making: An Int. J. Soc. Med. Decis. Making. 30, 341–354 (2010).
Lee, H. Y. et al. Estimating quality weights for EQ-5D (EuroQol-5 dimensions) health states with the time trade-off method in Taiwan. J. Formos. Med. Assoc. 112, 699–706 (2013).
Zeger, S. L. & Liang, K. Y. Longitudinal data analysis for discrete and continuous outcomes. Biometrics 42, 121–130 (1986).
Hardin, J.W. & Hilbe, J.M. Generalized Estimating Equations (1st ed.) (Chapman and Hall/CRC, 2002). https://doi.org/10.1201/9781420035285 (2021).
Wee, H. L., Yeo, K. K., Chong, K. J., Khoo, E. Y. H. & Cheung, Y. B. Mean rank, equipercentile, and regression mapping of world health organization quality of life brief (WHOQOL-BREF) to EuroQoL 5 dimensions 5 levels (EQ-5D-5L) utilities. Med. Decis. Making: An Int. J. Soc. Med. Decis. 38(3), 319–333. https://doi.org/10.1177/0272989X18756890 (2018).
Badia, X., Roset, M., Herdman, M. & Kind, P. A comparison of United Kingdom and Spanish general population time trade-off values for EQ-5D health states. Med. Decis. Making: An Int. J. Soc. Med. Decis. Making. 21, 7–16 (2001).
Roudijk, B., Donders, A., Stalmeier, P. & Cultural Values Group. Cultural Values: Can They Explain Differences in Health Utilities between Countries?. Med. Decis. Making : An Int. J. Soc. Med. Decis. Making. 39, 605–616 (2019).
Jo, M. W., Yun, S. C. & Lee, S. I. Estimating quality weights for EQ-5D health states with the time trade-off method in South Korea. Value Health: J. Int. Soc. Pharm. Outcomes Res. 11, 1186–1189 (2008).
Dolan, P. Modeling valuations for EuroQol health states. Med. Care 35, 1095–1108 (1997).
Acknowledgements
The study was funded by grants from the Ministry of Science and Technology (MOST107-2627-M-006-007, MOST 109-2314-B-006-091, MOST 110-2314-B-006-099). We are grateful to Chih-Hui Hsu for providing the excellent statistical consulting services from the Biostatistics Consulting Center, National Cheng Kung University Hospital.
Author information
Authors and Affiliations
Contributions
C.Z.C. was involved in the concept development and study design. All authors contributed to the data retrieval and patient recruitment. S.M.C. and C.H.Y. conducted the analysis. Paper drafting was mainly accomplished by and C.Z.C., and all authors contributed to revisions and approval of the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Yu, CH., Chang, SM., Hsu, CH. et al. Mapping algorithms for predicting EuroQol-5D-3L utilities from the assessment test of chronic obstructive pulmonary disease. Sci Rep 12, 20930 (2022). https://doi.org/10.1038/s41598-022-24956-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-022-24956-2
- Springer Nature Limited