Introduction

Systemic lupus erythematosus (SLE) is an autoimmune inflammatory connective tissue disease that affects multiple organs in young women1.The highest incidence of SLE is in North America, with a prevalence rate of 241/100,000/year. Asian and Hispanic ethnic groups have middle incidence. SLE is a chronic progressive autoimmune disease that is easily recurrent and affects multiple systems. Studies found that the death of the SLE patients in Chinese population had higher concurrent infection and serious complications2,3. Corticosteroids and immunosuppressants remain the main treatment strategies for SLE when disease progress. With the new development of stem cell transplantation or biological agents from cell or molecular biology in the treatment of the SLE, the clinical outcomes benchmark was catching attention4.

To evaluate the clinical outcomes of SLE, Quality of life (QOL) or patient-reported outcomes (PRO) is often been used to evaluate the effect for medical decision. Both generic and disease specific QOL instruments are used to investigate the changes of SLE patients in clinical practice. Although generic QOL instruments such as the Brief Version of World Health Organization Quality of Life WHOQOL-BREF, SF-36 are commonly adopted for chronic diseases or tumors patients, the results were found to be less sensitive or irrelevant to patients5. Consequently, some specific QOL/PRO instruments were developed for SLE patients, including SLE symptom checklist (SSC)6, the SLE Quality of Life Questionnaire (SLEQOL)7, Lupus Quality of Life (LupusQol)8, the SLE Quality of Life Scale (L-QoL)9 etc.

In China, the prevalence of SLE is higher than that in Western countries, and the mortality rate in the past is very high. The 10-year survival rate is greater than 80%, but there is no cure, and patients need long-term medication. There is also an urgent need for quality of life measurement and assessment. However, due to cultural dependency on quality of life, the existed QOL scales are not very suitable for Chinese. For example, the family relationship and kinship play very important roles in daily life in Chinese culture. Taoism and traditional medicine focus on good temper and high spirit. Good appetite, sleep, and energy are highly regarded in daily life. As a result, our QOL team developed a system entitled Quality of Life Instruments for Chronic Diseases (QLICD), which included a general module (QLICD-GM), and some specific modules for different diseases10,11. The latest version of the system QLICD (V2.0) contained 34 chronic disease-specific scales11, including QLICD-SLE(V2.0) for Systemic Lupus Erythematosus12 etc. Among them, the QLICD-SLE(V2.0) was developed by combining the 28 items general module of chronic diseases (QLICD-GM) and the 19 items specific module of SLE and confirmed good psychometric properties, with the QLICD-GM being classified into three domains of physical function (9 items), psychological function (11 items), and social function (8 items)12.

For evaluating of clinical outcomes, the interpretation of the Quality of life and clinical efficacy questionnaire score is particularly important13,14. However, the benchmark of QOL/PRO of calculation of scores that collected from subjective feeling somehow would still give statistical but not clinical meaningful changes. Therefore, the calculation of minimum clinical importance difference (MCID) by the anchor and distribution methods was highly recommended internationally13,14. Up to now, there is also great terminological confusion and controversy about the terms MCID, MID (Minimal importance difference) and MIC (Minimal important change) etc14,15,16,17. In this paper, we prefer to MCID and its classical definition in 1987 for it provides more powerful evidence for the interpretation of clinical conclusions, and it reflects the clinical significance of the smallest change in scores, in which MCID is defined as the minimum questionnaire dimension score change recognized by the patient without considering side effects and costs, it is a bridge between statistics and clinical establishment relationship13,14. According to the literature18,19,20,21,22, there are two traditional methods for calculating MCID, including anchor-based methods and distribution-based methods. The advantage of anchor-based methods is that clinical indicators can be used as anchors for calculating MCID, but the disadvantage is that it is difficult to find a suitable anchor having higher correlation with scores.

The MCID value has not been established for the QLICD-SLE (V2.0), although the MCID in determining the medical decision or important for predicting the prognosis of SLE is important. This study is aimed to explore the recently presented ROC method23 to calculate the MCID value of the QLICD-SLE (V2.0) for it has much more advantages23,24, in which an anchor item was used as the gold standard, and patients were grouped according to the results of the anchor item to construct ROC curve and calculate MCID of the QLICD-SLE (V2.0).

Methods

Participants

SLE patients who were diagnosed and followed up during their consultation at the Hospital of Kunming Medical University and Zhanjiang hospital were recruited to the participants. The recruitment was started from 2018.9 to 2019.5.

Inclusion criteria: (1) cognitive and physical capable of self-completion of questionnaire; (2) agreed to participate in the survey with the sought of informed consent from a trained research assistant.

Exclusion criteria: Subjects were excluded according to the following exclusion criteria: (1) diagnosed of any mental illness who could not facilitate the self-completion of the questionnaire; (2) cognitive impairment, those who cannot fill in the questionnaire themselves; (3) refused to participate in the research.

Survey methods

This project was reviewed by the Ethics Committee of Guangdong Medical University and Kunming Medical University and found that the qualifications and experience of the project investigator met the requirements, and the research plan and the subject selection plan were in line with scientific and ethical principles. The investigators explained the aims of the trial and the instrument to the patients older than 16 years and obtained informed consent from those patients who agreed to participate in the study and met with the inclusion criteria. The informed consent has been obtained from their parents for study participation for minors under 16 years old. All methods are carried out in accordance with relevant guidelines and regulations.

With the consent being sought of the subjects, they completed the QLICD-SLE (V2.0) scale and the anchor item on the day of admission. Respondents answered completely according to their actual situation. The completed questionnaires were checked immediately by research assistant to ensure no missing value. If any missing items were found, subjects would be asked for completion for follow up assessment. The patients were asked to complete the same questionnaire before the discharge.

The anchor item come from the first item Q1 in the SF-36 scale “overall, your health: (1) Excellent, (2) very good, (3) good, (4) fair, (5) poor”, and thus has five categories and five points.

Scoring methods

The QLICD-SLE (V2.0) contains 47 items, of which 19 items focused on the symptoms of SLE while 28 items for reflection of the physical psychological and social health12. Each question adopts a 5-level scoring method, in which forward questions (that is, as the score increases, the higher the quality of life) is directly based on the original score to calculate 1, 2, 3, 4, 5 points, and the reverse question (that is, as the score increases, the lower the quality of life), the score will be reversed, that is, 5, 4, 3, 2, 1, score. Add the scores of all the items included in each domain to get the score of this domain, and add the scores of all the items included in the scale to get the score of the total scale.

In order to compare the scores of different domains with each other, the raw scores of all domains and the overall are converted into standard scores ranging from 0 to 100 by the formula S = (X-Min) × 100/R, S is the standard score, and X is the raw score, Min is the minimum score of the domain or total scale, R is the range of the domain or total scale (that is, the difference between the maximum score and the minimum score)10,12.

Statistical methods

The primary outcome of this study was to establish the MCID for the QLICD-SLE(V2.0). The specific steps for formulating MCID using ROC curve as recommended and adopted here23:

Anchor-based grouping

If the patient had the same answer on the anchor item on the first day of hospitalization as the day before discharge, they were classified as "no change group". If the answer on the anchor item on the first day of hospitalization and the day before discharge of the patient had changed at least 1 point, they were classified as an "improvement group" or vice versa “deterioration group". Anchor groups were classified according to the changes in the options of anchor item before and after treatments, which are called anchor grouping.

As the first item Q1 in the SF-36 being the anchor, it was classified into three categories: "improvement group = 1 + 2", "no change group = 3 + 4", and "deterioration group = 5".

Determine what MCID to calculate

The aim of this study is to calculate MCID values, and this article clarifies that the minimum clinical importance improvement value will be established, for which patients in the improvement group and no change group will be included in the study. The upper 95% upper limit of the distribution map of the "no change" group is defined as the value with significant improvement (MCID), and the calculation formula is as follows: MCID = Meanchange + 1.645SDchange, and the minimum value of the no change group and its upper limit of 95% are the value range of the MCID value.

Construction of ROC curve

The anchor item was used as the gold standard of classification. The cut-off points in the range of values were used for the test classification, and the sensitivity and specificity of each cut-off point were calculated. Draw points with 1-specificity was used as the abscissa and sensitivity for the ordinate. The cut-off point when the Youden index in the ROC curve was the largest would be estimate as the MCID value.

Medcalc software is used for the statistical analysis of this study. In terms of sample size, according to the article of Hanley et al25., the minimum sample size required for ROC calculation is 172 patients (an error = 0.10; power = 0.90). It is estimated by the variation of the total score of the QLICD-SLE(V2.0) using Medcalc software, and can also basically meet requirement for all domains because of all of them ranging from 0 to 100. Considering a 5% loss rate, this part of the study requires 181 patients at least.

Results

Basic information on the sample

279 patients participated in two measurements before and after treatments and were used to establish MCID. The basic information of 279 patients was shown in Table 1. The sample consisted of 54.1% of age < 30 years old, 28.7% of age 30–39. Most SLE patients were female (92.1%), Han nationality (93.2%) and married (62.9%).

Table 1 Characteristics of the Sample (n = 279).
Full size table

The average time between the two assessment time points was 73 days with minimum 3 days and maximum 379 days.

QOL Scores of patients with SLE

The score of QLICD-SLE (V2.0) at baseline, follow-up assessment and the corresponding changes were shown in Table 2. It can be seen that the score of the QLICD-GM before treatments is 66.0 ± 13.8,, and the score after treatments is 65.6 ± 13.7. QLICD-GM was a score that reflects the physiological, psychological, and social functions of a patient, with a wide coverage, so the score was greatly affected by many factors. The score of the specific module was 66.7 ± 13.6. before treatment and 67.1 ± 13.4. The score of the patient-specific module has improved after treatment, which means that the patient’s disease has improved after treatment. The same goes for the total scale score.

Table 2 Standard score of QLICD-SLE(V2.0) and Correlations with Anchor (n = 279).
Full size table

Patient’s distribution and correlation by anchor

The number of patients in each category of the anchor at two assessment time points was presented in Table 3.

Table 3 The number of patients in each category of the anchor at two assessment time points (n = 279).
Full size table

The correlation coefficient between the anchor item and the total score of QLICD-SLE(V2.0) scale was 0.43, and the correlation coefficient with the physiological domain, psychological domain, social domain, and specific module for systemic lupus erythematosus were 0.46, 0.37, 0.37, 0.27, respectively. These correlation coefficients were also showed in Table 2 in detail.

Group and QOL scores based on the anchor

In this study, the correlation coefficient between the score of the patient anchor and the total score of the scale was 0.43. It is generally considered that the correlation is greater than 0.3. Depending on the changes in the scores of the anchor item before and after treatment, the patients were divided into 3 groups. 56 of the 279 patients, the anchor item score after treatment minus the anchor item score before treatment was greater than 0. Therefore, these patients were included in improvement group and the mean value of the score difference of the QLICD-SLE (V2.0) was 4.7 ± 10.5. 169 of the 279 patients, the anchor item score after treatment minus the anchor item score before treatment was 0. Therefore, these patients were included in No change group and the mean value of the score difference of the QLICD-SLE (V2.0) was 0.3 ± 8.1. 54 patients had a negative change after treatment with mean value of the score difference of the QLICD-SLE (V2.0) being − 3.4 ± 10.0.

The score information of different domains of the QLICD-SLE (V2.0) by groupings on anchor was shown in Table 4.

Table 4 Difference in scores of scale QLICD-SLE(V2.0) under anchor grouping (n = 279).
Full size table

MCID values estimate by the ROC for the QLICD-SLE (V2.0)

The area under the curve (AUC) of the QLICD-SLE (V2.0) were greater than 0.8 and the p < 0.001. The cut-off point was 3.2 of QLICD-SLE (V2.0) (sensitivity = 100%, specificity = 66.9%). In other words, before and after treatment, the QLICD-SLE (V2.0) score should increase by at least 3.2 points before the treatment was considered effective. Therefore, the MCID of the total score of QLICD-SLE (V2.0) was estimated as 3.2.

Similarly, if it is only for the Specific domain of patients with systemic lupus erythematosus, the score must be increased by at least 9.2 points (sensitivity = 75.0%, specificity = 82.8%) before clinical treatment was considered meaningful. The results of MCID values corresponding to different domains in the scale were shown in Table 5. It can be seen that the MCID of physical domain, psychological domain, social domain, QLICD-GM, specific domain and total of QLICD-SLE (V2.0) were 8.3, 2.3, 2.5, 2.7, 9.2 and 3.2, respectively.

Table 5 MCIDs estimation of the QLICD-SLE and relevant indicators by ROC curve method (n = 181).
Full size table

The ROC curves of the specific module and the total scale were shown in Figs. 1 and 2, respectively.

Fig. 1
figure 1

ROC graph of the QLICD-SLE (The sensitivity VS 100-specificity of the QLICD-SLE). Blue solid line: Fitted curve, Blue dotted line: 95% confidence interval of ROC curve, Red solid line: Random line, meaningless.

Full size image
Fig. 2
figure 2

ROC graph of the Specific domain (The sensitivity VS 100-specificity of the specific domain of the QLICD-SLE). Blue solid line: Fitted curve, Blue dotted line: 95% confidence interval of ROC curve, Red solid line: Random line, meaningless.

Full size image

Discussions

In China, there are few reports on the development of quality of life scales for patients with systemic lupus erythematosus. Therefore, our QOL research team developed the QLICD-SLE (V2.0) by modular approach with some advantages12. However, the MCID value is not established for this systemic lupus erythematosus scale. In this study, the MCID value was established by the ROC curve method, which further made up for the shortcomings in the actual application of the scale and facilitated the clinical evaluation of the patient's treatment effect through the change of the scale score.

There are four common methods for calculating MCID values, such as anchor-based method, distribution-based method, etc15,22. In recent years, due to the shortcomings of traditional methods, Roc curve method, multiple regression method, etc. were proposed. With the combination of illustrating the sensitivity and specificity in the same graph, the Roc curve method can facilitate the statistically interpretation with high accuracy26,27. And thus the method was recommended to formulate the MCID in other important PRO instruments such as Wu et al28. use ROC curve calculating MCID for work productivity and activity impairment (WPAI) questionnaire.

In the ROC curve, the cut-off point corresponding to the maximum Youden index has relatively high sensitivity and specificity. Compared with the traditional methods for formulating MCID values, the ROC curve-based method effectively combines the advantages of the anchor method and the distribution method. ROC curve-based method takes into account measurement errors and makes full use of the sample size as much as possible.

The anchor selection is very important for ROC curve method. In practice, it is difficult to find a very appropriate anchor and different anchors will also produce different results. Some literatures29 pointed out that using the data of the entire cohort to construct the ROC curve, the accuracy of formulating the MCID is higher. It is believed that the sufficiency of the anchor is strong if the correlation coefficient between the scale score and the anchor score is greater than 0.322. In this study, the correlation coefficient between the score of the anchor and the total score of QLICD-SLE (V2.0) before the treatment is 0.43. And the correlation coefficients between the score of the anchor and all domain scores was greater than 0.30 (exception of the specific module 0.27). Therefore, the anchor item is suitable for using as a comprehensive indicator to reflect the overall health of patients with systemic lupus erythematosus.

To formulate the minimum clinically significant improvement difference of the QLICD-SLE (V2.0) scale, patients were divided into three different groups according to the changes in the response of the anchor items before and after treatment. According to Table 4, the mean value for different groups of the total score of QLICD-SLE (V2.0) were: + 4.7 (improved group) >  + 0.30(no change group) > -3.4(deteriorated group). This trend is also the same for all domain scores and it implied the grouping of patients based on anchor item was feasible. For easy reference, the negative items in the scale were transformed positively, making the scale score more easily interpreted with the patient's health status. If the treatment is clinically meaningful to the patient, the patient's health will also be improved. After the treatment, the patient's score on the QLICD-SLE (V2.0) scale should be higher than before the treatment, that is, after the treatment. The scale's standardized score minus before treatment should be positive. In this study, the difference was used as the test variable, and the classification based on the anchor item was used as the gold standard to construct the ROC curve.

The MCID of the total score of QLICD-SLE (V2.0) is 3.2. The patient's scale score after treatment is at least 3.2 points higher than that before treatment, and the clinical treatment is considered meaningful. At the domain level, the MCID of physical domain, psychological domain, social domain, QLICD-GM, specific module were 8.3, 2.3, 2.5, 2.7, 9.2, respectively, and these values likewise have the meaning of the lowest score limit. It should be noted that these thresholds are meant for interpreting patient-level data (within-patient change), but not for interpreting mean differences between patient groups or mean change over time. Through MCID determination, it is possible to further explore the different treatment methods, which is more effective and other research.

It should be noted that this paper has also some limitations. First, this paper only focuses on the MCID of improvement, but not MCID of deterioration because of small sample size. Theoretically, ROC curve method can be also used to establish MCID of deterioration group if it has sufficient sample size. Second, this research only explores to establish MCID by ROC curve method. Other methods can also be used to formulate MCID such as anchor-based method, the distribution-based method and the response cumulative distribution function method20,30. Our next research will compute and compare all MCID results of these methods. Additionally, the samples of this study mainly come from the southern cities of Guangdong and Yunnan province with being no statistical difference in the scores of patients in the two regions before treatment (t = 1.600, p = 0.111). The economic situation in southern China is relatively good, and the population flow is relatively large. Selecting patients from the above areas for investigation can reflect the status of patients with lupus erythematosus in the Chinese system to a certain extent. Anyhow, a large sample of patients from other locations is needed to obtain more stable MCID values.

To sum up, formulating MCID value based on ROC curve as a new method has both advantages and disadvantages. The ROC curve intuitively combines sensitivity with specificity and accurately reflects the specificity and sensitivity of an analytical method. However, in the formulation of MCID, an appropriate anchor needs to be selected as the classification gold standard. The MCID value of the total score of QLICD-SLE (V2.0) developed in this study is 3.2, which can assist in determining whether the treatment has a clinical effect on SLE patients. However, due to the number of study subjects and the limitations of the survey points, the effectiveness of its promotion and use needs further verification.

Conclusion

The formulation of the scale MCID has always been the main concern of the questionnaire application. Existing methods for formulating MCID values, such as the anchor-based method and the distribution-based method, have advantages and disadvantages. Today, the ROC curve method has some significant advantages, but some patients will be lost due to the inclusion principle and require larger data support. Although imperfect, it is better than existing methods to a certain extent. The shortcomings of the ROC curve in formulating the MCID value are shown in the following aspects: (1) How to define the minimum change interval, whether to change one point or more points. (2) One of the keys to formulating MCID for ROC curve is the selection of an appropriate anchor. This does not have an accurate conclusion. Therefore, the formulation of the MCID value still needs to refer to the values of various methods for comprehensive evaluation.