Abstract
Background
The project to develop a new Japanese Orthopaedic Association (JOA) score rating system for low back disorders, the JOA Back Pain Evaluation Questionnaire (JOABPEQ), is currently in progress. Part 1 of the study selected 25 “candidateȝ items for use on the JOABPEQ. The purpose of this current Part 2 of the study was to verify the reliability of the questionnaire.
Methods
A total of 161 patients with low-back disorders of any type participated in the study. Each patient was interviewed twice at an interval of 2 weeks using the same questionnaire. The reliability of the questionnaire was evaluated by determining the extension of the kappa and weighted kappa coefficients.
Results
Both kappa and weighted kappa were more than 0.50 for all but one item, which was 0.48. The lower 95% confidence interval exceeded 0.4 in all but two items, which was 0.39. This implied that the test–retest reliability of JOABPEQ was acceptable as a measure of outcome.
Conclusions
The tentative questionnaire of the JOABPEQ with 25 items was confirmed to be reliable enough to describe the quality of life of patients who suffer low back disorders.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Measurement of the outcome is critical for any decision-making and results of evaluations in all medical circumstances. This should be applicable when managing patients who have lumbar spine-related problems. The Japan Orthopaedic Association (JOA) developed and published a specific instrument to measure outcomes for patients with low back problems in 1986.1 It was called the JOA score rating system for low back pain, with a full score being 29 points. Since then, the instrument has been widely utilized to evaluate the functional results of many types of intervention for patients with such problems. It has been referred to not only in articles by Japanese investigators2 but also in those by non- Japanese-speaking investigators.3,4 One of the major criticisms of this specific instrument, however, is that it is not a patient-oriented measurement but a physician-based one. It is now widely accepted that a patient’s perspective is essential for making medical decisions and for evaluating the results of interventions.5 Based on the current needs for measuring outcome, the JOA was urged to revise its original score rating system and to develop a new one. In 2002, a Subcommittee on Evaluation of Back Pain and Cervical Myelopathy was organized in the Clinical Outcome Committee of JOA, and work began on revising the original JOA scoring system.
This revision process consisted of four steps: Parts 1 to 4. As described in the previous literature concerning Part 1, the original JOA scoring system was revised and a new scoring system (the JOA Back Pain Evaluation Questionnaire — JOABPEQ) was developed.6 The key points of this revision were to make the original JOA score more patient-oriented. For the survey in the Part 1 study, we first created a preliminary questionnaire consisting of 60 items. The questionnaire was a self-administered, disease-specific measure that was created with reference to the Japanese editions of the short form health survey with 36 questions (SF-36)7 and the Roland-Morris Disability Questionnaire (RDQ)8 to assess health-related quality of life. From the survey, a total of 25 items were selected for tentative use on a draft of the JOABPEQ (Table 1).
The purpose of the Part 2 study in this project was to evaluate the reliability of the 25 items selected for the draft JOABPEQ; for this, test-retest reliability was ascertained.
Materials and methods
Recruitment of patients
Altogether, 460 of the 829 Japanese board-certified spine surgeons were randomly selected, and each was asked to recruit two patients to evaluate the JOABPEQ between January and June 2004. The recruited patients were scheduled to reply to the questionnaire twice at a 2-week interval. Patient criteria were as follows: (1) patients could be any age of either sex; (2) patients had any lumbar spine disorder and were currently visiting an outpatient clinic; (3) the severity of the symptoms was expected to be at the same level between the two interviews. Exclusion criteria were the presence of: (1) other musculoskeletal diseases requiring medical treatment; (2) psychiatric disease (e.g., dementia), potentially leading to inappropriate answers; (3) a postoperative condition; (4) having participated in previous surveys of the related study.
Testing the questionnaire
Each patient was asked to complete the same questionnaire twice at an interval of 2 weeks (±3 days). The attending surgeon filled out the patient information on the diagnosis and the presence or absence of concomitant diseases, followed by judging the severity of symptoms using a three-step rating scale (mild, moderate, severe). Symptom severity was determined subjectively by the attending surgeon, who was asked not to select a similar patient solely on the basis of severity. Patients who had the same level of severity as judged by all surgeons were then selected and analyzed to verify the reliability of the questionnaire.
This study was approved by the Ethics Committee of the Japanese Society for Spine Surgery and Related Research. Informed consent was obtained from each patient.
The reliability of the questionnaire was evaluated by determining the extension of the kappa coefficients. The weighted kappa coefficient was calculated in the items with three choices or more. The kappa and weighted kappa coefficients were calculated based on a formula using Microsoft Office Excel 2003. Kappa and weighted kappa coefficients of 0.4 or above were judged to be reliable.9 The 95% confidence intervals (95% CI) were calculated for all reliability coefficients using the bootstrap method.
Results
Patient characteristics
A total of 350 patients participated in this study and completed the questionnaire twice following the project’s plan. However, 135 patients were excluded because the severity of their symptoms had changed between the two interviews or they violated the interval period. Of the remaining 215 patients, 54 were ineligible because of other musculoskeletal diseases, such as knee and hip osteoarthrosis. As a result, a total of 161 patients were available for the analysis in this study: 86 men and 75 women with a mean age of 57.7 years (SD 16.3 years). The clinical diagnosis included degenerative lumbar canal stenosis in 49 patients, lumbar disc herniation in 44, spondylolisthesis in 20, spondylosis in 16, degenerative disc disease in 13, mechanical low back pain in 11, and miscellaneous in 8. The patients’ age varied from their twenties to their eighties, and symptom severity varied from mild to severe (Table 2). Neurological and physical status was evaluated for each patient using the current JOA score rating system and finger-floor distance (Table 3). Neurological deficits varied from mild to severe, and trunk flexibility varied among the subjects as well.
Face validity
Face validity was checked in terms of the completion rate for filling out the questionnaire. The distribution of the answers for all question items was then checked to ensure that there were no biased answers. Items remaining unanswered accounted for less than 5% in the first test, and there was no skewed distribution, such as “floor and ceiling” effects (Table 4).
Reliability
The test-retest reliability was confirmed by calculating the kappa and weighted kappa coefficients for each item (Tables 5A, 5B). Both kappa and weighted kappa were more than 0.50 in all items, except in one item with 0.48. The lower 95% CI exceeded 0.4 in all items, except in two items with 0.39. This implied that the test-retest reliability of JOABPEQ was acceptable as a measurement of outcome.
Discussion
Measurement of the outcome is generally divided into two categories: generic and disease-specific measures.5,10 SF-36 has been commonly used as representative of a measurement of generic health status.5,7,10 The RDQ and the Oswestry Disability Index are widely used as disease-specific measurements for back pain.8,11 The JOA score rating system for low back pain, developed in 1986, was also a disease-specific measuring instrument for back disorders and injuries and has been widely utilized in clinical research and the decision-making process in Japan. However, this is not a patient-based outcome measure reliable enough to describe the objective status of the function and quality of life (QOL) of patients with low-back disorders. There has, to date, been insufficient psychometric analysis to confirm the validity and reliability of this JOA score rating system.
The project for developing the new questionnaire, JOABPEQ, was initiated to create a self-administered, disease-specific method for measuring low back pain. This instrument should include functions of the lumbar spine as well as health-related QOL. The reliability of the questionnaire that includes the 25 suggested items was evaluated using psychometric analysis as Part 2 of this project. Kappa and weighted kappa coefficient were utilized to verify the test-retest reliability.12,13
In terms of external validity, biased data were inevitable because one criterion that was included was that the severity of the symptoms was expected to be at the same level between the two interviews. However, there was no bias on the choices of answer to each question. This implies that test-retest reliability was acceptable even if the subjects had symptoms of different severity. The older the patients were, the worse was the interpretation of each question. There were small numbers of patients of younger generations, such as those in their thirties and forties, in this study. Thus, the reliability would not deteriorate even if the number of young people were to increase.
In terms of English expression, there is a possibility of ambiguity in questions 1–2 and 1–11, where double negatives (two “no’s” in the answer) may be confusing. It is necessary to reconsider and revise the English expression so it is more easily understood by native English-language users. The number of choices for the answer in all questions varied from two to five, which is also a point to be reconsidered in the future.
The current study demonstrated that the 25 items had enough reliability to describe the QOL in patients suffering low back disorders. However, further studies are needed to complete the project, including a factor analysis to determine the underlying cluster of the questionnaire items, a formula for calculating the severity score, and confirmation of the responsiveness to the questionnaire.
Conclusions
The tentative JOABPEQ with 25 items was confirmed to be reliable enough to describe the QOL of patients suffering low back disorders.
References
S Izumida S Inoue (1986) ArticleTitleAssessment of treatment for low back pain J Jpn Orthop Assoc 60 391–4
K Takahashi H Kitahara M Yamagata M Murakami K Takata K Miyamoto et al. (1990) ArticleTitleLong-term results of anterior interbody fusion for treatment of degenerative spondylolisthesis Spine 15 1211–5 10.1097/00007632-199011010-00022 Occurrence Handle10.1097/00007632-199011010-00022 Occurrence Handle1:STN:280:DyaK3M%2Fps1SksA%3D%3D Occurrence Handle2267618
H Tiusanen H Hurri S Seitsalo K Osterman R Harju (1996) ArticleTitleFunctional and clinical results after anterior interbody lumbar fusion Eur Spine J 5 288–92 10.1007/BF00304342 Occurrence Handle10.1007/BF00304342 Occurrence Handle1:STN:280:DyaK2s%2Fnt1yltQ%3D%3D Occurrence Handle8915632
GK Jeong JA Bendo (2003) ArticleTitleLumbar intervertebral disc cyst as a cause of radiculopathy Spine J 3 242–6 10.1016/S1529-9430(02)00445-X Occurrence Handle10.1016/S1529-9430(02)00445-X Occurrence Handle14589206
JA Kopec (2000) ArticleTitleMeasuring functional outcomes in persons with back pain: a review of back-specific questionnaires Spine 25 3110–4 10.1097/00007632-200012150-00005 Occurrence Handle10.1097/00007632-200012150-00005 Occurrence Handle1:STN:280:DC%2BD3M7jt1Cnuw%3D%3D Occurrence Handle11124726
M Fukui K Chiba M Kawakami S Kikuchi S Konno M Miyamoto et al. (2007) ArticleTitleJOB Back Pain Evaluation Questionnaire: initial report J Orthop Sci 12 443–50 10.1007/s00776-007-1162-x Occurrence Handle10.1007/s00776-007-1162-x Occurrence Handle17909929
S Fukuhara S Bito J Green A Hsiao K Kurokawa (1998) ArticleTitleTranslation, adaptation, and validation of the SF-36 health survey for use in Japan J Clin Epidemiol 51 1037–44 10.1016/S0895-4356(98)00095-X Occurrence Handle10.1016/S0895-4356(98)00095-X Occurrence Handle1:STN:280:DyaK1M%2Fjt1Sisw%3D%3D Occurrence Handle9817121
Y Suzukamo S Fukuhara S Kikuchi S Konno M Roland Y Iwamoto et al. (2003) ArticleTitleValidation of the Japanese version of the Roland-Morris Disability Questionnaire J Orthop Sci 8 543–8 10.1007/s00776-003-0679-x Occurrence Handle10.1007/s00776-003-0679-x Occurrence Handle12898308
JR Landis GG Koch (1977) ArticleTitleThe measurement of observer agreement for categorical data Biometrics 33 159–74 10.2307/2529310 Occurrence Handle10.2307/2529310 Occurrence Handle1:STN:280:DyaE2s7jsFWqtA%3D%3D Occurrence Handle843571
C Bombardier (2000) ArticleTitleOutcome assessments in the evaluation of treatment of spinal disorders: summary and general recommendations Spine 25 3100–3 10.1097/00007632-200012150-00003 Occurrence Handle10.1097/00007632-200012150-00003 Occurrence Handle1:STN:280:DC%2BD3M7jt1CntQ%3D%3D Occurrence Handle11124724
M Roland J Fairbank (2000) ArticleTitleThe Roland-Morris Disability Questionnaire and the Oswestry Disability Questionnaire Spine 25 3115–24 10.1097/00007632-200012150-00006 Occurrence Handle10.1097/00007632-200012150-00006 Occurrence Handle1:STN:280:DC%2BD3M7jt1Cmsg%3D%3D Occurrence Handle11124727
H Brenner U Kliebsch (1996) ArticleTitleDependence of weighted kappa coefficients on the number of categories Epidemiology 7 199–202 Occurrence Handle1:STN:280:DyaK28vjtFCrsg%3D%3D Occurrence Handle8834562 Occurrence Handle10.1097/00001648-199603000-00016
B Hernandez-Cruz MH Cardiel (1998) ArticleTitleIntra-observer reliability of commonly used outcome measures in rheumatoid arthritis Clin Exp Rheumatol 16 459–62 Occurrence Handle1:STN:280:DyaK1czntFOitQ%3D%3D Occurrence Handle9706428
Author information
Authors and Affiliations
Additional information
This report was composed by the Subcommittee on Low Back Pain and Cervical Myelopathy Evaluation of the Clinical Outcome Committee of the Japanese Orthopaedic Association
Rights and permissions
This article is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.
About this article
Cite this article
Fukui, M., Chiba, K., Kawakami, M. et al. Japanese Orthopaedic Association Back Pain Evaluation Questionnaire. Part 2. Verification of its reliability. J Orthop Sci 12, 526–532 (2007). https://doi.org/10.1007/s00776-007-1168-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00776-007-1168-4