Background

Pain in children and youth with rheumatic conditions such as arthritis is common [1, 2]. Currently, there is no standardized approach guiding the clinical assessment of pain in children and youth presenting to pediatric rheumatologists and other allied health professionals. Consequently, large variations exist across rheumatology practices in the evaluation and treatment of pediatric pain. It has recently been recommended that pain assessment, along with other quality measures, should be routinely implemented in pediatric rheumatology practices as a means of tracking outcomes and generating quality improvement indices [3]. Thus, a consistent, comprehensive, and clinically feasible approach is needed to effectively assess, treat, and monitor pain outcomes in the pediatric rheumatology population.

As pain is a multi-dimensional experience, appropriate pain assessment requires evaluation of sensory, affective and cognitive dimensions and the impact of pain on aspects of a child's life. The Pediatric Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (PedIMMPACT) recently completed consensus guidelines for dimensions of pain to assess as outcomes in clinical trials [4]. These recommendations offer a foundation for evaluating the key components of pain to be integrated into clinical assessment, including pain intensity, global rating of satisfaction with pain treatment(s) received, additional symptoms and adverse events, physical functioning, emotional functioning, role functioning, sleep, and economic factors. However, no consensus guidelines exist for the assessment of these multiple dimensions of pain in routine rheumatology practice.

The ultimate aim of this study was to develop and test the feasibility of a Standardized Universal Pain Evaluation for pediatric rheumatology providers (SUPER-KIDZ). This aim was accomplished using a 2-phased approach. First, consensus was sought from rheumatologists and pediatric pain experts on the most important pain domains to assess during routine clinical rheumatology practice, using the domains recommended by PedIMMPACT as a preliminary guide. Second, feasibility was evaluated by administering the pain assessment tool in three formats (paper-, laptop-, and handheld computer-based applications) and determining acceptability, efficiency, and missed responses in four large pediatric rheumatology clinics. In the present study, we elected to evaluate medium feasibility before conducting validity testing as test validity has previously been shown to be sensitive to the test medium employed [5].

Methods

Phase 1: Developing consensus on the SUPER-KIDZ Pain Measure

A 2-stage Delphi technique was used to develop consensus amongst Childhood Arthritis and Rheumatology Research Alliance (CARRA) members (pediatric rheumatologists and other allied professionals) as to what aspects of pain assessment to include in the evaluation of children and youth presenting to rheumatology clinics. For purpose of this study, the word 'child' or 'children' refers to persons aged 4-7 years and the word 'youth' refers to persons aged 8-18 years. The Delphi method was originally developed by the RAND Corporation in the 1950s to obtain reliable consensus of opinion from a group of experts through a series of questionnaires interspersed with controlled opinion feedback [6]. This technique was recently used to develop consensus on the predictive factors of pediatric chronic pain and pain-related disability [7] and has been used successfully by CARRA, the Pediatric Rheumatology International Trials Organization (PRINTO), the Pediatric Rheumatology Collaborative Study Group (PRCSG) and the American College of Rheumatology (ACR) in the past [3, 8–13]. A two day consensus conference followed, with the goal of reaching a final consensus on the domains and items to be measured using SUPER-KIDZ.

Delphi Survey - Iteration 1

Following approval from the CARRA Pain Disease Specific Group and Steering Committee, an e-mail (with two subsequent reminder e-mails over a 3 week period) was sent to all CARRA members inviting their participation in a survey regarding the development of a uniform pain assessment for pediatric rheumatology clinics. Potential respondents were informed of the study purpose and participation requirements for the first iteration of the Delphi procedure and directed to the electronic survey site (Survey Monkey). Respondents rated the importance of the domains recommended by PedIMMPACT for inclusion in the pain assessment of children and youth presenting to rheumatology clinics using a 0 - 10 scale (0 = not at all important and 10 = extremely important) and generated any additional domains deemed important. Respondents were asked to give separate ratings of importance based on age group of the participant being assessed (ages 4-7 versus ages 8-18) and presumed diagnosis (arthritis, idiopathic musculoskeletal pain, or other rheumatic conditions). Respondents were also asked if and how their rating would change at different points of assessment (initial versus follow-up). Respondents gave importance ratings for each domain by every combination of time point, age and diagnosis. This resulted respondents answering 96 questions for Iteration 1.

Delphi Survey - Iteration 2

In the second iteration of the Delphi procedure, another e-mail was sent to all CARRA members as was done in the first iteration. Interested respondents then rated the importance of items (using the same 0 - 10 scale) within each assessment domain (generated from the first iteration) and then for each overall domain. Respondents also were asked to indicate whether and how (e.g. more important for youths than children) ratings would significantly increase or decrease based on the age or diagnosis of the patient. Because respondents were only prompted to indicate how their rating would change if they answered affirmatively to whether or not it would change, a range of questions (74-168) could be answered in Iteration 2.

Consensus Conference

A two-day consensus conference was held in May 2009 in Toronto, Canada. The meeting was attended by 8 pediatric pain experts of varying disciplines, 6 pediatric rheumatologists, 3 consumers, and 3 allied health professionals (physical and occupational therapists) from Canada and the United States. The overall goal of the meeting was to reach consensus on the pain domains and sub-domain items based on Delphi survey results based on age group of the participant being assessed. The meeting was facilitated by Drs. McGrath (PedIMMPACT meeting chair) and Walco (chair of the CARRA pain committee) using nominal group technique. Briefly, nominal group technique is a structured face-to-face meeting, with 2 round-robin voting guided discussions to facilitate reaching consensus among experts [10]. Attendees were provided with a brief synopsis regarding the goals of the conference and the technique that would be used to establish consensus. Each participant then had 1-2 uninterrupted minutes to speak to the group about which domains and sub-domains they felt were most relevant for inclusion in SUPER-KIDZ. Voting on the domains and sub-domains by all attendees then followed. Results from the vote were presented to attendees followed by a second opportunity to speak to the group regarding important domains and sub-domains. A second round of voting again followed. If a domain was endorsed by greater than 75% of participants, it was retained and the remainder were discarded.

Phase 2: Feasibility Testing of Three Methods of Administering the Pain Measures

Participant Selection

Participants were recruited from four large CARRA member rheumatology clinics in university affiliated pediatric tertiary care centers across North America. Children and youth were eligible to participate if they were: (a) between 4-18 years of age; (b) diagnosed with or being assessed for a rheumatic condition by a rheumatologist; and, (c) English-speaking. Parents were eligible to participate if they: (a) had a child that met the study inclusion criteria; and (b) were English-speaking. All participants were excluded if they had major cognitive impairments. Study sample size was based on numbers thought to be realistically attainable within the study timeframe and which would provide adequate representation of the population of interest. No formal sample size determination was conducted as this was a pilot feasibility study with no specific hypotheses. The study was approved by Institutional Review Boards at all sites.

Procedures

Prior to their rheumatology appointment, eligible children (and parents) and youth were approached by a health care team member, and those interested in participating met with a research assistant (RA) who obtained informed consent. Demographic and disease characteristics were then obtained from participants and medical chart review, and a secure patient account was created on the SUPER-KIDZ website. Children and their parents and youth were given brief training (requiring less than 5 minutes per participant) on the nature of the assessment (i.e. rheumatic pain), the functionality of the electronic devices and how to complete the assessments using each of the paper, handheld device and laptop computer mediums. Participants then completed the same SUPER-KIDZ pain assessment using each of the three mediums. In all cases the RA was present in the room as surveys were completed and ensured that the assessments were completed by participants themselves, without the aid of others (i.e. family members). During this process, RAs recorded time to complete each survey. The sequence in which each medium was completed was randomly assigned using an Internet-based randomization scheme generator. The paper version of the pain assessment and a concise summary report from the laptop computer were provided to physicians prior to their clinical assessment. The computer summary report included composite scores related to the affective and evaluative domains of pain and therefore has fewer items to review. Following the visit, youth, parents and physicians completed a questionnaire related to their satisfaction with each pain assessment method. Physicians also provided a global disease severity rating for each participant immediately following the visit.

Both electronic mediums for each age group were programmed to present questions in chronological order with one question displayed on the screen at a time. After answering a question, the participant selected the word 'next' on the screen to proceed. If the question was not answered and the participant selected 'next', the program would present the same question for completion. If the question was again not answered and the participant selected 'next', the program would proceed to the following screen. Participants were unable to access a previous question once they had proceeded. These features were included to minimize the number of missed responses. The handheld device used was an iPod Touch (second generation) developed by Apple Incorporated with an 8 GB flash-drive and a 3.5" (diagonal) multi-touch display. Each site used their own laptop computer with Internet Explorer 7.0. Both devices accessed the Internet through local Wi-Fi services. The paper questionnaires presented multiple questions per page and participants were neither encouraged nor discouraged to review or modify responses.

Data Analyses

The quantitative data from the questionnaires were coded, scored and entered into a Statistical Package for the Social Sciences (SPSS) database [14]. Descriptive statistics were used to describe the participant characteristics using means and standard deviations (for continuous variables) and frequencies and proportions (for categorical variables). Efficiency and missed response data were analysed using Friedman's tests of analysis of variance for non-parametric data for a repeated-measures design. Acceptability data were analysed using χ2-tests. Differences were considered statistically significant at p ≤ 0.05 unless otherwise indicated.

Results

Phase 1: Developing consensus on the SUPER-KIDZ Pain Measure

Delphi Survey - Iteration 1

Of the 251 CARRA members invited to participate, 115 (46%) completed the survey. Participants were predominantly female (61%) and physicians (97%), with fellows comprising 16% of the physician group. Table 1 presents mean (M) and standard deviation (SD) values for importance ratings for pain domains by diagnosis and age grouping (time point was excluded from the Iteration 2 questions). The majority of respondents rated most domains as highly important to evaluate (overall M ± SD: 7.55 ± 1.44). Appointment type (i.e. initial or follow-up) did not generally affect importance ratings, and "satisfaction with treatments received" was viewed by respondents as being most relevant at follow-up (M = 7.4 versus 6.7); t(72) = 3.85, p < 0.01). Respondents tended to rate domains as significantly more important to assess in youth versus children, with the exception of physical functioning which was rated equally important for both age groups. Importance ratings also differed on domains contingent on the nature of the presenting problem: importance of assessing symptoms and adverse effects was rated significantly lowest, and importance of assessing emotional functioning and sleep was rated significantly highest for patients with presumed idiopathic musculoskeletal pain. Free marginal kappas [15, 16] ranged from 0 (indicating poor agreement) to 0.82 (indicating substantial agreement) [17]. The domains consistently having the lowest ratings of importance and least agreement were "economic factors" (average importance across time point, disease-type and age group: 5.42 ± 2.85 [M ± SD]; average free marginal kappa across time point, disease-type and age group: -0.01) and "global treatment satisfaction" (average importance across timepoint, disease-type and age group: 6.83 ± 2.35 [M ± SD]; average free marginal kappa across timepoint, disease-type and age group: 0.24).

Table 1 Ratings of pain domain importance by diagnosis and age groupings
Full size table

Recommendations for additional areas to evaluate based on open-ended responses were classified into domains similar to PedIMMPACT (see Documents, Additional file 1 Digital Content 1). Based on the results of the first iterative survey, all PedIMMPACT domains other than "economic factors" were retained for possible inclusion in the final assessment tool. Potential specific items to assess within each general domain, based on open-ended responses, also were retained for rating in a second iterative survey. Given that importance ratings generally did not significantly differ by time point (initial versus follow-up assessment), no time point distinction was made in the second iterative survey.

Delphi Survey - Iteration 2

Of the 251 CARRA members sent requests for study participation, 157 (63%) completed the survey. The respondent group was made up of 83 Iteration 1 participants (73% of the initial respondents) plus additional CARRA members who had not completed the Iteration 1 survey. Participants were predominantly female (63%) and physicians (95%), with fellows comprising 20% of the physician group. Both average ratings and level of agreement were used to determine items for retention in this survey iteration. For level of agreement, responses were classified into minimal importance (0-3), moderate importance (4-6), and significant importance (7-10) and examined for agreement using multi-rater free-marginal kappa [16]. Items having an average rating of at least 7/10 for importance with a free-marginal kappa coefficient of at least 0.30 (indicating at least fair agreement on categorizing importance) [17] were retained.

The item with the highest average importance rating was the impact of pain on activities of daily living (ADL) (M ± SD: 9.04 ± 1.74); the item with the lowest average importance rating was pain unpleasantness (M ± SD: 5.19 ± 2.69). Low levels of agreement for importance were found for pain unpleasantness (κ = 0.00), sensory descriptors of pain (κ = 0.02), diet (κ = 0.05), comfort goal (κ = 0.06), and alcohol/drug use (κ = 0.11). The highest levels of agreement for importance were found for pain duration (κ = 0.53), stiffness (κ = 0.53), pain impact on physical activity (κ = 0.55), pain impact on school performance (κ = 0.79), and pain impact on ADL (κ = 0.83). The majority of participants (ranging from 56% to 93%) reported that their importance ratings would not significantly differ as a function of age or condition (see Documents, Additional file 2 Digital Content 2 for descriptive statistics for importance ratings by item).

Consensus Conference

Consensus conference discussion yielded general agreement on the following domains and items for inclusion: pain characteristics (current pain, average pain intensity over past 2 weeks, pain episode duration, pain frequency, pain location), associated symptoms (fatigue frequency), cognitive and emotional factors (catastrophizing, positive affect, sadness, anger, worry, stressors), and functioning (physical, social, and role). Areas agreed to be omitted included: pain sensory descriptors, pain aggravating/alleviating factors, pain unpleasantness, comfort goal, global pain treatment satisfaction rating, fatigue, appetite, pain self-efficacy, recent peer group changes or conflicts, and level of independence. During the consensus conference, lack of agreement over the specific questions and metrics to use for quantifying items (except for pain intensity) resulted in the assignment of workgroups to facilitate discussion and provide recommendations thereafter. Items for the functioning domain were obtained from the PROMIS pediatric pain interference scale [18]. Items for assessing the recommended cognitive dimension of pain (pain catastrophizing) were taken from the three highest loading items on the Pain Catastrophizing Scale for Children [19]. Pain location was captured using a body outline by von Baeyer and colleagues [20]. Remaining items were developed by investigators following literature review. The final measures included self-report measures for children (4 to 7 years) and youth (8 to 18 years) and a proxy-report for parents of children between 4 and 7 years (see Documents, Additional files Digital Content 3, 4, 5 for the final developmentally-appropriate SUPER-KIDZ measure for children, youth and parents).

Phase 2: Feasibility Testing of Three Methods of Administering the Pain Measures

Demographic and disease characteristics of child and youth participants

Seventy-seven youth and 24 children (and their parents) participated in the study across the 4 sites. Of the children, 20 participants were included the analyses as the parents of 4 children inadvertently completed the child self-report portion of the assessment. Demographic and disease characteristics for the child and youth participants are summarized in Table 2. Youth reported current pain as 3.0 ± 2.1/10 (M ± SD) and children reported 1.4 ± 2.3/10 (M ± SD). These scores are derived from those recorded on the SUPER-KIDZ pain tool via the computer. Youth assessed their pain intensity using an 11-point (0-10) numerical rating scale and children assessed their pain intensity using the Faces Pain Scale-Revised [21] (see Documents, Additional files Digital Content 3, 4, 5). There was no difference in current pain intensity reported on different mediums for either group (youth: p = 0.652; children: p = 0.094).

Table 2 Demographic and disease characteristics of child and youth participants
Full size table

Outcome measures

Primary outcome measures for feasibility testing included: efficiency, missed responses, and participant acceptability. Efficiency was defined as time (in minutes) taken to complete the questionnaire. Missed responses were defined as any omission or incomplete question, regardless of the reason for omission. Reasons for missed responses included participant error and technical difficulty. Missed response data are displayed as the percentage of questions missed when completing the survey by a given medium. Only questions that had discrete response possibilities were included. Therefore, the pain body map question was not included in the missed response analysis, as any number of selectable body parts could have been chosen by a child or parent thus making it impossible to differentiate between omissions and purposeful non-selections. Acceptability described a participant's likes and dislikes related to a given medium.

Efficiency

Variances existed in the amount of time required to complete the questionnaire by medium for youth (p = 0.001) and parents (p = 0.001) as shown in Table 3. In each of these groups, completing the survey using the handheld device took the greatest amount of time (5.90 ± 2.79 minutes for 8-18 year olds and 7.00 ± 4.08 minutes for parents). There was no significant difference in mean efficiency between mediums when completing the 2-item survey used by children. With this group, the time to completion via each medium ranged from 1.64 ± 1.50 and 1.91 ± 1.81 minutes.

Table 3 Efficiency and missed responses on age-appropriate SUPER-KIDZ pain measure for paper, computer and handheld mediums
Full size table

Missed Responses

Overall, the incidence of missed responses in all participant groups was low (Table 3). A significant difference in the number of missed responses between mediums was seen for youth (p = 0.047). Within this group, the largest percentage of missed responses occurred when using the handheld device. This phenomenon was largely driven by technical difficulties as the responses for 2 youth were not saved in the database despite survey completion. This resulted in 100% missed response rates for these participants. There were no significant differences in the number of missed responses between mediums for parents or children.

Acceptability

There was no difference in the overall preferred medium for youth or parents. There was however a significant difference in the preferred medium for children (p = 0.008) with 65% (n = 13) of parents reported their child preferred using the computer because the computer was the most simple and fun to use. A significant difference in medium least preferred existed only for youth (p = 0.001). Here 54% (n = 42) of youth disliked the handheld device citing its small size, unfamiliarity to the user and lengthy time to register responses. Not surprisingly, given the efficiency data, the computer or paper assessments were perceived to be quicker than the handheld device by the majority of youth (87%; n = 67; p = 0.001) and parents (91%; n = 21; p = 0.019). The majority of parents (91%; n = 21) also found the computer or paper to be easier to understand than the handheld device (p = 0.032) and 78% (n = 60) of youth found the coxmputer or paper more useful for describing pain than the handheld device (p = 0.027). There was also a significant difference in the medium youth felt was most appropriate for their age group (p = 0.004) with only 16% (n = 12) citing the paper medium as meeting this criterion.

Physician demographics and satisfaction

Fifteen physicians (73% female; 67% practicing longer than 10 years) participated in the study. A statistically significant difference existed in the method of data review thought to be the most time efficient (p = 0.022) with 67% (n = 10) of pediatric rheumatologists finding the concise computer-based summary the quickest to review. The majority of rheumatologists (67%; n = 10) also indicated no preference between the computer summary and paper questionnaire for developing pain management plans. There were no other significant differences in the method found most acceptable to physicians, however, the majority of physicians (60%; n = 9) would recommend the computer-generated summary to a colleague.

Missing data and technical problems

Although 101 youth and children (with parents) participated in the study, not all analyses include data from each participant. This is a result of data recording errors by RAs (e.g. collection of physician severity ratings), missing to measure time to survey completion by RAs and missed responses by participants on satisfaction surveys. In all cases, demographic and feasibility data presented are shown with the number of participants included in that analysis. During feasibility testing, we also experienced technical problems associated with the use of the handheld device and computer resulting from weak wireless network signals and leading to extended times to register responses and missed registration of entered responses.

Discussion

We present the development of a standardized measure, SUPER-KIDZ, to assess pain in children and youth presenting to rheumatology practices. This measure is the product of a validated iterative process that consisted of a 2-stage Delphi survey followed by a consensus conference using nominal group technique. The self- and proxy-report measures were evaluated for feasibility of administration using paper, laptop and handheld-based mediums. We found that either paper or computer-based administration of the SUPER-KIDZ pain measures is clinically feasible and acceptable to patients and health care providers.

CARRA members generally agreed with the larger pediatric pain literature (and specifically PedIMMPACT recommendations on important pain dimensions to assess in practice. In particular, characteristics of pain (intensity, location, frequency, and duration), known pain modifiers (cognitive/affective variables) and consequences (functional limitations) were identified as most highly important to assess in pediatric rheumatology practice. This was generally irrespective of diagnosis or type of visit (initial versus follow-up). Attending to these aspects of a child's pain presentation is known to be instrumental for identifying patients at risk of enduring physical and/or emotional disability, selecting pain interventions, and monitoring treatment response [1]. However, the multidimensional assessment of pain if often omitted or inconsistently applied in pediatric rheumatology practices due to various perceived barriers (e.g., time) despite pain assessment being regarded as a quality care indicator for pediatric rheumatology [3]. It is hoped that the brief consensus pain measure developed during the present study ultimately will be helpful for ensuring quality care in pediatric rheumatology practice while providing clinician-friendly data on what is viewed as important aspects of a child's pain experience.

In order to determine a clinically feasible method for routinely implementing the SUPER-KIDZ measure in pediatric rheumatology practice, the present study compared three different mediums for assessment delivery (paper, computer, and handheld device). In general, electronic approaches to self-report assessment previously have been found to be well-regarded by pediatric and adult respondents and to have improved validity over paper-based methods [22–24]. Use of electronic assessment methods also naturally integrate with other forms of health information technology (e.g., electronic health records) and thus fit within the vision for contemporary pediatric medicine [25]. Results of the present study suggested that although use of handheld devices or computers had advantages for pain assessment, there were potential problems as well. In particular, use of a handheld device was perceived as burdensome to some participants due to a small screen size, technical problems, and complexity. Further, data recorded on the handheld device occasionally were not wirelessly transferred to an online database as intended, resulting in missing data, which could be problematic in practice. In addition, although rheumatologists preferred viewing computer-based pain assessment reports in general, they also responded favorably to viewing raw answers on a paper-based form. Thus, the optimal medium for implementing pain assessment in pediatric rheumatology practice may vary depending on the technology available and provider preference.

Our study has several limitations that may temper interpretation of results and conclusions. During Phase 1, the response rate of CARRA members varied from 46-63% and therefore study data may not be fully representative of all members. Despite using standardized consensus methodologies to establish a comprehensive but brief measure, important items may have been left off the final instrument that may limit the usefulness of the tool in certain contexts. Furthermore, SUPER-KIDZ was tested for feasibility only; evaluation of predictive utility and responsiveness is required before wide scale implementation can be recommended. Future studies are planned with this focus.

Conclusions

It is feasible to implement a standardized, brief but multidimensional pain assessment (SUPER-KIDZ) using electronic or paper administration in routine pediatric rheumatology practice. In general, the computer-based administration of SUPER-KIDZ was most efficient, but the medium of pain assessment employed in practice may depend on age of the child, available resources and other administrative factors. Future research evaluating the reliability and predictive validity of the pain assessment tool is planned and is necessary prior to widespread implementation. Ultimately, it is hoped that the SUPER-KIDZ pain assessment tool will assist pediatric rheumatologists and allied health care professionals to use more consistent approaches to pain assessment such that pain management interventions and monitoring of pain outcomes in children and youth with rheumatic disease are more uniform across practices. In addition, a simple, comprehensive, and validated method to assess pain will facilitate inclusion as a critical outcome in clinical trials of rheumatic diseases of childhood.