Introduction

Care continuity is the repeated use of a specific ambulatory provider over time, whereas the inverse - care fragmentation - is the use of many ambulatory providers without a dominant provider [1]. Studies of care continuity or care fragmentation have typically used administrative claims as the gold standard method for measurement [2]. Claims contain comprehensive data on ambulatory visit utilization, which can be used to determine the components of the continuity or fragmentation scores, including the number of ambulatory visits, the number of ambulatory providers, and the distribution of visits across those providers. Claims are considered the gold standard because of their completeness. However, the disadvantages of claims are that they are generally not accessible to clinicians, are expensive for researchers to obtain, are not easy to analyze, and are often available only after a substantial time lag has occurred (typically several years).

It would potentially be useful if patients could report their own patterns of ambulatory utilization, as patient self-report would be faster and less expensive to obtain than claims. Perhaps even more importantly, it would be useful if patients could report whether they are aware of gaps in communication among providers, which can occur because of fragmented care; information about gaps in communication is not available in claims and is potentially actionable. However, the accuracy of patient-reported ambulatory utilization and the relationship between self-reported gaps in care coordination and claims-based fragmentation is unclear. Prior studies have attempted to determine the correlation between patient-reported continuity and claims-based continuity, but their results have varied, with studies finding a positive correlation, [3] no correlation, [4] or mixed results (with some components of self-reported measures being associated with claims-based measures but not all) [5,6,7]. In addition, previous studies have not collected data on whether patients thought the gaps in care coordination resulted in adverse consequences.

Thus, we sought to determine: (1) the association between self-reported and claims-based ambulatory utilization, (2) the association between claims-based fragmentation scores and self-reported gaps in care coordination, and (3) the association between claims-based fragmentation scores and self-reported adverse events that the participants thought could have been prevented with better care coordination.

Methods

Overview

We conducted a longitudinal study, capturing self-reported gaps in care coordination through a survey in 2017–2018 and comparing it to claims-based fragmentation. This study was conducted among REasons for Geographic and Racial Differences in Stroke (REGARDS) cohort study participants who were 65 years or older at the time of the survey. The study protocol was approved by the Institutional Review Boards of Weill Cornell Medicine and the University of Alabama at Birmingham.

Setting

Between 2003 and 2007, the REGARDS study enrolled 30,239 community-dwelling Black and White adults age ≥ 45 years [8]. Black adults and individuals living in the Southeastern US were oversampled, because the study was designed to elucidate reasons for racial and geographic differences in stroke mortality [8]. REGARDS participants, or their proxies if they could not be reached, were contacted by telephone every 6 months to detect study outcomes (e.g., stroke, myocardial infarction). A second in-home visit was conducted approximately 10 years after baseline. REGARDS has linked its data with fee-for-service Medicare claims [9]. All participants provided written informed consent.

Between August 2017 and November 2018, REGARDS included in its routine 6-month follow-up calls a 22-question survey module on experiences with healthcare, which has been published previously [10]. REGARDS participants could opt out of this survey module, even if they chose to participate in the rest of the follow-up call. Of the 22 questions, 8 were related to perceptions of care coordination, 8 related to preventable adverse events, and 6 related to healthcare utilization (including ambulatory visits, ambulatory providers, emergency department visits, and hospitalizations). Questions about perceptions of care coordination typically used a 6-month look-back period, and questions about adverse events and healthcare utilization used 12 months [10].

Data

Data from REGARDS’ primary data collection

We used the following self-reported variables collected at the REGARDS study baseline: biological sex, race (Black/White), educational attainment, history of stroke, region of the U.S., and type of census tract (rural, suburban, or urban), which had been derived by the REGARDS study from the participants’ addresses. We used the following variables from the REGARDS second in-home visit: age, annual household income, hypertension (self-reported, self-reported use of antihypertensive medication, systolic blood pressure ≥ 140 mmHg, or diastolic blood pressure ≥ 90 mmHg), hyperlipidemia (self-reported, use of lipid-lowering medication, total cholesterol ≥ 240 mg/dL, low density lipoprotein cholesterol ≥ 160 mg/dL, or high-density lipoprotein cholesterol ≤ 40 mg/dL), diabetes (self-reported, use of oral glucose-lowering medication, use of insulin, fasting glucose ≥ 126 mg/dL, or nonfasting glucose ≥ 200 mg/dL), myocardial infarction (self-reported or evidence on the study electrocardiogram), kidney disease (self-reported kidney failure, or estimated glomerular filtration rate < 60 mL/min/1.73 m2), and atrial fibrillation (self-reported, or evidence on the study electrocardiogram). We supplemented this information with REGARDS’ adjudicated events for stroke and myocardial infarction [11, 12]. We also used data from the healthcare experiences survey described above.

Data from REGARDS-linked medicare fee-for-service claims

We used Medicare beneficiary files to determine monthly enrollment status. We used deduplicated Medicare fee-for-service claims from the Carrier and Outpatient files to identify ambulatory visits for the 12-month period preceding the healthcare experiences survey for each respondent. Ambulatory visits were defined using a National Committee for Quality Assurance (NQCA) definition that was restricted to Clinical Procedure Terminology (CPT) codes for in-person, evaluation-and-management visits for adults in an office setting [13, 14]. The NCQA definition of ambulatory visits does not include emergency department (ED) visits. We identified unique ambulatory providers by using the identifiers in the claims, mapped to the National Plan and Provider Enumeration System (NPPES) dataset [15]. We also used Outpatient and Inpatient Medicare claims to capture ED visits and hospital admissions.

Derivation of the study cohort

We included individuals who: (1) had completed the healthcare experiences survey, (2) were 65 years or older at the time of the survey, (3) had consented to have their REGARDS data linked to Medicare claims, and (4) had continuous fee-for-service coverage for the 12 months prior to the survey. We further restricted the cohort to those with 4 or more ambulatory visits in the 12 months prior to the survey, because measuring fragmentation with fewer than 4 visits is unreliable [16]. We excluded those with claims-based outlier values (> 99th percentile) for number of ambulatory visits or number of ambulatory providers, as those values are likely erroneous. We also excluded those who did not complete the REGARDS second in-home visit.

Derived variables

Calculating self-reported gaps in care coordination

We used our previously published methods for calculating self-reported gaps in care coordination [17]. Briefly, for each of the eight questions on perceptions of care coordination, we dichotomized the response scales to identify problems with care coordination, which we refer to as “gaps” in care coordination. We combined two similar questions that asked about how often and how quickly a respondent received test results, leaving seven unique items. We calculated the frequency of responses that indicated a gap in care coordination for each of these seven items, as well as the frequency of reporting any gap. We also calculated the frequency of each of and any of four self-reported adverse events that respondents reported could have been prevented with better care coordination (i.e., a test that was repeated because the doctor did not have access to the first test, a drug-drug interaction that occurred in the context of multiple prescribers, an ED visit that could have been prevented with better communication across providers, and a hospital admission that could have been prevented with better communication across providers).

Calculating claims-based fragmentation

For each participant, we determined the number of ambulatory visits, the number of ambulatory providers, and the percentage of visits with the most frequently seen provider over the 12 months prior to the survey. We then calculated a fragmentation score using the previously validated Bice-Boxerman Index (BBI) [2, 16, 18,19,20]. This index captures both “dispersion” (the spread of ambulatory visits across providers) and “density” (the relative share of visits by each provider) [21]. Patterns of care characterized by high dispersion (many providers) and low density (a relatively low proportion of visits by each provider) receive worse scores (indicating more fragmentation) than patterns with low dispersion and high density. The original BBI ranges from 0 (each visit with a different provider) to 1 (all visits with same provider). To facilitate interpretation, we reversed the index, calculating 1 minus BBI, so that higher scores reflected more fragmentation [14, 22]. Note that this measure treats all physicians the same way, whether they are generalists or specialists.

Statistical analysis

We used descriptive statistics to characterize the study sample, self-reported healthcare utilization, and claims-based healthcare utilization. Using correlation coefficients (for continuous variables), we determined agreement between self-reported and claims-based numbers of ambulatory visits and, separately, between self-reported and claims-based numbers of ambulatory providers. Using kappa statistics (for dichotomous variables), we determined agreement between any (that is, ≥ 1 vs. 0) self-reported and claims-based ED visit and, separately, between any self-reported and claims-based hospitalization.

We then classified participants into three groups based on their fragmentation scores. We defined high fragmentation as a score of ≥ 0.85, based on previous work showing that this cut point is independently associated with an increased risk of hospitalization [20]. Because the distribution of fragmentation scores is skewed, [23] we further defined medium fragmentation as a score from 0.70 to < 0.85 and low fragmentation as a score < 0.70.

We used descriptive statistics to characterize claims-based ambulatory care patterns by fragmentation group. We also used descriptive statistics to show responses to survey questions by fragmentation group, making pairwise comparisons using Pearson’s chi-squared test or Fisher’s exact test (if a cell size was < 5).

We conducted unadjusted and adjusted logistic regression models, using fragmentation group as the independent variable and any self-reported gap in care coordination as the dependent variable. The adjusted models were built in a stepwise fashion, first adjusting for demographic characteristics and then adding adjustment for clinical characteristics. We repeated these models substituting any self-reported adverse event attributed to poor coordination as the dependent variable. Models were performed using a complete case approach and then re-run using multiple imputation for missing co-variates; the most frequently missing co-variate was annual household income (missing for 15%).

Analyses were conducted using SAS (version 9.4; Cary, NC). P-values ≤ 0.05 were considered significant.

Results

Study sample

We identified 4,296 individuals who met the inclusion criteria (Fig. 1). The average age of these participants was 78.1 years old (SD 6.7). Approximately half (54.3%) were female. Nearly one-fourth (24.4%) were Black. Nearly one-third (29.6%) had an annual household income of <$35,000, and more than one-fourth (28.1%) had a high school education or less. More than half (59.3%) lived in the Southeastern region of the U.S. Approximately one-tenth (13.2%) lived in a rural area. More than two-thirds had hypertension (70.2%) or hyperlipidemia (69.1%). The frequency of other chronic conditions is shown in Table 1.

Fig. 1
figure 1

Derivation of study cohort

Full size image
Table 1 Characteristics of the study sample at the time of the healthcare experiences survey*
Full size table

Agreement between self-reported and claims-based healthcare utilization

There was a weak correlation between self-reported and claims-based ambulatory visits and ambulatory providers (with correlation coefficients of 0.37 and 0.38 respectively, Table 2). There was moderate agreement between self-reported and claims-based ED visits and hospital admissions), with kappa statistics of 0.45 and 0.52, respectively. For the first three of these measures, the self-reported value was lower than the claims-based values. Participants reported an average of 6.0 visits over the past year, whereas by claims the same individuals had an average of 13.1 visits. Similarly, the participants reported having an average of 3.1 ambulatory providers, whereas claims found an average of 6.2 providers. Approximately one-fifth of respondents (19.5%) reported having at least one ED visit, whereas claims found ED visits for 29.6% of respondents. Similar proportions were found for hospital admissions (16.5% by self-report vs. 15.8% for claims).

Table 2 Agreement between self-reported utilization and claims-based utilization*
Full size table

Differences across groups by claims-based fragmentation score

Overall, those with 4 or more ambulatory visits had a median of 11 visits to 6 providers, with a median of 36% of visits with the most frequently seen provider, and a median fragmentation score of 0.84 (Table 3). The distributions of these variables are shown in the Appendix. Of the 4,296 participants in the study, 2,052 (47.8%) had high fragmentation, 1,543 (35.9%) had medium fragmentation, and 701 (16.3%) had low fragmentation (Table 3). Participants in the high fragmentation group had a median of 13 ambulatory visits to 7 unique providers (by claims), with the most frequently seen provider accounting for 27% of visits, yielding a median fragmentation score of 0.90. Participants in the medium fragmentation group had a median of 11 visits to 5 providers, with the most frequently seen provider accounting for 43% of visits, yielding a median fragmentation score of 0.80. By contrast, those with low fragmentation had a median of 8 visits to 3 providers, with the most frequently seen provider accounting for 64% of visits, with a median fragmentation score of 0.60.

Table 3 Claims-based patterns of ambulatory utilization, stratified into high, medium, and low fragmentation groups*
Full size table

Association between self-reported gaps in care coordination and claims-based fragmentation

There were statistically significant differences across claims-based fragmentation groups for 4 of the 7 unique questions on perceptions of care coordination (Table 4). For example, 11.8% of high fragmentation participants reported that they only somewhat received or did not receive help they needed from their doctor’s office to manage care among different providers and services, compared to 7.9% of low fragmentation participants (p < 0.01). Similarly, 10.8% of high fragmentation participants reported that they do not think their doctors communicate with each other about their healthcare, compared to 6.6% of low fragmentation participants (p = 0.001). The proportions of participants rating the coordination of care among all of the health professionals that they see as fair or poor were 8% of high fragmentation participants and 3.6% for low fragmentation participants (p < 0.001). The medium fragmentation group had proportions between the high and low fragmentation groups. Overall, in these unadjusted comparisons, those with high fragmentation were more likely than those with low fragmentation to report any of the 7 gaps in care coordination (37.8% vs. 33.7%, p = 0.049).

Table 4 Self-reported gaps in care coordination and self-reported adverse events, stratified by claims-based fragmentation (N = 4,296)
Full size table
Table 5 Unadjusted and adjusted associations between self-reported and claims-based measures
Full size table

This association persisted after adjustment for demographic and clinical characteristics, regardless of whether we used a complete case approach or multiple imputation for missing co-variates (Table 5). In the fully adjusted model that used multiple imputation, having high fragmentation was associated with a 24% increase in the odds of reporting any gap in care coordination, compared to low fragmentation (adjusted odds ratio [AOR] 1.24; 95% confidence interval [CI] 1.03, 1.48).

Association between claims-based fragmentation and self-reported adverse events

There were no statistically significant differences across fragmentation groups in the frequency with which respondents reported repeat tests (Table 4). Those with medium fragmentation reported more drug-drug interactions than the low fragmentation group, but the difference between high and low fragmentation was not statistically significant.

The high fragmentation group was more likely to have an ED visit than the low fragmentation group (21.7% vs. 11.8%, p < 0.001), but there was no statistically significant difference in the proportions of respondents who thought that their ED visit could have been prevented or could have been prevented with better communication among providers.

The high fragmentation group was also more likely to have a hospital admission than the low fragmentation group (19.4% vs. 10.3%, p < 0.001), but there was no statistically significant difference in the proportions of respondents who thought that their hospital admission could have been prevented or could have been prevented with better communication.

When any of the four adverse events were considered (that is, repeat test, drug-drug interaction, ED visit that could have been prevented with better communication, and hospital admission that could have been prevented with better communication), 7.9% of those with high fragmentation and 5.3% of those with low fragmentation reported any adverse event (p = 0.02). Adjusting for co-variates and using multiple imputation for missing co-variates, high fragmentation was significantly associated with a 61% increase in the odds of any self-reported adverse event, compared to low fragmentation (Table 5, AOR 1.61, 95% CI 1.11, 2.34).

Discussion

In this national study of 4,296 Medicare beneficiaries, we found a positive correlation between self-reported and claims-based ambulatory utilization, with correlation coefficients of 0.37 for the number of visits and 0.38 for the number of providers (p < 0.0001 for each). Participants substantially underestimated their ambulatory utilization, compared to claims; they reported an average of 6.0 visits to 3.1 providers, whereas claims identified an average of 13.1 visits to 6.2 providers (p < 0.0001 for each). Nearly half of participants (47.8%) had highly fragmented care by claims. More than a third of participants reported a gap in care coordination, and between 5% and 8% of participants reported adverse events that they attributed to poor care coordination. Those with high fragmentation by claims had a 24% increased adjusted odds of experiencing a gap in care coordination (95% CI 3%, 48%) and, separately, a 61% increased adjusted odds of a self-reported adverse event that they attributed to poor coordination (95% CI 11%, 134%).

In the context of the literature, our findings align with the previous study that found a positive correlation between self-reported and claims-based measures of ambulatory utilization, and the magnitude of association in our study was similar to theirs (r = 0.30) [3]. Our study is thus distinct from previous work that found mixed or null results [4,5,6,7]. We found slightly lower agreement between self-report vs. claims-based ED visits and hospitalizations (kappa of 0.45 for ED visits and 0.52 for hospitalizations), compared to the literature’s kappa of 0.65 for ED visits and 0.64 for hospitalizations, although the difference could be explained by our use of a 12-month look-back period compared to others’ use of a 6-month look-back [24]. Our study adds to the literature with the novel finding of a positive association between claims-based fragmentation and self-reported adverse events that respondents attribute to poor coordination of care.

The directions of the associations between self-reported and claims-based ambulatory utilization, claims-based fragmentation and self-reported gaps in care coordination, and claims-based fragmentation and self-reported adverse events were all consistent. Together, these findings suggest that patients are aware of when their care is fragmented, even if they underestimate the full extent of fragmentation. The findings further suggest that patient-reported gaps in care coordination and patient-reported adverse events could be considered important patient-centered outcomes.

Previous studies have measured patient-reported gaps in care coordination, but they have tended to use them as a measure of quality and satisfaction at the population level, [25, 26] not as a measure to be collected and addressed at the individual level in real time. If patients are aware of gaps in care coordination and can see preventable harm unfolding, it would be important for the medical community to elicit those observations and act to address them.

Although this study does not validate patient perceptions of harm against objective documentation of adverse events, another study found that patient reports are correct more often than not; in that study, 64% of safety concerns reported by patients were validated upon clinician review and that 57% of those confirmed problems resulted in a change in the patients’ record or care plan [27]. Patients’ perceptions of care coordination are not routinely elicited, and more research is needed to determine whether addressing patients’ concerns can favorably affect their care.

There are several limitations of this study. We do not have objective data on appropriateness of care or on communication among providers. In addition, this study took place within the context of a cohort study, and those who participate in cohort studies may be different from those who do not. However, empirical data suggest that REGARDS participants with linked fee-for-service Medicare claims are representative of fee-for-service Medicare beneficiaries broadly [9]. Strengths of this study include a national sample, large sample size, diverse sample, and use of previously tested measures for claims-based fragmentation and self-reported gaps in care coordination.

In conclusion, our study found positive associations among claims-based fragmentation, self-reported gaps in care coordination, and self-reported adverse events. While claims-based fragmentation may remain the gold standard for capturing ambulatory utilization, this study suggests that self-reported measures could play an important complementary role. Future studies could use self-reported gaps in coordination as a starting point for identifying opportunities for intervention at the individual level. The fact that people are reporting adverse events that they attribute to a failure of care coordination warrants more attention.