Abstract
Purpose
To analyse the reliability and validity of the Swedish indicator ‘Drugs that should be avoided in older people’.
Methods
From a previous study that included consecutive primary care patients ≥ 65 years of age, all patients ≥ 75 years of age were analysed. Two physicians independently screened their medication lists and medical records, applying the Swedish indicator which includes potentially inappropriate medications (PIMs): long-acting benzodiazepines, drugs with anticholinergic action, tramadol, propiomazine, codeine, and glibenclamide. The clinical relevance of identified PIMs was independently assessed. Thereafter, the physicians determined in consensus whether some medical action related to the drug treatment was medically justified and prioritised before the next regular visit. If so, the drug treatment was considered inadequate, and if not, adequate.
Results
A total of 1,146 drugs were assessed in 149 patients (75‒99 years, 62% female, 0‒20 drugs per patient). In 29 (19%) patients, at least one physician identified ≥ 1 PIM according to the indicator at issue; 24 (16%) patients were concordantly identified with ≥ 1 such PIM (kappa: 0.89). Of 26 PIMs concordantly identified, the physicians concordantly assessed four as clinically relevant and 12 as not clinically relevant (kappa: 0.17). After the consensus discussion, six (4%) patients had ≥ 1 PIM according to the studied indicator that merited action. Using the area under the receiver operating characteristic (ROC) curve, the indicator did not outperform chance in identifying inadequate drug treatment: 0.56 (95% confidence interval: 0.46 to 0.66).
Conclusion
The Swedish indicator has strong reliability regarding PIM detection but does not validly reflect the adequacy of drug treatment.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
Over the last decades, drug treatment in older people has received much attention, both in research and in the monitoring of health care quality. Drug treatment challenges in this age group include the presence of multiple diseases that require many medications and pharmacokinetic changes such as gradually declining kidney function and increased sensitivity to medications due to impaired physiologic compensatory mechanisms. In Sweden, more than 2 million people ≥ 65 years of age, 95% of all inhabitants in this age group, filled at least one drug prescription in 2023 [1].
Suboptimal drug treatment in older people is often described in terms of potentially inappropriate medications (PIMs), including, for instance, long-acting benzodiazepines or first-generation antihistamines. Several criteria sets to identify and analyse suboptimal drug treatment have been developed [2]; some are explicit, and others are implicit. Explicit criteria are usually described as drug-specific or disease-specific, while implicit criteria rely on clinical judgement [3]. An early example of an explicit criteria set is the Beers criteria from the United States, introduced in the 1990s [4] and repeatedly updated by the American Geriatrics Society, most recently in 2023 [5]. In Europe, some recognised criteria sets are the EU(7)-PIM list published in 2015 [6] and the Screening Tool of Older Persons’ Prescriptions (STOPP)/Screening Tool to Alert to Right Treatment (START), last updated in 2023, where STOPP describes PIMs and START potential prescribing omissions (PPOs) [7]. In 2004, the Swedish National Board of Health and Welfare introduced a criteria set, described as indicators for the quality of drug therapy in the elderly [8], with the latest update in 2017 [9]. Typically, the choice of PIMs and diagnosis/conditions to include in a criteria set to represent suboptimal drug treatment in older people relies on expert panel opinions [10].
The Swedish criteria set (see supplement) is applicable to people aged 75 years or older. It includes drug-specific and diagnosis-specific criteria. The former are distributed within nine subsets, of which ‘Drugs that should be avoided in older people unless specific reasons exist’ (hereafter called ‘Drugs that should be avoided in older people’) is the focus of this study. This criterion includes the following drugs and drug classes: long-acting benzodiazepines, drugs with anticholinergic action, tramadol, propiomazine, codeine, and glibenclamide [9]. Since 2011, the studied criterion has been used in Sweden as an indicator to monitor health care quality at national and regional levels, labelled as ‘inappropriate drugs’ [11]. Although also used in research [12,13,14,15], reliability and validity of this specific indicator have not been addressed in the international literature. In this study, we thus aimed to analyse these aspects.
Methods
This study was based on data collected in two previous studies that included 302 consecutive patients, aged 65 years or older, who had a scheduled medical visit in either of two primary care centres in Sweden over a 3-week period in 2017 and whose medication has been thoroughly assessed, for instance, by the application of several criteria sets [16, 17]. In the current study, we focused on a subset of the Swedish criteria set, i.e. ‘Drugs that should be avoided in older people’ [9], and the subgroup of patients for whom these are primarily intended, i.e. those who are ≥ 75 years of age.
The process of the assessments performed in the underlying studies is described in Fig. 1. Two specialist physicians (N.P.L., general practitioner; S.A.S., general practitioner/clinical pharmacologist) compiled the patients’ medication based on medical records up to 2½ years before the current medical visit. Based on the medication list and the medical records, they then independently identified PIMs/PPOs according to the STOPP/START criteria version 2 [18], the EU(7)-PIM list [6], and the Swedish criteria set [9] and subsequently assessed their clinical relevance, in one of the following categories: (i) clinically relevant; (ii) of uncertain clinical relevance, but with one or more related medical actions suggested; (iii) not clinically relevant; or (iv) of uncertain clinical relevance, with no related medical action suggested. The two former categories were collapsed into the category ‘clinically relevant’, and the two latter into the category ‘not clinically relevant’. Second, they independently, and then in consensus, categorised the patient’s overall drug treatment at the visit as adequate or inadequate. Adequate drug treatment reflected that no additional action related to the drug treatment would have been medically justified and prioritised before the next routine visit, such as the regular annual check-up for patients with chronic diseases. Inadequate drug treatment, on the other hand, reflected that one or more actions related to the drug treatment would have been medically justified and prioritised but had not been carried out. Called-for actions could include, for example, retrieving more information about the patient, withdrawing a drug, or ordering a laboratory test.
In the present study, focusing on the subgroup of patients who were ≥ 75 years old and ‘Drugs that should be avoided in older people’, we identified patients treated with medications included in the studied indicator. We recorded medical actions for these PIMs identified by any of the assessors. In cases where the overall drug treatment in consensus was categorised as inadequate, we also recorded whether the related action judged to be medically justified and prioritised concerned a drug included in the abovementioned indicator. When no action was deemed medically justified or prioritised, the underlying reason(s) were noted.
Statistical analysis
Descriptive analyses were performed using SPSS Statistics for Windows, version 27.0 (IBM Corp., Armonk, NY, USA). To assess the reliability of the studied indicator, inter‐rater agreement at the patient level, for the independent identification of ≥ 1 PIMs in the indicator ‘Drugs that should be avoided in older people’, was estimated using kappa statistics [19]. Based on concordantly identified PIMs, we then calculated the kappa value regarding the physicians’ assessment of their clinical relevance. The inter-rater agreement was interpreted according to the kappa value as follows: none (< 0.20), minimal (0.21–0.39), weak (0.40–0.59), moderate (0.60–0.79), strong (0.80–0.89), and almost perfect (> 0.90) [19]. To evaluate the ability of the studied indicator to classify drug treatment in older people as either adequate or inadequate, we used the area under the receiver operating characteristic (ROC) curve, reflecting the diagnostic capability of a test against a reference standard [20]. The reference standard was the consensus decision by the two specialist physicians, i.e. whether the drug treatment was adequate or inadequate. Finally, we calculated sensitivity and specificity, as well as positive and negative predictive values at the optimal cut-off point, defined as the highest sum of sensitivity and specificity [21]. For comparison, we estimated the area under the ROC curve and the other test accuracy measures based solely on the number of drugs in the medication list.
Results
In all, 149 patients were included in the study (Table 1). A total of 1,146 drugs were included in the medication lists of 146 patients; three (2%) patients had no prescribed drug treatment.
In 29 (19%) patients, at least one of the physicians identified ≥ 1 PIM according to the indicator ‘Drugs that should be avoided in older people’ (kappa: 0.89; Fig. 1). Of these patients, 24 (16% of all) were concordantly identified as having ≥ 1 such PIM.
A total of 32 PIMs according to the studied indicator were identified by at least one physician (Table 2), 18 (56%) of which were prescribed on an as-needed basis. Of these 32 PIMs, 26 were concordantly identified by both physicians. Of these 26, in turn, the physicians concordantly assessed four as clinically relevant and 12 as not clinically relevant; the remaining 10 were discordantly assessed regarding the clinical relevance (kappa: 0.17; Fig. 1). The six PIMs that were identified by one physician only concerned drugs that (i) were prescribed as needed (clemastine, n = 1; glycopyrronium, n = 1; meclozine, n = 1; tramadol, n = 1); (ii) had an unclear withdrawal decision at the end of the index visit (tramadol, n = 1); or (iii) were identified by both physicians as part of another indicator (codeine, n = 1).
In the consensus discussions, seven of the PIMs according to the studied indicator were considered medically justified and prioritised to act upon before the next regular physician visit: four PIMs were deemed suitable for discontinuation at the physician visit (hydroxyzine, n = 2; amitriptyline, n = 1; propiomazine, n = 1), and three were deemed suitable to withdraw or switch before the next routine visit (nitrazepam, clemastine, codeine/paracetamol, all n = 1) (Table 3). The remaining 25 PIMs did not merit action before the next routine visit based on the specific patient’s situation (amitriptyline, n = 3; diazepam, n = 3; tolterodine, n = 3; propiomazine, n = 3; codeine/paracetamol, n = 3; meclozine, n = 2; flunitrazepam, n = 2; tramadol, n = 2; glycopyrronium, glibenclamide, clomipramine, nitrazepam, all n = 1).
The drug treatment of 48 (32%) patients was in consensus considered inadequate. For six (4%) of these patients, the assessment was related to the presence of at least one PIM listed in the studied indicator. The area under the ROC curve, for this indicator’s ability to classify the overall drug treatment as adequate or inadequate, was 0.56 (95% confidence interval (CI): 0.46–0.66) (Fig. 2, Table 4). With a 95% CIs passing 0.5, the indicator cannot discriminate between adequate and inadequate drug treatment.
Discussion
This study shows that about one-fifth of primary care patients aged 75 years or older are treated with at least one PIM listed in the Swedish indicator ‘Drugs that should be avoided in older people’. The kappa values suggest strong agreement regarding the identification of patients with at least one PIM according to the studied indicator, but no agreement regarding physician assessments of their clinical relevance. In addition, the consensus assessments between the two physicians reveal that relatively few of these PIMs merit action from a medical perspective.
The area under the ROC curve shows that the studied indicator does not outperform chance in distinguishing between patients with adequate and inadequate drug treatment. An indicator based on the number of drugs in the medication list, on the other hand, performs better than chance. In this context, it must not be forgotten that the number of drugs reflects the burden of disease [22]. With greater complexity, the risk of overlooking something during a patient consultation is arguably higher. It may be worth noting that the optimal cut-off point in our study, to identify patients where action related to drug treatment could be medically justified and prioritised, is ≥ 6 drugs in the medication lists. Nonetheless, this threshold has low specificity.
The drugs included in the studied indicator have been determined by an expert panel to be generally unsuitable for older people [8]. However, like any other treatment, these medications require an individual assessment by the prescribing physician. Our results may thus illustrate the difference between what can be considered adequate/inadequate treatment at the population and individual levels, respectively; general treatment recommendations are based on an average individual, but no such individual exists in clinical practice. Still, the indicator may be a valuable tool to increase physicians’ attention to problematic medications and support them during the prescribing process. Indeed, the use of PIMs listed in the studied indicator has decreased in Sweden over time [14]. Nevertheless, our results suggest that there are still patients for whom these medications could be discontinued. Two drugs worth highlighting in this context are hydroxyzine for the treatment of anxiety and propiomazine for insomnia. In all cases, withdrawal of these drugs was considered medically justified, either immediately or in the long term.
In our study, 9% of the patients were prescribed drugs with anticholinergic action. The use of such drugs in those aged 75 years or older has increased over time [23]. Due to the side effects, however, such use may be particularly problematic. Indeed, the studied indicator, the Beers criteria [5], the EU(7)-PIM list [6], and the STOPP criteria [7] all include drugs with anticholinergic action (Table 5). Nevertheless, direct comparisons between criteria sets may be hampered by the fact that both drug-specific and diagnosis-specific criteria exist. Thus, knowledge of differences in criteria definitions is crucial when interpreting research results. Indeed, sets with drug-specific criteria may be suitable as screening tools and for prevalence studies, whereas complex diagnosis-specific criteria may be useful for prescribers in clinical practice and for educational purposes. Nevertheless, drug-specific criteria may have validity problems, supported by our finding that the studied indicator could not reflect the adequacy of drug treatment. Regarding complex diagnosis-specific criteria, on the other hand, both reliability and validity have been shown to be problematic as a substantial proportion have been reported not to be concordantly identified [24], and they also have limitations when it comes to their ability to reflect the adequacy of the drug treatment management [25].
The complexity of medical assessments becomes evident with the level of inter-rater agreement between the physicians. In the original study, 167 (55%) out of 302 patients were assessed similarly as regards overall adequacy of drug treatment (kappa: 0.33) [16]. The primary care setting, where the physician considers the patient’s overall situation and where more than one drug and disease often have to be taken into account, could be a contributing factor. In the hospital setting, on the other hand, acute cases have to be managed; thus, more targeted medical priorities are likely to be made. Therefore, it may not be surprising that assessments of the appropriateness of drug treatment at hospital admission show high inter-rater agreement [26].
Strengths and limitations
The main strength of this study is that it facilitates interpretation of results that are used in health care and research, both regarding the Swedish indicator per se and drug-specific PIMs in general. Indeed, although sometimes labelled as ‘inappropriate drugs’, they are in many cases not inappropriate for the specific patient when their individual circumstances are taken into account from a medical perspective. Furthermore, the results may illustrate the difference between adequately managed drug treatment according to general recommendations and what clinicians consider adequately managed treatment for a specific individual. However, the limited number of patients included in the study may be regarded as a limitation. Nevertheless, the consecutive inclusion of patients from two large primary care centres in both urban and rural settings [16, 17] may contribute to the generalisability of the results. Another limitation is that the data are from 2017; drug treatment changes over time, and three drugs within the current indicator have been withdrawn from the market in Sweden. Nonetheless, the entire criteria set of the Swedish indicator is still in use and has not been updated since 2017. Indeed, the studied indicator is incorporated in the electronic medical records of many primary health care centres and hospitals in Sweden, generating an alert during the prescribing process. Furthermore, almost all PIMs in the studied indicator are also present in the EU(7)-PIM list [6] and the Beers criteria [5]. A significant limitation is the absence of a reference standard to assess the quality of drug treatment. However, our reference standard relies on rigorous assessments of patients’ drug treatment and can be considered medically relevant, in particular as they were carried out first independently, preceded by the application of extensive explicit criteria sets, and then in consensus by two experienced physicians with relevant specialist expertise. Finally, it may be worth noting that the percentage of patients treated according to the studied indicator is higher in our study than in a national register study (19% versus 7%) [15]. Selection bias may contribute to this finding; our patients represented a consecutive sample attending a primary care centre. Furthermore, the medication lists in the current study were based on comprehensive information in the medical records, and drugs used as needed may not be captured when medication lists are estimated on filled prescriptions during a restricted time period [15, 27].
Conclusion
This study shows that the Swedish indicator ‘Drugs that should be avoided in older people’ has strong reliability for the identification of PIMs in the medication list but does not validly reflect the adequacy of drug treatment. For most, but not all, patients in primary care, treatment with drugs listed as PIMs in the studied indicator may be medically justified and not prioritised to change during the medical visit.
Data availability
The database from this study is not publicly available owing to Swedish data protection laws. The data can be shared with authorised persons after they have applied and obtained approval from the Swedish Ethical Review Authority.
References
The National Board of Health and Welfare (2024) Open-access database for dispensed prescription drugs. https://sdb.socialstyrelsen.se/if_lak/val.aspx. Accessed 22 Apr 2024
Pazan F, Kather J, Wehling M (2019) A systematic review and novel classification of listing tools to improve medication in older people. Eur J Clin Pharmacol 75(5):619–625
Spinewine A, Schmader KE, Barber N et al (2007) Appropriate prescribing in elderly people: how well can it be measured and optimised? Lancet 370(9582):173–184
Beers MH, Ouslander JG, Rollingher I et al (1991) Explicit criteria for determining inappropriate medication use in nursing home residents. UCLA Division of Geriatric Medicine. Arch Intern Med 151(9):1825–32
American Geriatrics Society (2023) updated AGS Beers Criteria® for potentially inappropriate medication use in older adults. J Am Geriatr Soc. 71(7):2052–2081
Renom-Guiteras A, Meyer G, Thurmann PA (2015) The EU(7)-PIM list: a list of potentially inappropriate medications for older people consented by experts from seven European countries. Eur J Clin Pharmacol 71(7):861–875
O’Mahony D, Cherubini A, Guiteras AR et al (2023) STOPP/START criteria for potentially inappropriate prescribing in older people: version 3. Eur Geriatr Med 14(4):625–632
Fastbom J, Johnell K (2015) National indicators for quality of drug therapy in older persons: the Swedish experience from the first 10 years. Drugs Aging 32(3):189–199
The National Board of Health and Welfare (2017) National indicators for quality of drug therapy in older persons [Indikatorer för god läkemedelsterapi hos äldre]
Motter FR, Fritzen JS, Hilmer SN et al (2018) Potentially inappropriate medication in the elderly: a systematic review of validated explicit criteria. Eur J Clin Pharmacol 74(6):679–700
The Swedish Association of Local Authorities and Regions and the National Board of Health and Welfare (2011) Open comparisons – care of the elderly [Öppna jämförelser 2011-Vård och omsorg om äldre]
Sjöberg C, Edward C, Fastbom J et al (2011) Association between multi-dose drug dispensing and quality of drug treatment–a register-based study. PLoS One 6(10):e26574
Bladh L, Ottosson E, Karlsson J et al (2011) Effects of a clinical pharmacist service on health-related quality of life and prescribing of drugs: a randomised controlled trial. BMJ Qual Saf 20(9):738–746
Hovstadius B, Petersson G, Hellström L et al (2014) Trends in inappropriate drug therapy prescription in the elderly in Sweden from 2006 to 2013: assessment using national indicators. Drugs Aging 31(5):379–386
Wastesson JW, Fritzell J, Burström B et al (2023) Regional variations in excessive polypharmacy and potentially inappropriate drug use among older adults in Sweden: trends from 2006 to 2020. Front Pharmacol 14:1030849
Parodi López N, Svensson SA, Wallerstedt SM (2021) Association between recorded medication reviews in primary care and adequate drug treatment management – a cross-sectional study. Scand J Prim Health Care 39(4):419–428
Parodi López N, Svensson SA, Wallerstedt SM (2022) Clinical relevance of potentially inappropriate medications and potential prescribing omissions according to explicit criteria – a validation study. Eur J Clin Pharmacol 78(8):1331–1339
O’Mahony D, O’Sullivan D, Byrne S et al (2015) STOPP/START criteria for potentially inappropriate prescribing in older people: version 2. Age Ageing 44(2):213–218
McHugh ML (2012) Interrater reliability: the kappa statistic. Biochem Med 22(3):276–282
Obuchowski NA, Lieber ML, Wians FH Jr (2004) ROC curves in clinical chemistry: uses, misuses, and possible solutions. Clin Chem 50(7):1118–1125
Habibzadeh F, Habibzadeh P, Yadollahie M (2016) On determining the most appropriate test cut-off value: the case of tests with continuous results. Biochem Med (Zagreb) 26(3):297–307
Brilleman SL, Salisbury C (2013) Comparing measures of multimorbidity to predict outcomes in primary care: a cross sectional study. Fam Pract 30(2):172–178
Grossi CM, Richardson K, Savva GM et al (2020) Increasing prevalence of anticholinergic medication use in older people in England over 20 years: cognitive function and ageing study I and II. BMC Geriatr 20(1):267
Parodi López N, Belfrage B, Koldestam A et al (2023) Revisiting the inter-rater reliability of drug treatment assessments according to the STOPP/START criteria. Br J Clin Pharmacol 89(2):832–842
Wallerstedt SM, Svensson SA, Lönnbro J et al (2022) Performance of 3 sets of criteria for potentially inappropriate prescribing in older people to identify inadequate drug treatment. JAMA Netw Open 5(10):e2236757
Lönnbro J, Holmqvist L, Persson E et al (2021) Inter-rater reliability of assessments regarding the quality of drug treatment, and drug-related hospital admissions. Br J Clin Pharmacol 87(10):3825–3834
Wallerstedt SM, Fastbom J, Johnell K et al (2013) Drug treatment in older people before and after the transition to a multi-dose drug dispensing system–a longitudinal analysis. PLoS One 8(6):e67088
Funding
Open access funding provided by University of Gothenburg. This study was supported by grants from the Swedish Research Council [2013–02639] and the Swedish state under the ALF agreement between the Swedish government and the county councils [ALFGBG-716941].
Author information
Authors and Affiliations
Contributions
N.P.L. and S.M.W. conceived the study, and all authors designed it. N.P.L. and S.A.S. performed medication list extractions and the drug treatment assessments. N.P.L. and S.M.W. performed the data analysis. N.P.L. and S.M.W. drafted the manuscript, and all authors revised it and have agreed to the final version.
Corresponding author
Ethics declarations
Some of the results presented in this article have previously been published in Swedish, in Läkartidningen 2023;120:23043 (PMID: 37964762).
Ethics approval
The study was approved by the Regional Ethical Review Board in Gothenburg, Sweden (DRN: 1046–15). Informed consent was waived by the Board.
Competing interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Parodi López, N., Svensson, S.A., Lönnbro, J. et al. Reliability and validity of the Swedish indicator ‘Drugs that should be avoided in older people’—an appraisal of a set of potentially inappropriate medications. Eur J Clin Pharmacol 80, 1285–1293 (2024). https://doi.org/10.1007/s00228-024-03700-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00228-024-03700-x