Abstract
Safety profiles of the influenza vaccine and its subtypes are still limited. We aimed to address this knowledge gap using multiple data mining methods and calculated performance measurements to evaluate the precision of different detection methods. We conducted a post-marketing surveillance study between 2005 and 2019 using the Korea Adverse Event Reporting System database. Three data mining methods were applied: (a) proportional reporting ratio, (b) information component, and (c) tree-based scan statistics. We evaluated the performance of each method in comparison with the known adverse events (AEs) described in the labeling information. Compared to other vaccines, we identified 36 safety signals for the influenza vaccine, and 7 safety signals were unlabeled. In subtype-stratified analyses, application site disorders were reported more frequently with quadrivalent and cell-based vaccines, while a wide range of AEs were noted for trivalent and egg-based vaccines. Tree-based scan statistics showed well-balanced performance. Among the detected signals of influenza vaccines, narcolepsy requires special attention. A wider range of AEs were detected as signals for trivalent and egg-based vaccines. Although tree-based scan statistics showed balanced performance, complementary use of other techniques would be beneficial when large noise due to false positives is expected.
Similar content being viewed by others
Introduction
Influenza is an infectious disease associated with high morbidity and mortality globally1,2, and vaccination is the most effective means of eradication3. In Korea, influenza vaccines have been provided as part of the National Immunization Program (NIP), and eligible groups have continuously expanded since their introduction in 1997. Accordingly, the number of adverse events (AEs) following immunization for influenza has been steadily increasing according to the Korea Adverse Event Reporting System (KAERS)4. Safety profiles are a collection of adverse events5, including the information identified from clinical trials or a post-marketing phase and unidentified to date. Especially, unidentified information or safety signal represents “reported information on a possible causal relationship between an AE and a vaccine or drug, of which the relationship is unknown or incompletely evaluated previously”6. In particular, concerns regarding serious AEs of influenza vaccination have been continuously highlighted, including Guillain–Barré syndrome (GBS), febrile seizure, anaphylaxis, narcolepsy, and Bell’s palsy7,8,9,10,11,12. However, only a few studies on AEs associated with influenza vaccines have been conducted in Korea7,8. Therefore, it is necessary to re-evaluate the safety information of influenza vaccines using the latest and widely collected Korean databases.
Influenza vaccines can be classified into two types in terms of the type of production method or virus immunization: (a) egg-based vaccines and cell-based vaccines for the former, and (b) trivalent influenza vaccines (TIVs) and quadrivalent influenza vaccines (QIVs) for the latter. Several studies have reported that the safety of influenza vaccines may vary according to subtypes13,14,15, but most of the evidence is dependent on the results of randomized clinical trials (RCTs) with a limited population and study period. Therefore, an additional study with a large-scale database is needed to establish safety profiles by subtype.
There are two types of methods to detect safety signals (quantitative and qualitative method), and a variety of quantitative data mining methods are available for post-marketing safety monitoring. Typically, disproportionality-based methods are widely used to detect safety signals based on an imbalance between the reporting rate of specific AEs for a specific drug and that of specific AEs for other drugs. The proportional reporting ratio (PRR), reporting odds ratio (ROR), and information component (IC) are the most common measurements in disproportionality-based methods16. Recently, tree-based scan statistics (TSS) have been applied in the signal detection of vaccines17,18, which were proposed to analyze a database with a hierarchical tree structure with adjustment for multiple comparisons19. Although various data mining methods have been developed and widely used for detecting signals, inconsistent results have been reported using different methods17,18, implying that the use of multiple data mining methods should be considered simultaneously.
Given the limited safety profiles on influenza vaccine and inconsistent results by different data mining methods, we conducted a nationwide, post-marketing surveillance study to detect adverse events following influenza vaccination using three different data mining methods. Firstly, we identified unexpected AEs which are potentially related to influenza vaccination, particularly focusing on five serious AEs of special interest (GBS, febrile seizure, anaphylaxis, narcolepsy, and Bell’s palsy). Secondly, we aimed to provide comparative safety profiles by the subtype of influenza vaccine (type of production method: egg-based vs. cell-based; type of virus immunization: QIVs vs. TIVs). Lastly, we compared the results of signal detection from each data mining methods to evaluate their performance on predicting safety signals.
Results
General characteristics
The basic characteristics of influenza vaccine-related AEs compared with those of other vaccines are provided in Table 1. More AEs were reported in females than in males for both influenza vaccines and all other vaccines, accounting for 68.77% and 57.30%, respectively. The proportion of children below 24 months in the influenza vaccine group was higher than that of other vaccines, while that of individuals 19 years or older in the influenza vaccine group was lower than that of other vaccines. When stratified by route of administration, most of the characteristics between the influenza vaccine and other vaccines were similar, except for the origin and year of the report: most came from a pharmaceutical company for all other vaccines (74.71%), while pharmaceutical companies with regional pharmacovigilance centers (RPVC) for influenza vaccines (52.84% and 42.57%, respectively) and the proportion of serious influenza vaccine-related AEs (5.18%) were less than that of other vaccines (8.06%).
Signal detection
Compared to all other vaccines, we identified 36 AEs as safety signals of influenza vaccines detected by any data mining method at least once (see Supplementary Material S2 online). Among them, seven AEs (injection site inflammation, muscle weakness, quadriplegia, paraplegia, hyperventilation, cachexia, and narcolepsy) were not included in the labeling information and were detected by all data mining methods, except for paraplegia (not in TSS). For serious AEs of special interest, anaphylaxis, neuritis (a preferred term [PT] of GBS), and narcolepsy were detected as signals compared to all other vaccines by all data mining methods, and paralysis (a PT of Bell’s palsy) was detected by IC and TSS (Table 2). Additionally, each method detected a similar number of safety signals (PRR, 29; IC, 30; TSS, 27).
Different findings have been reported for influenza vaccine subtypes using various methods. When using TIV as a comparator, 13 AEs were detected as safety signals by any data mining method at least once for QIV, and among them (see Supplementary Material S3 online), two safety signals (injection site bruising and cachexia) were not included in the labeling (Table 3). Moreover, each method generated a similar number of safety signals: 13 signals for PRR and TSS and 12 signals for IC. In contrast, we detected 59 AEs as safety signals at least once for TIV when compared to QIV; 26 safety signals were not included in labeling, but only 5 of these were overlapped by the three methods.
Compared to the egg-based vaccine, we identified 13 AEs as safety signals of cell-based vaccines using at least one data mining method (see Supplementary Material S4 online). Among them, four safety signals (injection site bruising, pneumonitis, encephalopathy, and tachycardia) were not included in the labeling, but only injection site bruising was detected by all three methods (Table 4). PRR, IC, and TSS detected 9, 13, and 6 safety signals, respectively. When using cell-based vaccines as a comparator, 30 AEs were detected as safety signals by any data mining method at least once for egg-based vaccines, 4 of which (leg pain, tremor, injection site mass, and cachexia) were not included in the labeling. PRR, IC, and TSS detected 20, 2, and 23 safety signals, respectively.
Performance evaluation
We measured the performance indicators of each data mining method based on the confusion matrix, where all detected signals were categorized into true positive (TP), false positive (FP), true negative (TN), or false negative (FN) based on the reference standard. Overall, TSS demonstrated well balance. TSS showed the highest sensitivity (77.8%), positive predictive value (PPV, 77.8%), accuracy (72.1%), and area under the curve (AUC, 70.1%) and the lowest specificity (62.5%). In contrast, IC showed the highest specificity (94.7%) and negative predictive value (NPV, 64.3%). In addition, except for specificity, all PRR measurements were the lowest (Fig. 1).
Discussion
We conducted a safety surveillance study on influenza vaccines and their subtypes using a nationwide spontaneous reporting database with three different data mining methods. The total number of signals detected by disproportionality-based analysis and TSS for the influenza vaccine compared to other vaccines were similar, and few unexpected AEs for the overall influenza vaccine were observed. Additional signal detection according to subtype was also conducted, where we noted that a wider range of AEs were detected as signals for TIVs and egg-based vaccines. Regarding the evaluation for signal detection, TSS showed better performance than the other data mining methods, with the optimal balance among all performance measurements.
From the results of the signal detection for the overall influenza vaccine, we identified that most signals were known AEs, such as application site disorders, body as a whole-general disorder, and musculoskeletal system disorders. All five serious AEs of special interest for influenza vaccine were detected as signals, except for febrile seizure. Similar findings were reported in two recent studies, both of which used two disproportionality-based methods (PRR and IC) for safety signal detection of influenza vaccines using the U.S. Vaccine Adverse Event Reporting System (VAERS), where GBS and paralysis were detected as signals by all three methods, respectively20,21. Among the seven detected signals not listed in the labeling in South Korea, injection-site inflammation and muscle weakness are reflected in the U.S. Food and Drug Administration (US-FDA) labeling. Accordingly, injection site inflammation and muscle weakness should be updated after signal evaluation. Of the remaining five unlabeled signals in both the U.S. and South Korea, narcolepsy seems to require urgent assessment. As mentioned earlier, narcolepsy cases grew rapidly following the 2009–2010 H1N1 vaccination in Finland and five other European countries (see Supplementary Material S5 online). Subsequent studies have shown that the AS03-adjuvanted pandemic vaccine Pandemrix (GlaxoSmithKline Biologicals, Wavre, Belgium), which was widely used in these countries during that period, contains a protein structurally similar to the orexin receptor, which is likely to induce an autoimmune response that destroys orexin cells in the brain and causes narcolepsy22. The H1N1 vaccine adjuvanted with AS03 was not used in Korea, but we identified narcolepsy as a safety signal for the influenza vaccine compared with other vaccines in this study, although there were only 11 reported cases. Therefore, signal validation through an in-depth evaluation of these cases is required.
Furthermore, we noted that the distribution of AEs differed between QIVs and TIVs. Local reactions at the injection site were more frequently reported in QIVs than in TIVs in our studies, which is consistent with previous studies. A meta-analysis of 5 RCTs reported that the incidence of injection site pain was higher with QIVs than TIVs with a pooled risk ratio of 1.18 (95% confidence interval: 1.03–1.35)23. In addition, a wider range of AEs were detected as signals for TIVs in our study, which showed inconsistent results with a previous RCT24 that demonstrated that QIVs and TIVs showed similar profiles. This may be attributed to the different distribution of age groups between QIVs and TIVs, with the proportion of the older group (65 years and above) being 0.47% and 10.1%, respectively. The aged population is vulnerable to adverse drug reactions due to age-related physiological changes25.
Moreover, we detected a wider range of AEs as signals with egg-based vaccines compared with cell-based vaccines, suggesting that safety profiles among these two subtypes may differ, which is inconsistent with the results of previous studies. Two RCTs demonstrated that cell-based and egg-based TIVs had comparable safety profiles in children and adolescents < 18 years of age26,27. We believe that this situation may be due to the following reasons. First, compared with cell-based vaccines, an inherent drawback of egg-based vaccines is that impurities or exogenous contamination may cause AEs28. In addition, Korea approved the first cell-based influenza vaccine in 2014, and egg-based influenza vaccines still dominate the majority of the market share29. In our database, cell-based vaccines only accounted for 15% of all cases; therefore, it is possible that some AEs of cell-based vaccines have not yet been revealed. Therefore, it is necessary to collect additional information.
In this study, TSS had the highest accuracy (72.1%), AUC (70.1%), PPV (77.8%), and sensitivity (77.8%). TSS allows the simultaneous evaluation of a large number of individual AE terms and related AE groups, thereby resolving multiple testing problems of disproportionality-based methods30. Two previous studies of the BCG vaccine and pneumococcal vaccine using the KAERS database also showed balanced performance in TSS17,18. However, we found that the sensitivity of TSS in this study was much higher than that in the BCG and pneumococcal vaccine studies (35.3% and 31.7%, respectively). There are two possible explanations for this observation. First, safety profiles are relatively well-established in the influenza vaccine, which targets the entire population as more extensive AEs have been reported and studied, compared to BCG and pneumococcal vaccines, both of which have limited recipients. Furthermore, the ability to detect signals through data mining techniques is largely influenced by the size of the dataset, which differs significantly between influenza and other vaccines (current study, 17,378 AE reports; BCG, 833 AE reports; pneumococcal vaccine, 1135 AE reports).
This study had several limitations. First, quantitative signal detection using passive surveillance systems, such as the KAERS database, has the inherent limitations of missing information, inconsistent quality of individual case safety reports, duplicated reporting, and under-reporting due to lack of awareness31. Therefore, our results do not reflect a causal relationship between vaccines and AEs, and further signal validation and assessment are needed. Second, in this study, the reference standard was established based on Ministry of Food and Drug Safety of South Korea (MFDS)-approved labeling for performance evaluation; thus, the results could be largely affected by the reference standard32. However, we believe that the labeling included all related AEs since influenza vaccines have been used for a long time. Third, our findings should not be overstated that TSS is the best-balanced method to detect safety signals in other vaccines or even in drugs as individual vaccine or drugs has different medical or clinical context. Therefore, further studies should be conducted to confirm our finding on the performance TSS across different vaccines and drugs. Lastly, although TSS showed the well-balanced performance compared to other data mining methods, TSS did not show the best values in all performance measurements, particularly in specificity. This result could be explained by the inverse relationship between sensitivity and specificity when considering the highest sensitivity in TSS, which were about threefold higher compared with the other methods (TSS: 77.8%, PRR: 21.1%, IC: 25.5%). This result suggests that the utilization of multiple data mining methods is reasonable choice to detect safety signals conservatively. In addition, the low values of accuracy and AUC were observed across all methods (TSS: 72.1%, 70.1%; PRR: 63.2%, 57.1%; IC: 66.1%, 60.1%). These results might be influenced by the reference standard that was used for constructing a confusion matrix for performance measurement. However, as the bias from reference standard could affect the performance of all methods, we believe that this bias is unlikely to distort the results of performance. Another possibility is that these methods are not precise or advanced to detect safety signals appropriately. Recently, machine learning algorithms have been utilized in the pharmacovigilance field with relatively high performance, compared with the traditional data mining methods33. However, we did not utilize this method in this study. Therefore, further research involving a variety of new methods such as machine learning is needed.
Conclusions
In this post-marketing surveillance study, we identify 7 new safety signals for influenza vaccine including narcolepsy as a serious AE, and observed TIVs and egg-based influenza vaccine showed a wide range of new safety signals when compared to QIVs and cell-based influenza vaccine, respectively. Further studies are needed to confirm the potential relationship between influenza vaccine and new safety signals, and meanwhile, healthcare providers should evaluate whether vaccinees are susceptible to experiencing these safety signals by an in-depth review of comorbidities or AE-related history to prevent an avoidable risk. In addition, although TSS showed balanced performance in this study, there is still discrepancy in the identified safety signals among different methods, and some measurements were lower than the others. This result suggests that the complementary use of multiple data mining methods should be utilized to detect new safety signals conservatively.
Methods
Data source
We used the Korea Institute of Drug Safety and Risk Management-Korea Adverse Event Reporting System Database (KIDS-KAERS database)34 from 2005 to 2019. This database includes all spontaneous AE reports in South Korea and detailed information on demographics, vaccination, AEs, causality assessment, and clinical information. Information on the vaccine product was also provided. All drugs and AEs are coded according to the Anatomical Therapeutic Chemical Classification (ATC) system and the World Health Organization-Adverse Reaction Terminology (WHO-ART) dictionary35,36, respectively. WHO-ART is established as a tree structure, including system organ class (SOC), high-level terms (HLT), PTs, and included terms (IT) from top to bottom, and the signals were determined at the PT levels in this study.
The need for informed consent was waived, as this study was conducted using a spontaneous reporting database which does not include personal information. This study was approved by the Institutional Review Board of Chung-Ang University in South Korea (1041078-202008-ZZ-209-01). The Institutional Review Board of Chung-Ang University has waived informed consent for this study. All methods were carried out in accordance with the relevant guidelines and regulations.
Study vaccine
All vaccines available in Korea were included. The study vaccine was an influenza vaccine, and all other vaccines were used as comparators for signal detection. Nineteen influenza vaccine products have been approved and used in Korea, with 2 cell-based vaccines, 17 egg-based vaccines, 8 TIVs, and 11 QIVs. A full list of influenza vaccine products used in our study is given in Supplementary Material S1 online.
Selection of AE reports
From all AEs following vaccination collected in our database from 2005 to 2019, we excluded the following: (1) those that were reported as a concomitant vaccine, not a suspected vaccine; (2) those that were not final reports; (3) those that were invalid reports (missing or unspecified drugs or AE codes), and (4) those with report errors (logical error).
From a total of 42,211 reports in our database, we identified 38,221 reports after vaccination. Among these, 17,378 (45.5%) were reported for the influenza vaccine. We identified 17,280 reports, of which 7639 (44.2%) were reported for QIV, 9641 (55.8%) for TIV, 2744 (15.9%) for cell-based influenza vaccine, and 14,536 (84.1%) for egg-based influenza vaccine. A flowchart of the study is presented in Fig. 2.
Selection of five AEs of special interest
Based on literature REVIEWS, we selected the following five possible serious and important AEs observed after influenza vaccination: GBS, febrile conversion, anaphylaxis, narcolepsy, and Bell’s palsy. Detailed information is provided in Supplementary Material S5 online.
Statistical analyses
Descriptive analyses
We calculated the frequency and proportion of baseline demographic characteristics (e.g., sex and age group) and reports (e.g., report type, source, reporter, seriousness, and causality assessment). Chi-squared tests were performed to compare the distribution of baseline characteristics between the influenza vaccine and all other vaccines or among individual subtypes of influenza vaccine (TIVs vs. QIVs and egg-based vs. cell-based vaccines).
Algorithms for signal detection
To detect the signals, three signal detection algorithms were used in this study, including PRR, IC, and TSS.
Disproportionality-based analysis
In this study, we used PRR and IC to generate signal scores for all AE pairs for the influenza vaccine versus all other vaccines, QIVs versus TIVs, and cell-based versus egg-based influenza vaccine. The Medicines and Healthcare Products Regulatory Agency (MHRA) in the United Kingdom and the WHO use PRR and IC, respectively37. The thresholds of signals for each algorithm were defined as follows: PRR ≥ 2, chi-squared test ≥ 4, number of reported AEs ≥ 3, IC with a lower limit of 95% confidence interval ≥ 0.
Tree-based scan statistic (TSS)
TSS is based on log-likelihood ratio (LLR) statistics and is suitable for analyzing a hierarchical structure variable. In this study, a diagnosis tree defined by the WHO-ART hierarchical structure was constructed, and the signals were determined at the PT level to compare them with the signals detected by the other algorithms. The unconditional Bernoulli model was selected to compare the number of AE reports between the two drugs. The expected and observed counts at each leaf were generated under the null hypothesis of an independent relationship between the vaccine and AE. AEs were defined as signals when the p-value of LLR was lower than 0.05, and p-values were generated using a Monte Carlo simulation. The analysis was performed using TreeScan Software38.
Performance evaluation of signal detection algorithms
The performance of disproportionality analysis (PRR, IC) and TSS was evaluated based on performance indicators such as sensitivity, specificity, PPV, NPV, accuracy, and AUC to identify which algorithm was more appropriate for detecting safety signals for influenza vaccines. To evaluate the performance indicators, a reference standard was established based on the MFDS-approved package inserts downloaded on September 13, 2020, from the NeDRUG website of the MFDS39.
Data availability
The data underlying this study are from the KAERS which has been transferred to the KIDS. The Korean government prohibits the release of the spontaneous reporting dataset to the public domain. Interested researchers can obtain the data through formal application to the KIDS. Korea Institute of Drug Safety & Risk Management (Ministry of Food and Drug Safety), Korean (https://open.drugsafe.or.kr/original/invitation.jsp).
References
Iuliano, A. D. et al. Estimates of global seasonal influenza-associated respiratory mortality: A modelling study. Lancet 391, 1285–1300 (2018).
Simonsen, L. et al. Global mortality estimates for the 2009 Influenza pandemic from the GLaMOR project: A modeling study. PLoS Med. 10(11), e1001558 (2013).
Osterholm, M. T., Kelley, N. S., Sommer, A. & Belongia, E. A. Efficacy and effectiveness of influenza vaccines: A systematic review and meta-analysis. Lancet Infect. Dis. 12(1), 36–44 (2012).
Yoon, D., Kim, J. H., Lee, H. & Shin, J. Y. Updates on vaccine safety and post-licensure surveillance for adverse events following immunization in South Korea, 2005–2017. Yonsei Med. J. 61, 623–630 (2020).
Aronson, J. Safety. BMJ 351, h3521 (2015).
Edwards, I. R. & Aronson, J. K. Adverse drug reactions: Definitions, diagnosis, and management. Lancet 356, 1255–1259 (2000).
Kim, C. et al. Pandemic influenza A vaccination and incidence of Guillain-Barré syndrome in Korea. Vaccine 33, 1815–1823 (2015).
Kim, J. H. et al. Adverse events following immunization (AEFI) with the novel influenza a (H1N1) 2009 vaccine: findings from the national registry of all vaccine recipients and AEFI and the passive surveillance system in South Korea. Jpn. J. Infect. Dis. 65, 99–104 (2012).
Leroy, Z., Broder, K., Menschik, D., Shimabukuro, T. & Martin, D. Febrile seizures after 2010–2011 influenza vaccine in young children, United States: a vaccine safety signal from the vaccine adverse event reporting system. Vaccine 30, 2020–2023 (2012).
Mutsch, M. et al. Use of the inactivated intranasal influenza vaccine and the risk of Bell’s palsy in Switzerland. N. Engl. J. Med. 350, 896–903 (2004).
Nohynek, H. et al. AS03 adjuvanted AH1N1 vaccine associated with an abrupt increase in the incidence of childhood narcolepsy in Finland. PLoS ONE 7, e33536 (2012).
Zhou, W. et al. A potential signal of Bell’s palsy after parenteral inactivated influenza vaccines: Reports to the vaccine adverse event reporting system (VAERS)–United States, 1991–2001. Pharmacoepidemiol. Drug Saf. 13, 505–510 (2004).
Baxter, R. et al. Evaluation of the safety, reactogenicity and immunogenicity of FluBlok® trivalent recombinant baculovirus-expressed hemagglutinin influenza vaccine administered intramuscularly to healthy adults 50–64 years of age. Vaccine 29, 2272–2278 (2011).
King, J. C. Jr. et al. Evaluation of the safety, reactogenicity and immunogenicity of FluBlok trivalent recombinant baculovirus-expressed hemagglutinin influenza vaccine administered intramuscularly to healthy children aged 6–59 months. Vaccine 27, 6589–6594 (2009).
Szymczakiewicz-Multanowska, A. et al. Safety and immunogenicity of a novel influenza subunit vaccine produced in mammalian cell culture. J. Infect. Dis. 200, 841–848 (2009).
Duggirala, H. J. et al. Use of data mining at the food and drug administration. J. Am. Med. Inform. Assoc. 23, 428–434 (2016).
Kim, J. H., Lee, H. & Shin, J. Y. Bacillus calmette-guerin (BCG) vaccine safety surveillance in the Korea adverse event reporting system using the tree-based scan statistic and conventional disproportionality-based algorithms. Vaccine 38, 3702–3710 (2020).
Lee, H., Kim, J. H., Choe, Y. J. & Shin, J. Y. Safety surveillance of pneumococcal vaccine using three algorithms: Disproportionality methods, empirical bayes geometric mean, and tree-based scan statistic. Vaccines 8, 242 (2020).
Kulldorff, M., Fang, Z. & Walsh, S. J. A tree-based scan statistic for database disease surveillance. Biometrics 59, 323–331 (2003).
Kamath, A., Maity, N. & Nayak, M. A. Facial paralysis following influenza vaccination: A disproportionality analysis using the vaccine adverse event reporting system database. Clin. Drug. Investig. 40, 883–889 (2020).
Lee, H., Kim, H. J., Choe, Y. J. & Shin, J. Y. Signals and trends of Guillain-Barré syndrome after the introduction of live-attenuated vaccines for influenza in the US and South Korean adverse event reporting systems. Vaccine 38, 5464–5473 (2020).
Ahmed, S. S. et al. Antibodies to influenza nucleoprotein cross-react with human hypocretin receptor 2. Sci. Transl. Med. 7, 294ra105 (2015).
Moa, A. M., Chughtai, A. A., Muscatello, D. J., Turner, R. M. & MacIntyre, C. R. Immunogenicity and safety of inactivated quadrivalent influenza vaccine in adults: A systematic review and meta-analysis of randomised controlled trials. Vaccine 34, 4092–4102 (2016).
Treanor, J. T. et al. Immunogenicity and safety of a quadrivalent inactivated influenza vaccine compared with two trivalent inactivated influenza vaccines containing alternate B strains in adults: A phase 3, randomized noninferiority study. Vaccine 35, 1856–1864 (2017).
Carroll, C. & Hassanin, A. Polypharmacy in the elderly-when good drugs lead to bad outcomes: A teachable moment. JAMA Intern. Med. 177, 871 (2017).
Diez-Domingo, J. et al. Safety and tolerability of cell culture-derived and egg-derived trivalent influenza vaccines in 3 to <18-year-old children and adolescents at risk of influenza-related complications. Int. J. Infect. Dis. 49, 171–178 (2016).
Oh, C. E. et al. A randomized, double-blind, active-controlled clinical trial of a cell culture-derived inactivated trivalent influenza vaccine (NBP607) in healthy children 6 months through 18 years of age. Pediatr. Infect. Dis. J. 37, 605–611 (2018).
Pandey, A., Singh, N., Sambhara, S. & Mittal, S. K. Egg-independent vaccine strategies for highly pathogenic H5N1 influenza viruses. Hum. Vaccin. 6, 178–188 (2010).
Pérez Rubio, A. & Eiros, J. M. Cell culture-derived flu vaccine present and future. Hum. Vaccin. Immunother. 14, 1874–1882 (2018).
Kulldorff, M. et al. Drug safety data mining with a tree-based scan statistic. Pharmacoepidemiol. Drug Saf. 22, 517–523 (2013).
Rosenthal, S. & Chen, R. The reporting sensitivities of two passive surveillance systems for vaccine adverse events. Am. J. Public Health. 85, 1706–1709 (1995).
Whiting, P. et al. Development and validation of methods for assessing the quality of diagnostic accuracy studies. Health Technol. Assess. 8, 1–234 (2004).
Bae, J. H. et al. Machine learning for detection of safety signals from spontaneous reporting system data: Example of nivolumab and docetaxel. Front. Pharmacol. 11, 602365 (2021).
Korea Institute of Drug Safety & Risk Management . Introduction of KAERS (Korea Adverse Event Reporting System) https://www.drugsafe.or.kr/iwt/ds/en/report/WhatIsKAERS.do (2022).
World Heatlth Organization. Anatomical Therapeutic Chemical (ATC) Classification https://www.who.int/tools/atc-ddd-toolkit/atc-classification (2022).
World Heatlth Organization. WHO-adverse reaction terminology (WHO-ART) in Dictionary of Pharmaceutical Medicine 192–193 (Springer, 2009).
Sakaeda, T., Tamon, A., Kadoyama, K. & Okuno, Y. Data mining of the public version of the FDA adverse event reporting system. Int. J. Med. Sci. 10, 796–803 (2013).
TreeScan. TreeScan User Guide https://www.treescan.org/techdoc.html (2022).
Ministry of Food and Drug Safety. Integrated System for Safety Information of Drug https://nedrug.mfds.go.kr/index (2022).
Acknowledgements
This work was supported by a grant (21153MFDS607) from the Ministry of Food and Drug Safety of South Korea in 2021-2025 (to J-YS), and a grant (20200509312-00) from the Ministry of Food and Drug Safety of South Korea in 2020 (to S-YJ). Moreover, this work was supported by the Bio Industry Technology Development Program (No. 20015086) By the Ministry of Trade, Industry & Energy (MOTIE, Korea) and Government-wide R&D Fund project for infectious disease research (GFID), Republic of Korea (grant No HG18C0068) (to J-YS). The funders had no role in the study design, data collection and analysis, interpretation of data, writing of the report, or decision to submit the article for publication. The authors thank the Korea Institute of Drug Safety and Risk Management (KIDS) for their cooperation in providing access to the Korea Adverse Event Reporting System database. We would like to thank Editage (www.editage.co.kr) for English language editing.
Author information
Authors and Affiliations
Contributions
H.L., B.H., S.-Y.J., and J.-Y.S. were responsible for the study concept and design. S.-H.K. and J.-H.K. were responsible for the literature search. S.-Y.J. and J.-H.K. were responsible for acquisition of data. J.-H.K. and B.H. were responsible for analysis and interpretation of data. H.L. and B.H. were responsible for drafting of the manuscript. N.-K.C., S.-Y.J., and J.-Y.S. were responsible for critical revision of the manuscript for important intellectual content. The corresponding authors attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Corresponding authors
Ethics declarations
Competing interests
J-YS received grants from the Ministry of Food and Drug Safety, the Ministry of Health and Welfare, the National Research Foundation of Korea, and pharmaceutical companies, including Daiichi Sankyo, GSK, and Pfizer, outside the submitted work. S-YJ received grants from the Ministry of Food and Drug Safety, the Ministry of Health and Welfare, the National Research Foundation of Korea. All other authors have no conflicts of interest to report.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Lee, H., Hong, B., Kim, S. et al. Post-marketing surveillance study on influenza vaccine in South Korea using a nationwide spontaneous reporting database with multiple data mining methods. Sci Rep 12, 20256 (2022). https://doi.org/10.1038/s41598-022-21986-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-022-21986-8
- Springer Nature Limited