Abstract
Structured reporting contributes to the completeness of radiology reports and improves quality. Both the content and the structure are essential for successful implementation of structured reporting. Contextual structured reporting is tailored to a specific scenario and can contain information retrieved from the context. Critical findings detected by imaging need urgent communication to the referring physician. According to guidelines, the occurrence of this communication should be documented in the radiology reports and should contain when, to whom and how was communicated. In free-text reporting, one or more of these required items might be omitted. We developed a contextual structured reporting template to ensure complete documentation of the communication of critical findings. The WHEN and HOW items were included automatically, and the insertion of the WHO-item was facilitated by the template. A pre- and post-implementation study demonstrated a substantial improvement in guideline adherence. The template usage improved in the long-term post-implementation study compared with the short-term results. The two most often occurring categories of critical findings are “infection / inflammation” and “oncology”, corresponding to the a large part of urgency level 2 (to be reported within 6 h) and level 3 (to be reported within 6 days), respectively. We conclude that contextual structured reporting is feasible for required elements in radiology reporting and for automated insertion of context-dependent data. Contextual structured reporting improves guideline adherence for communication of critical findings.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Structured reporting is a proven concept in radiology [1]. It contributes to an improved inter-reader agreement [2], improved communication [3], guideline compliance [4] and improved economic efficiency [5]. It facilitates data extraction [6], epidemiological research [7], and the development of deep learning algorithms [8]. Structured reporting is important in quality improvement programs, as indicated by 75% of respondents in an international survey [9].
Despite its broad application, different definitions for structured reporting are used interchangeably [10]. First, it can refer to standardized reporting where a template with well-established required components is used to make a structured report. This type of structured reporting improves quality by ensuring coherent and complete reports. The second definition of structured reporting refers to the way the content is arranged. Integrated into an IT-based workflow, this can contribute to improved usability and acceptance. Both components (content and structure) are needed to ensure that the structured reporting template contributes to quality improvement and contributes to efficient workflow. Suboptimal usability lowers adherence, and the potential quality improvement is not fully reached.
A drawback of structured reporting is the perceived inflexibility, limiting radiologists in using their creativity [11]. Radiologists can be forced to use a template that is not fully adapted to a specific situation. A solution for this is contextual structured reporting that is tailored to the clinical scenario [12]. In contextual structured reporting, different templates are used for various specific clinical scenarios. It can even refer to a more modular approach for different findings within clinical scenarios. With contextual structured reporting, a radiologist can use modular “building blocks” to customize a report to particular situations.
In this study, we worked out two concepts for context-dependent structured reporting, for both content and structure:
-
1.
Structured reporting for a specific task within a radiology report using a sub-template
-
2.
Automated insertion of data within this sub-template, dependent on contextual information
We report on the context-dependent structured reporting for the documentation of the communication of critical findings.
Critical findings are defined as findings in a radiological examination that need urgent patient management [13, 14]. Referring physicians need to be made aware of these findings as soon as possible. According to our national guideline, this communication should be documented in the radiology report and should contain three elements: a. by what method the communication took place, b. when the communication occurred, and c. to whom the findings were communicated. These items need to be included only in cases where critical findings are present and when this specific communication has taken place. In free-text reporting, there is the risk of omitting items. Structured reporting could help to increase adherence to guidelines [15], but its role in improving the communication of critical findings has not yet been investigated.
The purpose of this study is to:
-
1.
develop a structured radiology report template with the automated insertion of context-dependent data elements.
-
2.
implement this template in clinical practice to improve guideline adherence in the communication of critical findings.
-
3.
evaluate the usability of this template by assessing long-term usage
Methods
Report template
In June 2016, a local quality improvement project was started to facilitate radiologists in documenting their communication of critical findings in radiology reports according to our national guideline. For this purpose, an in-house structured reporting template was developed in the local PACS (IDS7 Sectra AB, Sweden) using the template editor of Sectra (Fig. 1a). The template was created by a local radiologist (AO) and made available to all radiologists of the three hospital locations of the Treant healthcare group.
The template consisted of three types of text: fixed, variable automated, and variable manual.
-
Fixed text: text describing the method of communication, to ensure the ‘How’ -item of the guideline.
-
Variable automated text dependent on the context:
-
code element that automatically inserts the actual date and time to ensure the ‘When’-item of the guideline.
-
code element that automatically inserts the name of the reporting radiologist.
-
-
Variable manual data-element: name of the contacted physician, to ensure the ‘Who’-item of the guideline.
The template can be activated anywhere in a radiology report or in an addendum to ensure usability. To do so, the radiologist gives a voice-command to the reporting system (Fig. 1b) at the time of communication of critical findings to the referring physician. Radiologists had the choice to use this template or free-text to document this communication. The critical finding itself is not included in the template, but is described in the sections ‘findings’ and ‘impression’ of the report.
Evaluation study
We performed a pre-implementation and post-implementation assessment. The collected data was restricted to reading process-related information not containing patient identifiers. Therefore, the local institutional review board waived the necessity of obtaining informed consent. The processing of the data was not “likely to result in a high risk”, according to the criteria in the Guidelines on Data Protection Impact Assessment (DPIA). Therefore a DPIA was not required [16].
We collected and compared data from three different time periods: July 2014 – June 2015 (before the publication of the national guideline), July 2016 – June 2017 (short-term post-implementation) and January 2020–13 March 2020 (long-term post-implementation).
Radiology reports were retrieved from the PACS to Excel by a text-mining query on Dutch words related to the communication of critical findings. The translated words/phrases are ‘by phone’, ‘called’, ‘verbally’, ‘passed on’, ‘telephone’, ‘discussed’, and ‘email’.
The total number of reports per time period were recorded to calculate the percentage of reports with documented communication of critical findings. Reports with undocumented communication, or follow-up on critical findings without documented communication were beyond the scope of our study. Figure 2 indicates the hierarchical subgroups ranging from all reports to reports with documented critical findings using the structured reporting template.
The retrieved radiology reports were manually checked, and reports containing critical findings were included. A radiologist (AO) scored the included reports for the presence of the three required items according to the guideline:
-
WHO: the name or role of the person to whom was communicated.
-
WHEN: the date and time of the communication.
-
HOW: the method of communication (direct verbal communication, telephone, or e-mail). These options for communication are based on the institutional working agreement. In case of e-mail communication, an acknowledgment of receipt should be recorded in the radiology report as well.
We scored whether or not the template was utilized. The flowchart of data collection is provided in Fig. 3.
The overall completeness of the information provided in the reports was assessed by adding the scored items per report ranging from 0 (none of the required items described), to 3 (all three items described). The results per radiologist were calculated by averaging the summed scores per time-period.
The content of the radiology reports was analysed by scoring the type of critical finding and the urgency level. According to the national guideline, level 1 represents life threatening critical findings that need to be communicated without delay within 60 min. Level 2 findings are to be communicated within 6 h, and level 3 findings within 6 days.
Statistical analyses were performed using SPSS. The presence of the items WHO, WHEN, and HOW in radiology reports pre-implementation were compared with short-term and long-term post-implementation using the Chi-Square Test. A p < 0.05 was considered to be statistically significant.
Results
The proportion of radiology reports containing communication of critical findings increased after the implementation of a structured reporting template from 0.79% in the pre-implementation period to respectively 1.10% and 1.44% in the short-term and long-term post-implementation periods (Fig. 3).
The reporting template usage increased from 30% of reports in the short-term post-implementation group to 83% of reports in the long-term post-implementation group (Table 2). Reports containing all three elements (WHO, WHEN, and HOW) were nearly absent in the pre-implementation period and increased by template usage in the short-term and long-term post-implementation periods (Fig. 4). When the template was used, the completeness of the description of the communication of critical findings increased significantly in the short-term and long-term post-implementation period for all three items. The highest increase was seen for the WHEN-item (from 2.5% to 100%, Table 1).). The WHO-item was present in 99.7% of cases when the template was used short-term post-implementation, despite requiring input from the radiologist. This increased to 100% long-term post-implementation. The HOW-item was around 50% in all groups without and around 100% in all groups with the template.
A visual impression of the changes between the study periods is presented in Table 2 by color-coding the average number of items and the usage percentages of the template ranging from low (red) to high (green).
Figure 5 demonstrates an overview of the content and the process of the critical findings: the types of critical findings, the urgency level categories, and the method of communication. For both reports with and without structured reporting template, the two most often occurring categories are “infection/inflammation” and “oncology”, corresponding to a large part of urgency level 2 and level 3, respectively. For all three levels e-mail communication was nearly absent.
Discussion
In this work, we present an implementation of contextual structured reporting to document the communication of critical findings in radiology reports. The template aides radiologists in documenting by the automated insertion of two context-dependent data elements (WHEN and HOW), and by guiding the radiologist to include the third required item (WHO). The voice command ensures the seamless integration in radiologist workflow. The rounded 100% score for all items indicates the technical feasibility of the system. The radiologists were required to dictate the name of the referring physician whom they contacted. The 99.7% adherence to this short-term post-implementation indicates that guiding of the radiologists to the specific insert-field was sufficient to facilitate them to perform the required action.
Clinical implementation was successful. A complete description of the communication of critical findings with all three required items (WHO, WHEN, HOW) was achieved in nearly 100% of the reports with template usage, compared to less than 5% of the reports without the template. This result was not only valid for a pre- versus post-implementation comparison (which is to be expected), but also in a post-implementation comparison between reports with and without a template. This difference illustrates the value of system-level interventions to improve quality.
The initial low compliance of template usage in the short-term post-implementation indicates that broad usage is not guaranteed. Technical feasibility does not immediately lead to broad usage. The high compliance in the long-term demonstrates that compliance was not restricted because of usability problems of the template itself. Increased awareness of guidelines and positive feedback probably contributed to the increased adherence.
Regarding the types of pathology and the urgency levels, the visual representation of fig. 5 demonstrates a similar pattern for reports with and without structured reporting template. This not unexpected because the radiologists were free to use the template for all types of pathology. We found no other comparative studies with a similar design.
For the process of communication, it is striking that even though non-direct communication, like e-mail, is allowed, radiologists preferred communication by phone. Probably the lack of an automated recording of an acknowledgement or receipt attributed to this. Closed-loop communication could otherwise only be obtained by manually checking the confirmation that the message was received and understood, or by implementing an alert notification system [17].
Comparison with the literature
The DICOM Structured Reporting (SR) specification has been in use for over 20 years [18]. DICOM SR describes the technical specifications of integrating data elements in radiology reports, but implementation studies have been scarce in the years after introduction. Structured reporting studies usually emphasize clinical utility, with limited attention to technical details. Structured reports often consist of several template fields, with or without picklists, with a number of predetermined options for text within that field [19]. For our purpose, picklists would be less efficient compared to the automated insertion of data. A technical paper on Management of Radiology Report Templates (MRRT) illustrates the application of templates in the exchange of information in a clinical scenario of critical findings. Findings are coded to facilitate processing and automated notification of the referring physician [20]. Although the purpose and methodology differ from our study, we support the notion that technical solutions may improve efficiency and quality.
Our study is the first to describe an improved completeness of the communication of critical findings by implementing a structured reporting template. Our results are in line with previous studies, indicating that structured reporting implementation leads to more comprehensive radiology reports. Aase et al. demonstrate in a pre- and post-implementation study that template usage resulted in a higher percentage of required items in the description of incidental pulmonary nodules [4].
Another study on the documentation of the communication of critical findings (in neuroradiology only) also used a reporting template, but had a different purpose and methodology [21]. In this study a predefined list of critical findings was used and a random sample of selected reports was used. This allowed them to get insight into all reports with and without critical findings with the drawback of a relatively small sample of reports with critical findings (n = 195). Although our study lacks information about critical findings where the communication is not documented, this was not our study objective.
A broad range of guidelines is available for the management of different types of incidental and critical findings [22,23,24,25,26,27]. In addition to recommendations on managing incidental findings, several reports have been published on whether these recommendations improve the communication of critical findings [21, 28,29,30,31,32], and on the level of adherence to guidelines. For example, Clark et al. used free-text query software to identify studies investigating guideline adherence for two types of incidental findings (gallbladder polyps and thyroid nodules) [33]. As ours, this study also contributes to the evidence that quality and safety is improved by implementing guidelines and dedicated procedures, although the selected types of pathology and the methodology are different from our study.
We found no previous studies on the relationship between long-term usage and usability of structured reporting. Nevertheless, it is likely that these are associated, because in a study on usability measurement tools in eHealth, task completion was associated with usability [34].
In this study we analyzed the content of the critical findings by using manual annotation of radiology reports. Even though manual annotation can be regarded as gold standard, it is less suitable for continuous monitoring. Natural language processing is a promising method for automated monitoring and quality assurance of critical finding communication compliance [35].
Limitations
The periods examined in our study have different durations. The pre-implementation and the short-term post-implementation periods are one year, compared to 2.5 months duration for long-term post-implementation. This difference is a limitation but did not have an impact on the results because of the sufficient large sample size compared with the difference between the groups. The short-term data demonstrated the significant change caused by the template. The long-term data proved usability and adherence.
Radiologists were offered the opportunity to use a structured reporting template, but they were not obliged to do so. This method could have introduced bias, in that radiologists that favour the use of structured reporting might beforehand be more likely to adhere to guidelines of critical findings than their colleagues who used free text.
We observed that post-implementation, not all radiologists used structured reporting, and radiologists who did use structured reporting did not do so for all of their reports with critical findings, at times opting to use free-text instead. Feedback during the evaluation period could have improved compliance with using the structured reporting template (Table 2).
Selection bias might have occurred as the communication of critical findings could also have been described in other, less-common words. It is unlikely that these cases impacted the results because this bias would have been the same in all three study groups.
Even though we based our data retrieval on a selection of Dutch words, our method of data extraction can be replicated in any language.
Conclusion
Contextual structured reporting is feasible for required elements in radiology reporting and for automated insertion of context-dependent data.
Contextual structured reporting improves guideline adherence for communication of critical findings.
References
Rocha DM, Brasil LM, Lamas JM, Luz GVS, Bacelar SS (2020) Evidence of the benefits, advantages and potentialities of the structured radiological report: An integrative review. Artif Intell Med. https://doi.org/10.1016/j.artmed.2019.101770
Dimarco M, Cannella R, Pellegrino S, et al (2020) Impact of structured report on the quality of preoperative CT staging of pancreatic ductal adenocarcinoma: assessment of intra- and inter-reader variability. Abdom Radiol 45:437–448
Khurana A, Nelson LW, Myers CB, et al (2020) Reporting of acute pancreatitis by radiologists-time for a systematic change with structured reporting template. Abdom Radiol 45:1277–1289
Aase A, Fabbrini AE, White KM, Averill S, Gravely A, Melzer AC (2020) Implementation of a Standardized Template for Reporting of Incidental Pulmonary Nodules: Feasibility, Acceptability, and Outcomes. J Am Coll Radiol 17:216–223
Chung CY, Makeeva V, Yan J, Prater AB, Duszak R, Safdar NM, Heilbrun ME (2020) Improving Billing Accuracy Through Enterprise-Wide Standardized Structured Reporting With Cross-Divisional Shared Templates. J Am Coll Radiol 17:157–164
Goel AK, DiLella D, Dotsikas G, Hilts M, Kwan D, Paxton L (2019) Unlocking Radiology Reporting Data: an Implementation of Synoptic Radiology Reporting in Low-Dose CT Cancer Screening. J Digit Imaging 32:1044–1051
Pinto dos Santos D, Scheibl S, Arnhold G, Maehringer-Kunz A, Düber C, Mildenberger P, Kloeckner R (2018) A proof of concept for epidemiological research using structured reporting with pulmonary embolism as a use case. Br J Radiol 091:20170564
Pinto dos Santos D, Brodehl S, Baeßler B, Arnhold G, Dratsch T, Chon S-H, Mildenberger P, Jungmann F (2019) Structured report data can be used to develop deep learning algorithms: a proof of concept in ankle radiographs. Insights Imaging 10:93
Dick J, Darras KE, Lexa FJ, et al (2020) An International Survey of Quality and Safety Programs in Radiology. Can Assoc Radiol J. https://doi.org/10.1177/0846537119899195
Nobel JM, Kok EM, Robben SGF (2020) Redefining the structure of structured reporting in radiology. Insights Imaging. https://doi.org/10.1186/s13244-019-0831-6
Yousem DM (2019) In Opposition to Standardized Templated Reporting. Acad Radiol 26:981–982
Mamlouk MD, Chang PC, Saket RR (2018) Contextual radiology reporting: A new approach to neuroradiology structured templates. Am J Neuroradiol 39:1406–1414
Larson PA, Berland LL, Griffith B, Kahn CE, Liebscher LA (2014) Actionable Findings and the Role of IT Support: Report of the ACR Actionable Reporting Work Group. J Am Coll Radiol 11:552–558
Radiological Society of the Netherlands (2015) Guideline Critical Findings. https://www.radiologen.nl/secties/nvvr/documenten/leidraad-kritieke-bevindingen. Accessed 1 Jun 2016
European Society of Radiology (ESR) (2018) ESR paper on structured reporting in radiology. Insights Imaging 9:1–7
The working party on the protection of individuals with regard to the processing of personal data (European Commission) (2017) Guidelines on Data Protection Impact Assessment (DPIA). https://ec.europa.eu/newsroom/article29/item-detail.cfm?item_id=611236. Accessed 13 Jun 2019
Lacson R, O’Connor SD, Sahni VA, Roy C, Dalal A, Desai S, Khorasani R (2016) Impact of an electronic alert notification system embedded in radiologists’ workflow on closed-loop communication of critical results: a time series analysis. BMJ Qual Saf 25:518–524
Bidgood Jr. WD (1998) Clinical importance of the DICOM structured reporting standard. Int J Card Imaging 14:307–315
Herts BR, Gandhi NS, Schneider E, Coppa CP, Mody RN, Baker ME, Remer EM (2019) How we do it: Creating consistent structure and content in abdominal radiology report templates. Am J Roentgenol 212:490–496
Chen JY, Sippel Schmidt TM, Carr CD, Kahn CE (2017) Enabling the next-generation radiology report: Description of two new system standards. Radiographics 37:2106–2112
Babiarz LS, Lewin JS, Yousem DM (2015) Continuous Practice Quality Improvement Initiative for Communication of Critical Findings in Neuroradiology. Am J Med Qual 30:447–453
Hoang JK, Langer JE, Middleton WD, Wu CC, Hammers LW, Cronan JJ, Tessler FN, Grant EG, Berland LL (2015) Managing incidental thyroid nodules detected on imaging: White paper of the ACR incidental thyroid findings committee. J Am Coll Radiol 12:143–150
Khosa F, Krinsky G, Macari M, Yucel EK, Berland LL (2013) Managing incidental findings on abdominal and pelvic CT and MRI, part 2: White paper of the ACR Incidental Findings Committee II on vascular findings. J Am Coll Radiol 10:789–794
Patel MD, Ascher SM, Paspulati RM, Shanbhogue AK, Siegelman ES, Stein MW, Berland LL (2013) Managing incidental findings on abdominal and pelvic CT and MRI, Part 1: White paper of the ACR incidental findings committee II on adnexal findings. J Am Coll Radiol 10:675–681
Gore RM, Pickhardt PJ, Mortele KJ, Fishman EK, Horowitz JM, Fimmel CJ, Talamonti MS, Berland LL, Pandharipande P V. (2017) Management of Incidental Liver Lesions on CT: A White Paper of the ACR Incidental Findings Committee. J Am Coll Radiol 14:1429–1437
Megibow AJ, Baker ME, Morgan DE, Kamel IR, Sahani D V., Newman E, Brugge WR, Berland LL, Pandharipande P V. (2017) Management of Incidental Pancreatic Cysts: A White Paper of the ACR Incidental Findings Committee. J Am Coll Radiol 14:911–923
Smereka P, Doshi AM, Ream JM, Rosenkrantz AB (2017) The American College of Radiology Incidental Findings Committee Recommendations for Management of Incidental Lymph Nodes: A Single-Center Evaluation. Acad Radiol 24:603–608
Trotter SA, Babiarz LS, Viertel VG, Nagy P, Lewin JS, Yousem DM (2013) Determination and communication of critical findings in neuroradiology. J Am Coll Radiol 10:45–50
Honig SE, Honig EL, Babiarz LB, Lewin JS, Berlanstein B, Yousem DM (2014) Critical findings: Timing of notification in neuroradiology. Am J Neuroradiol 35:1485–1492
Babiarz LS, Trotter S, Viertel VG, Nagy P, Lewin JS, Yousem DM (2013) Neuroradiology critical findings lists: Survey of neuroradiology training programs. Am J Neuroradiol 34:735–739
Shahriari M, Liu L, Yousem DM (2016) Critical Findings: Attempts at Reducing Notification Errors. J Am Coll Radiol 13:1354–1358
Prevedello LM, Little KJ, Qian S, White RD (2017) Automated Critical Test Findings Identification and Online Notification System Using Artificial Intelligence in Imaging. Radiology 285:923–931
Clark TJ, Coats G (2016) Adherence to ACR Incidental Finding Guidelines. J Am Coll Radiol 13:1530–1533
Broekhuis M, van Velsen L, Hermens H (2019) Assessing usability of eHealth technology: A comparison of usability benchmarking instruments. Int J Med Inform 128:24–31
Heilbrun ME, Chapman BE, Narasimhan E, Patel N, Mowery D (2019) Feasibility of Natural Language Processing-Assisted Auditing of Critical Findings in Chest Radiology. J Am Coll Radiol. https://doi.org/10.1016/j.jacr.2019.05.038
Availability of data and material
The data of this research can be requested from the corresponding author.
Code availability
Not applicable.
Author information
Authors and Affiliations
Contributions
Allard Olthof: Conceptualization, Methodology, Data collection, Formal analysis, Writing - Original Draft.
Anne Leusveld: Conceptualization, Writing - Review & Editing.
Jan Cees de Groot: Formal analysis, Writing - Review & Editing.
Petra Callenbach: Formal analysis, Writing - Review & Editing.
Peter van Ooijen: Formal analysis, Writing - Review & Editing, Supervision.
All authors approved the final article
Corresponding author
Ethics declarations
Conflicts of interest/competing interests
The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.
Ethics approval
Institutional Review Board approval was not required because of the retrospective nature of the study.
Consent to participate
Written informed consent was not required for this study because patient data were anonymously used for the study and no interventions took place for the study.
Consent for publication
Not applicable.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article is part of the Topical Collection on Systems-Level Quality Improvement
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Olthof, A.W., Leusveld, A.L.M., de Groot, J.C. et al. Contextual Structured Reporting in Radiology: Implementation and Long-Term Evaluation in Improving the Communication of Critical Findings. J Med Syst 44, 148 (2020). https://doi.org/10.1007/s10916-020-01609-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10916-020-01609-3