Abstract
Automated Static Analysis Tools (ASATs) are an additional tool available to developers in their pursuit of high-quality software. ASATs match source code against configured rules and produce a warning when a rule is violated. However, the evaluation of the warnings by developers as well as the resolution of warnings requires time. This raises the question of whether we are able to evaluate the usefulness of ASATs empirically. Within this chapter, we present the results of four case studies, which investigate different aspects regarding the impact of ASATs on software quality and the perception of the developers thereof. We present results regarding the evolution of ASAT warnings from a longitudinal study of 54 open-source projects. To evaluate the impact on defects, we present results from two studies. The first study is evaluating predictive models in the context of defect prediction with ASAT-based features. The second study provides a statistical investigation of the differences between changes that induce a defect and all other changes. In order to observe the developer’s perspective regarding ASAT warnings and other software quality metrics, we include the results of a study of developer intent, which compares changes where the developers intend to improve the quality of the code base with all other changes to see which quality metrics and ASAT warnings change in which way. We employ methods of empirical software engineering research to investigate these relationships and provide evidence-based information for researchers and practitioners alike. Within our studies, we can show empirically that we are able to measure an impact on quality. However, the effect is surprisingly small. Moreover, our investigation of developer intents yield information about the magnitude of bug fixing as a driver for complexity in software. Our results can help practitioners estimate the possible impact of introducing an ASAT on defects, as well as provide guidelines for managing the complexity of software.
Chapter PDF
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2024 The Author(s)
About this chapter
Cite this chapter
Trautsch, A. (2024). Usefulness of Automatic Static Analysis Tools: Evidence from Four Case Studies. In: Bodden, E., et al. Ernst Denert Award for Software Engineering 2022. Springer, Cham. https://doi.org/10.1007/978-3-031-44412-8_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-44412-8_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44411-1
Online ISBN: 978-3-031-44412-8
eBook Packages: Computer ScienceComputer Science (R0)