Abstract
Due to the fast-paced nature of the software industry and the success of small agile projects, researchers and practitioners are interested in scaling agile processes to larger projects. Agile software development (ASD) has been growing in popularity for over two decades. With the success of small-scale agile transformation, organizations started to focus on scaling agile. There is a scarcity of literature in this field making it harder to find plausible evidence to identify the science behind large scale agile transformation. The objective of this paper is to present a better understanding of the current state of research in the field of scaled agile transformation and explore research gaps. This tertiary study identifies seven relevant peer reviewed studies and reports research findings and future research avenues.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Transformation from a traditional software engineering process to an agile process is not well understood. This area of agile transformation is relatively new, and researchers are working on different aspects to gain new understanding. Industry organizations that transformed from waterfall to agile implemented multiple strategies such as altering quality assurance practices [1, 2] and training staff in agile methods [3, 4]. However, organizations must consider additional factors when scaling such transformations.
In this paper, we present a tertiary study in scaled agile transformations. Our focus is on large projects defined by the number of people working on the project team. We are not aware of another tertiary study specific to scaled agile transformation. This study identifies gaps in the literature of scaled agile transformation that will help the community to identify potential research avenues in future.
We could not identify tertiary studies that present a meta-analysis of relevant secondary studies. Additionally, we observed there are not many secondary studies compared with more mature research areas. The goal for this study is to collate instead of synthesize; in this way we hope to identify gaps as opportunities for further study and understanding. As a relatively new area, researchers have been trying to answer different questions related to the scaled agile transformation. The contribution of this paper is to assemble evolving early large-scale agile transformation evidence.
The main goal of this tertiary study is to synthesize the research goal and findings of peer-reviewed secondary studies in scaled agile transformation. Under this goal, we addressed two specific questions:
-
RQ1: What success factors and challenges in scaled agile transforms have been identified by prior secondary studies?
-
RQ2: What gaps exist in existing studies that should be prioritized by the research community moving forward?
2 Research Methodology
We use [5] and [6] as guidelines for performing this tertiary study. The research methodology was primarily conducted by the first author and reviewed by the second author.
2.1 Search Process
The steps of the search process are shown in Fig. 1 and summarized as follows:
-
1.
Digital libraries from IEEE (29), ACM (12), SpringerLink (35), and ScienceDirect (46) were searched for journal, conference, and workshop papers.
-
2.
Duplicate studies were removed manually.
-
3.
For a tertiary study, we only consider secondary studies for analysis. Specifically, only systematic literature reviews (SLRs) were included beyond this step.
-
4.
The abstracts from step 3 were read. Reviewers searched for keywords and for verification that the study was an SLR on the topic of scaled agile transformation. Manual review was necessary as an initial keyword scan on terms such as ‘agile’, ‘large’, ‘scale’, ‘change’ OR ‘transformations’ resulted in too many false positives.
-
5.
The final set of papers was determined by a full-text manual review of the studies from step 4, conducted by both authors, focusing on research questions and paper quality. Paper quality review may be subjective [5]; we considered the publication venue, date of publication, citations of the paper, and citation indices of the venue and authors. Further, we also considered whether enough primary studies were included in the secondary study to validate claims of significance. Disagreements between the authors on inclusion or exclusion were resolved through discussion.
2.2 Summary of Included Studies
Agile adoption has rapidly increased over the past two decades. A few preliminary studies and calls for more research in scaled agile transformation started appearing in the 2000s as practitioners in large organizations are moved towards large-scale agile adoption (cf. [7,8,9,10]). Scholarly study of this phenomenon largely started appearing since 2015. Given the relatively small number of studies that met our inclusion criteria, we summarize these papers here.
Dikert, Paasivaara & Lassenius [12] present an extensive secondary study of 52 primary studies on the challenges and success factors of agile transformation. The study reported 35 challenges classified into 9 categories and 29 success factors classified into 11 categories. As per the authors, the most important success factors were management support, choosing the right agile model, mindset, and alignment with the organization’s value. The challenges included resistance to change, lack of training, and misunderstanding agile. This study is very influential and a widely referenced work, however, all of the primary studies used for evidence were from 2010 and earlier, and almost all describe transformation from a waterfall-like process to an agile process.
Uludag et al. [13] conducted a structured literature review of 73 papers related to the challenges of scaling agile from a stakeholder perspective. The study reported 79 challenges in 11 categories. The top 3 challenges were coordinating multiple teams working on the same project, considering dependencies in integration, and coordination among geographically distributed teams. This recent study reports new challenges in scaled agile transformations. Notably, it claims the majority of challenges in large-scale agile development (38 of 79) still exist and are “typical”. The stakeholder perspective focuses on development team roles, with only a few higher-level roles included.
An action research approach to a meta-review was taken by Kalenda, M., Hyna, P., & Rossi, B [11]. The authors identified 8 common features of scaling frameworks SAFe and LeSS [10] and used this to drive a focused literature review. 12 papers were selected, 10 of which mapped to at least one of the 8 common features, and then described challenges and success factors of each paper. The paper is influential due to its recency and use of scaling frameworks (whose awareness and adoption are becoming more prevalent in industry) as an organizing principle. However, the authors acknowledge this approach is not comprehensive, so there may be more evidence in the literature not included in the study. We also wonder if the approach of identifying common practices between SAFe and LeSS is appropriate, due to scaling agile not being a prescriptive formulaic process, and also because multiple frameworks were excluded (notably DaD [10] and SoS [10]). Nevertheless, this paper is highly influential and an ambitious action research for understanding scaled agile transformation.
Klünder et al. [14] answer 4 questions on large-scale agile transformations. On the first question ‘Does any transformation model for large companies exist that-in particular-preserves already existing SPLs?’ The authors state that no model can be used to transform large organizations into agile. The second question ‘What preconditions should be met before starting transformation toward in a large company?’ indicates success factors reported by studies which are the same as [11], [12], and [17] such as management commitment, training, knowledge, and one additional precondition that is risk planning. Question 3 ‘What tasks are recommended to be fulfilled during the agile transformation on development team and management level?’ reports that the distribution of tasks and setting up the infrastructure are key steps that need to be done by development teams and management during the transformation phase. Question 4 ‘What tasks are required on organizational level to finalize the transformation in a large company?’ reports that management should start the transformation with a pilot team so they can get feedback to improve the pilot team’s transformation and also other teams.
A different set of challenges related to scaling agile software development has been reported by Ozkan & Tarhan [15]. The study reports physical dependencies, fragmentation, feudalism, narrow focus on product, construction, and bottlenecks from one to many. This paper is relatively new, but reported some challenges that we could not find any other studies we analyzed.
Shameem et al. [16] reports success factors for large scaled agile projects. The authors report a set of 15 success factors grouped into six categories. The paper also classified these into two major categories, client and vendor. This provides a broader picture of agile processes and factors related to its respective success factors.
Saeeda et al. [17] reports that 24% of the research states that documentation is one of the limitations of agile, 22% reported time period issues and 14% talks about budget overflow. It also reports that 33% of studies report communication as a challenge and 25% report distributed teams as a challenge. The authors also report that researchers are working to find the limitations of agile scalability and its remedial ways.
2.3 Data Extraction
We extracted detailed information from the 7 studies in Table 1 including research goals and questions, findings, discussion, and limitations and reviewed this information for our analysis. Our analysis focused on the success factors and/or challenges in scaled agile transformation presented in each paper, though we note this was not always the primary focus of every study. For example, study [14] presents literature review findings from primary studies a bit differently; this paper identifies preconditions and tasks from the primary studies that exist for large-scale agile transformation to be successful. We mapped preconditions and tasks to success factors to facilitate analysis of these studies. Tables 2 and 3 below shows 23 different challenges and 22 success factors reported by the 7 studies from Table 1.
We manually combined similar ideas with different verbiage into one for the purpose of this study. For example, ‘Change Resistance’ from [12] and ‘Dealing with doubts in people about changes’ [13] have been combined as ‘Resistance to change’.
2.4 Limitations
There are challenges in conducting a tertiary review in a topic as recent and fluid as scaled agile transformation. First, there is a lack of general agreement on what is scale. The term can describe the size of an organization, the size of software projects, the breadth of application and system domains, or the range of organizational roles participating in the transformation. Second, the recency of industry adoption and published research presented a challenge both for identifying relevant literature and for applying a systematic process for analysis. Admittedly, we had to soften our inclusion criteria somewhat to identify even the small number of recent studies due to this limitation. Analyzing the papers from a common perspective was also difficult as often the studies focused on different aspects. For example, [13] focused on stakeholder perspectives, [11] started with scaled agile frameworks (SAFe and LeSS), [14] focused on software product line engineering, and [15] focused on a design perspective. Therefore, our identification of common success and/or challenge factors is bounded by the perspectives of the included secondary studies. Finally, this study was conducted by one Ph.D. student as the primary researcher, and a single secondary researcher. In a situation where the research is sparse and there is an above average reliance on subjective interpretation due to the subject matter, a third researcher may have improved the arbitration process.
3 Analysis and Discussion
In this section we present the answers to the research questions.
-
RQ1: What success factors and challenges in scaled agile transforms have been identified by prior secondary studies?
From Tables 2 and 3 we see that 5 out 7 studies identified challenges while 4 out 7 studies presented success factors in scaled agile transformations. Two studies [11] and [17] reported transformation frameworks and limitations. We could find only one study [14] that suggests organization support is a key factor in scaled agile transformation.
Figure 2 (left) shows the coverage of challenges by different studies as listed in Table 2. In this format, we can see there is not widespread agreement on the challenges, though the few that are agreed upon include coordination, employee mindset, management support, resistance to change and quality assurance.
Figure 2 right shows the coverage of success factors by different studies listed in Table 3. There is somewhat more agreement in factors here as compared to challenges, though again there are still some (such as training, coordination, training and knowledge) that are emphasized in most all, if not all, studies.
-
RQ2: What gaps exist in current studies that should be prioritized by the research community moving forward?
The answer to this question will shed light on the future research avenues. Scaled agile transformation is a relatively new area so we could only identify a few relevant studies. Most of the studies focus on challenges and success factors of scaled agile transformation. Given the scarcity of literature and the answers from RQ1 these are the research gaps we identified that the community needs to address to move forward.
There are some challenges to scale agile projects that have been reported by a significant number of studies (Table 2). However, there are still many challenges that appeared only in one of the seven studies in Table 1 (CH7, CH8, CH10, CH11, CH13–CH22). Success factors are similar; some were reported by multiple studies (Table 3) while others (SF8–SF11, SF13, SF14, SF16–SF22) were reported by only one study.
In our opinion the identification of challenges and success factors by these studies offer guidance to real-world practitioners and identify areas for future research. Further research is needed to identify common perspectives as more software engineering organizations go from “agile-in-the-small” to “agile-in-the-large” transformation. The ultimate goal is to coalesce understanding into a reference framework for practitioners such a machine learning or statistical model. The goal of the framework would be to help practitioners to make decisions during scaling agile. These are the possible avenues that we think need to be explored.
References
Huo, M., Verner, J., Zhu, L., Babar, M.A.: Software quality and agile methods. In: Proceedings of the 28th Annual International Computer Software and Applications Conference, COMPSAC 2004, pp. 520–525. IEEE (2004)
Ambler, S.: Quality in an agile world. Softw. Qual. Prof. 7(4), 34 (2005)
Conboy, K., Coyle, S., Wang, X., Pikkarainen, M.: People over process: key people challenges in agile development (2011)
Da Silva, F.Q., Santos, A.L., Soares, S., França, A.C.C., Monteiro, C.V., Maciel, F.F.: Six years of systematic literature reviews in software engineering: an updated tertiary study. Inf. Softw. Technol. 53(9), 899–913 (2011)
Kitchenham, B., et al.: Systematic literature reviews in software engineering–a tertiary study. Inf. Softw. Technol. 52(8), 792–805 (2016)
Lindvall, M., et al.: Empirical findings in agile methods. In: Wells, D., Williams, L. (eds) XP/Agile Universe 2002. LNCS, vol. 2418, pp. 197–207. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45672-4_19
Fry, C., Greene, S.: Large scale agile transformation in an on-demand world. In: Agile 2007 (AGILE 2007), pp. 136–142. IEEE (2007)
Beavers, P.A.: Managing a large “Agile” software engineering organization. In: Agile 2007 (AGILE 2007), pp. 296–303. IEEE (2007)
Lee, E.C.: Forming to performing: transitioning large-scale project into agile. In: Agile 2008 Conference, pp. 106–111. IEEE (2008)
Alqudah, M., Razali, R.: A review of scaling agile methods in large software development. Int. J. Adv. Sci. Eng. Inf. Technol. 6(6), 828–837 (2016)
Kalenda, M., Hyna, P., Rossi, B.: Scaling agile in large organizations: Practices, challenges, and success factors. J. Softw.: Evol. Process 30(10), e1954 (2018)
Secondary Studies identified for this paper:
Dikert, K., Paasivaara, M., Lassenius, C.: Challenges and success factors for large-scale agile transformations: A systematic literature review. J. Syst. Softw. 119, 87–108 (2016)
Uludag, Ö., Kleehaus, M., Caprano, C., Matthes, F.: Identifying and structuring challenges in large-scale agile development based on a structured literature review. In: 2018 IEEE 22nd International Enterprise Distributed Object Computing Conference (EDOC), pp. 191–197. IEEE (2018)
Klünder, J.A.C., Hohl, P., Prenner, N., Schneider, K.: Transformation towards agile software product line engineering in large companies: a literature review. J. Softw.: Evol. Process 31(5), 1–23 (2019)
Ozkan, N., Tarhan, A.K.: Investigating causes of scalability challenges in agile software development from a design perspective. In: 2019 1st International Informatics and Software Engineering Conference (UBMYK), pp. 1–6. IEEE (2019)
Shameem, M., Kumar, C., Chandra, B., Khan, A.A.: Systematic review of success factors for scaling agile methods in global software development environment: a client-vendor perspective. In: 2017 24th Asia-Pacific Software Engineering Conference Workshops (APSECW), pp. 17–24. IEEE (2017)
Saeeda, H., Khalid, H., Ahmed, M., Sameer, A., Arif, F.: Systematic literature review of agile scalability for large scale projects. Int. J. Adv. Comput. Sci. Appl. 6(9), 63–75 (2015))
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2021 The Author(s)
About this paper
Cite this paper
Das, S., Gary, K. (2021). Agile Transformation at Scale: A Tertiary Study. In: Gregory, P., Kruchten, P. (eds) Agile Processes in Software Engineering and Extreme Programming – Workshops. XP 2021. Lecture Notes in Business Information Processing, vol 426. Springer, Cham. https://doi.org/10.1007/978-3-030-88583-0_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-88583-0_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88582-3
Online ISBN: 978-3-030-88583-0
eBook Packages: Computer ScienceComputer Science (R0)