Keywords

1 Introduction and Related Works

Kanban in software engineering [1] remains a relatively new approach compared to Scrum or XP, despite its origins in Lean principles [2] and inspiration from the Toyota Production System’s efficiency practices from the 1960 s [3]. In 2010, this idea specifically adapted to software engineering was described in a book by J. Anderson [4]. Despite being introduced significantly later than well-established methodologies like Scrum, which gained prominence [5] following the Agile Manifesto, Kanban’s adoption has steadily increased as surveys have shown [6].

With the growth of different agile approaches, the need for measurement of maturity in implementing agile or other agile-oriented practises has emerged [7]. This need has taken the form of various Maturity Models (MM). The initial approaches [7,8,9] were often based on the Capability Maturity Model [10] and its successors. Newer and often more specialised MM have proposed their own classification suitable for their specific domain. During the comparison study over Agile MM by A. Schmitt et al. [8] researchers grouped the models into different categories based on their focus. The first group was the generic Agile MM [11,12,13], which did not focus on a specific Agile practice, methodology, or framework. The second group concentrated on specific agile approaches, such as Scrum MM [14]. The last group focused on engineering practices associated with Agile, such as testing. The Kanban Maturity Model (KMM) [15], described by J. Anderson and T. Bozheva in their 2018 book, can be classified as another specialised MM. The authors specify seven Maturity Levels (ML) from ML zero, “Oblivious”, to ML six, “Build for Survival”. A set of “Consolidation” and “Transition” practices, culture indicators, and expected outcomes are described for each ML.

To the authors’ knowledge, no known research paper presents a methodological case study on the application of the KMM in agile transformation. However, recent case studies of different MMs are available. A case study described by N. Freedrikson Arifin et al. [16] applies the Scrum MM to assess the current Scrum ML of the investigated software development team (SDT) and proposes recommendations for future improvements. Another case study applies both Scrum MM and Agile MM to SDTs in a start-up environment in work by H. Muzakkiy and Y. Sucahyo [17], and to teams in state-owned banks in Indonesia in a case study presented by H. Zelfia et al. [18]. This case study explores the use of the KMM as guidelines for the agile transformation of a SDT within a mid-sized cybersecurity organisation. In this short paper, we aim to lay the ground for future KMM studies and its broader applications reaching beyond single teams. To guide our work, we have defined the following research questions:

RQ1: How can the Kanban Maturity Model be used to facilitate the agile transformation of a software development team?

RQ2: What are the benefits of implementing Kanban practices using the Kanban Maturity Model as guidelines?

RQ3: What are the challenges of applying Kanban Maturity Model guidelines to the agile transformation of a software development team?

2 Method and Setting

This case study follows the guidelines from P. Runeson and M. Höst for reporting case study research in software engineering [19]. The time frame for the case study was set from March 2023 to the end of June 2023. It describes the application of the KMM in the agile transformation of a SDT within a medium-sized company based in Poland. The investigated team consists of six experts in one or several software domains such as front-end, back-end, cloud infrastructure, system architecture and cyber security. The software developed by the team is integrated with other parts of the platform, requiring continuous communication and collaboration with other company departments such as Product Team, Customer Success, Customer Support, Technical Operations and Cyber Security. In the previous year the team has experienced an unsuccessful agile transformation using Scrum. Some practices, such as Daily Scrums (however irregular), work in Sprints, but without Planning or Review meetings have remained. A few possible reasons for the failed Scrum transformation have been identified. The SDT received tasks from multiple different departments as well as further developed their application. Some of this work needed to be delivered outside of the standard Sprint cycle. Additionally, the multiplicity of work sources made finding a suitable Product Owner and managing the Product Backlog very difficult. The last identified factor was the smaller level of support for the change within the SDT at the time. This, coupled with the inexperienced Scrum Master led to an unsuccessful transformation. At the time of the case study, the support for the change was much stronger. The SDT openly stated that they were overwhelmed with work and are open to new solutions. The delivery of application releases was significantly delayed. The quality of the products delivered was below the expectations of both stakeholders within the company and the customers themselves. As a result, a strong willingness to change was present both within the SDT, connected departments and top management. The new transformation was initiated and supported by the team consisting of Chief Information Officer, Head of Engineering, Product Manager and Agile Coach. After analysis of the current situation and results of previous attempts, the Kanban Method was selected as a way to address multiple sources of work and limit work in progress without the need to introduce new roles to the SDT.

The data collection was based on direct data obtained from systems used within the organisation and the team’s observation performed by the researchers during the 3 month Kanban implementation period. To allow for triangulation of the results the data based performance analysis focused on two separate aspects: the team’s productivity and the software quality. The team’s productivity has been measured by its throughput using data from the Jira system used by the team. The software quality was measured by examining the number of bugs reported by the company’s customers through a Help Desk tool, following the release of new software. These results were compared to the incidence of bugs reported during the same period in the previous year. The company releases major updates in a stable yearly cycle, allowing for this comparison.

During the first month of the study, the initial assessment of the team’s maturity was performed using the KMM and the team’s performance prior to agile transformation was measured. Additionally, the performance data of the investigated team was recorded for future analysis. In the next month, the transformation started following the KMM as guidelines for introduced practices within the team. For the first 3 months after the start of the Kanban implementation, direct data on the team’s productivity were collected every month. During the data collection phase, the research team actively engaged with the SDT to assess the effectiveness of the agile transformation by using KMM and adjust the actions according to observed progress and encountered problems.

3 Results and Discussion

The initial team’s assessment performed with the use of the KMM indicated that the team was applying most of the practices specified on ML 0, called “Oblivious”, therefore it was classified as this ML. By applying the KMM as guidelines, consecutive practices were implemented within the team in an order defined by the KMM. As shown in Table 1 the first month of agile transformation, April, was focused on introducing remaining ML 0 practices, transitional 0/1 ML practices (transitional from ML 0 to ML 1), and initial ML 1 practices, “Team Focused”. The only exception was the introduction of higher ML practices in flow management and collaboration improvement to facilitate the transformation. After visualising work-related information through detailed tickets and the use of avatars to visualise individuals’ workloads, the per-person Work in Progress (WIP) limits were established, which aimed to balance the workflow and prevent overloading team members. Establishing basic policies and flow-related metrics, together with introduced Feedback Loops in the form of Daily Kanban Meetings and regular Retrospective and Replenishment Meetings allowed for identification of sources of dissatisfaction and delay, laying the groundwork for future improvement and helping to establish the order of implementation of other practices.

Table 1. The sequence of implementing Kanban practices based on the Kanban Maturity Model within the SDT and their corresponding Maturity Levels (ML).
Full size table

The second month’s goal was to fully reach Kanban ML 1 and to start further transformation towards 2 of the KMM, “Customer-Driven”. During this month, the main focus remained on more advanced visualisation of work, further improvement in flow management and collaboration. The number of introduced practises declined from 23 in April to only 12 in May. The reduction in the number of new practices introduced within the team continued to decline in June to just 6. The focus of the last month of the case study was to build upon feedback and data gathered during the previous two months. During the review conducted with the team members, the newly described problems and proposed solutions aligned with the further implementation of the KMM. After the rapid adoption of multiple Kanban practices, the transformation slowed down and focused on smaller improvements based on built adaptation mechanisms.

The number of observed obstacles remained relatively low despite the rapid transformation of multiple aspects of the observed SDT, sometimes reaching beyond the team. The primary challenge was establishing suitable Work-In-Progress (WIP) limits. Initially, low limits caused frequent task blockages. Raising these limits to a much higher level did not resolve the issue and impacted the efficiency of code reviews, but after weeks of experimentation, the team identified optimal WIP limits, ensuring a smooth flow of work. Another challenge was the need for cultural change described in the KMM. The emphasis on transparency and continuous improvement called for a shift towards a culture of openness and constant evolution within the team. Quickly observed positive results helped with facilitating the cultural change within the SDT. Lastly, such a complex transformation impacted the way the SDT interacted with other product teams and management not yet implementing Kanban. Despite this, they were led through a series of Kanban training sessions to help them understand how to cooperate with the transformed SDT.

To capture team’s productivity, a single metric in the form of average daily throughput showing the number of tasks finished by the entire team each work day has been selected. The team’s average daily throughput, measured before and during the agile transformation is presented in Fig. 1. Data for March 2023 show the average throughput of 1.1 tasks per day for the entire team during the month before the Kanban implementation started. Data for the next three months show a steady increase in team’s average daily throughput. In April, it increased slightly to 1.33 tasks per work day. In May, the result came up to 2.5 tasks per workday, and in the last month of the Kanban implementation, it reached 3.65 tasks per work day. It can be concluded that during the Kanban method implementation, a clear upward trend was observed in the number of tasks completed by the SDT. The results after the first month of agile transformation already indicates a gradual improvement in the team’s work productivity, despite the introduction of multiple new practices and additional burden of training and meetings facilitating the change. The results of the next two months show a significant increase in throughput resulting in more than a tripling of initial productivity when compared to the month before the Kanban implementation has started. This can indicate that the implementation of the KMM practices during the agile transformation in the SDT had a significant impact on increasing the number of delivered tasks.

Fig. 1.
figure 1

Software development team’s throughput before and during the agile transformation between April and July 2023.

Full size image

To analyse the quality of the delivered software, the number of bugs reported by the customers using investigated team’s product was assessed from April to September 2022 and 2023. This data is presented in Fig. 2. Between April and September 2022, customers of the investigated company reported a total of 42 bugs, whereas in the same period in 2023, only 24 bugs were reported. The biggest difference between months can be observed after the new full release at the beginning of July. This is the moment when the main results of work performed during the investigated agile transformation were released to the end users. Additionally, by September 2023, only one patch version of the application was released, indicating an improvement in the quality of the SDT’s work, especially in comparison with the previous version of the application, for which as many as 11 patches were released. This data shows a significant improvement in the quality of the delivered software when compared to the same period in 2022. This can indicate additional benefits of the performed agile transformation in the form of increased quality of delivered software. Additionally, the positive change in the number of reported bugs disproves the concern that the increase in productivity comes at the cost of software quality.

Fig. 2.
figure 2

Number of bugs reported by customers from April to September in 2022 and 2023.

Full size image

4 Conclusions and Future Work

The presented case study investigates the agile transformation of a SDT in a mid-sized organisation. The transformation used the KMM as guidelines to implement Kanban within the team. During the case study the team was observed by researchers, and its performance was measured using productivity and developed software quality metrics. The gathered results were compared with historical data for the investigated team. During the study the following research questions were answered:

RQ1: How can the Kanban Maturity Model be used to facilitate the agile transformation of a software development team? During the presented successful agile transformation, the KMM was used as a source of Kanban practices and a guideline for the sequence of their implementation. It is worth mentioning, that the provided practices should be implemented gradually with adjustments to the team’s needs and circumstances.

RQ2: What are the benefits of implementing Kanban practices using the Kanban Maturity Model as guidelines? As a result of the described Kanban implementation, the productivity of the team tripled and the quality of the developed software improved compared to the previous month and the previous year, respectively. Additionally, implementing the Kanban practices level by level, provides a gradual change, which could result in a smaller negative impact on team’s productivity when the transformation starts. It also can provide a clear roadmap for future transformations, aiding in decision-making.

RQ3: What are the challenges of applying Kanban Maturity Model guidelines to the agile transformation of a software development team? The KMM does not specify the order of implementation for specific practices. It remains a helpful guideline, not a ready step-by-step implementation plan. During the described agile transformation, the exact order of practice implementation was decided based on gathered feedback and observed progress. Instead of immediate implementation of advanced practices e.g. from ML 2 the practices were implemented gradually. Our observations suggest that this transformation still required deep understanding of Kanban practices and adjustments during implementation. In our opinion, the introduction of advanced flow management practices from ML 1/2 and ML 2 in the first month helped with work balancing within the team. A rapid transformation e.g. directly from ML 0 to ML 2 in KMM remains a matter for future research.

Although the guidelines from P. Runeson and M Höst [19] were followed, several threats to validity must be taken into consideration. To minimise the possible influence of human bias in the study, the impact of the described transformation was measured using the number of delivered tasks and reported bugs. The four-eyes principle and working in pairs were applied to data gathering and analysis. Furthermore, during the analysed period, no changes to the granularity of work items and the complexity of bugs or the product were observed. However, the existence of other hidden factors impacting teams throughout and software quality can not be excluded. This case study has several limitations for further generalisation. It focused on a single SDT with only experienced personnel, that remained unchanged throughout the entire data-gathering period. The applications of KMM in agile transformation on a larger scale remain a matter for future research. Together with possible utilisation of KMM in bolstering senior management’s confidence in the proposed agile transformation. Given that the research was done and relates to a single case within an organisation, further research and application of the Kanban Maturity Model in different contexts, including different team structures, different organisations, sizes of organisations, industries, and cultures, is required.