1 Introduction

A notable trend in learning and teaching that has been accentuated by the Covid pandemic is a move towards more online, blended or hybrid learning scenarios. Learning in diverse technology-supported learning environments poses certain challenges to students and teachers. There are for instance higher demands on students’ abilities to plan, manage and reflect on their learning in such environments—abilities that are part of students’ competence to self-regulate their own learning (Azevedo, 2009).

The importance of self-regulated learning (SRL) has long been recognized in research and practice. Theories of SRL, i.e., the ways learners actively set goals, monitor, reflect on and regulate their motivation, cognition, metacognition and emotions in order to learn (Pintrich, 2000) have been developed and refined over a number of years since they emerged from educational psychology approximately 20 years ago (Panadero, 2017). Research has shown positive correlations between the use of SRL-strategies, learning processes and academic outcomes specifically in online and blended learning environments (Broadbent & Poon, 2015). Compared to face-to-face settings, SRL seems to be even more important in online settings because students have to work more often without teacher support and thus need to be able to learn autonomously (Xu et al., 2023a).

However, the relationship between the development of SRL skills, computer-supported learning environments and learning outcomes is complex and multi-directional. On the one hand, developments in online learning, specific digital tools, and functionalities as well as new applications of artificial intelligence (AI) provide more opportunities for promoting the development of SRL skills (Molenaar, 2022). On the other hand, opportunities, and affordances for SRL might also be restricted in such environments, depending on whether the supporting features act as useful scaffolds or result in unnecessary restrictions. In this context, ways in which learners, teachers and digital tools can act synergistically to support SRL have yet to be investigated (Molenaar, 2022).

In recent years, research into SRL and how it may be supported by technologies has proliferated, resulting in many systematic reviews. There are now close to 100 systematic reviews and meta-analyses addressing various aspects of SRL in computer-supported and online learning environments. The focus of these reviews ranges across the identification and clustering of SRL-related behaviours in online-learning, the determination of the effectiveness of specific tools (e.g., dashboards, pedagogical agents) and methodologies (e.g. learning analytics) to the importance of co-regulated learning. However, the methodological and contextual diversity of systematic reviews makes it difficult to derive clear evidence-based research and design recommendations. At the same time, conceptual and technological developments in the field of SRL are continuing apace. There is a need for orientation with respect to the state of the art in research on computer-assisted SRL (CA-SRL) support. We need a better understanding of the challenges of SRL in computer-supported and online learning environments for different learners, and of how to design CA-SRL supports that effectively promote specific SRL processes and phases. Furthermore, to understand how SRL can be promoted, we need to clarify key concepts of SRL and how they relate to specific CA-SRL supports (Järvelä et al., 2023a).

Therefore, we used an umbrella review approach (Aromataris et al., 2015) to integrate the evidence from systematic reviews and meta-analysis to provide an overall assessment of the available information on the support of SRL in computer-assisted, online and blended learning environments. Based on this analysis, this paper identifies challenges regarding the design of CA-SRL supports and future research needs in this area. Specifically, we aim to answer the following research questions:

  • RQ1: What are the scope and the main research themes of previous systematic reviews on CA-SRL supports?

  • RQ2: What theoretical SRL models have been used to study the CA-SRL supports?

  • RQ3: What types of CA-SRL support exist and how do they support different phases and components of SRL?

  • RQ4: How effective are CA-SRL supports in terms of their impact on learning outcomes and how do learner characteristics moderate the effectiveness of the CA-SRL supports?

  • RQ5: What is the role of learning analytics in current CA-SRL supports?

  • RQ6: What recommendations for design and future research do previous systematic reviews draw from their analysis?

2 Methods

The aim of this umbrella review is to provide a synthesis of and critical reflection on the approaches and results of systematic reviews and meta-analyses published between 2008 and 2024 on computer-assisted SRL (CA-SRL) supports. To ensure high methodological quality, we applied the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) guidelines (Moher et al., 2009) and the recommendations for conducting umbrella reviews proposed by Aromataris et al. (2015).

2.1 Search Procedure

Based on our research questions, we defined four general concepts/categories to search for relevant studies: (1) SRL/metacognition, (2) technology context, (3) educational context, and (4) review approach. The search string included key terms and synonyms for each category to cover possible variations. For instance, the first category included the two terms “SRL” and “metacognition” because both concepts are closely related and often used interchangeably (Dinsmore et al., 2008). This resulted in the following Boolean search string: “self-regulat* OR selfregulat* OR self regulat* OR SRL* OR metacogn* OR meta-cogn* “. The full query covering the terms in all four categories can be found in Online Resource 1. We searched four databases (Scopus, Eric, Psychinfo, and ACM). These specific databases were selected to include key journals relevant to our research questions, were suggested by librarian experts, and were frequently used in related previous systematic reviews. Additionally, we conducted a supplemental search in Google Scholar and manually searched the reference lists of the reviews identified to ensure that we included all relevant papers. The supplemental search resulted in one additional article. The database search was conducted on 26th of October 2023. It included only English-language, peer-reviewed journals, and conference proceedings published between 2008 and 2024. The search resulted in 445 articles. After removing 116 duplicates, a total of 329 articles remained (see Fig. 1).

Fig. 1
figure 1

PRISMA flowchart visualising the study selection process

Full size image

2.2 Study Selection

First, the titles and abstracts of the remaining 329 articles were screened in accordance with the Step 1 exclusion criteria (see Fig. 1). This led to the exclusion of 254 papers.

Second, the remaining 76 articles were read in full by two researchers each and assessed for eligibility based on the Step 2 exclusion criteria. Six articles were excluded because they did not contain a systematic collection of studies or did not follow accepted guidelines for systematic reviews. Seven articles were excluded because of the absence of theoretical concepts of SRL/metacognition. In a further step, we analysed whether the articles described and investigated the role or the effectiveness of CA-SRL supports. This resulted in the exclusion of 28 articles. Finally, the methodological quality of the remaining 35 articles was assessed, by applying quality criteria based on the critical appraisal checklist for systematic reviews by Aromataris et al. (2015). This led to an exclusion of four further articles, which did not meet the minimum requirements.

We calculated the inter-rater agreement for all papers in the eligibility-phase by means of Cohen’s kappa (Cohen, 1960). The coding consistency for inclusion or exclusion of articles between the two raters was κ = 0.83, which corresponds to a high level of agreement (Cohen, 1960). Articles that did not result in consensus were rated by a third reviewer and decisions were negotiated.

2.3 Data Analysis

To address our research questions, we performed a quantitative and qualitative content analysis of the 31 selected systematic reviews and meta-analysis. To systematically capture all relevant information, we developed a framework for analysing and coding the information from the articles. The development of the framework was based on the identification and clustering of main research themes of the selected 31 reviews. Each theme defined a main coding category. In an iterative process of reading and coding, different sub-categories were defined. The framework was validated through several rounds of discussions and refinement. The final framework was then used for the analysis of the papers.

We collected the information for each subcategory in a spreadsheet and indicated whether a certain subcategory was systematically investigated as a studied element or only mentioned, for example in the discussion of the paper. This allowed, on the one hand, for a descriptive quantitative analysis, providing an overview of the frequency of occurrence for each theme. On the other hand, specific studies focussing on the investigation of certain themes, were then content analysed by the researchers to gain more insight into the reasoning and conclusions drawn by the authors of the reviews. The analyses were conducted in teams of two for each main theme. Validation of results was ensured through a series of discussions. The results are presented in the next section.

3 Results

3.1 Scope and Research Themes of Previous Reviews

This umbrella review analysed 27 systematic reviews and four meta-analyses from 20 different journals (including three conference proceedings). All 31 papers are listed in the reference section (marked with asterisk and ID) and an overview of their descriptive characteristics can be found in Online Resource 2.

The reviews cover a total of 1206 studies, some of which are included in more than one article. The reviews analysed between five (ter Beek et al., 2018) and 149 studies (Xu et al., 2023a), with an average of 39 studies per review. The period covered by the studies included in the reviews extends from 1994 to 2022. Over 70% of the reviews analysed only studies published after 2004.

Almost all reviews consisted of studies involving students in higher education (HE). Seven reviews limited the selection of studies to HE only. Only three reviews focused exclusively on K-12. Most of the 31 reviews did not focus on a specific subject (n = 24), and those that did have a subject focus were mostly on STEM subjects. There was only one review that examined the effectiveness of CA-SRL supports in a different domain, that of reading. Regarding the research design of the studies, a total of 13 reviews limited their selection to studies with an experimental or quasi-experimental design. Most reviews focused on online learning and/or computer-based learning environments (CBLE). Only two studies focused specifically on blended learning.

In order to answer RQ1, we categorised the research foci of the reviews into seven main themes. As illustrated in Online Resource 2 (see “research themes included”), almost all the reviews investigated the specific types of CA-SRL support used in the studies (n = 29) and the SRL phases and components promoted (n = 20). 23 studies examined the effectiveness of different CA-SRL supports on academic outcomes and/or SRL skills. Ten reviews focused on the affordances and limitations of learning analytics. Moderators of certain effects of CA-SRL supports such as learner characteristics were only investigated in seven reviews. Only a few reviews (n = 5) analysed the theoretical models underlying the research design of the studies.

3.2 Applied Theoretical SRL Models

Due to our inclusion criteria, all the systematic reviews and meta-analyses referred to theoretical models of SRL. However, some articles only provided an overview of SRL theories in their introduction, while the majority of reviews used a specific theory or a combination of theories to frame their analysis. These SRL frameworks typically allowed them to identify and categorise SRL processes into phases (e.g., forethought, performance, reflection) and components (e.g., cognition, metacognition, motivation). As shown in Table 1, the reviews used a variety of different theoretical models and combinations of approaches. Nearly all of the reviews focused only on individual SRL rather than socially shared regulation of learning (SSRL), with the exception of Shao et al. (2023). Depending on the specific theories and their level of granularity for analysing SRL processes, the applied SRL frameworks can be categorised into three main approaches. Eleven reviews used more basic models to distinguish between the three main phases or the three main components of SRL (see Table 1). Five reviews applied the model by Pintrich (2000) and analysed both phases and components of SRL and their interactions. Eight reviews used even more fine-grained models to distinguish different SRL processes, for instance the model by Zimmerman and Martinez-Pons (1986) and the model by Winne and Hadwin (1998).

Table 1 Theoretical SRL models /approaches used for the analysis of studies
Full size table

Despite the significant number of theoretical SRL models available, their guidance for analysing and promoting SRL processes may be limited. For example, according to Araka et al. (2020), some studies claimed that existing theoretical models did not provide a defined framework to guide researchers and designers on how to measure and promote SRL. Nevertheless, reviews found justifiable reasons for using particular models of SRL. For example, Valle et al. (2021) applied Pintrich’s (2000) model because it encompasses critical phases of learning as well as important areas of regulation. Garcia et al. (2018) used Zimmerman and Martinez Pons’s (1986) model because of its comprehensive list of specific strategies. Matcha et al. (2020) argued in favour of Winne and Hadwin’s (1998) model because of its detailed description of cognitive and metacognitive components and its emphasis on external feedback.

Only five reviews systematically analysed the theories and models underlying the studies reviewed (see Online Resource 2). Perez-Alvarez et al. (2022) found that 68% of the studies explicitly mentioned a specific model, with Pintrich’s and Zimmerman’s being the most popular. However, they reported that “current approaches do not use theoretical frameworks for establishing a link between the functionalities supported by the tool and the processes each one supports” (Perez-Alvarez et al., 2022, p. 515). Alonso-Mencía et al. (2020) concluded that most of their reviewed studies did not specify their chosen SRL model.

Interestingly, this lack of theoretical grounding in SRL theory seems to be particularly pronounced in learning analytics (LA) research as highlighted by the analysis of two reviews. Avila et al. (2022) reported that only a small number of papers mention the SLR theory used to deliver and evaluate LA-feedback. Matcha et al. (2020) found that none of their 29 analysed papers on LA-dashboards (LAD) explicitly referred to SRL models. Thus, the theoretical foundation for the design of tools to promote SRL is limited, especially in the field of LAD.

3.3 Types of CA-SRL Supports Investigated

29 reviews examined the CA-SRL supports used in the reviewed studies. Two reviews focused on the measurement of SRL and related this to the promotion of SRL processes (see Online Resource 2). Some reviews focused on one specific type of support, such as prompts (Guo, 2022), LAD (Matcha et al., 2020), recommender systems (Du & Hew, 2022), or concept mapping techniques (Stevenson et al., 2017). However, the majority of the reviews investigated a variety of different types.

SRL supports can be categorised according to their different functions and features. However, many reviews did not use a systematic framework to categorise CA-SRL supports, but rather listed the supports found in the studies reviewed. In addition, certain terms, such as scaffolds or feedback, were used with different meanings, sometimes to refer very broadly to a variety of supports, sometimes focused on a set of well-defined features. The different meanings and levels of granularity in defining and analysing SRL supports make it difficult to make comparisons across studies. Therefore, based on Dignath and Veenman’s (2021) theoretical distinction between direct and indirect promotion of SRL and the analysis of the 31 reviews, we developed a framework to categorise different types of CA-SRL supports. As shown in Table 2, the CA-SRL supports emphasise either a more direct or indirect approach to promoting SRL. Direct promotion of SRL can be explicit (e.g., teaching SRL strategies with videos) or implicit (e.g., encouraging metacognitive processes through prompts). Indirect promotion of SRL can be achieved by providing learners with tools that enable them to perform certain SRL strategies (e.g., note-taking tool) or through the pedagogical-instructional design of a learning environment. The latter refers to basic principles of effective learning, such as giving learners the opportunity to control and adapt their learning paths.

Table 2 Types of CA-SRL supports mentioned or investigated in the reviews
Full size table

Table 2 summarizes the types of CA-SRL support mentioned or investigated in the reviews. We have marked those reviews that analysed these specific types and reported on the evidence, or even have a specific focus on this particular type of support. While direct explicit approaches are mentioned in only eight reviews, direct implicit promotion (e.g., prompts: n = 17) is the most studied form of CA-SRL support. Many reviews also reported on indirect approaches to promote SRL such as providing learners with planning or note-taking tools that might encourage the use of SRL strategies (n = 14) or the presentation of information about one’s own learning progress (e.g., visualised in a dashboard) (n = 15). Furthermore, collaboration tools can support help-seeking behaviour or co-regulation of SRL activities (n = 8). Surprisingly, only very few reviews address the pedagogical features of the CBLEs as a way of fostering SRL. For example, Van Laer and Elen (2017) identify seven attributes for designing blended learning environments that support SRL: authenticity, personalisation, learner control, scaffolding, interaction and cues for reflection and calibration. However, they were unable to determine the relationship between each of these attributes and SRL-behaviour. A recent trend in research has been the investigation of integrated support systems which combine different types of supports. This integration of direct and indirect forms of support has been reported to be particularly effective (Zheng, 2016) (see Sect. 3.4).

The various types of CA-SRL support (see Table 2) can be further sub-categorised according to their specific design features (see Table 3). These features specify how a particular type of CA-SRL support is designed to support SRL within the learning process. For example, a prompt can be either domain-specific or domain-general, it can promote SRL within a specific task (micro-level) or based on an entire learning cycle (macro-level), and it can be presented only once or continuously throughout the learning period. These specific features might impact the effectiveness of a particular type of CA-SRL support. Therefore, a systematic distinction of these features allows for a detailed investigation of their effectiveness and enables concrete design recommendations (Devolder et al., 2012; Wong et al., 2019). Less than a third of the reviews analysed specific design features. Table 3 summarises the features analysed in the reviews. It can also serve as a guide on what to consider when investigating or designing CA-SRL supports.

Table 3 Design features of CA-SRL supports investigated in reviews
Full size table

Furthermore, CA-SRL supports can be used for the promotion of different phases and components of SRL. 20 reviews included the support of particular SRL phases and components in their research focus (see Online Resource 2), although only 16 reviews reported conclusive results on how the different phases and components were facilitated by the CA-SRL supports (see Table 4). Twelve reviews found that the performance phase in Zimmerman’s model and the control phase in Pintrich’s model, respectively, are the most supported SRL phases in research. Accordingly, the preparation and reflection phases received significantly less support. Although most reviews reported that the majority of the reviewed studies investigated CA-SRL supports in different phases, some reviews also concluded that only a small number of studies supported the entire SRL cycle (e.g., Du & Hew, 2022; Heikkinen et al., 2023). Due to the interconnectedness of SRL phases and components, several reviews argued that limiting support to single SRL phases may not be effective in inducing changes in learners’ SRL processes and skills (Edisherashvili et al., 2022; Wong et al., 2019). For example, monitoring processes are unlikely to occur in the performance phase if the definition of learning goals was not supported in the forethought phase (Matcha et al., 2020). The meta-analysis by Xu et al. (2023b) showed that the combination of supports in all phases had the greatest impact on academic outcomes.

Table 4 Most supported SRL phases and components in the reviews
Full size table

Cognition and metacognition were the most supported and analysed SRL components in the reviewed studies (see Table 4), whereas motivation regulation (Devolder et al., 2012; Eggers et al., 2021; Valle et al., 2021; Yang & Kortecamp, 2020) and emotion regulation (Edisherashvili et al., 2022; Hooshyar et al., 2020) are reported to be the least supported components.

Regarding the relationship between particular types of CA-SRL support and specific promoted phases and components, no specific pattern could be identified. The only result found was a higher frequency of the support type "prompt" for promoting metacognition (Devolder et al., 2012; Edisherashvili et al., 2022; Xu et al., 2023b).

3.4 Effectiveness of CA-SRL Supports and the Moderating Role of Learner Characteristics

23 systematic reviews and meta-analysis defined the investigation of the effects of CA-SRL supports on learning outcomes as an explicit research question (see Online Resource 2). One more review reported effects on learning outcomes in their result section and was therefore also included in the summary of the results in Table 5.

Table 5 Effects of CA-SRL supports on learning outcomes
Full size table

Our analysis of learning outcomes revealed different approaches taken by the reviews to synthesise the empirical evidence (see Table 5). 12 of the systematic reviews integrated exemplary results in a more narrative approach. On the one hand, this makes it difficult in some cases to understand and validate the conclusions. On the other hand, such an approach highlights the complex relationships between different factors and learning outcomes. Nine of the systematic reviews analysed the evidence more systematically by rating, counting, and weighing the different results of the studies. One review used both approaches (see Table 5). Some of the meta-analyses also controlled for different types of moderators. The learning outcomes investigated in the reviews can be classified into two categories: academic achievement outcomes and SRL-related outcomes (see Table 5). Academic achievement outcomes are, for example, the impact of CA-SRL support on knowledge acquisition, conceptual understanding, or academic performance in general (e.g., grades). SRL-related outcomes refer to the acquisition and performance of specific SRL processes and skills.

Overall, the reviews mainly reported positive effects of CA-SRL supports on SRL-related outcomes (n = 14) and academic achievement (n = 13) (see Table 5). Six reviews reported ambiguous results, with an inconclusive mixture of positive and negative effects across the studies. This was the case for studies on LADs (Heikkinen et al., 2023; Matcha et al., 2020) and MOOCs (Lee et al., 2019). Some reviews on LADs (Viberg et al., 2020) or recommender systems (Du & Hew, 2022) identified too little evidence to synthesise their findings into conclusive results.

Prompts are one type of CA-SRL support that has been extensively studied (see Table 2) and will therefore serve as an example of how the effectiveness of a CA-SRL support depends on its specific design and on characteristics of the learners. Five of the reviews concluded that prompts are an effective support to promote SRL and other learning outcomes (Devolder, et al., 2012; Edisherashvili et al., 2022; Guo, 2022; Verschaffel et al., 2019; Zheng, 2016). However, these reviews also found a large variance in effects and reported that some studies found prompts not to be effective. Wong et al. (2019) emphasised that several aspects need to be considered to demonstrate the effectiveness of prompts, such as how prompting was implemented (e.g., question, advice, instruction), the intention of the implementation (e.g., to create metacognitive awareness or to conceptually guide learners) and the specificity of the prompt (e.g. general prompts or domain-specific prompts).

Therefore, we analysed the reviews that investigated the moderating role of prompt design features in order to learn what an effective prompt design looks like. The results paint a complex picture. On the one hand, the reviews agreed on the benefits of adaptive prompts and adequate feedback. Adaptive prompts were found to be more effective than fixed prompts (Guo, 2022; Wong et al., 2019). Prompts with (immediate) feedback were more effective than those without (Guo, 2022; Viberg, et al., 2020; Wong et al., 2019). Furthermore, Zheng (2016) concluded that a combination of direct and indirect prompts and scaffolds is most effective, suggesting a combination of different types of prompts. On the other hand, the results regarding the specificity of prompts were inconclusive. Guo (2022) and Devolder et al. (2012) found domain-specific prompts to be more effective than generic ones. In contrast, Zheng’s (2016) meta-analysis revealed that domain-general scaffolds were more effective, but that the combination of both domain-specific and domain-general prompts had the greatest impact on SRL processes and academic achievement. Furthermore, the provision of prompts throughout the entire SRL process appears to be crucial (Wong et al., 2019).

In addition to the importance of design features, four reviews argued that the effectiveness of prompts also depends on learner experience and other learner characteristics (Devolder et al., 2012; Edisherashvili et al., 2022; Guo, 2022; Wong et al., 2019). This may further explain why some studies failed to find the assumed effects of prompts or identified only very specific combinations to be effective (e.g. Devolder et al., 2012). For example, Wong et al. (2019) concluded that the effectiveness of direct forms of SRL support (e.g., pretraining) combined with indirect forms (e.g., prompts) depends on learners’ prior knowledge. While learners with low prior knowledge benefit most from a combination of pretraining and prompts, their peers learn best with prompts alone. In addition, Edisherashvili et al., (2022, p. 15) summarised that "more detailed and more frequent prompts have been found to be efficient with more inexperienced learners, prompts that are more strategic and generic in nature have been proven to work better with more experienced learners, avoiding unnecessary overload and distraction”. These complex interactions highlight the need to consider learner characteristics in order to understand and design effective CA-SRL supports. However, only four of the 31 reviews investigated the role of learner characteristics (Devolder et al., 2012; Edisherashvili et al., 2022; Sun et al., 2022; Wong et al., 2019) and only the latter did so in detail. These reviews addressed three categories of learner characteristics: (1) motivational orientations (e.g., goal orientation, learning preferences, intrinsic motivation), (2) cognitive ability (e.g., prior knowledge, general cognitive ability) and (3) SRL competence (e.g., SRL skills and learning strategies).

3.5 Role of Learning Analytics in Current CA-SRL Supports

Learning analytics (LA) and artificial intelligence (AI) can be used to assess and support SRL. There is a growing body of evidence on their value as well as their challenges in supporting SRL (Molenaar, 2022). Therefore, it is not surprising that 20 of the 31 reviews discuss the role of LA in some way (see Table 6). 10 articles explicitly investigated the effectiveness or role of LA in supporting SRL. Seven reviews of LA also explicitly addressed AI in some way. However, most of these reviews did not clarify nor specifically investigate the role of AI in the applied LA-methods and technologies.

Table 6 Reviews that discuss or investigate LA in current CA-SRL supports
Full size table

LA-Dashboards (LAD) have been found to be the most common LA-tool for supporting SRL, through personalised information about learning behaviour, predictions or recommendations. For example, Matcha et al. (2020) presented different types of dashboards and the indicators they were based on. A less common form of LA support are intelligent pedagogical agents, which are considered to be effective because of their provision of timely feedback and prompts in response to learner behaviour, progress, and self-assessment (Shao et al., 2023).

Among LADs, visualisation of information was the most commonly used LA-method to support SRL (Avila et al., 2022; Heikkinen et al., 2023; Matcha et al., 2020; Viberg et al., 2020). Feedback support and recommendations have been much less implemented in LADs. In the review by Heikkinen et al. (2023), less than 30% of LADs provided recommendations, but almost 60% included visualisations. Four reviews emphasised the potential of combining LA-methods with massive data analysis to calculate, assess and predict learning behaviours and to provide adaptive assistance (Avila et al., 2022; Heikkinen et al., 2023; Perez-Alvarez et al., 2022; Viberg et al., 2020). These four reviews stressed that current applications do not exploit their full potential in this respect. Although LA for SRL is usually based on two elements, calculation and recommendation, Perez-Alvarez et al. (2022) and Viberg et al. (2020) showed that most studies focused only on calculation, which does not automatically lead to improvements in learning and therefore showed little success in supporting SRL-related and academic outcomes. Furthermore, Valle et al. (2021) also reported that LADs were rarely designed to support affective and motivational regulation.

As already mentioned in Sect. 3.4, the results on the effectiveness of LA support were often inconclusive. In Heikkinen’s et al. (2023) review only 43% of the LAD interventions positively impacted learning. In this respect, several reviews argued that LA is not yet designed to facilitate relevant changes in students’ SRL behaviour. Matcha et al. (2020) criticised current LA-methods for primarily capturing data that is readily available (e.g., number of logins or posts) without considering what information is actually required to provide meaningful feedback and support. There was a general claim across the reviews, that SRL theory and pedagogical approaches should better inform the design of LADs to support the full range of SRL processes (e.g., Valle et al., 2021).

3.6 Recommendations for Design and Future Research

Despite the varying contexts of the reviews, the recommendation for design and future research regarding CA-SRL support converge, encapsulating the following key issues.

Research-Based Recommendations for Effective CA-SRL Supports for Diverse Learners: Our analysis of design recommendations for CA-SRL support revealed that four reviews explicitly state that their findings are not strong enough to make design recommendations (Araka et al., 2020; Eggers et al., 2021; Heikkinen et al., 2023; ter Beek et al., 2018). Most reviews agreed that more research is needed to identify effective CA-SRL support designs for specific learners and contexts (e.g., Wong et al., 2019). This applies for instance to the use of effective design features of CA-SRL supports such as the fading of scaffolds (Devolder et al., 2012) or the combination of different types of CA-SRL supports such as prompts and scaffolds (Du & Hew, 2022; Shao et al., 2023; Zheng, 2016). To this end, the use of integrated support systems allows the combination of numerous features including indirect promotion (e.g., visual diagrams) and direct promotion (e.g., SRL modelling) to support individual learners (Wong et al., 2019). The reviews suggested that supports should be designed and evaluated with attention to human factors (Devolder et al., 2012; Guo, 2022; Wong et al., 2019), task characteristics (Devolder et al., 2012) and instructional design (Xu et al., 2023a). Furthermore, research should also address ways to support emotion and motivation regulation (Lee et al., 2019; Yang & Kortecamp, 2020) and socially shared learning (Shao et al., 2023).

Developing Innovative CA-SRL Supports for Promoting the Entire SRL Process: It is not surprising that the most frequently mentioned future research direction was the exploration of new technologies to promote SRL. Many reviews emphasised the further development of LA-tools (Heikkinen et al., 2023; Sun et al., 2022; Viberg et al., 2020), dashboards (Avila et al., 2022; Matcha et al., 2020; Valle et al., 2021), and the effective use of educational data mining methods (Araka et al., 2020; Saint et al., 2022). In addition, developers should pay attention to the inclusion of all phases of SRL in the design of CA-SRL tools (e.g., Du & Hew, 2022; Shao et al., 2023; Zheng, 2016).

Theoretical Grounding for Effectively Researching and Designing CA-SRL Supports: Since it was found that many studies did not use a clear theoretical framework for designing their CA-SRL support, a strong recommendation is to strengthen and justify the theoretical framework for tool design. This is particularly true for research on LA and MOOCs.

Enhancing Theoretical Frameworks and Measurements of SRL: Future research should refine and extend SRL-frameworks and measurement techniques, by adapting them to new contexts. This includes developing comprehensive SRL-models for different learning environments such as MOOCs (Alonso-Mencía et al., 2020; Lee et al., 2019), e-learning contexts (Araka et al., 2020) and blended learning (e.g., Eggers et al., 2021; Van Laer & Elen, 2017). In addition, the reviews suggested exploring challenges in measuring SRL and developing new methods for comprehensive SRL-assessment in digital contexts (Du & Hew, 2022; Eggers et al., 2021).

Investigating (Long-Term) Effects of CA-SRL Supports Across Diverse Learner Populations: Many reviews agreed that future research should aim to better understand SRL strategies and CA-SRL supports across diverse demographics and educational contexts (e.g., HE and K-12, online and blended learning, formal and informal learning environments) and for various domains (e.g., Devolder et al., 2012; Verschaffel et al., 2019; Viberg et al., 2020). The developmental spectrum from childhood through adulthood should also be considered (Xu et al., 2023a) to enhance understanding and support of SRL for all learners (e.g., Wong et al., 2019). Finally, long-term studies are needed because improving SRL takes time (Du & Hew, 2022; Stevenson et al., 2017; Xu et al., 2023b; Yang & Kortecamp, 2020).

4 Discussion

This umbrella review analysed 31 systematic reviews and meta-analyses that investigated CA-SRL supports to provide an overview of previous research with a specific focus on the design of CA-SRL supports and also an outlook for future research.

While most studies aimed at investigating the effectiveness of different CA-SRL supports on learning outcomes in a more or less systematic way, little attention was paid to analysing the theoretical models underlying the research design of the studies and the moderators of certain effects of CA-SRL supports such as learner characteristics (RQ1). The majority of the reviews consisted of studies involving students in HE while K-12 was clearly underrepresented. Because SRL and its supports are highly dependent on the age and educational level of students, the applicability to K-12 remains an important issue for future development and research.

In regard to RQ2, Zimmerman’s and Pintrich’s models were the most frequently used theoretical approaches, both in the analytical frameworks used by the reviews and in the reviewed studies. Zimmerman’s (2000) model depicts SRL as a cyclical process consisting of three phases (forethought, performance, self-reflection), while Pintrich’s (2000) model distinguishes four phases (forethought/planning/activation, monitoring, control, reaction/reflection) and relates them to four areas of regulation (cognition, motivation, behaviour, context). These and other widely used SRL-models are at a relatively high level of abstraction and therefore might not always be suitable to inform the development and research of CA-SRL supports. This limitation calls for the use of more detailed models that clearly identify specific SRL processes and strategies. The few reviews that analysed the theoretical frameworks used in research showed that even though many studies mentioned some SRL-model as the basis of their research, they often failed to provide a clear theoretical rationale for their design of CA-SRL supports. This lack of theoretical grounding appears to be particularly pronounced in LA research. In addition, very few studies used models of co-regulation or socially shared regulation of learning. Thus, studies based on theories that support the community and interactional nature of online and blended learning (Greenhow et al., 2022) are underrepresented.

The analysis of the reviews revealed a broad spectrum of CA-SRL supports designed to enhance various SRL processes (RQ3). However, not all SRL phases and components are supported equally. The preparation and reflection phase and the emotion and motivation components are underrepresented, while the performance phase and cognition and metacognition are most frequently promoted. In addition, only a few studies supported the entire SRL cycle. Due to the interconnectedness of SRL phases and components, limiting support to individual SRL phases or components might be less effective, thus future studies should pay more attention to more comprehensive approaches.

Despite the significant number of various CA-SRL supports, most reviews lacked systematic frameworks for categorising different support types and design features (RQ3). The different meanings and levels of granularity in defining and analysing CA-SRL supports make it difficult to make cross-study comparisons. Based on a theoretical distinction between direct and indirect promotion of SRL (Dignath & Veenman, 2021), we therefore developed a framework for categorising different types of CA-SRL support. The analysis of the reviews revealed a predominance of studies on direct, implicit promotion of SRL such as the use of prompts, and indirect CA-SRL supports, such as note-taking or group awareness tools. In contrast, direct explicit CA-SRL promotion methods (e.g., video instruction on SRL strategies) were less frequently applied. This might be a problem, especially for younger and inexperienced learners who do not know how, when and why to perform a certain SRL strategy yet. In addition, only a few reviews addressed the pedagogical design underlying the learning environment in which the CA-SRL supports were embedded. This relationship should be considered in future research, as these pedagogical designs are an important basis for enabling or hindering SRL processes (e.g., by giving more or less autonomy to learners).

Regarding RQ4, we identified different methodological approaches that vary in the quality of their synthesis of evidence from the individual studies in order to clarify the effectiveness of CA-SRL supports. About half of the reviews only integrated exemplary results in a narrative approach, thus making it difficult to validate their conclusion. However, narrated examples can provide important insights into the complex ways in which CA-SRL supports affect learning behaviours and outcomes.

Most reviews reported clear positive effects of CA-SRL supports on SRL-related outcomes and academic achievement (RQ4). Inconclusive results were often reported in research on LADs and MOOCs. Furthermore, the effects of CA-SRL supports varied across contexts and learners. It is crucial to consider and investigate the specific design features of CA-SRL supports for different learners, which most reviews did not do. Only a systematic distinction of the design features allows for a detailed investigation of their effectiveness for diverse learners and enables concrete design recommendations. Moreover, the interaction between prompt design and learner characteristics, such as prior knowledge and SRL skills, highlights the importance of tailoring CA-SRL supports to individual learner needs, suggesting a more personalised approach to the design and implementation of educational technologies. However, even though adaptive CA-SRL supports have been shown to be crucial, research on the role of learner characteristics is scarce. More research is needed to gain a better understanding of how to design effective adaptive CA-SRL supports for diverse learners and educational settings.

Interestingly, none of the reviews specifically investigated how the presented supports are perceived, understood and used by learners and teachers, and how this use affects learning outcomes. Future research and development should pay more attention to the active role of learners and teachers in utilising and co-designing effective CA-SRL supports. Furthermore, it is also important to monitor the impact of CA-SRL supports not only on learners’ academic outcomes, but also on students’ and teachers’ emotional well-being.

The large number of studies discussing or investigating LA highlights the growing interest in this area. Regarding RQ5, our analysis showed that while LA has the potential to enrich our understanding of student learning and improve SRL support, current tools do not yet fulfil this potential. One problem identified is the theoretical grounding of LA methods—especially with respect to learning theories and pedagogical approaches—which does not yet inform the comprehensive design of LADs to support the full range of SRL processes. Many LA technologies use easily available data and indicators to assess SRL behaviours that may not accurately represent learners SRL skills or SRL profiles. In addition, although LA for SRL is based on two elements, calculation and recommendation, most studies focused only on the calculation part and mainly presented visualisations of learning progress information to the learner. These visualisations could trigger metacognitive processes and regulation, but learners also need feedback and recommendations on how to adjust their behaviour. Future challenges lie in the identification of meaningful indicators of SRL processes from log data, the use of this information for tailored recommendations and the investigation of their interpretation and utilisation by teachers and students.

In regard to RQ6, we found that evidence-based design recommendations for CA-SRL support are still limited and the need for further research is evident. Findings suggest that integrated SRL support systems which can be deployed with different functions at appropriate times throughout all phases of SRL to support students’ individual needs may be of great value. An integration of direct and indirect forms of support seems to be particularly effective. CA-SRL supports should be designed and evaluated with special attention to human factors, task characteristics and instructional design. Furthermore, the findings call for greater precision in defining and categorising CA-SRL supports and their underlying theoretical grounding to enhance the clarity and comparability of research.

Finally, few studies have focused on group work, collaboration and socially shared regulation of learning (SSRL) even though online and blended learning are often greatly enhanced by collaborative learning. A promising theoretical development for integrating AI into learning regulation is the human-AI shared regulation in learning (HASRL) model (Järvelä et al., 2023b) which may inform the design of AI-based support for learning regulation. Identifying different levels of regulation in group-work is another crucial aspect of tool deployment since groupwork typically involves SSRL, CoRL and individual SRL (Ouyang et al., 2023).

5 Conclusion

Supporting learners’ SRL processes and skills is crucial for effective learning, especially in digital learning environments. The synthesis of the reviews highlights the critical importance of adopting comprehensive approaches to designing and implementing CA-SRL supports that integrate a variety of direct and indirect CA-SRL supports across the entire SRL cycle. Although still at an early stage, integrated approaches that focus on developing SRL through a combination of digital tools, AI support and human support may be promising (Molenaar, 2022). There are several issues that researchers and developers should consider when designing and investigating CA-SRL supports, including (1) the application of an integrated SRL framework to ensure that all relevant SRL sub-processes are supported, (2) the consideration of the specific types, functions and design features of CA-SRL supports and their interactions with learner characteristics, and (3) the theoretically sound and responsible implementation of LA and AI for a human-AI shared regulation in learning. Finally, on a more general note, given the growing number and relevance of systematic reviews in this field, their methodological quality should be a focus of attention in future research.