Abstract
Collaborative problem solving (CPS) is widely recognized as a prominent 21st-century skill to be mastered. Until recently, research on CPS has often focused on problem solution by the individual; the interest in investigating how the theorized problem-solving constructs function as broader social units, such as pairs or small groups, is relatively recent. Capturing the complexity of CPS processes in group-level interaction is challenging. Therefore, a method of analysis capturing various layers of CPS was developed that aimed for a deeper understanding of CPS as a small-group enactment. In the study, small groups of teacher education students worked on two variations of open-ended CPS tasks—a technology-enhanced task and a task using physical objects. The method, relying on video data, encompassed triangulation of analysis methods and combined the following: (a) directed content analysis of the actualized CPS in groups, (b) process analysis and visualizations, and (c) qualitative cases. Content analysis did not show a large variation in how CPS was actualized in the groups or tasks for either case, whereas process analysis revealed both group- and task-related differences in accordance with the interchange of CPS elements. The qualitative cases exemplified the interaction diversity in the quality of coordination and students’ equal participation in groups. It was concluded that combining different methods gives access to various layers of CPS; moreover, it can contribute to a deeper articulation of the CPS as a group-level construct, providing divergent ways to understand CPS in this context.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
This paper presents the development of a method capturing various layers of collaborative problem solving (CPS) for a deeper understanding of CPS as a small-group enactment. CPS has received substantial interest as one of the key competencies of 21st-century learners in educational reforms of national curricula worldwide and international, large-scale assessments (i.e., assessment and teaching of 21st Century skills ATC21S, http://www.atc21s.org; the Organisation for Economic Co-operation and Development OECD, Programme for International Student Assessment PISA 2015, http://www.oecd.org/pisa/). Because of the origins of CPS constructs, which stem from assessing individual problem solving, until recently, research on CPS has often focused on problem solving and the skills individual learners bring to the joint problem space. Against this background, an interest in investigating how the theorized CPS constructs function as broader social units, such as small groups, is relatively recent (Dowell et al., 2020; Funke et al., 2018; Graesser et al., 2018; Scoular et al., 2017).
CPS is a complex and dynamic construct, where the “complexity label” is not based on cognitive demands but refers to the definition of the CPS comprising multiple interacting elements (Scoular et al., 2017). Grounded in the socio-cognitive approach to learning, CPS lies in a two-dimensional space of social and cognitive domains that are tightly coupled and intermingle in the problem-solving process (e.g., Avry et al., 2020; Care et al., 2016; Dowell et al., 2020; Hesse et al., 2015; Swiecki et al., 2020). Given the social nature of CPS, the socio-cognitive properties reside and evolve in interaction between the problem solvers (Dowell et al., 2020). Yet, because of the inherent complexity of the construct, determining how to identify the manifested behaviors of CPS and capture the processes at the group level is challenging (Dowell et al., 2020), especially in open-ended problem spaces that lack clarity of the problem solution (Scoular et al., 2017). Despite their usefulness in certain situations, methods that rely primarily on participants’ perceptions of collaboration and CPS via questionnaires, self- or peer evaluations (for an overview, see Kyllonen et al., 2018), or analyzing the contents of communication—irrespective of their semantic and temporal relations—can afford limited understanding around the actual processes of collaboration (Dowell et al., 2020; Swiecki et al., 2020).
The CPS construct and its social dynamics have been simulated in structured, computer-based environments that automatically generate large-scale data samples (e.g., Dowell et al., 2020; Graesser et al., 2018; Swiecki et al., 2020). These data allow for analysis based on computational linguistics frameworks (i.e., group communication analysis, GCA; Dowell et al., 2018, 2020) that quantify, for example, social dynamics in groups involved in CPS. However, together with simulated CPS activities and automated analysis at a large scale, searching for ecologically valid measures based on detailed capturing of the micro-interaction processes at the small-group level (e.g., Davis et al., 2015)—in authentic learning contexts that large-scale studies can also account for (Reimann, 2021)—is similarly desired (Gauvain, 2018). Consequently, if CPS processes are seen to consist of intertwined and interdependent interactions among the group that evolve over time, adequate methods are needed that allow for detailed capturing and analysis of these interaction-related events (Reimann, 2021) and the complex mechanisms of “coupling minds” (Cress et al., 2018) involved in CPS in this context.
In this study, to better understand CPS as a group-level enactment, a method is developed for capturing various layers of CPS by relying on triangulating analysis techniques (Humble, 2009; Meadows & Morse, 2001). In triangulation, at least two analysis approaches are integrated to form a deeper picture of the object of investigation (Flick, 2004; Kelle & Erzberger, 2004). In this study, triangulation combines the following: (a) directed content analysis on the actualized CPS in groups (e.g., Hsieh & Shannon, 2005; Humble, 2009), (b) process analysis and visualizations (e.g., Abbott, 1990; Kapur, 2011; Reimann, 2009, 2021), and (c) qualitative cases (e.g., Davis et al., 2015; Stahl, 2017). All the analyses are based on the communication stream as video recordings from CPS sessions in small groups (Barron et al., 2015).
In the study, small groups of teacher education students solve open-ended CPS tasks during an experiential mathematics education course in geometry. The tasks include a technology-enhanced task using dynamic software, and a task using physical objects. In a co-located CPS, participants work together in the same physical location and coordinate their understanding and actions by exchanging ideas, knowledge, or resources to collaboratively converge on the correct understanding of the shared problem (Alterman & Harsch, 2017). However, groups often fail to make use of their full potential (Barron, 2000, 2003). Therefore, to better ensure that productive CPS processes evolve, the pedagogical design in this study incorporates the task characteristics (as sufficient complexity of the task) to enhance the process gains of collaboration (Sears & Reagin, 2013) and includes external support as a macro-script, an activity model to engage learners in group interaction processes (e.g., Dillenbourg & Hong, 2008). However, because this paper emphasizes the methodological aspects of capturing various layers of CPS as a small-group enactment, the scope does not cover how the pedagogical design affects the quality of CPS. Next, the two focal points of this study, the CPS construct and the method, are described in more detail.
Collaborative problem solving: a construct definition
In CPS, collaboration and problem solving intermingle, requiring both the social and cognitive contributions of the participants (Care et al., 2016; Funke et al., 2018; Hesse et al., 2015; Scoular & Care, 2020; Scoular et al., 2017). Basically, the social components of CPS are related to how participants coordinate and communicate with one another (e.g., Richardson et al., 2007), as well as how they regulate and resolve differences among the collaborating partners (e.g., Järvelä et al., 2016). The cognitive elements, in turn, are related to how effectively and efficiently participants solve the problem (e.g., Mayer, 1998). To be successful, CPS requires engagement in shared processes and exchanges of ideas, knowledge, or resources, as well as externalizing understandings and efforts in the group.
In observed group interaction, CPS is actualized using verbal and nonverbal indications and constant updates that are transparent to most group members (Graesser et al., 2018). This means that for the group to be aware of most of the substantial problem parts and to work successfully and reach a mutual understanding, the participants are obliged to communicate, exchange, and share knowledge over problem-solving processes at the social and cognitive levels (Graesser et al., 2018; Hesse et al., 2015). This requires a willingness to collaborate and share expertise, as well as the ability to manage interpersonal conflicts (Hesse et al., 2015). To be productive, CPS not only makes use of the full potential of the group’s expertise but also necessitates the full set of social skills coming into force.
In this study, for detailed articulation of the CPS construct in small groups of learners, a comprehensive CPS framework by Hesse et al., (2015; see also Care et al., 2016; Scoular & Care, 2020; Scoular et al., 2017), originally developed for the assessment of CPS skills in the ATC21S project, is applied (Care et al., 2016; Griffin & Care, 2015). The framework is chosen because it profoundly covers both social and cognitive elements of the CPS construct (cognitive, social, and regulatory aspects), and it amalgamates existing theoretical knowledge from social psychology and problem solving (Hesse et al., 2015). Expanding from the original usage, the aim is to explore how the CPS elements intermingle and interact in the group-level processes and over time. The framework gives a detailed description of how the social and cognitive elements of the CPS construct can come into force in a collaborative context (see Appendix 1), and therefore, it is suitable for the purposes of directed content analysis (Hsieh & Shannon, 2005) as employed in this study.
In the framework, CPS is defined as a set of elements consisting of “social” and “cognitive” as the first-order components (see Table 1). As the second-order components, the framework covers three social process elements (participation, perspective taking, social regulation) and two cognitive process elements (task regulation, knowledge building). The five elements are further divided into 19 specific sub-elements as third-order components.
From the social set, “participation” (sub-elements: action by individual and interaction with the partner or partners) refers to the willingness and eagerness to share information, externalize one’s thoughts, and be involved in the different stages of problem solving. “Perspective taking” (Hayashi, 2018; sub-elements: adaptive responsiveness, audience awareness) means the ability to understand and consider others’ perspectives and contributions and to tailor one’s contributions to others. “Social regulation” points to the knowledge and awareness of the strengths and weaknesses of the other group members (sub-elements: transactive memory, Lewis & Herndon, 2011; negotiation, Thompson et al., 2010; self-evaluation [metamemory], Wegner, 1986; and responsibility initiative, which signifies taking responsibility for the progress of [the parts of] the task by the group).
From the cognitive set, “task regulation” refers to planning and monitoring skills for developing strategies for problem solving and shared problem representation (“joint problem space”; Roschelle & Teasley, 1995). The sub-elements of task regulation incorporate problem analysis, setting goals, resource management, tolerance for ambiguity, collection of information, and systematicity as four aspects of planning, one of the fundamental activities in CPS (Eichmann et al., 2019). Moreover, from the cognitive set, “knowledge building” refers to the ability to learn and build knowledge through group interaction (Scardamalia & Bereiter, 2006). In CPS, steps in knowledge building include the sub-elements of identification of relationships, cause and effect, and testing of hypotheses (Griffin, 2014). Taken together, these five CPS elements and their 19 sub-elements are considered essential for successful CPS activity. In the framework, CPS may not proceed linearly; rather, the CPS elements can overlap and run parallel between the different stages of the process.
Applying analysis method triangulation for capturing various layers of CPS as a group-level enactment
The concept of triangulation (e.g., Flick, 2002, 2004; Kelle & Erzberger, 2004) has multiple meanings in the literature, but it is generally viewed as a cumulative way of validating the results or as “an enlargement of perspectives that permit a fuller treatment, description and explanation of the subject area” (Kelle & Erzberger, 2004, p. 174). Here, the emphasis is on the latter purpose—to increase the scope, depth, and consistency of research (Flick, 2002). The same problem can be approached using various methods; alternatively, triangulation can be employed to treat different aspects of the same phenomenon (Kelle & Erzberger, 2004). In this study, three different methods of analysis (directed content analysis, process analysis and visualizations, and case examples) are used to capture various layers of CPS as a group-level enactment. All the methods are valued equally and can add to a unified depiction in understanding CPS in this context. Next, the set of methods chosen is discussed and presented with examples of related analysis approaches.
Using a directed approach (e.g., Hsieh & Shannon, 2005; Humble, 2009), content analysis is a deductive category application. It is guided by a structured process where an existing theory or conceptual frame is applied to a novel context, and the categories and the determined operational definitions of each category are based on the theory or framework (Hsieh & Shannon, 2005). In the current work, directed content analysis applies the ATC21S CPS framework and the codebook by Hesse et al. (2015) in analyzing interactional data. The evidence from the coding can be presented, for example, by offering descriptive evidence, or in this study, first calculating the frequency distributions of the categories that enter into the broader framework (Vogel & Weinberger, 2018).
A related approach, quantitative content analysis (e.g., Neuendorf, 2011), also referred to as the “coding and counting approach” in analyzing interaction, is a summarizing, quantitative analysis (Jeong et al., 2014); in contrast to directed content analysis, it applies inductive coding (Kennedy, 2018). In quantitative content analysis, a coding scheme is developed and applied in the data corpus (De Wever et al., 2006). When studying collaborative learning and CPS, the analysis has typically resulted in frequency counts of different categories as a means of comparing different experimental conditions (i.e., concerning the processes or outcomes; Csanadi et al., 2018; Swiecki et al., 2020).
“Coding and counting” analyses are seen as useful in explaining the distribution of process categories, that is, regarding the variations of the outcomes (Kapur, 2011; Reimann, 2009). However, when frequency-based methods are applied in studies of collaborative learning and CPS, it has been found, for example, that groups with similar frequency distribution of categories may well have different temporal dynamics of these categories (Kapur, 2011). As Hesse et al. (2015) stated, CPS processes unfold over time and can vary over the course of problem solving. Therefore, to avoid treating CPS as a set of isolated events, an issue that cumulative accounts or frequencies are often criticized for (e.g., Csanadi et al., 2018; Kapur, 2011; Reimann, 2009; Swiecki et al., 2020), the temporal dimension of the data is also acknowledged. As Kapur (2011) pointed out, “learning in general, and problem solving in particular, is a continuous, dynamic process that evolves over time” (p. 39). In the current work, studying temporality in collaborative learning and CPS processes is inspired by the previous work of several authors (e.g., Avry et al., 2020; Csanadi et al., 2018; Kapur, 2011; Reimann, 2009, 2021; Reimann et al., 2011; Stahl, 2017; Swiecki et al., 2020).
Various methods are applied for analyzing processes and temporality in collaborative learning and CPS studies (for an overview, see Lämsä et al., 2021a; Reimann, 2009, 2021). For example, in epistemic network analysis (ENA), which is a discourse analysis technique that models the temporal co-occurrences of codes in discourse, temporality is studied by focusing on the recent temporal context and immediate events of CPS in the groups (e.g., Andrist et al., 2018; Csanadi et al., 2018; Swiecki et al., 2020; Williamson Shaeffer, 2017).
Lag-sequential analysis (LsA; e.g., Bakeman & Gottman, 1997; Bakeman & Quera, 2011) has been employed for modeling CPS processes using transitional probabilities between sequencies of collaborative action; in this way, researchers have demonstrated, for example, the significant relationship between variation in temporal patterns and variation in group performance (Kapur, 2011). A related sequential analysis approach, sequential pattern mining (e.g., Febrer-Hernández & Hernández-Palancar, 2012), has been applied, for example, for modeling students’ ill-defined problem solving (Norm Lien et al., 2020). Both methods model sequences, but if compared to LsA, sequential pattern mining has been used to mine sequences of events that are frequent (i.e., occur more often than a minimal level of frequency) in a dataset (e.g., Chen et al., 2017).
Temporality not only comes into play in quantitative terms (i.e., duration) but also has significance in terms of the order of events (Reimann, 2009, 2021). In the current study, the indicators of temporal events were the order and intensity of the social and cognitive states and state changes during task completion. The social and cognitive states refer to the coded, timestamped (social and cognitive) sub-elements of CPS (see Appendix 2), which are amalgamated back into broader social and cognitive strands (i.e., the first-order components of the CPS construct). The intensity of the states is related to how many of the different sub-elements of the social or cognitive strands appear at the same point in the CPS session. The state changes are based on calculations of inter-event distances as temporal distances between social states and cognitive states over the CPS session, whereas the calculated averages of states and state changes cover the entire CPS session (e.g., Abbott, 1990; see also Reimann, 2009); all are identified from the timestamped coded interaction of the groups. Accordingly, in this study, the indicators—the order and intensity of the social and cognitive states and the (averages of) states and state changes—are expected to pinpoint the interchange of joint efforts and related communication with the partners, which form the basis of collaboration and CPS (e.g., Dillenbourg et al., 2016; Roschelle & Teasley, 1995).
Finally, via qualitative cases, attention is turned back to the micro-interaction processes in the groups (Davis et al., 2015; Stahl, 2017). All groups are unique in terms of how the actions and interactions come together to form the shared practice (Stahl, 2017). In addition, case examples can provide intensification and offer explanations to help in the interpretation of the previous phases of analysis (Flick et al., 2004). Here, qualitative cases can show whether the differences in the temporal occurrence of CPS from the previous phase of analysis are visible at the micro-interaction level. In general, when carefully justified, cases can provide a thorough view of a complex social phenomenon (Baškarada, 2014). Cases do not aim to generalize to populations, but they can arrive, for example, at conceptualizing the regularities of small-group processes in CPS (Stahl, 2017).
Aims
This study applies analysis method triangulation to capture various layers of CPS enactment in small, co-located groups during a mathematics education course in teacher education. The research questions are as follows:
RQ1: How are the theorized CPS construct (comprising social and cognitive elements and their sub-elements) actualized in the groups’ interactions in two different task designs (i.e., technology-enhanced vs. using physical objects), interpreted as cumulative accounts?
RQ2: How do the elements of the CPS construct vary over time during problem solving in the different groups and different tasks? What differences are there in temporal occurrence in this regard?
RQ3: In what ways are the differences in the temporal occurrence of CPS elements visible in the qualities of interaction in the groups (exemplified as case examples)?
Methods
Participants and context of study
The participants were teacher education students (N = 15, 11 women, 4 men, mean age 27 years) from a Finnish university. At the outset, the participants were divided into four comparable groups of three to four students. Participation in the study was voluntary. As one student declined to participate in the research, only three groups were included in the analysis.
Course design and tasks
The course under study was an applied mathematics education course in experiential geometry. The course included seven meetings (90 min each) and working in groups outside the meeting hours. The meetings included the following sessions:
-
Session 1: Introductory session;
-
Sessions 2 and 3: CPS task demos;
-
Session 4: reflection session to discuss and reflect on the experiences of the previous CPS task demos;
-
Session 5: developing and sharing ideas for forthcoming workshops in schools;
-
Session 6: working in small groups to design, implement, and report on a CPS teaching experiment within a school context; and
-
Session 7: presenting and discussing the workshops with co-students.
The work in small groups resulted in jointly prepared reports that were assessed by the teachers using a scale of 1–5. In this paper, the focus is on the two CPS task demos (Sessions 2 and 3).
During the demos, the groups experienced CPS in terms of two different CPS task designs—a technology-enhanced task relying on virtual manipulatives using the dynamic GeoGebra software and a task using physical objects (giant tiles made from cardboard). The GeoGebra software was employed to support students’ learning by providing more opportunities to explore, analyze, and test their knowledge (Santos‐Trigo & Espinosa-Perez, 2002). Physical objects, or manipulatives, are physical materials (e.g., blocks and tiles) that are used to support learning and have tactile and manipulative properties (Manches et al., 2010).
The tasks (see Table 2) were open ended (Chan & Clarke, 2017), embracing experiential mathematics learning with math–art connections. In open-ended problem solving, the starting situation is not clearly defined, there are multiple correct answers to the problem, or there are many approaches to solving the problem. The difficulty level of the tasks was defined by the teachers in accordance with the preceding math course level.
The tasks were designed under the topic of tessellations. A tessellation (or tiling) means an arrangement of figures (i.e., tiles) such that they cover the plane without any gaps or overlaps of the tiles. The tile used in the course was Paul Gailiunas’s tile design, “Princess” (Fig. 1). This was selected because a wide range of different symmetrical patterns can be created based on this tile (Gailiunas, 2007).
For the CPS task demos, two settings were applied (Table 2). During the first CPS session, students were advised to use the software program (GeoGebra), and physical objects (giant tiles) were provided as a complementary tool. During the second CPS session, to create giant tessellations, students were advised to use the cardboard tiles as the primary tool and GeoGebra as the complementary tool.
The macro-script
As part of the pedagogical design, a macro-script was applied (Dillenbourg & Hong, 2008; Dillenbourg & Tchounikine, 2007). The script, as a coarse-grained pedagogical model, was aligned with the principles of the socio-cognitive learning approach defined by Zimmerman (2010); this approach emphasizes learning as a complex metacognitive and social process. The script (Näykki et al., 2015, 2017) was orchestrated by the teachers and implemented as physical cards that included question prompts to be answered as a group. In short, the design rationale included three different phases: (a) the forethought phase, where groups set their goals for their learning (“Orientation” card); (b) the monitoring phase (“Checkup”); and (c) reflection (“Reflection” card; see Fig. 2 for the phases and question prompts used). At the onset of the session, teachers gave instructions on how to use the cards and noted the approximate timepoints at which students could focus on them. During the sessions, the groups were gently prompted by the teachers if it seemed they did not follow the given guidelines.
Data collection
The data collection included observations, taking notes, and video recordings of group interactions as the primary source of data. Small GoPro® cameras, equipped with suction cups that enabled them to be flexibly positioned in the classroom, were used for video recordings. In addition, when utilizing GeoGebra, CamStudio™ software (http://camstudio.org) was used for the screen activity recordings during the sessions. The full dataset comprised video recordings of the sessions (26 h), screen capture videos (3 h), and students’ written reports on their teaching experiments. In this paper, the emphasis is on video data.
Data analysis
The main analysis process combined directed content analysis (Phase 1), followed by process analysis and visualizations (Phase 2); this set the stage for the qualitative cases (Phase 3; for an overview of the analysis process, see Fig. 3). As a preliminary phase, based on observations during the data collection, a time-indexed narrative was produced that allowed us to go through the data corpus rapidly (e.g., Barron et al., 2015).
Phase 1: directed content analysis of the interaction data
In the first phase, to identify the interacting CPS elements at the group level, directed content analysis was applied (e.g., Hsieh & Shannon, 2005; Humble, 2009). The categorization matrix and the operational definitions of each category were modified from the CPS codebook by Hesse et al., (2015; see also Care et al., 2016; Scoular et al., 2017). Since the framework was designed for the targets of automated analysis of CPS skills, it included a large number of code variables. For manual coding, the number was too large. The condensed frame and the selection of codes were based on previous research and the current understanding of the key elements related to the socio-cognitive approach to CPS (Appendix 1).
In this phase, the interest was in how the groups enacted the CPS tasks in the two task designs. That is, the aim of this phase was to uncover how different or similar the groups (Groups B, C, D) were regarding the appearance of the selected CPS elements in the different tasks (technology-enhanced vs. using physical objects) by relying on simple descriptive counts in the coded data.
In brief, the categories applied for analyzing the social elements set (Appendix 1) included “Participation” (originally the “interaction” sub-element) as the willingness to share information, externalize one’s thoughts, and participate in the different stages of problem solving. “Perspective Taking” means, on the one hand, the ability to understand and consider others’ perspectives and contributions (originally named “responding skills,” here labeled “Perspective Taking A”), and on the other, the ability to tailor one’s contributions to others (here called “Perspective Taking B”).
From the cognitive elements set (Appendix 1), “task regulation” (based on the “resource management” sub-element, here called “Coordination”) is included. Sub-element “collects elements of information” is here called “Task Exploration,” and “rules: ‘if…then’” is called “Problem Analysis.” The sub-element “reflects and monitors” is here renamed “Problem Reanalysis.”
The video recordings of the two problem-solving sessions (26 h of video data for all three groups) were analyzed with the assistance of Atlas.ti® data analysis software (Friese, 2014). The first author trained a research assistant for the coding with several data examples until an agreement was reached, and during the process of analysis, they jointly discussed the interpretations and unclear cases. For qualitative rigor (Thomas & Magilvy, 2011), the deductive coding procedure was carried out three times. The initial coding was conducted on the video recordings. Next, to reduce coding errors and to look for similarities within and across groups, the video recordings were transcribed, and the data were re-coded in parallel with transcribing the videos; the codes and the timestamped segments in the video recordings were elaborated and rechecked during this process. Finally, a third round of elaboration of the coding was performed on the transcribed, written data. In the analysis, when reading the transcriptions line-by-line, an episode or passage with natural borders was captured, delimited by a thematic shift. Thus, a minimum unit, where a certain criterion of the predetermined category was observed, was adjusted for analysis (Chenail, 2012). The analysis did not follow a time-based segmentation of events (Sinha et al., 2015), but the length of the coded segments was based on their contents. Therefore, the segments all had unique lengths. Those parts of the video where any criteria segments could not be found were left uncoded. (For representative examples of the coded data and data captures, see Appendix 2).
The coded data were exported from the Atlas.ti® data analysis software to Microsoft Excel for organizing, analyzing, and visualizing the data. Based on the categorized data, descriptive statistics were used to summarize the appearance of CPS elements in different groups and task designs. First, the proportional distribution of the coded CPS elements was calculated in terms of the following: (a) different types of tasks (technology-enhanced vs. utilizing physical objects) and (b) different groups (Groups B, C, and D). In addition, the standard deviation (SD), average, and median were estimated for the CPS elements in the different groups and task designs. These cumulative accounts did not point to individuals but were conceived as group-level constructs (Kapur et al., 2008).
Phase 2: focusing on the temporality in the data
In the second phase, the focus was on the temporal occurrence of CPS elements as inter-event temporal distances between social and between cognitive states over the two task sessions (e.g., Abbott, 1990; see also Reimann, 2009). Accordingly, the order and intensity of the appearance of the social and cognitive states over the course of the two problem-solving tasks was first calculated. Second, to compare the groups, and especially to see whether the general characteristics of the temporal nature of CPS activity in the groups were different in the two tasks, definite ratios were calculated as follows: (a) the averages of the social and cognitive states, (b) the averages of state changes, and (c) the proportional differences between the averages of the state changes at the group level in the two task designs.
The social and cognitive states were created by amalgamating the coded CPS social sub-elements and cognitive sub-elements into broader social and cognitive strands. When calculating the intensity of the social state, the state level takes a value of 1 if all three sub-elements of the social strand (Participation, Perspective Taking A, Perspective Taking B) appear at the same point. (For the values for calculating social states, see Table 3). For the cognitive state, the state level takes a value of 1 if all four of the sub-elements of the cognitive strand (Coordination, Task Exploration, Problem Analysis, Problem Reanalysis) appear at the same point. (For the values for calculating cognitive state, see Table 4). In the visualizations, the intensity values for the social or cognitive state give a quick overview of the possible (co)-occurrence of the CPS sub-elements over the CPS process.
For calculating the averages of the social and cognitive states, the sum of values calculated for social (or cognitive) states (see Tables 3 and 4) were divided by the number of these values. The average state change simply describes the average amount of change (i.e., “travel”) from one social (or cognitive) state to another. (For the values for calculating the distances between social states and the distances between cognitive states, see Tables 5 and 6).
An example of the coded interaction and calculations of the social and cognitive states, as well as the social and cognitive state changes (labeled “travel”), are given in Fig. 4. The example is from the onset of the phase when the script card is discussed (“Orientation”).
This phase also incorporates visualizing the temporal interchange of social and cognitive states in the groups and the tasks. This visualized interchange, showing the alignment of CPS actions over time, is purported to provide an overview of the actualized CPS activity during the tasks. The visualizations, developed for this study and called “CPS circles,” display (clockwise) the state changes and their intensity during the sessions. It also shows the period when the question prompts of the script cards were discussed (see Fig. 5 for an example of a CPS circle).
The dark gray color points to the social states of CPS skills, whereas the light gray color points to the cognitive states. The three areas marked with the transparent, lightest gray color represent the timepoints at which the script cards were discussed. The intensity of the appearance, reflecting the number of social or cognitive sub-elements activated at a specific point, is visualized by the height of the “ray.”
Phase 3: qualitative cases based on selected interactions
Based on the results of the previous phase (i.e., the proportional differences between the averages of state changes in the two tasks), in Phase 3, the focus returned to the interaction data to better understand the group-level differences in this regard. In the analysis, the aim was to concentrate on identifying the characteristics of interaction when the groups were completing the second task and searching for chains of events that would exemplify the differences.
Results
RQ1: How did the theorized collaborative problem-solving elements actualize in the groups’ interactions?
In general, the cumulative accounts did not indicate a large variation between the different groups in either task design. When focusing on the sub-elements of the CPS, Problem Reanalysis from the cognitive set formed the most frequent sub-element in both the technology-enhanced task (see Table 7) and the task utilizing physical objects (see Table 8). In Task 1, the Participation sub-element from the social set was the second largest sub-element, whereas Task Exploration from the cognitive set was the third largest sub-element, except in Group D, where Task Exploration formed the second largest element from all the sub-elements, followed by Participation as the third largest sub-element. In Task 2, the largest element, Problem Reanalysis, was followed by Participation as the second largest and Coordination from the cognitive set as the third largest sub-element. In both task designs, it seems that cognitive elements were more identifiable from the interaction data but with slightly different task-related emphases: Task 1 involved more task exploration than coordination, while Task 2 involved more coordination than task exploration.
From the descriptive statistics, when focusing on the dispersion of the CPS sub-elements, the low SD implies a more or less equal appearance of CPS elements in the different groups and task designs, as indicated in Tables 9 and 10. The measures of central tendency (average, median) indicate the same result.
RQ2: How did the collaborative problem-solving elements vary over time?
Because the results from the first phase of analysis did not show a large variation in different groups or tasks, the temporal information of these coded data was elaborated further. Here, it was asked how the elements of CPS varied over time by focusing on the order and intensity of social and cognitive states (Figs. 6, 7, 8, 9, 10, 11) and comparing the (average) alignment of the social and cognitive states and state changes in the groups regarding the two different tasks (Table 11).
The results suggest that in Group B, the average occurrence of social states is slightly higher in Task 2 than it is in Task 1 (Task 1: 0.115, Task 2: 0.122), but the value for state changes regarding social elements is remarkably lower in Task 2 compared with Task 1 (see Table 11). (The calculated proportional difference between the tasks is 35.2% less). It seems that at the group performance level, more social states appeared in the task with the physical objects than with GeoGebra, but the continued “social moments” in the group interaction, on average, were longer. Figures 6 and 7 visualize the actualized interchange of the social and cognitive states in Group B.
For Group C (see Table 11), there was an increase in the average occurrence of social states in Task 2 compared with Task 1 (Task 1: 0.135, Task 2: 0.164). In addition, the value for the average of social state changes was higher in Task 2 (using physical objects) compared with technology-enhanced Task 1. (For Task 2, the calculated proportional difference is 10.7% more in this regard). This indicates that although more social states occurred on average, the social moments were slightly shorter in Task 2 compared with Task 1. Figures 8 and 9 visualize the actualized interchange of the social and cognitive states in Group C.
In Group D (Table 11), there was a decrease in the averages of the cognitive states (Task 1: 0.246, Task 2: 0.208), as well as a decrease in the average of the cognitive state changes in Task 2. The calculated proportional difference between Tasks 1 and 2 was 24.6% less in this regard. This means that, in Task 2, fewer “cognitive moments” occurred, but on average, these moments lasted longer than they did in Task 1. Figures 10 and 11 visualize the actualized interchange of the social and cognitive states in Group D.
RQ3: In what ways were the differences in the temporal occurrence of CPS elements visible in the qualities of interaction in the groups?
In the third phase, to search for explanations for the results from the second phase of analysis, the focus returned to exploring the interaction processes in the groups. With the case examples, as “a sensible follow-up” to the previous levels of analysis (Flick et al., 2004), the interest was in seeking clarifications for the proportional, group-related differences between the averages of the state changes regarding the two task designs.
Accordingly, different qualities of interaction were found, varying from the lowest quality as chains of individual members’ autonomous actions (Group B, Vignette 1) to strings of synchronized and coordinated actions (Group D, Vignette 2), and finally, contemplative, reflective interactions as a group (Group C, Vignette 3), which could be labeled as the highest quality in terms of interaction in the groups. These different characteristics of the interaction are depicted in Vignettes 1–3.
Vignette 1. Group B: chains of individual members’ autonomous actions
As Table 11 shows, in Group B, the second task resulted in a decrease of the average of state changes of social aspects (− 35.2%). The microlevel perspective on interaction data provides a view of a situation comprising chains of localized moments of independence (Davis et al., 2015). Students seemed to have shared intentions (a conceptual goal to solve the problem), but at many moments, they went off individually or acted as varied subgroups to solve the problem (Stahl et al., 2014). While the subgroups worked in parallel, the members communicated their individual efforts. Whereas two of the group members (Lotta and Samuel) focused on task completion individually, Mia did not develop an individual solution, but instead, moved between the other two members, her contributions remaining at a rather superficial level. She mainly asked questions and commented on the work of Lotta and Samuel, without participating in developing their ideas further. Finally, as Samuel failed with his tessellation, Lotta’s work was taken as the final product of their group. In general, the group unsystematically manipulated the tiles to build giant tessellations. The conceptual goals were not communicated, and the mechanical coordination of the group did not seem intentional (Davis et al., 2015).
Vignette 1
Example from a chain of individual members’ autonomous activities in group interaction
Actor | Quote |
---|---|
1. Student 1 (Mia) (asking Lotta): | “Anything taking shape?” |
2. Student 2 (Lotta): | “Nothing significant.” |
3. Student 1 (Mia) (asking Samuel): | “Do you have something, then somehow everything is, like, positioned differently?” |
4. Student 3 (Samuel): | “I guess so. I don’t know if you can just, like, keep going on with this sort of thing somehow. (Now there’s something starting to –) from there I set out.” |
5. Student 1 (Mia) (to Samuel): | “So, as I watched that you made –” |
6. Student 3 (Samuel): | “Now it started to come up here, by this kind of a formula.” |
[Julia is shaping her pattern; Mia moves to watch Samuel’s working] | |
7. Student 1 (Mia): | “That’s, like, the same (part of the same pattern somehow), isn’t it?” |
The teacher interrupted the work and reminded the group that they should continue working on the final version of the tessellation. At this stage, Lotta and Samuel continued with their individual tessellations in parallel, communicating over their progression.
Actor | Quote |
---|---|
8. Student 2 (Lotta): | “Well then.” |
9. Student 3 (Samuel): | “This will be sort of random.” |
10. Student 2 (Lotta): | “This will be a flame.” |
11. Student 3 (Samuel): | “Yes, this can be extended.” |
12. Student 2 (Lotta): | “Help, you’re (Samuel) now coming over to my pattern.” |
13. Student 2 (Lotta): | “I got inspired to make a flame. I should just, there’s now a partly red and blue flame, then.” |
14. Student 3 (Samuel): | “[whistling] This is working.” |
15. Student 2 (Lotta): | “Is it?” |
16. Student 3 (Samuel): | “Perhaps. Yes, there’s something for all the holes.” |
Lotta then asks whether the group could merge their individual efforts (mainly her and Samuel’s drafts).
Actor | Quote |
---|---|
17. Student 2 (Lotta): | “Hey, shall we still combine these?” [a short laugh] |
18. Student 1 (Mia): | “I was thinking that if they should be combined, so that.” |
19. Student 3 (Samuel): | “A bit difficult, perhaps.” |
20. Student 2 (Lotta): | “Yes, perhaps.” |
21. Student 3 (Samuel): | “Because they are at different distances from each other. We would need to move the whole pattern. It wouldn’t really be feasible.” |
22. Student 1 (Mia) (to Lotta): | “Take some yellow from there now. (Would it be) still like the same pattern, with different colors? As there’s a blue center –” |
23. Student 3 (Samuel): | “Hey, now there’s a problem. Now it’s not working anymore.” |
Vignette 2. Group D: strings of synchronized and coordinated actions
Table 11 shows how, in Group D, the second task resulted in a decrease in the average of cognitive states and the average of state changes (− 24.6%). When zooming back in on the group interaction, it seems that the group stayed together over the process and jointly discussed and decided what they were doing or should do; however, the goals were less targeted and immediate. The group members constantly externalized their thoughts, which was not the case in Group B. At the beginning, the group discussed their previous session and decided to set more goals and work more logically than previously. Here, they discovered a “diamond problem,” as they called it; that is, they found themselves in a situation in which they would have needed extra, diamond-shaped tiles to fill all the gaps in their designed patterns (see also Hähkiöniemi et al., 2016).
Vignette 2
Example from synchronized and coordinated actions of the group (Group D)
Actor | Quote |
---|---|
1. Student 1 (Leo): | “((laughs)) Then we would only need a sort of small diamond shape so that we could in a way make this work.” |
2. Student 4 (Anna): | “((laughing)) Yes.” |
3. Student 2 (Lisa): | “So, this isn’t really like working in any way now, is it?” |
4. Student 1 (Leo): | “Nope.” |
5. Student 4 (Anna): | “No, it isn’t.” |
6. Student 1 (Leo): | “No, as it calls for a sort of diamond. Then it would become like that.” |
7. Student 3 (Nea): | “Diamond-shaped sweets.” |
8. Student 1 (Leo): | “Or, yes, diamond shapes for here, so then it would become like that.” |
9. Student 4 (Anna): | “Yes.” |
10. Student 3 (Nea): | “Well what if, could that one be set somehow in another direction, or like, could we get it here somehow?” |
11. Student 2 (Lisa): | “Yes.” |
12. Student 4 (Anna): | “But isn’t that-” |
13. Student 1 (Leo): | “That’s quite an impossible angle; you cannot really do anything about it. ((Notices Nea’s solution)) No, I mean, you actually can do it that way.” |
14. Student 4 (Anna): | “Yes, you actually can.” |
15. Student 2 (Lisa): | “But I don’t know, I think that the problem will always come up at some point, the problem that -” |
16. Student 4 (Anna): | “That it requires some supplementary job there.” |
17. Student 3 (Nea): | “The diamond.” |
Vignette 3. Group C: contemplative, reflective interactions as a group
As Table 11 shows, in Group C, the second task resulted in an increase in the average of social states and a slight increase in the social state changes (10.7%). Accordingly, the microlevel perspective on the interaction data of Group C provides us with a view of contemplative interactions in which the group members vocalized both immediate conceptual goals and goals going beyond the moment-to-moment interactions (Davis et al., 2015). The overarching goals for their discoveries were verbalized explicitly by relying on mathematical concepts, and the group was willing to verify the final pattern with GeoGebra. As a “warm-up” exercise, the group started by forming a pattern similar to the one made in the previous session with GeoGebra. Next, the group created a new, star-shaped pattern. The students used mathematical terms (i.e., reflection, rotation, translation), but they also wanted to make the pattern esthetic via the use of colors. To ensure the final pattern was mathematically correct, the group decided to test it with GeoGebra. The next excerpt is from the phase after the warm-up exercise, when the group started working on the real pattern. The group was oriented toward joint problem solving and communication, with individual members building on the words and ideas of their co-members (Stahl et al., 2014). The central, primary idea seems to have originated from Student 1 (Linus).
Vignette 3
Example from Contemplative, Reflective Interactions in the Group (Group C)
Actor | Quote |
---|---|
1. Student 1 (Linus): | “I was thinking that, if you make a straight line between these points, you know. No, but it won’t work because (–).” |
2. Student 3 (Tina): | “So, is it like, so that—” |
3. Student 1 (Linus): | “The way it goes, yes. So, then it, the line, would go with a 90-degree angle through the midsection between these points here, wouldn’t it? So, then this would be mirrored perfectly opposite to each other, wouldn’t it? If it’s like this angle here (even -) and then the line goes exactly there, like in between, like just in between that line, so then this would be mirrored on top of that, yes. And then it could be turned like there, I guess.” |
4. Student 3 (Tina): | “Does it go like this?” |
5. Student 1 (Linus): | “Yes, because this (–) there. No, no it won’t! No, because now, now it is mirrored like this because now this is the (-).” |
“Okay, and then this deflects this from there in parallel with this one in this way.” | |
6. Student 4 (Sara): | “(It is deflected in this direction.) It is deflected to this direction as here it was deflected from here to there. So, wouldn’t it be the same way there, then?” |
7. Student 2 (Hanna): | For this [it is] again the same, the line to the middle.” |
8. Student 1 (Linus): | “Yes.” |
9. Student 3 (Tina): | “Yes, so will it be then -” |
10. Student 4 (Sara): | “Is this now deflected here, to this direction?” |
11. Student 1 (Linus): | “Yes, so then this is deflected in relation to this. In relation to this line. And then, wait a sec, whereabouts should we find it (symmetry) again?” |
12. Student 4 (Sara): | “But then from there again.” |
13. Student 2 (Hanna): | “Look, we should get this one here -” |
14. Student 3 (Tina): | “[Yes, and if we would have there;] Just put that one there.” |
15. Student 2 (Hanna): | “Here in relation to this.” |
16. Student 3 (Tina): | “Hmm-m. But will this shape be like, you know.” |
17. Student 1 (Linus): | “Hmm-m.” |
18. Student 4 (Sara): | “Like what?” |
19. Student 3 (Tina): | “Just put those there. And then, was it like, which way did it go now?” |
20. Student 1 (Linus): | “Now it goes to its direction.” |
21. Student 2 (Hanna): | “Yes, so far, so good!” |
22. Student 2 (Hanna): | “(There’s a certain logic.)” |
23. Student 4 (Sara): | “Yes, it has.” |
24. Student 2 (Hanna): | “How the next layer would start, that’s what I don’t (still really) ((laughs)).” |
25. Student 3 (Tina): | “Yes, that’s what I was just thinking that -” |
26. Student 2 (Hanna): | “Should we take a picture of this phase? Until now it seems to work.” |
27. Student 1 (Linus): | “Yes.” |
(The group takes pictures of their product.) |
Discussion
This paper presented a method, together with empirical investigations, to capture various layers of CPS as a small-group enactment during open-ended, co-located CPS. The data were collected during an experiential mathematics course in geometry in the context of initial teacher education and included two variations of the CPS tasks. The method was based on analysis method triangulation and combined three phases. In the first phase, the descriptive counts on the coded data did not show a large variation in how the CPS construct was actualized in the different groups or tasks. Yet, in the second phase, when further elaborating on the coded data regarding the embedded temporal information, the results revealed both group- and task-related differences. In the third phase, via qualitative cases, the analysis zoomed back in on the microlevel, and exemplified diversity in the interaction quality of the groups.
When focusing on the empirical outcomes, the analysis method triangulation provided access to the diversity of the CPS enactment processes in the different groups and tasks. In terms of the CPS construct, the study made the processes visible as exchanges of actions and understandings that merge into unique CPS situations (Stahl, 2017). To a large extent, as visible in the case-based examples, the processes did not follow linear or rational models of problem solving as “logical deductions in individual minds” (Stahl, 2017, p. 117); instead, they included “breakdown” situations yielding actions, negotiations, and finally, explicit rounds of agreement as a group (Stahl, 2017). Accordingly, collaborative situations are often exploratory, where no clear plans or procedures exist for problem solving or organizing joint efforts (Baker, 2015). However, in these situations, students explore the problem space, and by doing so, they can gain a deeper conceptual understanding of the problem via co-elaboration of knowledge and understanding (Baker, 2015).
In general, the empirical notions from the different layers pinpoint the complexity of the CPS and the unpredictability of its enactment. This resonates well with the many challenges pronounced in earlier research, such as issues related to coordination of joint interactions and contributions (e.g., Andrist et al., 2018; Baker, 2015) and the students’ equal participation in the groups (e.g., Barron, 2000, 2003). Moreover, the different methods implemented in this study were not opposed to each other, and as argued by Kelle and Erzberg (2004), “the results of qualitative and quantitative approaches can converge, complement or contradict one another and each of these possibilities can be beneficial to the research process” (p. 176). Here, the process of quantifying the results of the qualitative analysis, focusing on the temporal characteristics in these data, and returning to qualitative explorations permitted a fuller treatment of the empirical material and its layers, with all the analysis phases still relating to the same concept or phenomenon.
Despite the many critiques of coding and quantifications (i.e., pointing to the fragmentation of the data and the phenomenon; see Ludvigsen et al., 2018), the first approach, combined with simple descriptive statistics (Jeong et al., 2014), was relevant because it allowed an overall understanding of the appearance of the CPS elements in the groups’ interaction to be developed and basic information about the group- and task-related differences to be obtained (Swiecki et al., 2020). In addition, these data formed the input material for the next phases of the analyses. However, manual coding is time-consuming and resource intensive (Graesser et al., 2018). Until now, a variety of automated text-analysis methods based on natural language processing have been developed to analyze the contents of communication, including latent semantic analysis (e.g., Dowell et al., 2020; Landauer et al., 1998, 2007) and automated content analysis for online (e.g., Mu et al., 2012) and manually transcribed face-to-face communication (Lämsä et al., 2021a, 2021b). Although these methods aid and expand the manual approach, ensuring the accuracy of the outcomes is still requisite (Graesser et al., 2018). That is, to guarantee that the linguistic “fine points” of human communication involved in collaboration would not be lost in the analysis; at the same time, to pay attention to the situational context of collaboration (Wise & Schwartz, 2017).
The second approach, relying on the temporal information embedded in the coded interactions, showed differences in the group-related temporal qualities regarding the tasks. When carefully designed, methods that take temporality into account can test and add to the process theory of collaborative learning and CPS (Kapur, 2011; Reimann, 2009; Wise & Schaeffer, 2015). However, Kapur (2011) recalls the limitations of the quantitative, event-based analysis as reductions of the richness and complexity of group processes, and thus, advocates using a comprehensive analytical scheme and triangulating the findings with microlevel, qualitative analysis. In the current study, the second approach directed the analysis toward the selection of units for further investigation via qualitative cases.
Whereas the second phase provided evidence of the differences between the groups and the tasks, the third approach contributed to more detailed clarifications of these analysis outcomes. Accordingly, a detailed interaction analysis of carefully selected cases has the full potential to add to the theoretical understanding of CPS (Gauvain, 2018; Reimann, 2021; Stahl, 2017). In general, providing access to different layers of CPS, the method can intensify and extend our understanding of CPS as a dynamic, group-level construct. However, to reach the theoretical underpinnings of CPS as a group-level enactment requires repeated and refined application of the measures that formed the analysis triangulation. That is, the analysis can be performed on a larger amount of data (both in terms of the sample size and timescale) with advanced, computational methods that make the analysis more automated and thereby more scalable.
Conclusion
Taken together, the paper provided a detailed description of the various layers of CPS enactments as a group-level construct, extending from individually oriented research approaches in CPS. To accomplish this, the study triangulated three analysis approaches. Each approach was beneficial in the process of deepening our understanding of CPS and contributing methodologically to the growing body of research on temporality in data. While the sample size cannot be generalized to wider populations, the case examples revealed diverse group adoption patterns that reproduced the previous findings regarding interpersonal interaction in the groups as being largely explorative, intuitive, and less aligned with linear problem solving. All these aspects relate to the high complexity of the actualized CPS construct and point to the need to consider its manifestations in long-term study designs of wider populations of students, researched with advanced methods that can grasp the causal mechanisms of the multiple interacting elements that enter into successful CPS processes in small groups, aiming for increased and relevant theoretical understanding of CPS.
References
Abbott, A. (1990). A primer on sequence methods. Organization Science, 1(4), 375–392.
Alterman, R., & Harsch, K. (2017). A more reflective form of joint problem solving. International Journal of Computer-Supported Collaborative Learning, 12, 9–33.
Andrist, S., Ruis, A. R., & Williamson Shaeffer, D. (2018). A network analytic approach to gaze coordination during a collaborative task. Computers in Human Behaviour, 89, 339–348. https://doi.org/10.1016/j.chb.2018.07.017
Näykki, P., Pöysä-Tarhonen, J., Järvelä, S., & Häkkinen, P. (2015). Enhancing teacher education students’ collaborative problem-solving and shared regulation of learning. In O. Lindwall, P. Häkkinen, T. Koschman, P. Tchounikine, & S. Ludvigsen, (Eds.), Exploring the material conditions of learning: The Computer Supported Collaborative Learning (CSCL) Conference 2015, Volume 1 (pp. 514–517). The International Society of the Learning Sciences.
Hähkiöniemi, M., Fenyvesi, K., Pöysä-Tarhonen, J., Tarnanen, M., Häkkinen, P., Kauppinen, M., Martin, A., & Nieminen, P. (2016). Mathematics learning through arts and collaborative problem-solving: The princess and the diamond-problem. In E. Torrence, B. Torrence, C. H. Séquin, D. McKenna, K. Fenyvesi, & R. Sarhangi (Eds.), Conference proceedings of BRIDGES mathematical connections in art, music, and science (pp. 97–104). Tessellations Publishing.
Näykki, P., Isohätälä, J., Järvelä, S., Pöysä-Tarhonen, J., & Häkkinen, P. (2017). Facilitating socio-cognitive and socio-emotional monitoring in collaborative learning with a regulation macro script: An exploratory study. International Journal of Computer-Supported Collaborative Learning, 12(3), 251–279. https://doi.org/10.1007/s11412-017-9259-5
Avry, S., Molinari, G., Bétrancourt, M., & Chanel, G. (2020). Sharing emotions contributes to regulating collaborative intentions in group problem-solving. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2020.01160
Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to sequential analysis. Cambridge University Press. https://doi.org/10.1017/CBO9780511527685
Bakeman, R., & Quera, V. (2011). Sequential analysis and observational methods for the behavioral sciences. Cambridge University Press.
Baker, M. J. (2015). Collaboration in collaborative learning. Interaction Studies, 16(39), 451–473.
Barron, B. (2000). Achieving coordination in collaborative problem-solving groups. The Journal of the Learning Sciences, 9(4), 403–436.
Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359.
Barron, B., Pea, R., & Engle, R. A. (2015). Advancing understanding of collaborative learning with data derived from video records. In C. E. Hmelo-Silver, C. A. Chinn, C. Chan, & A. M. O’Donnell (Eds.), The international handbook of collaborative learning (pp. 203–219). Routledge.
Baškarada, S. (2014). Qualitative case study guidelines. The Qualitative Report, 19(24), 1–18.
Care, E., Scoular, C., & Griffin, P. (2016). Assessment of collaborative problem solving in education environments. Applied Measurement in Education, 29(4), 250–264.
Chan, M. C. E., & Clarke, D. (2017). Structured affordances in the use of open-ended tasks to facilitate collaborative problem solving. ZDM Mathematics Education, 49, 951–963. https://doi.org/10.1007/s11858-017-0876-2
Chen, B., Resendes, M., Chai, C. S., & Hong, H.-Y. (2017). Two tales of time: Uncovering the significance of sequential patterns among contribution types in knowledge-building discourse. Interactive Learning Environments, 25(2), 162–175. https://doi.org/10.1080/10494820.2016.1276081
Chenail, R. J. (2012). Conducting qualitative data analysis: Qualitative data analysis as a metaphoric process. The Qualitative Report, 17(1), 248–253.
Cress, U., Stahl, G., Rose, C., Law, N., & Ludvigsen, S. (2018). Forming social systems by coupling minds at different levels of cognition: Design, tools, and research methods. International Journal of Computer Supported Collaborative Learning, 13(3), 235–240. https://doi.org/10.1007/s11412-018-9284-z
Csanadi, A., Eagan, B., Kollar, I., Shaffer, D. W., & Fischer, F. (2018). When coding-and-counting is not enough: Using epistemic network analysis (ENA) to analyze verbal data in CSCL research. International Journal of Computer-Supported Collaborative Learning, 13(4), 419–438. https://doi.org/10.1007/s11412-018-9292-z
Davis, P., Horn, M., Block, F., Phillips, B., Evans, M. E., Diamond, J., & Shen, C. (2015). “Whoa! We’re going deep in the trees!” Patterns of collaboration around an interactive information visualization exhibit. International Journal of Computer-Supported Collaborative Learning, 10, 53–76.
De Wever, B., Schellens, T., Valcke, M., & van Keer, H. (2006). Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers and Education, 46, 6–28.
Dillenbourg, P., & Hong, F. (2008). The mechanics of CSCL macro scripts. International Journal of Computer-Supported Collaborative Learning, 3(1), 5–23.
Dillenbourg, P., Lemaignan, S., Sangin, M., Nova, N., & Molinari, G. (2016). The symmetry of partner modelling. International Journal of Computer-Supported Collaborative Learning, 11, 227–253.
Dillenbourg, P., & Tchounikine, P. (2007). Flexibility in macro-scripts for CSCL. Journal of Computer Assisted Learning, 23(1), 1–13.
Dowell, N. M., Lin, Y., Godfrey, A., & Brooks, C. (2020). Exploring the relationship between emergent sociocognitive roles, collaborative problem-solving skills, and outcomes: A group communication analysis. Journal of Learning Analytics, 7(1), 38–57. https://doi.org/10.18608/jla.2020.71.4
Dowell, N. M., Nixon, T., & Graesser, A. C. (2018). Group communication analysis: A computational linguistics approach for detecting sociocognitive roles in multi-party interactions. Behavior Research Methods, 51(3), 1007–1041. https://doi.org/10.3758/s13428-018-1102-z
Eichmann, B., Goldhammer, F., Greiff, S., Pucite, L., & Naumann, J. (2019). The role of planning in complex problem solving. Computers & Education, 128, 1–12. https://doi.org/10.1016/j.compedu.2018.08.004
Febrer-Hernández, J. K., & Hernández-Palancar, J. (2012). Sequential pattern mining algorithms review. Intelligent Data Analysis, 16(3), 451–466.
Flick, U. (2002). An introduction to qualitative research. Sage.
Flick, U. (2004). Triangulation in qualitative research. In U. Flick, E. von Kardorff, & I. Steinke (Eds.), A companion to qualitative research (pp. 178–183). Sage.
Flick, U., von Kardorff, E., & Steinke, I. (2004). What is qualitative research? An introduction to the field. In U. Flick, E. von Kardorff, & I. Steinke (Eds.), A companion to qualitative research (pp. 1–11). Sage.
Friese, S. (2014). Qualitative data analysis with ATLAS.ti. Sage.
Funke, J., Fischer, A., & Holt, D. V. (2018). Competencies for complexity: Problem solving in the twenty-first century. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications (pp. 41–53). Springer.
Gailiunas, P. (2007). Some monohedral tilings derived from regular polygons. In R. Sarhangi & J. Barrallo (Eds.), Conference proceedings of BRIDGES mathematical connections in art, music, and science (pp. 9–14). Mathartfun.
Gauvain, M. (2018). Collaborative problem solving: Social and developmental considerations. Psychological Science in the Public Interest, 19(2), 53–58. https://doi.org/10.1177/1529100618813370
Graesser, A., Fiore, S. M., Greiff, S., Andrews-Todd, J., Foltz, P. W., & Hesse, F. W. (2018). Advancing the science of collaborative problem solving. Psychological Science in the Public Interest, 19(2), 59–92. https://doi.org/10.1177/1529100618808244
Griffin, P. (2014). Performance assessment of higher order thinking. Journal of Applied Measurement, 15(1), 1–16.
Griffin, P., & Care, E. (2015). The ATC21s method. In P. Griffin & E. Care (Eds.), Assessment and teaching of 21st century skills: methods and approach (pp. 1–33). Springer.
Hayashi, Y. (2018). The power of a “maverick” in collaborative problem solving: An experimental investigation of individual perspective-taking within a group. Cognitive Science, 42, 69–104. https://doi.org/10.1111/cogs.12587
Hesse, H., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). Framework for teachable collaborative problem solving skills. In P. Griffin & E. Care (Eds.), Assessment and teaching of 21st century skills. Methods and approach (pp. 37–56). Springer.
Hsieh, H., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288.
Humble, A. M. (2009). Technique triangulation for validation in directed content analysis. International Journal of Qualitative Methods. https://doi.org/10.1177/160940690900800305
Järvelä, S., Järvenoja, H., Malmberg, J., Isohätälä, J., & Sobocinski, M. (2016). How do types of interaction and phases of self-regulated learning set a stage for collaborative engagement? Learning and Instruction, 43, 39–51. https://doi.org/10.1016/j.learninstruc.2016.01.005
Jeong, H., Hmelo-Silver, C. E., & Yu, Y. (2014). An examination of CSCL methodological practices and the influence of theoretical frameworks 2005–2009. International Journal of Computer-Supported Collaborative Learning, 9, 305–334. https://doi.org/10.1007/s11412-014-9198-3
Kapur, M. (2011). Temporality matters: Advancing a method for analyzing problem-solving processes in a computer-supported collaborative environment. International Journal of Computer-Supported Collaborative Learning, 6(1), 39–56.
Kapur, M., Voiklis, J., & Kinzer, C. K. (2008). Sensitivities to early exchange in synchronous computer-supported collaborative learning (CSCL) groups. Computers & Education, 51(1), 54–66.
Kelle, U., & Erzberger, C. (2004). Qualitative and quantitative methods: Not in opposition. In U. Flick, E. von Kardorff, & I. Steinke (Eds.), A companion to qualitative research (pp. 172–177). Sage.
Kennedy, B. (2018). Deduction, induction, and abduction. In U. Flick (Ed.), The Sage handbook of qualitative data collection (pp. 49–64). Sage.
Kyllonen, P. C., Zhu, M., & von Davier, A. (2018). Introduction: Innovative assessment of collaboration. In A. von Davier, M. Zhu, & P. C. Kyllonen (Eds.), Innovative assessment of collaboration: Methodology of educational measurement and assessment (pp. 1–18). Springer.
Lämsä, J., Hämäläinen, R., Koskinen, P., Viiri, J., & Lampi, E. (2021a). What do we do when we analyse the temporal aspects of computer-supported collaborative learning? A systematic literature review. Educational Research Review. https://doi.org/10.1016/j.edurev.2021.100387
Lämsä, J., Uribe, P., Jiménez, A., Caballero, D., Hämäläinen, R., & Araya, R. (2021b). Deep networks for collaboration analytics: Promoting automatic analysis of face-to-face interaction in the context of inquiry-based learning. Journal of Learning Analytics, 8(1), 113–125. https://doi.org/10.18608/jla.2021.7118
Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25, 259–284.
Landauer, T. K., McNamara, D. S., Dennis, S., & Kintsch, W. (2007). Handbook of latent semantic analysis. Taylor & Francis.
Lewis, K., & Herndon, B. (2011). Transactive memory systems: Current issues and future research directions. Organizational Science, 22(5), 1254–1265.
Ludvigsen, S., Cress, U., Rosé, C. P., Law, N., & Stahl, G. (2018). Developing understanding beyond the given knowledge and new methodologies for analyses in CSCL. International Journal Computer-Supported Collaborative Learning, 13, 359–364. https://doi.org/10.1007/s11412-018-9291-0
Manches, A., O’Malley, C., & Benford, S. (2010). The role of physical representations in solving number problems: A comparison of young children’s use of physical and virtual materials. Computers & Education, 54(3), 622–640. https://doi.org/10.1016/j.compedu.2009.09.023
Mayer, R. E. (1998). Cognitive, metacognitive, and motivational aspects of problem solving. Instructional Science, 26, 49–63.
Meadows, L., & Morse, J. M. (2001). Constructing evidence within the qualitative project. In J. M. Morse, J. Swanson, & A. Kuzel (Eds.), The nature of evidence in qualitative inquiry (pp. 187–202). Sage.
Mu, J., Stegmann, K., Mayfield, E., Rosé, C., & Fischer, F. (2012). The ACODEA framework: Developing segmentation and classification schemes for fully automatic analysis of online discussions. International Journal of Computer-Supported Collaborative Learning, 7(2), 285–305.
Neuendorf, K. A. (2011). Content analysis: A methodological primer for gender research. Sex Roles, 64, 276–289. https://doi.org/10.1007/s11199-010-9893-0
Norm Lien, Y.-C., Wu, W.-J., & Lu, Y.-L. (2020). How well do teachers predict students’ actions in solving an ill-defined problem in STEM education: A solution using sequential pattern mining. IEEE Access, 8, 134976–134986. https://doi.org/10.1109/ACCESS.2020.3010168
Reimann, P. (2009). Time is precious: Variable- and event-centred approaches to process analysis in CSCL research. Computer-Supported Collaborative Learning, 4, 239–257. https://doi.org/10.1007/s11412-009-9070-z
Reimann, P. (2021). Methodological progress in the study of self-regulated learning enables theory advancement. Learning and Instruction. https://doi.org/10.1016/j.learninstruc.2019.101269
Reimann, P., Yacef, K., & Kay, J. (2011). Analyzing collaborative interactions with data mining methods for the benefit of learning. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL. Computer-Supported Collaborative Learning Series. (Vol. 12). Springer.
Richardson, D. C., Dale, R., & Kirkham, N. Z. (2007). The art of conversation is coordination. Psychological Science, 18(5), 407–413. https://doi.org/10.1111/j.1467-9280.2007.01914.x
Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem solving. In C. O’Malley (Ed.), Computer supported collaborative learning series. NATO ASI series (Series F: computer and systems sciences) (Vol. 128, pp. 69–97). Springer.
Santos-Trigo, M., & Espinosa-Perez, H. (2002). Searching and exploring properties of geometric configurations using dynamic software. International Journal of Mathematical Education in Science and Technology, 33(1), 37–50.
Scardamalia, M., & Bereiter, C. (2006). Knowledge building. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 97–115). Cambridge University Press.
Scoular, C., & Care, E. (2020). Monitoring patterns of social and cognitive student behaviors in online collaborative problem solving assessments. Computers in Human Behavior. https://doi.org/10.1016/j.chb.2019.01.007
Scoular, C., Care, E., & Hesse, F. (2017). Designs for operationalizing collaborative problem solving for automated assessment. Journal of Educational Measurement, 54(1), 12–35.
Sears, D. A., & Reagin, J. M. (2013). Individual versus collaborative problem solving: Divergent outcomes depending on task complexity. Instructional Science, 41, 1153–1172.
Sinha, S., Kempler Rogat, T., Adams-Wiggins, K. R., & Hmelo-Silver, C. (2015). Collaborative group engagement in a computer-supported inquiry learning environment. International Journal Computer-Supported Collaborative Learning, 10, 273–307.
Stahl, G. (2017). Group practices: A new way of viewing CSCL. International Journal Computer-Supported Collaborative Learning, 12, 113–126.
Stahl, G., Law, N., Cress, U., & Ludvigsen, S. (2014). Analyzing roles of individuals in small-group collaboration processes. International Journal of Computer-Supported Collaborative Learning, 9, 365–370. https://doi.org/10.1007/s11412-014-9204-9
Swiecki, Z., Ruis, A. R., Farrell, C., & Williamson Shaffer, D. (2020). Assessing individual contributions to collaborative problem solving: A network analysis. Computers in Human Behavior. https://doi.org/10.1016/j.chb.2019.01.009
Thomas, E., & Magilvy, J. K. (2011). Qualitative rigor or research validity in qualitative research. Journal for Specialists in Pediatric Nursing, 16, 151–155.
Thompson, L. L., Wang, J., & Gunia, B. C. (2010). Negotiation. Annual Review of Psychology, 61, 491–515.
Vogel, F., & Weinberger, A. (2018). Quantifying qualities of collaborative learning processes. In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, & P. Reimann (Eds.), International handbook of the learning sciences (pp. 500–510). Routledge.
Wegner, D. M. (1986). Transactive memory: A contemporary analysis of the group mind. In B. Mullen & G. R. Goethals (Eds.), Theories of group behavior (pp. 185–205). Springer.
Williamson Schaeffer, D. (2017). Quantitative ethnography. Cathcart Press.
Wise, A. F., & Schwarz, B. B. (2017). Visions of CSCL: Eight provocations for the future of the field. International Journal of Computer-Supported Collaborative Learning, 12(4), 423–467. https://doi.org/10.1007/s11412-017-9267-5
Wise, A. F., & Shaffer, D. W. (2015). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2(2), 5–13.
Zimmerman, B. J. (2010). Self-regulation involves more than metacognition: A social cognitive perspective. Educational Psychologist, 30, 217–221.
Acknowledgements
We are thankful for the teacher education students participating in the study and the teacher educators for their collaboration. We would especially like to thank Anne Martin for her assistance in the data collection and preliminary coding. We also thank Jaana Isohätälä for preparing materials for the study and Martti Minkkinen for the graphical design of the materials.
Funding
Open Access funding provided by University of Jyväskylä (JYU). This study was funded by the Academy of Finland (Grant Number 273970).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1
Categorization matrix for the social elements set of collaborative problem solving, modified from Hesse et al. (2015)
Code | Description | Indicator/criterion |
---|---|---|
Participation | Is a participant interacting with, prompting, and responding to the contributions of others? | Initiates and promotes interaction or activity, responds to cues in communication |
Perspective taking | (A) Does a participant accept and adapt contributions by other group members? | Contributions or prompts of others are used to suggest possible problem-solving paths; contributions or prompts of others are adapted and incorporated |
(B) Is a participant (aware of) adapting her/his behavior in relation to other group members’ needs/intellectual capabilities? | Contributions are tailored to recipients based on the interpretation of recipients’ understanding, and contributions are modified for recipient understanding in light of deliberate feedback |
Categorization matrix for the cognitive elements set of collaborative problem solving, modified from Hesse et al. (2015)
Code | Description | Indicator/criterion |
---|---|---|
Coordination | Is a participant (a group) able to manage resources or people to complete a task? | Suggests that people or resources be used, allocates people or resources to a task |
Task exploration | Does a participant a (group) explore and understand elements of the task? | Identifies need for information related to current, alternative, and future activity, identifies the need for information related to immediate activity |
Problem analysis | Does a participant (a group) analyze and describe a problem? | Identifies the necessary sequence of subtasks, and the problem is divided into subtasks |
Problem reanalysis | Does a participant (a group) adapt reasoning/course of action as information or circumstance change? | Reconstructs and reorganizes understanding of the problem in search of new solutions, tries additional options considering new information or lack of progress |
Appendix 2
An example of coded interaction (Task 2, Group C).
Actor | Quote | Code |
---|---|---|
1. Student 1 (Linus): | “I was thinking that, if you make a straight line between these points, you know. No, but it won’t work because (–).” | Perspective taking B, Problem reanalysis |
2. Student 3 (Tina): | “So, is it like, so that—” | |
3. Student 1 (Linus): | “The way it goes, yes. So, then it, the line, would go with a 90-degree angle through the midsection between these points here, wouldn’t it? So, then this would be mirrored perfectly opposite to each other, wouldn’t it? If it’s like this angle here (even -) and then the line goes exactly there, like in between, like just in between that line, so then this would be mirrored on top of that, yes. And then it could be turned like there, I guess.” | Perspective taking B, Problem reanalysis |
4. Student 3 (Tina): | “Does it go like this?” | Participation |
5. Student 1 (Linus): | “Yes, because this (–) there. No, no it won’t! No, because now, now it is mirrored like this because now this is the (-).” | Perspective taking B, Problem reanalysis |
“Okay, and then this deflects this from there in parallel with this one in this way.” | Problem reanalysis | |
6. Student 4 (Sara): | “(It is deflected to this direction.) It is deflected to this direction as here it was deflected from here to there. So, wouldn’t it be the same way there, then?” | |
7. Student 2 (Hanna): | For this again the same, the line to the middle.” | |
8. Student 1 (Linus): | “Yes.” | |
9. Student 3 (Tina): | “Yes, so will it be then -” | |
10. Student 4 (Sara): | “Is this now deflected here, to this direction?” | |
11. Student 1 (Linus): | “Yes, so then this is deflected in relation to this. In relation to this line. And then, wait a sec, whereabouts should we find it (symmetry) again?” | |
12. Student 4 (Sara): | “But then from there again.” | |
13. Student 2 (Hanna): | “Look, we should get this one here -” | Participation, Problem reanalysis, coordination |
14. Student 3 (Tina): | “[Yes, and if we would have there;] Just put that one there.” | |
15. Student 2 (Hanna): | “Here in relation to this.” | |
16. Student 3 (Tina): | “Hmm-m. But will this shape be like, you know.” | Participation, Problem reanalysis |
17. Student 1 (Linus): | “Hmm-m.” | |
18. Student 4 (Sara): | “Like what?” | |
19. Student 3 (Tina): | “Just put those there. And then, was it like, which way did it go now?” | Coordination, Participation |
20. Student 1 (Linus): | “Now it goes to its direction.” | Problem reanalysis |
21. Student 2 (Hanna): | “Yes, so far, so good!” | Participation, Problem reanalysis |
22. Student 2 (Hanna): | “(There’s a certain logic.)” | |
23. Student 4 (Sara): | “Yes, it has.” | |
24. Student 2 (Hanna): | “How the next layer would start, that’s what I don’t (still really) ((laughs)).” | Problem reanalysis |
25. Student 3 (Tina): | “Yes, that’s what I was just thinking that -” | |
26. Student 2 (Hanna): | “Should we take a picture of this phase? Until now it seems to work.” | |
27. Student 1 (Linus): | “Yes.” | |
(The group is taking pictures of their product.) |
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Pöysä-Tarhonen, J., Häkkinen, P., Tarhonen, P. et al. “Anything taking shape?” Capturing various layers of small group collaborative problem solving in an experiential geometry course in initial teacher education. Instr Sci 50, 1–34 (2022). https://doi.org/10.1007/s11251-021-09562-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11251-021-09562-5