Keywords

1 Introduction

Children approaching primary school are asked to meet several standards and find themselves at facing several challenges when starting their entrance to the school system. In this scenario, the concept of school readiness becomes a fundamental factor to consider [1]. School readiness is a label used to describe children’s mathematical, literacy and socio-emotional skills that are strong predictors of their academic success and personality development [2,3,4].

The importance of these skills is widely recognized, and it has become a key element in the Italian National Guidelines for Kindergarten since 2012 issued by the Italian Ministry of Education [5]. School readiness is a set of abilities that represents the basic achievements, from a cognitive and emotional point of view, to face the transition between kindergarten and primary school.

Considering the different abilities which are included in the concept of school readiness, we focused on two main branches stemming from them, or numerical and spatial thinking and socio-emotional skills. These two cores have been used as the root for the development of two parallel versions of the Diligo 2.0 mobile app [6]. The version which focuses on emotional skills is designed to assess these key abilities: awareness of the emotions, use and comprehension of emotion-related vocabulary, recognition of facial expressions and their link to the emotions, comprehension of the situations that elicit emotions, knowledge of the cultural rules for displaying emotion and regulation and management of one’s own and others’ emotions. The other version, which is the one which has already been tested in schools and whose results are discussed in this paper, is focused on these skills: knowledge of the geometrical figures, acquisition of big and small concepts, recognition of number representation, spatial concepts of in-out and up-down, temporal order in terms of before and after, spatial directions left and right.

These mobile apps not only allow to monitor the school readiness abilities, but they also evaluate the cognitive preference for slow or fast thinking activities as described in Kahneman’s theory [7] who describes human thinking based on two main reasoning systems which uses different strategies, where “System 1” is fast, automatic, unconscious, and emotional and “System 2” is slow, logical, conscious, and effortful. So, the hypothesis underlying this study is that children are much more used to engage in fast thinking activities that in slower ones, and this may be due to the experience children have more chances to encounter in modern entertainment forms and environments.

2 The Structure of the App

Diligo 2.0 can be described as a serious game, developed for children of age 5 to 6 years old. The game has been developed using the STELT (Smart Technologies to Enhance Learning Technologies) [8] development framework and it follows an agent-based approach [9], since it is based on the continuous interaction between a natural agent (the kid) and the main character of the game, which is the artificial agent.

The game is focused on helping the character of “Leo the Explorer” to solve puzzles in eight different islands to collect treasures. To solve the puzzles, children must answer to 32 different questions, divided by eight sets of items, one for every school readiness skill related to numerical and spatial thinking. For every interaction, the Explorer has been fully voiced by a professional voice actor, and if needed, the child can listen any number of times the instructions by just pressing an on-screen button. For every group of four items, there have been programmed four alternative forms, which are randomly selected for every game session, allowing the game to be various enough for children if played more than once with a total number of 128 possible items.

At the beginning of every level, which is represented by an island on a treasure map, the child can choose between a slow or fast approach by selecting the preferred mode.

The fast mode is characterized by presenting the four items of the skill in quick sequence, without any storytelling, encouraging the child to play as fast as possible, as it is shown in Fig. 1.

Fig. 1.
figure 1

An item from the number recognition skill. In this case, the Explorer voice asks the child to say if the two depicted number are the same or not.

Full size image

The slow mode is based on a narrative approach, since the same item of the fast mode are mixed with other requests for the child that does not deal with numerical or spatial abilities but are necessary for the story undergoing in which the child and the Explorer are involved. The idea is that a narrative approach may improve children’s performances [10,11,12,13], since it activates Kahneman’s System 2, allowing the children to better focus on the task, and achieving a higher score in the game. It is possible to see a “narrative” item from slow mode in Fig. 2.

Fig. 2.
figure 2

This is an example of narrative items from the slow mode. In this case, the Explorer’s voice asks the children to choose an item to run away from the shark during an immersion. This kind of item does not affect the final score.

Full size image

The app can perform both online and offline, requiring an online connection to send data to the database where they have been stored for this study, and where they can be accessed only by researcher and in anonymous mode. The only score the child can see in the app is the preference for fast or slow mode, and this is because Diligo is still not validated as an assessment tool, so we did not want it to become misunderstood.

The concept behind Diligo is both to create the structure for an ipsative tool and to make a normative one. The game can be an ipsative tool for teachers to collect cumulative data on a specific child, and at the same time, teachers can monitor the entire classroom considering it as a normative tool to understand the level of the entire group.

Both numerical-spatial cognition and emotional skills versions of Diligo are currently available in Italian language for Android devices and can be found on the Google Play Store [14, 15], but the one we have used for this pilot study is the one focused on the numerical and spatial cognition.

3 Methods

This pilot study involved n = 44 Italian children from kindergarten with age between 5 and 6 years old from two different schools in Center Italy.

Before administering the mobile app, a prototype of the app has been given to teachers, allowing them to familiarize with it, and to collect feedbacks from teachers through a checklist. These feedbacks have been since used to correct some minor issues and to improve the UI and the overall experience of the app.

Before the experiment, teachers were given access keys to associate to children profiles to collect data anonymously but allowing children to login as distinct users. All data collected has been anonymously and safely stored in an online database.

The experiment has been conducted with the help of teachers, who administered the mobile app through Android tablets in small groups, so that every child had the opportunity to play a complete session, receiving help from teachers if they had technical issues.

4 Results and Discussion

We can start describing results by observing Table 1, where descriptive statistics have been reported. Every sub/scale had a minimum possible score of 0 and a maximum of 4 per-scale, while the total maximum score for the complete session was 32. The mean score of the sample is slightly above the mode, which shows that most of the children answered correctly to the 58% of the items.

This may be explained by the difficulty of the items, or by the dimensions of the sample, which may be not enough big to shed light on this data. Even if we consider the possible difficulty of items, we could attribute it to different factors. Since we used a mobile app, UI usability, devices performances and graphics may have a major role in children’s performances, so it would be interesting to repeat this experiment together with other available evaluation methods for numerical and spatial skills. We can also compare the different scores in subscales. One notable difference is the low score on the “Inside-Outside” and “Up-down” scales, which have the lowest scores on all descriptive measures.

Taking account that other spatial thinking abilities shows higher scores, it is possible to hypothesize that these low scores may be due to some interface or graphics factors, like interface readability or level design elements. This aspect may play an important role, since the environments of the game are realized in 2D, especially in these two subscales, which may benefit more than the others of a 3D environment to improve UI clarity.

Table 1. Descriptive statistics for the n = 44 children sample considering correct answers
Full size table

We can now discuss data related to the preference for fast or slow thinking modes. Considering Table 2, we can notice that all the measures are very much closer to 1 than to 2, which means that if 1 is the score assigned for the fast mode and 2 the one for the slow preference, most of the children showed a strong tendency towards the use of fast thinking cognitive strategies.

This data is even more consistent when considering that standard deviation for this measure is quite low and even smaller than the one observed in the score for the different subscales in Table 1, showing that variance is low, and it is very evident the preference for fast mode. This result is quite close to the hypothesis, expressed in the previous paragraph, that today’s children may have a stronger preference for fast thinking activities due to the exposition to experiences and entertainment products that privilege this mode at the expense of slow cognitive activities.

Table 2. Descriptive statistics for the n = 44 children sample considering fast/slow preferences, where 1 = fast and 2 = slow
Full size table

5 Conclusions

The first results coming from this pilot study on the usage of Diligo 2.0 open some interesting scenarios to better understand.

The first point to deepen is the impact of the game design, UI and UX on the performances of children. This theme is highly relevant when designing a serious game which has the aim to become an assessment tool, since the user experience of the design elements of the app/game inevitably add a layer between the user and the evaluation tool, having an impact on the final performance that can be improved or slowed down by the ease of use and clarity of app design. It would be interesting, for example, to redesign the UI for the two subscales which have shown lower average scores, because it would allow to better understand if a 2D or 3D environment can affect children performances on cognitive tasks.

Further data analysis is needed to be realized on a bigger sample, since preliminary data analysis showed a weak reliability of item scales (α = .61) with different methodologies (KR-20 = 0.59 and KR-21 = 0.56), and this may easily be due to the small sample or the shortness of the scale.

Even correlation seems to be quite low between the preference for fast mode and average scores on all subscales, but for the same reasons above, it is difficult to consider appropriately this data on a small size sample. Even a preliminary one sample t-test has not shown significant differences in average scores between children with preference towards fast and slow thinking activities, and this may be linked to the size of the sample, so further analysis and a bigger study are needed to better understand the role of cognitive mode preference and performance, which seems, by now, not significant.

For the future development of the Diligo 2.0 app it will be important to improve game design, UI and UX and, moreover, it is soon expected to start a new pilot study with the other version of the game, which is focused on social and emotional skills.

This may give even more information to understand the role of the design of the app on children performances, other than giving information about the specific skills on which the game is focused.

Summarizing all these points, future studies with Diligo 2.0 need to include a bigger sample, to consider possible differences related to gender, and to improve the game UX to make it even more accessible and usable in kindergarten.