Keywords

1 Introduction

The world has changed and with it, jobs, tools and transportation. In 2007, 76% of schools in the USA had to ban mobile phones (Obringer & Coffey, 2007). Unfortunately, in 2019, there was still no official protocol to deal with the situation, sending contradictory messages to students (Dennen & Rutledge, 2019). Mobile phones need to be part of education, as they are a part of life. Schools have to harness the potential of technology in the classroom to address the digital divide (Scarani et al., 2020) and give students the skills they need in today's connected world (OECD, 2015).

Moreover, the current pandemic has had an enormous impact on global education, with more than 1.3 billion learners being unable to go to school (Hagan, 2020). In order to ensure the continuity of learning, educational institutions have embarked on setting up technological platforms to deliver online learning programmes to students. These include the Soundcool system.

Soundcool is a free system for musical, sound and visual collaborative creation through mobile phones, tablets and other interfaces. It consists of a set of modules such as virtual instruments, players (audio and video), live audio (microphone) and video (camera), audio and video effects, and mixers that work on Mac and PC computers. Soundcool modules are opened from the app and can be interconnected with each other. For example, a sound source, such as a virtual instrument and a microphone, can be connected to a delay effect, which is applied in real-time to that source. The main advantage of Soundcool is that these modules can be controlled via users’ mobile phones or tablets, enabling collaborative creation, with both educational and professional applications.

The app was designed to create audiovisual content between two or more people, although there have been times where 100 people have intervened in a particular artistic project. This means that all users can give their ideas at the same time to create audiovisual content under the supervision of a main computer operator who mixes the elements. The system is intuitive and very easy to use; for example, when children use it, they do not need instructions about controls, they just learn by constructing the project.

1.1 Opportunities Afforded by the Soundcool System

Soundcool has applications in fields like education, special needs education, music performances, artistic performances, dance and therapy for people with special needs and for patients with neurodegenerative diseases. It started out as a tool to complement music performances, but music teachers quickly saw the potential of the app for musical education. It has had successful results in this area, with at least 40 schools working with the tool and 6,000 monthly downloads. Thanks to the implementation of audiovisual content, Soundcool can now also be used in other subjects such as science, maths and languages, and it could also be interesting for other arts.

Soundcool is suitable for users with special needs, and the app is available for functional diversity schools. For example, the HoloLens glasses opened the door to dance schools, ballet companies and their performances. Research is currently underway on the possibilities of using the system for people with neurodegenerative illnesses, like Alzheimer's and Parkinson's.

1.1.1 Education

The educational part of Soundcool was developed by a team of doctors in pedagogy, UPV members, professionals in the performing arts and technology experts. Children and young people like the system very much because it speaks the technological language of their generation and they intuitively understand it without the need for instructions. As all the students participate in the project, they feel part of it, are motivated, kept busy and concentrated, which holds class discipline together.

The value of the system is that it enables children to explore different abilities, embracing multiple intelligence (Gardner, 1993) and giving them the opportunity to learn from their strong points. The key to Soundcool is that what kids construct through information comes through creativity and not from learning content off by heart.

In addition, the system uses resources that are available to many individuals in many educational centres, such as PC and Mac computers, microphones, webcams and speakers. The mobile phones and tablets used were selected because they are widely available among students and their families, eliminating the need for schools to acquire them.

1.1.2 Training for Teachers

Conversely to students, teachers need to be well informed about the system’s possibilities to make the most of the tool. Soundcool has free online training in the shape of two MOOCs (Massive Open Online Courses): “Musical creation with Soundcool: Introduction” and “Soundcool 2: Video modules and creative proposals”, available on edX, the international online course platform. Moreover, these course videos are also on the project’s YouTube channel. These courses are endorsed by UPV certificates and they benefit teachers in terms of new educational technologies and methodologies, which also improves their resume.

1.1.3 Online Learning

Soundcool also facilitates collaborative creation with the app for distance education (Sastre et al., 2020). Some artistic experiences took place during lockdown with users being able to participate in artistic projects from home. These included:

  • Collaborative project in the 21st Century International Festival.

  • Seoul International Computer Music Festival, during which a collaborative piece was played with online participants from around the world.

  • Presentation of collaborative creation with Soundcool for online education (Sastre et al., 2020), in the Korean Electro-Acoustic Music Society’s Annual Conference.

  • Online course to train teachers on musical and audiovisual collaborative creation projects in times of social distancing with Soundcool.

1.1.4 Special Needs Education

Soundcool was also adapted by the special needs educational team working with the performing arts and technology so it can be used for students with functional diversity. All the Soundcool tools are adapted to people who are blind, deaf, autistic or have other intellectual disabilities. It also gained funding to develop two projects related to this type of education which are described in the financial section.

After the experience of working with this community, it is important to highlight that this technology has improved aspects related to the users’ psychomotor skills, their cognitive stimulation, improvements in social relations and well-being, increased self-esteem and the enhancement of each user’s own abilities in the musical project (Briceño et al., 2014).

This technology could benefit countries where there are inclusion programmes, which means that children with special needs are in the same classroom as students without them, embracing diversity. Unfortunately, teachers are not trained in functional diversity. Out of 194 countries analysed, there are only six (three of which are Latin American) which have inclusive education laws, catering for all students. They are Chile, Colombia, Italy, Luxembourg, Paraguay and Portugal (Llorente, 2020).

1.1.5 Movement

Soundcool goes beyond the finger-eye movement functions used by the majority of apps. The research group introduced improvements in the app which involve using it with body movements instead of controls on the telephone. This encourages users to move around and explore other virtual experiences (Flavián et al., 2018). This part of the app started with Kinect, but after production of the device was halted, it continued its transformation to augmented reality and has recently evolved to virtual reality (Neira, 1993), i.e. an immersive multi-sensory experience (Gigante, 1993).

The Kinect is a camera-based sensor, primarily used to directly control computer games through body movement. The Kinect tracks limb and body position without the need for handheld controllers or force platforms. The use of a depth sensor also enables the Kinect to capture three-dimensional movement patterns (Galna et al., 2015).

The Kinect was used to adapt Soundcool to functional diversity students, specifically blind students, enabling them to use the app (Herrero et al., 2018). More adaptations were made, such as accessories and other tools, so that other special needs users could access Soundcool. Unfortunately, production of the Kinect stopped (Willson, 2017) and another alternative had to be found.

The alternative found was the HoloLens, which is an untethered mixed reality device that delivers an immersive experience. Users can see the reality and space around them through the HoloLens glasses, but also can see and touch a hologram of the Soundcool controllers in the same space, enabling them to move around, dance, create and have fun with the app. The only problem of the HoloLens is the price tag of $3,500 per unit (Microsoft, 2020) which makes it unfeasible for commercial purposes.

Given the price of the HoloLens glasses, a cheaper option was found in virtual reality. Users do not see the space around them, they only see what the glasses show them, but this modification reduces the budget to €200 for the virtual reality device.

In spite of the mixed reality and virtual reality, these developments have not replaced the functions of Kinect in terms of usability for blind people. Nevertheless, it has opened the door to other areas such as games, dance, ballet and degenerative illnesses.

1.1.6 Performance Art

The app started as a technological complement for professional musicians, but the research group has continued to create different activities to extend the content and options available on the system. It has also released artistic works to give visibility to the app and open the doors to participation in music festivals, and artistic and technology shows, where artistic collectives come together and network.

The most significant production is the multimedia opera La Mare dels Peixos (The Mother of Fishes), a free adaptation of a story by Enric Valor, with music by Jorge Sastre and Roger Dannenberg. Soundcool is ideal for music schools because it can be played by an orchestra of young musicians and singers, with the help of less than 10 professional musicians, which makes the performance affordable and interesting. The Soundcool part can be played by new students because they do not need to read music or play an instrument. The set is very simple and choreographies can be created by a group of children. This idea enabled a large number of students to participate in the musical show which was performed at the Palau de les Arts in Valencia (Spain) twice, in Mexico City with the Monterrey Institute of Technology and in Pittsburgh (USA) in 2020.

In the professional field, Soundcool has been used in festivals such as the Sonar Electronic Music Festival in Barcelona (Spain) and the Kikk Festival in Namur (Belgium). Other works have been presented, such as Floating in the Deep Blue for percussion and Soundcool live electronics and narrator.

An event that gave Soundcool a lot of visibility among artists was El Guerrer de la Valltorta, a circus showpiece that had an enormous band of musicians and electronic effects which was created with the app using over 100 devices. This event brought 4,130 music students together from 193 musical societies who, besides taking part in the show, were introduced to the Soundcool system.

1.1.7 Neurodegenerative Illnesses

Soundcool’s new project seeks to work with people with neurodegenerative diseases and the team has already won funding for two projects to study the benefits of the system for this group (see Annex).

The first project consists of alternative therapies based on new audiovisual technologies for neurodegenerative diseases in times of social distancing due to COVID-19. Its aim is to promote preparatory actions to support the exploration and formulation of future research projects, looking for innovative scientific and technical memory through the UPV’s Fisabio and Polysabio programmes. The other project centres on new music and audiovisual technologies to treat neurodegenerative diseases. This project has received a grant to consolidate AICO research groups.

2 Sustainability of the System

Up until now, Soundcool has been a non-profit project. It has been forged in the UPV, thanks to the willingness and generosity of the teachers and students that have worked on the initiative. Soundcool’s most valuable asset is its highly qualified team, the majority of whom are teachers from the UPV and other universities that are part of the performing arts and technology research group. Most of the teachers are doctors in their field and contribute to the system in different ways. Some use the tool to improve the quality of their lessons, others write papers, others give ideas and time, leading to product innovations, some organise conferences that give visibility to the app, some bring contacts that can offer advice, and some let the group know about public tenders (see Annex).

For university lecturers, participating in this kind of projects brings a range of benefits including helping in the development of society, building credibility and prestige within the scientific community and society, and the chance to obtain funding for researchers. In addition, the UPV benefits by increasing its scores in world rankings, opting at possible subsidies for its research institutes, providing evidence of the productivity of the institution and future economic benefits (Lameda et al., 2015).

The UPV has scholarships for students that work in the research group or link their studies to benefit the group. These are available for graduate, Master and doctoral programmes. In graduate programmes, students carry out small projects and help in daily tasks; in the Masters’ programmes, they perform small projects, and the PhD students normally go the extra mile to improve the system or use it in a different way (Ospina, 2019).

Other sources to pay for the team come from public tender awards to spend on research projects (see Annex). The Spanish Youth Job Scheme helps young people to access the job market, the Spanish Ministry of Employment pays some young people to help research and business, and these kinds of projects enable Soundcool to pay developers (Spanish Ministry of Employment, 2020).

There are all kinds of public tender awards available (see Annex). Some consist of money to spend on the project, some offer opportunities to test the product, others help with visibility and advice from experts on the matter, but most of them provide tools to continue developing related projects and new ways to explore the tool.

3 Conclusions

This paper details the entire process followed to create the Soundcool app. It explains opportunities in different markets and economic fields such as education, music performances, artistic performances, dance and therapy for special needs people and patients with neurodegenerative diseases.

Soundcool has been a non-profit project since the beginning, but now has a tight budget and lack of staff. If the app is to continue in the future, it needs to incorporate a business model supported by clients, so it can offer more, improved content in the platform in the long term.

The system is trying to be sustainable by developing a business plan with the help of experts in the UPV. It has recently entered the Co-lab Las Naves initiative, which is an incubator programme that aims to help the system reach the next level and hopefully start to sell the product in the midterm. Soundcool has carried out market research, market analysis and financial analysis in the field of education, which is the most developed part of the app, and the results are very promising.