Abstract
Auditory recognition is a crucial cognitive process that relies on the organization of single elements over time. However, little is known about the spatiotemporal dynamics underlying the conscious recognition of auditory sequences varying in complexity. To study this, we asked 71 participants to learn and recognize simple tonal musical sequences and matched complex atonal sequences while their brain activity was recorded using magnetoencephalography (MEG). Results reveal qualitative changes in neural activity dependent on stimulus complexity: recognition of tonal sequences engages hippocampal and cingulate areas, whereas recognition of atonal sequences mainly activates the auditory processing network. Our findings reveal the involvement of a cortico-subcortical brain network for auditory recognition and support the idea that stimulus complexity qualitatively alters the neural pathways of recognition memory.
Similar content being viewed by others
Introduction
Encoding and recognizing sounds that are structurally complex is a cognitive challenge relying on neural mechanisms that are not yet fully elucidated. To memorize complex sound sequences, we likely depend on the temporal organization of a stimulus’ components and memory functions1.
Memory encoding takes place in the hippocampus2,3,4, whereas subsequent processes related to recognition memory are supported by a functional network of interconnected regions in the medial temporal lobe, including the hippocampus, insula, and inferior temporal cortex2,5,6. For memory consolidation, communication between hippocampal and neocortical areas is needed7,8,9. Much evidence comes from studies using static visual stimuli, such as pictures of objects, faces, or natural scenes10,11,12. However, information and meaning also unfold over time as the brain attempts to predict upcoming stimuli based on prior memory representations. Hence, to better understand memory recognition and its underlying fast brain dynamics, novel methods must be adopted that highlight the temporal properties of dynamic stimuli. This can be done by studying the neural activity underlying the processing of sound sequences that acquire meaning through their evolution over time, such as music13,14,15.
Few neuroscientific studies have explored the neural underpinnings of musical memory, as reviewed by Campo and Brattico16. For example, using functional resonance imaging (fMRI) and a naturalistic music listening paradigm, Alluri et al.17 investigated the neural correlates of music processing and reported activation of cognitive, motor, and limbic brain networks for the continuous processing of timbral, tonal, and rhythmic features. Subsequently, using the same stimuli, Burunat et al.18 reported the recruitment of memory-related and motor brain regions during the recognition of musical motifs. Despite their contributions, these studies fail to identify the fine-grained temporal mechanisms of sound encoding and memory processes.
More recently, we combined the high temporal resolution of magnetoencephalography (MEG) with the high spatial resolution of magnetic resonance imaging (MRI) to study music recognition. These studies accentuated the temporal involvement of a widespread cortico-subcortical brain network comprising the primary auditory cortex, superior temporal gyrus, frontal operculum, cingulate gyrus, orbitofrontal cortex, and hippocampus during recognition of auditory (musical) sequences19,20,21. Overall, these investigations have provided unique insight into the neural mechanisms underlying the recognition of temporal sequences. What remains to be addressed is how these mechanisms are modulated by stimulus complexity.
Here, we used melodic sequences, where meaning emerged from the sequential combination of individual tones over time21, and varied the tone distribution to obtain new, complex musical sequences. In this scenario, encoding and recognition of the musical sequences largely depend on the sequential order of the tones that comprise it. We first selected musical sequences based on the rules of tonality, which is the dominant musical system in Western popular music22. Second, by modifying the tone intervals (i.e., the distances between pitches) while keeping all other variables (e.g., rhythm, tempo, timbre) constant, we generated matched atonal musical sequences. The stimulus manipulation was based on previous literature, which reported that tonal rather than atonal musical sequences are overall easier to process23,24,25,26,27 and more appreciated by non-expert listeners27,28,29. Unlike tonal music, atonal music is characterized by the absence of a clear tonal center and hierarchical stability, which significantly reduces its predictive value and gives rise to increased prediction errors23,24,25,26,30. Thus, we expected that the alteration of tonal intervals would reduce the predictability of the atonal sequences, leading to increased difficulty to recognize them.
To summarize, in the current study we used MEG and a musical recognition task19,20,21 while participants listened to and recognized auditory (musical) sequences of varying complexity and aimed at describing the fine-grained spatiotemporal dynamics of auditory recognition memory. Following the previous studies17,18,19,20,21, we expected that the recognition of auditory sequences would activate a widespread brain network that includes both auditory (e.g., primary auditory cortex, superior temporal gyrus, Heschl’s gyrus, planum temporale, insula) and memory processing areas (e.g., hippocampus, cingulate gyrus). We further hypothesized that neural activity would be distributed along two main frequency bands that reflected the occurrence of two different cognitive processes: a slow frequency band related to the recognition of the full musical sequence in memory processing areas, and a faster frequency band associated with the processing of each individual tone of the musical sequence in auditory regions. More importantly, we hypothesized that, based on stimulus complexity, tonal music would be more efficiently processed than atonal music, which would be reflected in different behavioral responses and distinct neural pathways during recognition of tonal and atonal sequences.
Our behavioral and neural results showed clear differences between the recognition of tonal and atonal sequences. Source reconstruction analyses indicated different activation clusters for tonal and atonal sequences. Overall, the neural activity was stronger in memory processing areas for memorized tonal sequences and in auditory processing regions for memorized atonal sequences.
Results
Behavioral data
Participants performed an old/new auditory recognition task (Fig. 1). They first listened to a full musical piece (encoding) and subsequently identified which musical sequences were memorized or novel. During recognition, the response accuracy and reaction time of the participants were recorded using a joystick. These behavioral data were statistically analyzed to examine the differences between the four experimental conditions (memorized tonal sequences, novel tonal sequences, memorized atonal sequences, novel atonal sequences).
A one-way analysis of variance (ANOVA) showed that the differences in response accuracy were statistically significant, F(3, 280) = 6.87, p = 0.002. Post-hoc analyses indicated that the average number of correct responses was significantly lower for memorized atonal sequences (M = 30.98, SD = 5.46) than for novel atonal (M = 34.51, SD = 4.26, p < 0.001), memorized tonal (M = 34.34, SD = 5.95, p = 0.002) and novel tonal sequences (M = 34.41, SD = 6.04, p = 0.001). Figure 2a shows the differences between conditions using raincloud plots31.
Regarding the mean reaction time, there was a statistically significant difference between conditions as determined by one-way ANOVA, F(3, 280) = 4.94, p = 0.002. Post-hoc analyses revealed that the average reaction time was significantly lower for memorized tonal sequences (M = 1735.17, SD = 259.91) compared to memorized atonal (M = 1879.44, SD = 259.34, p = 0.005) and novel atonal sequences (M = 1873.78, SD = 250.48, p = 0.007), but not compared to novel tonal sequences (M = 1799.52, SD = 267.14, p = .450). Figure 2b displays the differences between conditions.
Numerical source data for behavioral responses is provided in Supplementary Data 1.
MEG sensor data
The MEG data (204 planar gradiometers and 102 magnetometers) were analyzed at the MEG sensor level, using the broadband signal. Although the emphasis of the study lays in identifying the brain areas involved in recognizing tonal versus atonal musical sequences, the MEG sensor data were examined to assess whether the neural signal was significantly different for memorized than for novel trials and thus would corroborate the results of previous studies19,20.
After averaging the epoched data of correct trials for each experimental condition and combining the planar gradiometers, paired-samples t tests were performed to identify which condition (memorized or novel) generated a stronger neural signal for each time sample and MEG sensor. Cluster-based MCS were then calculated to correct for multiple comparisons. This was performed independently for both tonal and atonal data (see Methods for details).
First, paired-samples t tests (α = 0.01) were calculated for the tonal data in the time interval 0–2500 ms (from the onset of the trial) using combined planar gradiometers as these sensors are less affected by external noise than magnetometers32. Next, multiple comparisons were corrected by using cluster-based MCS on the significant t tests’ results (α = 0.001, 1000 permutations). Three main significant clusters of activity were identified in three specific time intervals when contrasting memorized versus novel sequences, as reported in Table 1, Supplementary Fig. 1, and Supplementary Data 2. Additionally, two main significant clusters of activity were detected when contrasting novel versus memorized sequences (Table 1, Supplementary Fig. 2, and Supplementary Data 2).
Regarding the atonal data, paired-samples t tests (α = 0.01) were calculated in the same time interval (0–2500 ms) using combined planar gradiometers. Next, multiple comparisons were corrected for using MCS on the significant t tests’ results (α = 0.001, 1000 permutations). This procedure identified three main significant clusters of activation when contrasting memorized versus novel excerpts (Table 2, Supplementary Fig. 3, and Supplementary Data 2). In the case of the novel versus memorized contrast, three main significant clusters of activity were found (Table 2, Supplementary Fig. 4, and Supplementary Data 2).
Source reconstruction
After examining the strength of the neural signals at the MEG sensor level, we focused on the main aim of the study, namely to investigate the neural differences underlying the recognition of tonal versus atonal musical sequences in MEG reconstructed source space. To perform this analysis, we localized the brain sources of the event-related fields recorded by the MEG channels. This was performed for both the tonal and atonal sequences and for two frequency bands (slow [0.1–1 Hz] and faster [2–8 Hz]) that were used in Bonetti et al.19,20 and linked to the processing of single components (slow) relative to a whole sequence (fast). Additional analyses were computed to contrast the brain activity between the slow and fast frequency bands and to examine the pattern of activity at the standard delta (1–4 Hz) and theta (5–8 Hz) frequency bands.
Slow frequency band (0.1–1 Hz)
The neural sources were calculated using a beamformer approach. First, a forward model was computed by considering each brain source as an active dipole and calculating its strength across the MEG sensors. Second, a beamforming algorithm was used as an inverse model to estimate the brain sources of neural activity based on the MEG recordings.
After computing the neural sources, a GLM was calculated at each timepoint and dipole location. A series of t tests (α = 0.05) was carried out at the first and group levels to estimate the main effect of memorized and novel conditions and their contrast for both the tonal and atonal data independently. Cluster-based MCS (α = 0.001, 1000 permutations) were computed to correct for multiple comparisons and to determine the brain activity underlying the development of the musical sequences. These analyses were carried out for five specific time intervals that corresponded to each of the tones comprising the sequences: first tone (0–250 ms), second tone (251–500 ms), third tone (501–750 ms), fourth tone (751–1000 ms), and fifth tone (1001–1250 ms). This was estimated for the memorized versus novel contrast for both tonal and atonal sequences independently and for memorized tonal versus memorized atonal sequences.
Significant clusters of activity (p < 0.001) were located across a number of brain voxels (k) for each tone of the tonal sequences, as reported in Supplementary Data 3. For memorized tonal sequences, the neural activity was overall stronger for the third (k = 69), fourth (k = 266), and fifth tones (k = 229). The largest differences were localized in the middle cingulate gyrus, right supplementary motor area, precuneus, and left lingual gyrus for the third tone; the left amygdala, left parahippocampal gyrus, left lingual gyrus, left hippocampus, and middle cingulate gyrus for the fourth tone, and the anterior and middle cingulate gyrus and left lingual gyrus for the last tone. For novel tonal sequences, the brain activity was stronger for the first (k = 54) and second tones (k = 29). In particular, the difference between novel and memorized sequences was strongest in the left calcarine fissure, left lingual gyrus, left hippocampus, left precuneus, and left superior temporal gyrus for the first tone, and the right fusiform gyrus, right lingual gyrus, and right inferior occipital gyrus for the second tone. The contrast between memorized and novel tonal sequences for the slow band is depicted in Fig. 3a.
In the case of atonal sequences, significant activity clusters were found primarily in auditory and medial temporal lobe regions in the right hemisphere for memorized sequences, and the neural activity was stronger for memorized than novel sequences across all five tones (k1 = 132, k2 = 163, k3 = 130, k4 = 140, k5 = 64), as reported in Supplementary Data 3. In particular, the brain activity was strongest in the right Rolandic operculum, right superior temporal gyrus, right Heschl’s gyrus, right supramarginal gyrus, and right insula for the first tone; the right Heschl’s gyrus, right superior temporal gyrus, right Rolandic operculum, right middle temporal gyrus, and right insula for the second tone; the right putamen, right insula, right Rolandic operculum, right Heschl’s gyrus, and right thalamus for the third tone; the parahippocampal gyrus, right fusiform gyrus, right hippocampus, and putamen for the fourth tone; and the anterior cingulate cortex, middle frontal gyrus, and caudate nucleus for the last tone. No significant clusters of activity were located in the slow band for novel atonal sequences. Figure 3b pictures the contrast between memorized and novel atonal sequences in the slow band.
Finally, to examine the differences during recognition of tonal versus atonal sequences, the brain activity during recognition of memorized tonal and memorized atonal sequences was contrasted. Significant activity clusters were located for both types of musical sequences across all tones (see Supplementary Data 3). For tonal sequences, the number of significant voxels was higher for the third (k = 70) and fifth tones (k = 79), whereas for atonal sequences the number of significant brain voxels was higher for the first (k = 98), second (k = 80), and fourth tones (k = 103). Overall, the brain activity was stronger in left cingulate and medial temporal lobe regions for tonal sequences, and in right auditory processing areas for atonal sequences. Specifically, in the case of memorized tonal sequences, the neural activity was localized in the supplementary motor area, left median cingulate gyrus, and superior frontal gyrus for the third tone, and the left hippocampus, left superior temporal gyrus, left thalamus, left insula, left putamen, and left parahippocampal gyrus for the fifth tone. For memorized atonal sequences, the neural activity was localized in the left lingual gyrus, left precuneus, left calcarine fissure, middle temporal gyrus, and right insula at the first tone; the inferior frontal gyrus, right precentral gyrus, right Rolandic operculum, and right superior temporal gyrus for the second tone; the right Rolandic operculum, right middle frontal gyrus, right postcentral gyrus, right putamen, and right insula for the fourth tone; and the right middle frontal gyrus, right angular gyrus, and right thalamus for the last tone. The contrast between memorized tonal and atonal sequences in the slow band is shown in Fig. 3c.
Faster frequency band (2–8 Hz)
The same procedure was carried out for assessing the brain activity underlying the recognition of musical sequences in the faster frequency band (2–8 Hz). Once the GLM was computed, cluster-based MCS (α = 0.001, 1000 permutations) were calculated for five time intervals corresponding to each of the five tones that formed the sequence. Again, this was estimated for the memorized versus novel contrast for both tonal and atonal sequences and memorized tonal versus memorized atonal sequences.
Regarding the contrast for tonal sequences, significant clusters of activity (p < 0.001) were located in multiple brain voxels for both memorized and novel sequences, as reported in Supplementary Data 3. For memorized tonal sequences, the neural activity was overall stronger for the first tone (k = 74), whereas it was stronger for novel tonal sequences for the second (k = 36), third (k = 200), fourth (k = 196), and fifth tones (k = 70). The brain activity was localized in the right Rolandic operculum, right insula, right Heschl’s gyrus, and right superior temporal gyrus for the first tone for memorized tonal sequences. For novel tonal sequences, the main active areas were the left superior temporal gyrus, insula, Heschl’s gyrus, and left hippocampus for the second tone; Heschl’s gyrus, superior temporal gyrus, insula, and putamen (k = 11) for the third tone; right Heschl’s gyrus, right insula, right Rolandic operculum, and right superior temporal gyrus for the fourth tone; and the right Rolandic operculum, right Heschl’s gyrus, right hippocampus, and right thalamus for the fifth tone. Figure 4a displays the contrast between memorized and novel tonal sequences.
For the contrast between memorized and novel atonal sequences, the majority of significant clusters of activity were localized at the first (k = 166) and second tones (k = 104) for memorized sequences, and at the third (k = 189), fourth (k = 118), and fifth tones (k = 156) for novel sequences, as reported in Supplementary Data 3. For memorized atonal sequences, the neural activity was strongest at the calcarine fissure, lingual gyrus, and right Rolandic operculum for the first tone, and the cingulate gyrus, right supplementary motor area, and superior frontal gyrus for the second tone. For novel atonal sequences, the neural activity was strongest at the insula, putamen, superior temporal gyrus, and Heschl’s gyrus for the third tone; the right insula, right putamen, right Heschl’s gyrus, right Rolandic operculum, and right superior temporal gyrus for the fourth tone; and the right insula, right Rolandic operculum, right Heschl’s gyrus, right putamen, and right superior temporal gyrus for the fifth tone. The contrast between memorized and novel atonal sequences for the faster band is shown in Fig. 4b.
Finally, the significant clusters of activity in the tonal versus atonal contrast for the faster band are reported in Supplementary Data 3. In the case of memorized tonal sequences, the number of significant brian voxels was higher for the first (k = 20), second (k = 44), fourth (k = 45), and fifth tones (k = 71). The neural activity was located in the right inferior frontal gyrus and right middle frontal gyrus at the first tone; the right middle temporal gyrus, right inferior parietal gyrus, right angular gyrus, middle occipital gyrus, and left superior occipital gyrus at the second tone; the frontal gyrus for the fourth tone; and the right middle occipital gyrus, right frontal gyrus, and right middle temporal gyrus for the fifth tone. For memorized atonal sequences, the number of significant voxels was higher at the third tone (k = 38) and the neural activity was mainly localized in the left inferior frontal gyrus, left middle frontal gyrus, left supramarginal gyrus, and right supplementary motor area. Figure 4c shows the contrast between tonal and atonal sequences for the fast band.
Additional analyses were computed to calculate the difference in neural activity between the slow and faster frequency bands. The results are reported in Supplementary Fig. 5 and Supplementary Data 4.
Altogether, we found significant differences between tonal and atonal sequences, especially for the slow frequency band. Recognition of memorized tonal sequences elicited stronger neural activity in left medial temporal lobe and cingulate regions in the last three tones of the sequences, whereas recognition of memorized atonal sequences was supported by activation in right auditory regions from the second tone onwards.
Delta (1–4 Hz) and theta (5–8 Hz) frequency bands
Finally, to obtain a complete picture of the brain activity underlying memory recognition of auditory sequences and to make our results comparable with previous literature, additional analyses were performed in two standard frequency bands: delta (1–4 Hz) and theta (5–8 Hz). The same procedure was carried out for assessing the brain activity underlying the recognition of musical sequences in each frequency band independently. Once the GLM was computed, cluster-based MCS (α = 0.001, 1000 permutations) were calculated for five time intervals, corresponding to each of the five tones that formed a sequence. Again, this was estimated for the memorized versus novel contrast for both tonal and atonal sequences. The significant clusters of activity for the 1–4 Hz and 5–8 Hz frequency bands are reported in Supplementary Data 5.
For the delta band, significant clusters of activity (p < 0.001) were located across a number of brain voxels (k) for each tone of the tonal sequences. For memorized tonal sequences, the neural activity was overall stronger at the first (k = 39), third (k = 72), and fourth tones (k = 92). The largest differences were localized in the left putamen, left lingual gyrus, left inferior frontal operculum, and right Heschl’s gyrus. For novel tonal sequences, the brain activity was stronger at the second tone (k = 129). In particular, the difference between novel and memorized sequences was strongest in the left superior temporal gyrus, right insula, and right putamen. For memorized atonal sequences, the neural activity was stronger at the first (k = 149), second (k = 167), and third (k = 50) tones. This activity was strongest in the right precuneus and right calcarine fissure. For novel atonal sequences, the fourth (k = 55) and fifth (k = 41) tones elicited more activity, particularly in the right Heschl’s gyrus and right Rolandic operculum. The contrast for the 1–4 Hz band is shown in Supplementary Fig. 6.
In the case of the theta band, we found significant activity at the first (k = 47) tone of memorized tonal sequences, namely in the left inferior frontal gyrus and right Heschl’s gyrus. The neural activity was stronger for novel tonal sequences at the second (k = 13), third (k = 89), fourth (k = 19), and fifth (k = 161) tones. The largest differences were localized in the right inferior frontal gyrus, left caudate, and right insula. For memorized atonal sequences, the neural activity was stronger at the first (k = 129), second (k = 167), third (k = 50) and fourth (k = 93) tones. This activity was strongest at the left calcarine fissure and right precuneus. For novel atonal sequences, the neural activity was stronger at the fifth tone (k = 33), particularly in the right insula and right putamen. The contrast for the 1–4 Hz band is shown in Supplementary Fig. 7.
Correlation with familiarity ratings
Data from the familiarity ratings regarding the tonal piece were correlated to the source analysis on the recognition of memorized tonal sequences to examine whether familiarity with the piece could predict neural activity during recognition.
Minor (t value <0.4) but significant positive correlations were found between familiarity ratings and activation in the left middle temporal gyrus and left middle occipital gyrus for the last three tones of the memorized tonal sequences. Detailed statistics and a graphical depiction of the results are reported in supplementary materials (Supplementary Data 6 and Supplementary Fig. 8).
Discussion
This study set out to investigate the brain activation underlying the recognition of auditory musical sequences characterized by different levels of complexity (tonal and atonal). Behavioral data showed clear differences between the recognition of tonal and atonal sequences and significant clusters of activation were observed at the MEG sensor level. Source reconstruction analyses indicated different activation clusters for tonal and atonal sequences, particularly in the slow frequency band. Overall, the neural activity was stronger in memory processing areas for memorized tonal sequences and in auditory processing regions for memorized atonal sequences.
Prior to focusing on the differences in brain activity related to distinct levels of recognition complexity, we verified that the current results were consistent with previous studies. Indeed, the brain areas activated during the recognition of tonal sequences confirmed the involvement of a widespread brain network including both auditory and memory processing regions17,18,19,21,33,34. Specifically, we observed increased activation in auditory regions, such as the superior temporal gyrus35 and Heschl’s gyrus36, and in the cingulate gyrus37 and medial temporal lobe structures (hippocampus, parahippocampal cortex) associated with memory recognition10,38,39. Furthermore, in accordance with previous research, the neural activity was clearly distributed in two frequency bands19,20. The slow band (0.1–1 Hz) was linked to the recognition of the whole auditory sequences (global processing), which was reflected by the stronger activation occurring in this band for the recognition of the memorized sequences. Conversely, the faster band (2–8 Hz) was associated with the processing of the individual tones (local processing), as suggested by the stronger neural activity in auditory regions during processing of novel sequences.
Regarding the recognition of tonal and atonal sequences, we observed distinct neural pathways when processing and recognizing the two types of auditory stimuli. While the recognition of tonal sequences mainly recruited a widespread brain network involving cingulate gyrus and hippocampus in the right hemisphere, the recognition of atonal sequences was mainly associated with a sustained, slow activity in the left auditory cortex. These results can be interpreted in light of different theoretical frameworks, namely predictive coding of music40,41, harmonicity42, global neuronal workspace43,44, and temporo-spatial theory of consciousness45,46.
According to the predictive coding of music theory, the brain’s predictive model is being continuously updated while listening to music to decrease precision-weighted prediction errors40,47,48. Specifically, bottom-up sensations evoked by auditory stimuli are processed in primary cortices and contrasted with top–down predictions in higher-order cortices to generate musical expectations and minimize hierarchical prediction errors48,49,50. Predictive mechanisms rely on long- and short-term memory functions, familiarity, and listening strategies to create musical expectations48. This theory provides a framework for studying music perception51,52,53,54, training55,56, action57,58, synchronization59,60,61, and emotion62,63,64.
Previous research has shown the predictive value of atonal music is weaker than tonal music, increasing its complexity and prediction errors23,24,25,26,27,30. Additionally, this change in stimulus predictability undermines the processing23,24,25,26 and enjoyment27,28,29 of atonal music. This was apparent when examining the behavioral results of the current study, as memorized atonal sequences were more slowly and less accurately recognized. Furthermore, the distribution of the neural activity in two frequency bands suggests a combination of top–down predictions in the slow band, which is related to the recognition of memorized sequences, and bottom-up predictions in the faster band, as the prediction error increases with novel sequences. It is important to note that, while results can be interpreted in the framework of predictive coding, the current study did not aim to test the predictive coding of music theory (e.g., by measuring the prediction error generated during recognition of the musical sequences). Therefore, future studies could replicate and expand these results within the predictive coding of music framework.
An alternative explanation for these results focuses on the harmonicity of auditory stimuli. Tonal music has been closely linked to the harmonic series, a natural sequence of sound frequencies that are integer multiples of a fundamental. Environmental sounds are typically nonharmonic, whereas both human and animal vocalizations contain harmonic structures42. The tonotopic organization of the human auditory cortex is particularly sensitive to harmonic tones, suggesting that this region developed to process harmonics due to their relevance for social communication65,66. These results indicate that distinct neural pathways are activated when recognizing auditory stimuli that are not coherent with the natural harmonic series and thus arguably more complex to process. Indeed, we found that for memorized tonal sequences, the brain activity was primarily located at the cortico-hippocampal network in the right hemisphere, and for memorized atonal sequences the auditory network in the left hemisphere. As mentioned above, atonal music is more difficult to process23,24,25,26,27 and less appreciated27,28,29 than tonal music. Thus, to recognize atonal sequences, auditory regions might require additional processing of the sounds to extract the relevant information carried by the first tones of the sequences. Then, this information might be broadcasted to high-order areas for the recognition of the sequences. One possible approach to further test the harmonicity hypothesis in relation to sequence recognition would be to create a collection of pieces that are systematically varied in terms of their similarity to the natural harmonic series. Future studies are called to investigate such perspectives.
The current results are also consistent with the global neuronal workspace hypothesis43,44,67,68. According to this theory, stimuli become conscious when they ignite late, high-order regions in response to the activation of sensory cortices involved in perceptual representation. Conversely, unconscious information does not reach high-processing brain areas and neural activity is limited to sensory cortices44,67,69. Importantly, we found that tonal sequences induced a late and robust activation of memory processing regions. Although it is unclear why atonal sequences were differently processed by the brain, we can confirm that the complexity of the stimuli modulates the transition from primary sensory areas to the global neuronal workspace, providing additional information to this comprehensive theoretical framework.
Finally, our results are in line with the temporo-spatial theory of consciousness, which focuses on four spatiotemporal mechanisms of the brain activity (expansion, globalization, alignment, and nestedness) to account for four corresponding consciousness dimensions (phenomenal content, access, form/structure, and level/state)45,46. In the current and previous studies19,20,70, we found that activity in the slow frequency band (0.1–1 Hz) was linked to memory recognition of auditory (musical) temporal sequences. These results support the theory’s notion of a temporal receptive window that allows for the integration of stimulus sequences into one cognitive unity in slow frequency bands45. In both tonal and atonal memorized sequences, we found stronger activation in memory processing regions in the slow frequency band.
Although the aim of this study was memory recognition of auditory sequences extracted from musical pieces of varying complexity, it would be interesting to examine the differences in brain activity between encoding and recognition of simple and complex auditory sequences. In the current study, the recognition task included 160 five-tone auditory sequences, whereas the complete musical pieces were played only four times in the encoding task, which prevents direct comparison between the two. In an attempt to link the encoding and recognition phases, we computed correlation analyses between the familiarity ratings provided by participants and the brain activity underlying the recognition of tonal memorized sequences. Greater familiarity with the tonal piece was correlated with increased activation in the middle temporal gyrus and middle occipital gyrus in the left hemisphere for the last three tones of the memorized tonal sequences. This result supports previous studies linking these two regions to recognition memory71 and suggests that deep encoding of music plays a role in the recognition process. Future studies should replicate and expand the current results by examining the neural correlates underlying not only memory recognition but also memory encoding of auditory sequences of varying complexity.
The current study provides valuable insights into the brain mechanisms underlying the recognition of auditory sequences. The results are consistent with those of previous studies and evidence the engagement of a large brain network that comprises both memory processing and auditory regions when recognizing music. Results further highlight the importance of stimulus complexity for the processing of temporal sequences and hint that the brain employs different strategies to account for this complexity.
Methods
Participants
The participant sample consisted of 71 volunteers (38 males and 33 females) aged 18 to 42 years old (mean age: 25 ± 4.10 years). All participants were healthy and reported normal hearing. Participants were recruited in Denmark and came from Western countries with matching socioeconomic and educational backgrounds.
The project was approved by the Ethics Committee of the Central Denmark Region (De Videnskabsetiske Komitéer for Region Midtjylland, ref. 1-10-72-411-17). The experimental procedures were carried out in compliance with the Declaration of Helsinki—Ethical Principles for Medical Research. All participants gave informed consent before starting the experimental procedure.
Experimental stimuli and design
Two musical compositions were used in the experiment: the right-hand part of Johann Sebastian Bach’s Prelude No. 1 in C Major, BWV 846 (hereafter referred to as the tonal piece), and an atonal version of the prelude (hereafter referred to as the atonal piece). MIDI versions were created using Finale (MakeMusic, Boulder, CO) and both pieces lasted 2.5 min each, with the same duration for all tones. LB composed the atonal piece based on the tonal piece. In particular, new tones were assigned to each of the tones comprising Bach’s original prelude. These new tones were one or two semitones higher or lower than the original tones, and the same tone conversion was applied throughout the entire tonal piece to obtain the atonal piece (e.g., every C tone in the tonal piece was converted into a C sharp in the atonal piece). Thus, both compositions were identical in terms of the sequential presentation of the tones (i.e., if C was positioned as 1st, 7th, and 8th tone in the tonal piece, C sharp occupied the same positions in the atonal piece), their rhythmic pattern, dynamics, and duration. Thus, the crucial difference between the two pieces was that the tonal piece was in the key of C Major, whereas the atonal piece did not have a musical key. The first two bars of each piece are displayed in Fig. 1a, showing similarities and correspondence between the two pieces.
Forty musical excerpts (i.e., short melodies or sequences) were extracted from each of the pieces. All excerpts consisted of the first five notes of each bar and lasted for 1250 ms (250 ms per note). In addition, 40 new excerpts were created for each piece based on the original ones. These new sequences were matched to the original ones among several variables, to prevent potential confounds. Specifically, they were matched for rhythm, volume, timbre, tempo, meter, and tonality.
To quantify the differences in complexity between tonal and atonal sequences, we computed information content (IC) and entropy (H) values for each tone of the musical sequences using the Information Dynamics of Music (IDyOM) model56,72. IC and H provide an estimation of the predictability of each tone and uncertainty with which it can be predicted73. The IC represents the minimum number of bits required to encode \({e}_{i}\) and is described by the Eq. (1):
where \(p({e}_{i}|{e}_{\,\left(i-n\right)+1}^{i-1})\) is the probability of the event \({e}_{i}\) given a previous set of \({e}_{\,\left(i-n\right)+1}^{i-1}\) events.
The H gives a measure of certainty/uncertainty about an upcoming event given the previous set of \({e}_{\,\left(i-n\right)+1}^{i-1}\) events and is calculated by the Eq. (2):
Equation (2) shows that, if the probability of a given event \({e}_{i}\) is 1, the probability of the other events in \(A\) will be 0 and therefore \(H\) will be equal to 0 (maximum certainty). Conversely, if all the events are equally likely, \(H\) will be maximal (maximum uncertainty). Higher levels of IC and H indicate less predictability of an upcoming tone and therefore a higher level of complexity.
The \({{{{{\rm{IC}}}}}}\) and H of each note of the tonal and atonal sequences were calculated using a model trained on a corpus of prototypical Western musical pieces employed by IDyOM56. After obtaining the IC and H scores for the individual tones, we performed paired-samples t tests to compute the difference in IC and H between tonal and atonal sequences. There was a statistically significant difference in IC (t73 = −8.389, p < 0.001) and H (t73 = −3.374, p = 0.001) values between tonal and atonal sequences, meaning that atonal sequences (mean IC: 6.69 ± .86; mean H: 4.74 ± .17) are more complex than tonal ones (mean IC: 5.66 ± 0.83; mean H: 4.65 ± .11).
The stimuli were employed in an old/new auditory recognition paradigm that was administered to the participants while their brain activity was recorded using MEG (Fig. 1b). The paradigm consisted of two parts, encoding and recognition, and was performed twice, once for the tonal piece and once for the atonal piece. The order of tonal/atonal was counterbalanced across participants. During the encoding part, participants actively listened to four repetitions of the entire musical piece (tonal or atonal) and tried to memorize it as much as possible. Afterwards, they were presented with the 80 musical excerpts (40 memorized and 40 novel excerpts, randomly ordered) and stated whether the excerpts belonged to the piece they had previously listened to (memorized) or whether they were new excerpts (novel). Response accuracy and reaction time were recorded using a joystick.
Behavioral data analysis
The behavioral data were statistically analyzed to examine the differences between the four experimental conditions (memorized tonal sequences, novel tonal sequences, memorized atonal sequences, novel atonal sequences) in response accuracy and reaction times. Two ANOVAs were computed (α = 0.05). Post-hoc analyses with Bonferroni correction (adjusted α = 0.0125) were used to compare each pair of experimental conditions.
Neural data acquisition
The MEG recordings were acquired in a magnetically shielded room at Aarhus University Hospital (Denmark) with an Elekta Neuromag TRIUX MEG scanner with 306 channels (Elekta Neuromag, Helsinki, Finland). The data were recorded at a sampling rate of 1000 Hz with an analog filtering of 0.1–330 Hz. Before starting the recordings, the head shape of the participants and the position of four Head Position Indicator (HPI) coils with respect to three anatomical landmarks (nasion and fiducials) were registered using a 3D digitizer (Polhemus Fastrak, Colchester, VT, USA). This information was later used to co-register the MEG data with the MRI anatomical scans. During the MEG recordings, the HPI coils registered the continuous head localization, which was subsequently used for movement correction analyses. Additionally, two sets of bipolar electrodes were used to record eye movements and cardiac rhythm for later removing electrooculography (EOG) and electrocardiography (ECG) artifacts.
The MRI scans were recorded on a CE-approved 3 T Siemens MRI-scanner at Aarhus University Hospital (Denmark). The data were recorded using a structural T1 with a spatial resolution of 1.0 × 1.0 × 1.0 mm and the following sequence parameters: echo time (TE) = 2.96 ms, repetition time (TR) = 5000 ms, reconstructed matrix size = 256 × 256, bandwidth = 240 Hz/Px.
The MEG and MRI recordings were acquired in two separate sessions.
Neural data pre-processing
The raw MEG sensor data (204 planar gradiometers and 102 magnetometers) were first preprocessed by MaxFilter32 in order to suppress external interferences. The following MaxFilter parameters were applied: spatiotemporal signal space separation (SSS), movement compensation using cHPI coils (default step size: 10 ms), down-sample from 1000 Hz to 250 Hz, correlation limit of 0.98 between inner and outer subspaces used to reject overlapping intersecting inner/outer signals during spatiotemporal SSS. In addition, the data were corrected for head motion and downsampled to 250 Hz. The data were then converted into Statistical Parametric Mapping (SPM)74 format and analyzed in MATLAB (MathWorks, Natick, MA, USA) with the Oxford Centre for Human Brain Activity (OHBA) Software Library (OSL) (https://ohba-analysis.github.io/osl-docs/), a freely available software that builds upon Fieldtrip75, FSL76, and SPM toolboxes. The signal was high-pass filtered (0.1 Hz of cutoff) to remove external frequencies and a notch filter was subsequently applied (48–52 Hz) to correct for inferences of the electric current. The signal was further downsampled to 150 Hz and the continuous MEG data were visually inspected to remove artifacts using the OSLview tool. An independent component analysis (ICA) was performed to remove EOG and ECG components. After reconstructing the signal with the remaining components77, the data were epoched into 160 trials (80 excerpts from each musical piece). Each trial lasted 1350 ms (1250 ms plus 100 ms of baseline time) and further analyses were performed on correctly identified trials only (see Fig. 1c).
MEG sensor data analysis
The primary focus of this study was on detecting differences in the brain activity underlying the recognition of tonal versus atonal musical sequences. However, the data were first analyzed at the MEG sensor level to verify that the neural signal was stronger for memorized versus novel musical sequences. This first step was essential to replicate previous findings obtained using a very similar experimental setting and paradigm and thus assess the quality of our data19,20.
Following the pre-processing of the neural data, and in accordance with MEG analysis guidelines78, all trials belonging to one condition were averaged together. This procedure resulted in four mean trials: one for memorized trials and one for novel trials for each musical piece (i.e., memorized tonal, novel tonal, memorized atonal, novel atonal). Next, each pair of planar gradiometers was combined by a sum root square. Paired-samples t tests (α = 0.01) were then calculated to contrast the memorized and novel conditions for both the tonal and atonal pieces, independently. This was performed for each combined planar gradiometer and each timepoint in the time-range 0–2500 ms (from the onset of the first tone of the musical sequences) in order to determine which condition generated a stronger neural signal. The analyses were calculated for planar gradiometers, since these sensors are less affected by external noise and thus highly reliable when computing analyses at the MEG sensor level78,79,80,81. Multiple comparisons were corrected using cluster-based Monte Carlo simulations (MCS)82 (α = 0.001, 1000 permutations) on the significant t tests’ results. Specifically, for each timepoint, a 2D matrix was generated reproducing the spatial location of the MEG channels and the results of the t tests of each MEG channel binarized according to their p values (0 s for not significant tests and 1 s for significant tests [i.e., p < 0.01]). The elements of the resulting 3D matrix were then submitted to 1000 permutations. For each permutation, we identified the maximum cluster of permuted 1 s, and we built a reference distribution using the maximum cluster sizes detected for each of the 1000 permutations. Finally, the original clusters that had a larger size than 99.9% of the maximum cluster sizes of the permuted data were considered significant.
Source reconstruction
After examining the strength of the neural signals at the MEG sensor level, we focused on the main aim of the study, which was to investigate the neural differences underlying the recognition of tonal versus atonal musical sequences in MEG reconstructed source space. To perform this analysis, we localized the brain sources of the neural signal recorded in the MEG channels. This procedure required designing a forward model, computing the inverse solution (in this case, using a beamforming approach), and identifying the statistically significant brain sources underlying the recognition of tonal and atonal sequences and their contrasts over time. Figure 1d shows the graphical depiction of the source reconstruction analyses.
Before computing the source reconstruction algorithm, the continuous data were bandpass filtered into two frequency bands: a slow band (0.1–1 Hz) and a faster band (2–8 Hz). The data collected and analyzed were event-related fields, not induced responses or oscillations. These bands were selected based on prior findings by Bonetti et al.19,20 which suggest that event-related fields in the faster band were responsible for a sensorial elaboration of each object (tone) in a sequence, while event-related fields in the slow band accounted for the recognition of the whole temporal sequence. To make it comparable with previous literature, additional analyses were computed at the delta (1–4 Hz) and theta (5–8 Hz) frequency bands. Finally, a contrast between the slow and faster frequency bands was calculated.
First, an overlapping-spheres forward model was computed using an 8-mm grid. This theoretical head model considers each brain source as an active dipole and describes how the unitary strength of such a dipole is reflected across the MEG sensors83. Using the information collected with the 3D digitizer, the MEG data and individual T1-weighted images were co-registered and the forward model was subsequently computed. An MNI152-T1 template with 8-mm spatial resolution was used in four cases in which the individual anatomical scans were not available. Second, a beamforming algorithm was employed as the inverse model. This is one of the most widely used algorithms for estimating the brain sources from MEG channels’ data and consists of utilizing a different set of weights which are sequentially applied to the source locations (dipoles) for isolating the contribution of each source to the activity recorded by the MEG channels for each timepoint78,84,85.
After estimating the brain sources of the signal recorded on the MEG channels, a General Linear Model (GLM) was estimated sequentially for each timepoint at each dipole location. At the first level, the main effect of memorized and novel conditions, as well as their contrast, was computed independently for each participant. At the group level, t tests were carried out for each dipole location to obtain the main effect of tonal, atonal, and their contrast computed on all aggregated participants. The GLMs were estimated independently for both the tonal and atonal data and for both frequency bands.
Brain activity underlying the development of the musical sequences
To determine the temporal evolution of the brain activity underlying musical sequences’ recognition, cluster-based MCS were estimated for five specific time windows that corresponded to each of the five tones comprising the musical sequences. This procedure was carried out independently for both tonal and atonal data and for both frequency bands. Thus, ten cluster-based MCS were calculated for each musical piece (five tones x two frequency bands) on the results of the group-level analysis with an adjusted alpha level of 0.001 (α = 0.01/10 = 0.001). This procedure allowed detecting the spatial clusters of significant brain sources underlying the recognition of the tonal and atonal musical sequences. For each of the MCSs, the data were sub-averaged in the time-window of interest (e.g., the time-window for the first tone of the musical sequences) and then submitted to 1000 permutations to build a reference distribution of the maximum cluster sizes detected in the permuted data. Then, using the same procedure as with the MEG channels, the original cluster sizes were compared to the reference distribution and were considered significant if their size was bigger than 99.9% of the maximum cluster sizes of the permuted data.
Importantly, further analyses were conducted to assess the differences between tonal and atonal data when recognizing memorized trials for both the slow and fast frequency bands. For each participant, a t test (α = 0.01) was computed for each source location and for the five time windows corresponding to each musical tone, contrasting the brain activity underlying the recognition of tonal versus atonal music. Multiple comparisons were corrected for using cluster-based MCS, as described above. In this case, ten MCS (α = 0.001, 1000 permutations) were calculated on the significant t test results (five tones × two frequency ranges).
Correlation between familiarity ratings and brain activity underlying the development of the tonal sequences
We examined whether familiarity with J. S. Bach’s Prelude No. 1 in C Major, BWV 846 was correlated with the brain activity during recognition of the tonal sequences extracted from the piece. Outside the scanner, participants were asked to rate their familiarity with the tonal piece on a scale from 1 to 7 (1 = “I never heard it”, 2 = “I occasionally heard it”, 3 = “I sometimes listen to it”, 4 = “I usually listen to it”, 5 = “I played it”, 6 = “I played it in front of an audience”, 7 = “I taught it”).
We computed Pearson’s correlations between participants’ familiarity scores and the brain activity during recognition of each of the five tones comprising the tonal memorized sequences and corrected for multiple comparisons using cluster-based MCS. After obtaining the spatial clusters that significantly correlated (α = 0.05) with participants’ familiarity ratings, the data were sub-averaged in five time windows. The significant brain voxels were shuffled in space and the maximum cluster size was measured. Then, we built a reference distribution of the maximum cluster sizes and compared the original cluster sizes to the reference distribution. Clusters were considered significant only if their size was bigger than 95% of the maximum cluster sizes of the permuted data.
Statistics and reproducibility
The statistical analyses for assessing IC and H levels of the stimuli employed paired-sample t tests. The behavioral data were analyzed using two one-way ANOVAs. The MEG sensor data were analyzed using paired-sample t tests and corrected for multiple comparisons using cluster-based MCS. The first (subject) level analysis for the MEG source data was calculated using a GLM. The group level was computed using paired-sample t tests and corrected for multiple comparisons with cluster-based MCS. The relationship between brain data and previous familiarity with the music was assessed using Pearson’s correlations and corrected for multiple comparisons through cluster-based MCS. Details of these procedures are extensively reported throughout the Methods section. The analyses involved 71 participants.
Reporting summary
Further information on research design is available in the Nature Research Reporting Summary linked to this article.
Data availability
The neuroimaging data that support the findings of this study have been deposited in Zenodo: https://zenodo.org/record/7249065#.Y1fuES8RppR.
Code availability
The codes used for the analyses reported in this study are available on GitHub: https://zenodo.org/badge/latestdoi/560818133.
References
Gabrieli, J. D. Cognitive neuroscience of human memory. Annu. Rev. Psychol. 49, 87–115 (1998).
Daumas, S., Halley, H., Frances, B. & Lassalle, J. M. Encoding, consolidation, and retrieval of contextual memory: differential involvement of dorsal CA3 and CA1 hippocampal subregions. Learn. Mem. 12, 375–382 (2005).
Greicius, M. D. et al. Regional analysis of hippocampal activation during memory encoding and retrieval: fMRI study. Hippocampus 13, 164–174 (2003).
Meltzer, J. A. & Constable, R. T. Activation of human hippocampal formation reflects success in both encoding and cued recall of paired associates. NeuroImage 24, 384–397 (2005).
Bird, C. M. The role of the hippocampus in recognition memory. Cortex 93, 155–165 (2017).
Eichenbaum, H., Yonelinas, A. P. & Ranganath, C. The medial temporal lobe and recognition memory. Annu. Rev. Neurosci. 30, 123–152 (2007).
Wiltgen, B. J., Brown, R. A., Talton, L. E. & Silva, A. J. New circuits for old memories: the role of the neocortex in consolidation. Neuron 44, 101–108 (2004).
van Kesteren, M. T., Fernandez, G., Norris, D. G. & Hermans, E. J. Persistent schema-dependent hippocampal-neocortical connectivity during memory encoding and postencoding rest in humans. Proc. Natl Acad. Sci. USA 107, 7550–7555 (2010).
Mehta, M. R. Cortico-hippocampal interaction during up-down states and memory consolidation. Nat. Neurosci. 10, 13–15 (2007).
Aggleton, J. P. & Brown, M. W. Interleaving brain systems for episodic and recognition memory. Trends Cogn. Sci. 10, 455–463 (2006).
DiCarlo, J. J., Zoccolan, D. & Rust, N. C. How does the brain solve visual object recognition. Neuron 73, 415–434 (2012).
Sato, W. & Yoshikawa, S. Recognition memory for faces and scenes. J. Gen. Psychol. 140, 1–15 (2013).
Dehaene, S., Meyniel, F., Wacongne, C., Wang, L. & Pallier, C. The neural representation of sequences: from transition probabilities to algebraic patterns and linguistic trees. Neuron 88, 2–19 (2015).
Kang, H., Agus, T. R. & Pressnitzer, D. Auditory memory for random time patterns. J. Acoust. Soc. Am. 142, 2219 (2017).
Peretz, I., & Zatorre, R. J. The cognitive neuroscience of music. 2003: OUP Oxford.
Campo, F., & Brattico, E. Remembering sounds in the brain: from locations findings to connectivity research (in press). Ital. J. Clin. Neurosci. (2022).
Alluri, V. et al. Large-scale brain networks emerge from dynamic processing of musical timbre, key and rhythm. NeuroImage 59, 3677–3689 (2012).
Burunat, I., Alluri, V., Toiviainen, P., Numminen, J. & Brattico, E. Dynamics of brain activity underlying working memory for music in a naturalistic condition. Cortex 57, 254–269 (2014).
Bonetti, L., et al. Spatiotemporal brain dynamics during recognition of the music of Johann Sebastian Bach. bioRxiv, https://www.biorxiv.org/content/10.1101/2020.06.23.165191v2 (2020).
Bonetti, L. et al. Brain recognition of previously learned versus novel temporal sequences: a differential simultaneous processing. Cer. Cort. bhac439, https://doi.org/10.1093/cercor/bhac439 (2022).
Bonetti, L. et al. Rapid encoding of musical tones discovered in whole-brain connectivity. NeuroImage 245, 118735 (2021).
Serra, J., Corral, A., Boguna, M., Haro, M. & Arcos, J. L. Measuring the evolution of contemporary western popular music. Sci. Rep. 2, 521 (2012).
Mencke, I. et al. Prediction under uncertainty: dissociating sensory from cognitive expectations in highly uncertain musical contexts. Brain Res. 1773, 147664 (2021).
Vuvan, D. T., Podolak, O. M. & Schmuckler, M. A. Memory for musical tones: the impact of tonality and the creation of false memories. Front. Psychol. 5, 582 (2014).
Ockelford, A. & Sergeant, D. Musical expectancy in atonal contexts: musicians’ perception of “antistructure”. Psychol. Music 41, 139–174 (2013).
Mencke, I., Omigie, D., Wald-Fuhrmann, M. & Brattico, E. Atonal music: can uncertainty lead to pleasure? Front. Neurosci. 12, 979 (2018).
Daynes, H. Listeners’ perceptual and emotional responses to tonal and atonal music. Psychol. Music 39, 468–502 (2011).
Nieminen, S., Istók, E., Brattico, E. & Tervaniemi, M. The development of the aesthetic experience of music: preference, emotions, and beauty. Musica. Sci. 16, 372–391 (2012).
Proverbio, A. M. et al. Non-expert listeners show decreased heart rate and increased blood pressure (fear bradycardia) in response to atonal music. Front. Psychol. 6, 1646 (2015).
Krumhansl, C. L. & Cuddy, L. L. A theory of tonal hierarchies in music, in Music perception. 51–87 (Springer, 2010).
Allen, M., Poggiali, D., Whitaker, K., Marshall, T. R., & Kievit, R. A. Raincloud plots: a multi-platform tool for robust data visualization. Wellcome Open Res. 4, 63 (2019).
Taulu, S. & Simola, J. Spatiotemporal signal space separation method for rejecting nearby interference in MEG measurements. Phys. Med. Biol. 51, 1759–1768 (2006).
Pando-Naude, V., Patyczek, A., Bonetti, L. & Vuust, P. An ALE meta-analytic review of top–down and bottom-up processing of music in the brain. Sci. Rep. 11, 1–15 (2021).
Criscuolo, A., Pando-Naude, V., Bonetti, L., Vuust, P. & Brattico, E. An ALE meta-analytic review of musical expertise. Sci. Rep. 12, 1–17 (2022).
Humphries, C., Liebenthal, E. & Binder, J. R. Tonotopic organization of human auditory cortex. NeuroImage 50, 1202–1211 (2010).
Warrier, C. et al. Relating structure to function: Heschl’s gyrus and acoustic processing. J. Neurosci. 29, 61–69 (2009).
Bubb, E. J., Kinnavane, L., & Aggleton, J. P. Hippocampal - diencephalic - cingulate networks for memory and emotion: an anatomical guide. Brain Neurosci. Adv. 1, https://doi.org/10.1177/2398212817723443 (2017).
Brown, M. W., Warburton, E. C. & Aggleton, J. P. Recognition memory: material, processes, and substrates. Hippocampus 20, 1228–1244 (2010).
Aggleton, J. P. & Brown, M. W. Episodic memory, amnesia, and the hippocampal-anterior thalamic axis. Behav. Brain Sci. 22, 425–444 (1999).
Vuust, P., Ostergaard, L., Pallesen, K. J., Bailey, C. & Roepstorff, A. Predictive coding of music–brain responses to rhythmic incongruity. Cortex 45, 80–92 (2009).
Vuust, P., Dietz, M. J., Witek, M., & Kringelbach, M. L. Now you hear it: a predictive coding model for understanding rhythmic incongruity. Ann. N. Y. Acad. Sci. https://doi.org/10.1111/nyas.13622 (2018).
Lewicki, M. S. Efficient coding of natural sounds. Nat. Neurosci. 5, 356–363 (2002).
Dehaene, S., Charles, L., King, J. R. & Marti, S. Toward a computational theory of conscious processing. Curr. Opin. Neurobiol. 25, 76–84 (2014).
Dehaene, S. & Changeux, J. P. Experimental and theoretical approaches to conscious processing. Neuron 70, 200–227 (2011).
Northoff, G. & Huang, Z. How do the brain’s time and space mediate consciousness and its different dimensions? Temporo-spatial theory of consciousness (TTC). Neurosci. Biobehav. Rev. 80, 630–645 (2017).
Northoff, G. & Zilio, F. Temporo-spatial Theory of Consciousness (TTC)–Bridging the gap of neuronal activity and phenomenal states. Behav. Brain Res. 424, 113788 (2022).
Vuust, P., Heggli, O. A., Friston, K. J., & Kringelbach, M. L. Music in the brain. Nat. Rev. https://doi.org/10.1038/s41583-022-00578-5 (2022).
Koelsch, S., Vuust, P. & Friston, K. Predictive processes and the peculiar case of music. Trends Cogn. Sci. 23, 63–77 (2019).
Schroger, E., Marzecova, A. & SanMiguel, I. Attention and prediction in human audition: a lesson from cognitive psychophysiology. Eur. J. Neurosci. 41, 641–664 (2015).
Rohrmeier, M. A. & Koelsch, S. Predictive information processing in music cognition. A critical review. Int. J. Psychophysiol. 83, 164–175 (2012).
Quiroga-Martinez, D. R. et al. Decomposing neural responses to melodic surprise in musicians and non-musicians: evidence for a hierarchy of predictions in the auditory system. NeuroImage 215, 116816 (2020).
Quiroga-Martinez, D. R. et al. Reduced prediction error responses in high-as compared to low-uncertainty musical contexts. Cortex 120, 181–200 (2019).
Lumaca, M., Trusbak Haumann, N., Brattico, E., Grube, M. & Vuust, P. Weighting of neural prediction error by rhythmic complexity: A predictive coding account using mismatch negativity. Eur. J. Neurosci. 49, 1597–1609 (2019).
Koelsch, S., Fritz, T., Schulze, K., Alsop, D. & Schlaug, G. Adults and children processing music: an fMRI study. NeuroImage 25, 1068–1076 (2005).
Putkinen, V., Tervaniemi, M. & Huotilainen, M. Musical playschool activities are linked to faster auditory development during preschool-age: a longitudinal ERP study. Sci. Rep. 9, 11310 (2019).
Pearce, M. T. Statistical learning and probabilistic prediction in music cognition: mechanisms of stylistic enculturation. Ann. N. Y. Acad. Sci. https://doi.org/10.1111/nyas.13654 (2018).
Witek, M. A., Clarke, E. F., Wallentin, M., Kringelbach, M. L. & Vuust, P. Syncopation, body-movement and pleasure in groove music. PloS One 9, e94446 (2014).
Matthews, T. E., Witek, M. A., Heggli, O. A., Penhune, V. B. & Vuust, P. The sensation of groove is affected by the interaction of rhythmic and harmonic complexity. PloS One 14, e0204539 (2019).
Konvalinka, I., Vuust, P., Roepstorff, A. & Frith, C. D. Follow you, follow me: continuous mutual prediction and adaptation in joint tapping. Q. J. Exp. Psychol. 63, 2220–2230 (2010).
Heggli, O. A., Konvalinka, I., Kringelbach, M. L. & Vuust, P. Musical interaction is influenced by underlying predictive models and musical expertise. Sci. Rep. 9, 1–13 (2019).
Pecenka, N. & Keller, P. E. The role of temporal prediction abilities in interpersonal sensorimotor synchronization. Exp. Brain Res. 211, 505–515 (2011).
Vuust, P. & Kringelbach, M. L. The pleasure of making sense of music. Interdiscip. Sci. Rev. 35, 166–182 (2010).
Gebauer, L., Kringelbach, M. L. & Vuust, P. Ever-changing cycles of musical pleasure: the role of dopamine and anticipation. Psychomusicology 22, 152 (2012).
Brattico, E. & Pearce, M. The neuroaesthetics of music. Psychol. Aesthet. Creat. Arts 7, 48 (2013).
Norman-Haignere, S. V., Kanwisher, N., McDermott, J. H. & Conway, B. R. Divergence in the functional organization of human and macaque auditory cortex revealed by fMRI responses to harmonic tones. Nat. Neurosci. 22, 1057–1060 (2019).
Hackett, T. A. Anatomic organization of the auditory cortex. Handb. Clin. Neurol. 129, 27–53 (2015).
Mashour, G. A., Roelfsema, P., Changeux, J.-P. & Dehaene, S. Conscious processing and the global neuronal workspace hypothesis. Neuron 105, 776–798 (2020).
Dehaene, S., Kerszberg, M. & Changeux, J.-P. A neuronal model of a global workspace in effortful cognitive tasks. Proc. Natl Acad. Sci. 95, 14529–14534 (1998).
Dehaene, S. & Naccache, L. Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework. Cognition 79, 1–37 (2001).
Fernández-Rubio, G., Carlomagno, F., Vuust, P., Kringelbach, M. L., & Bonetti, L. Associations between abstract working memory abilities and brain activity underlying long-term recognition of auditory sequences. PNAS Nexus 1, https://doi.org/10.1093/pnasnexus/pgac216 (2022).
Yonelinas, A. P., Hopfinger, J. B., Buonocore, M. H., Kroll, N. E. A. & Baynes, K. Hippocampal, parahippocampal and occipital-temporal contributions to associative and item recognition memory: an fMRI study. Neuroreport 12, 359–363 (2001).
Pearce, M. T. & Wiggins, G. A. Auditory expectation: the information dynamics of music perception and cognition. Top. Cogn. Sci. 4, 625–652 (2012).
Sears, D. R., Pearce, M. T., Caplin, W. E. & McAdams, S. Simulating melodic and harmonic expectations for tonal cadences using probabilistic models. J. N. Music Res. 47, 29–52 (2018).
Penny, W. D., Friston, K. J., Ashburner, J. T., Kiebel, S. J., & Nichols, T. E. Statistical parametric mapping: the analysis of functional brain images. 2011: Elsevier.
Oostenveld, R., Fries, P., Maris, E. & Schoffelen, J. M. FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput. Intell. Neurosci. 2011, 156869 (2011).
Woolrich, M. W. et al. Bayesian analysis of neuroimaging data in FSL. NeuroImage 45, S173–S186 (2009).
Mantini, D. et al. A signal-processing pipeline for magnetoencephalography resting-state networks. Brain Connectivity 1, 49–59 (2011).
Gross, J. et al. Good practice for conducting and reporting MEG research. NeuroImage 65, 349–363 (2013).
Bonetti, L., Haumann, N., Vuust, P., Kliuchko, M. & Brattico, E. Risk of depression enhances auditory Pitch discrimination in the brain as indexed by the mismatch negativity. Clin. Neurophysiol. 128, 1923–1936 (2017).
Bonetti, L. et al. Auditory sensory memory and working memory skills: association between frontal MMN and performance scores. Brain Res. 1700, 86–98 (2018).
Bonetti, L. et al. Brain predictive coding processes are associated to COMT gene Val158Met polymorphism. NeuroImage 233, 117954 (2021).
Kroese, D. P., Taimre, T., & Botev, Z. I. Handbook of Monte Carlo Methods. (John Willey & Sons. Inc., Hoboken, NJ, 2011).
Huang, M. X., Mosher, J. C. & Leahy, R. M. A sensor-weighted overlapping-sphere head model and exhaustive head model comparison for MEG. Phys. Med. Biol. 44, 423–440 (1999).
Brookes, M. J. et al. Beamformer reconstruction of correlated sources using a modified source model. NeuroImage 34, 1454–1465 (2007).
Hillebrand, A. & Barnes, G. R. Beamformer analysis of MEG data. Int. Rev. Neurobiol. 68, 149–171 (2005).
Acknowledgements
The Center for Music in the Brain (MIB) is funded by the Danish National Research Foundation (project number DNRF117). L.B. is supported by Carlsberg Foundation (CF20-0239), Lundbeck Foundation (Talent Prize 2022), Center for Music in the Brain, Linacre College of the University of Oxford, and Society for Education and Music Psychology (SEMPRE’s 50th Anniversary Awards Scheme). M.L.K. is supported by Center for Music in the Brain and Centre for Eudaimonia and Human Flourishing, which is funded by the Pettit and Carlsberg Foundations. Additionally, we thank the Fundación Mutua Madrileña for the economic support provided to the author G.F.R. and the University of Bologna for the economic support provided to student assistants Giulia Donati, Riccardo Proietti, and Giulio Carraturo. We thank Giulia Donati, Riccardo Proietti, Giulio Carraturo, Mick Holt, Holger Friis for their assistance in the neuroscientific experiment.
Author information
Authors and Affiliations
Contributions
L.B., E.B., M.L.K., and P.V. conceived the hypotheses, designed the study, and recruited the resources for the experiment. L.B. collected the data. L.B. and G.F.R. performed pre-processing and statistical analysis. E.B., S.A.K., L.B., M.L.K., and P.V. provided essential help to interpret and frame the results within the neuroscientific literature. G.F.R. and L.B. wrote the first draft of the manuscript and prepared the figures. All the authors contributed to and approved the final version of the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Fernández-Rubio, G., Brattico, E., Kotz, S.A. et al. Magnetoencephalography recordings reveal the spatiotemporal dynamics of recognition memory for complex versus simple auditory sequences. Commun Biol 5, 1272 (2022). https://doi.org/10.1038/s42003-022-04217-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s42003-022-04217-8
- Springer Nature Limited
This article is cited by
-
Age-related neural changes underlying long-term recognition of musical sequences
Communications Biology (2024)
-
Spatiotemporal brain hierarchies of auditory memory recognition and predictive coding
Nature Communications (2024)