Abstract
Optimal mental workload plays a key role in driving performance. Thus, driver-assisting systems that automatically adapt to a drivers current mental workload via brain–computer interfacing might greatly contribute to traffic safety. To design economic brain computer interfaces that do not compromise driver comfort, it is necessary to identify brain areas that are most sensitive to mental workload changes. In this study, we used functional near-infrared spectroscopy and subjective ratings to measure mental workload in two virtual driving environments with distinct demands. We found that demanding city environments induced both higher subjective workload ratings as well as higher bilateral middle frontal gyrus activation than less demanding country environments. A further analysis with higher spatial resolution revealed a center of activation in the right anterior dorsolateral prefrontal cortex. The area is highly involved in spatial working memory processing. Thus, a main component of drivers’ mental workload in complex surroundings might stem from the fact that large amounts of spatial information about the course of the road as well as other road users has to constantly be upheld, processed and updated. We propose that the right middle frontal gyrus might be a suitable region for the application of powerful small-area brain computer interfaces.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Introduction
Each year approximately 1.35 million people die because of road traffic accidents while another 20–50 million suffer from non-fatal injuries1. The vast majority of these accidents are caused by human error2,3,4. Many of these errors occur as a result of excessive demands due to complex traffic situations. To address this issue, more and more driver-assisting systems like advanced cruise and electronic stability control as well as lane and distance-keeping features have been included in cars over the last decades. Already in today’s cars, advanced driver-assisting systems allow to drive for hours without manual interference. In future conditionally automated cars will even allow the driver to ignore the traffic completely during automation and prompt the driver for takeover only in certain situations. It has however, been argued that both too high and too low levels of workload lead to a decline in driving performance5,6,7,8. Thus, until complete automation is achieved in driving, driver-assisting systems that automatically adept to the drivers’ current mental workload might provide the best improvements regarding traffic safety.
Mental workload has been defined as the portion of processing capacity and resources of an individual that a given task demands9,10. For example, in respect to estimate mental workload in driving, it is not sufficient to track the difficulty of the driving environment via indicators like acceleration, deceleration and steering wheel movement. The reason for this is that specific driving maneuvers may challenge the resources of individual drivers in a different manner depending on factors like driving experience, fatigue and present distractions. Thus, mental workload is typically measured with several questionnaires like the NASA TLX11 or the ISA12. However, especially during difficult tasks it is often not feasible to ask operators directly for their current workload because simultaneously having to make such estimations would impair task performance13. To bypass subjective ratings, researchers have tried to find objective correlates of mental workload. Besides unspecific peripheral physiological correlates like heart rate, heart rate variability, blood pressure, respiration, eye blinks and skin conductivity14 researchers have tried to pinpoint neuronal structures and processes directly contributing to the processing of demanding tasks.
Thus, there have been attempts to measure the neural correlates of mental workload in driving (-like) tasks with EEG15,16,17 and fMRI18,19. However, both fMRI and EEG are quite restrictive in regards to interference caused by participant’s movements and surrounding sources, which makes it hard to implement naturalistic settings in studies that make use of these techniques.
A relatively new method to measure cortical activity is functional near-infrared spectroscopy (fNIRS). fNIRS allows the indirect measurement of neuronal activity via, optically detected changes in oxygenated hemoglobin [oxyHB] and deoxygenated hemoglobin [deoxyHB] concentrations. Both a rise in concentration of oxygenated blood and a decline in concentration of deoxygenated blood can be regarded as the result of neuronal activity. In a laboratory setting, fNIRS-studies have mainly reported two regions that show workload-related activity changes in the n-back task, the ventrolateral prefrontal cortex (VLPFC)20,21 and the dorsolateral prefrontal cortex (DLPFC)21,22,23. fNIRS however does not suffer from the same restrictions as fMRI and EEG regarding movement or environmental interferences and thus as well has gained more and more popularity in driving research. In the last decade amongst others, the neuronal correlates of specific driving maneuvers24,25,26,27,28, drowsiness and fatigue29,30,31,32,33,34,35,36,37,38 habituation39 and frustration during driving40 have been examined with fNIRS. Further, several fNIRS studies have examined the neural correlates of mental workload during different driving operations. These studies in accord with laboratory paradigms have most prominently (though not exclusively) reported a rise of prefrontal activity with rising workload. This link has been reported for several isolated driving maneuvers (lateral prefrontal cortex41), secondary tasks during driving (DLPFC, inferior frontal gyrus/ IFG42,43), narrow vs. wide rode driving (DLPFC42) and driving with differing amounts of automation44,45,46.
To build driver-assisting systems that successfully adapt to the general demand imposed on a driver, rather than transient correlates of single driving maneuvers, it would be useful to find correlates of the general workload imposed on a driver by their environment. So far, not much research has been done in this field. One notable exception is a study by Foy and Chapman47 who designed a course comprising four different track types (arterial A-roads, city center multi-lane routes, suburban roads and dual carriageway) and found that prefrontal cortex activity significantly differed between all track types and rose with the mental demand each track type imposed. This study however did not further examine, which prefrontal structures underlie the reported changes. Additionally a rather small sample size of two course segments per track type was used, putting the generalizability of the results into question.
The present study
In the future, online measurements of neural activity during driving might become a useful tool to gain insight into the driver’s cognitive state. Such information, gathered with fNIRS or similar devices, could be used to automatically engage and disengage driver-assisting systems as required. As a necessary prerequisite however, we need to gain more insight into the neural correlates of driving-related cognitive processes. The goal of this study was to identify specific prefrontal structures activated by mental workload imposed on a driver by their surroundings (i.e. independent of specific driving maneuvers). To this end, we designed two sets of experimental tracks. Within each set, the courses were heterogeneous in regards to routing and thus all required different driving maneuvers. However, routes were designed to have a homogeneous difficulty level within each set. More specifically, we designed a set of country courses, which required standard driving maneuvers like accelerating, breaking, steering and making turns, and a set of city courses, which required the same basic driving maneuvers but additionally required much more stimulus processing, presumably more attention, as well as the regular updating of conceived action plans due to the more complex scenario. Additionally to these experimental tracks, we designed a straight transition track, serving as a conjunction in between each two consecutive experimental track and as a baseline for the change in neural activity in the experimental conditions. Following previous driving research as well as laboratory findings, we measured middle frontal gyrus (MFG) activity with fNIRS during driving. In a first analysis, like previous research, we comprised data in larger regions of interest (ROI), i.e. the left anterior DLPFC (laDLPFC), the left posterior DLPFC (lpDLPFC), the right anterior DLPFC (raDLPFC) and the right posterior DLPFC (rpDLPFC). Additionally (and contrary to most previous research), we also analyzed neural activity separately for each of the 18 channels of our optode mounting. This approach allows us to draw more precise conclusions about specific brain regions involved in driving. Firstly, it allows us to draw comparisons to high resolution neural laboratory research and in turn allows insight into specific processes of action control involved in driving. Secondly, identifying spatially confined brain areas with high sensitivity to mental workload in driving might aid in the development of economical, single-channel BCIs that could be used to control adaptive driver assisting systems in the future.
We hypothesized that both conceived mental workload, as well as MFG activity would be higher during city than during country tracks.
Results
Behavioral results
The medium duration it took participants to finish city courses was 80.05 s (SD = 20.50 s). The medium duration it took participants to finish country courses was 63.12 s (SD = 9.35 s). The medium duration it took participants to finish the transition track was 27.76 s (SD = 4.04 s).
Repeated measurement t-tests revealed significant differences in workload ratings between country and city tracks for both the ISA t(21) = 10.75, p < 0.001, dz = 2.32 (mean country = 1.62, SD = 0.36; mean city = 2.39, SD = 0.47) and the NASA TLX t(21) = 5.17, p < 0.001, dz = 1.09 (mean country = 7.26, SD = 2.01; mean city = 10.41, SD = 3.17). See Fig. 1A for behavioral results.
Neuronal results
All four regions of interest (ROI) showed significant differences between conditions in [deoxyHB] indicating higher activity during city courses than during country courses. No ROI showed significant differences between conditions in [oxyHB]. See Table 1 and Fig. 1B for ROI-wise hemodynamic results (see Supplementary Table S1 for the analysis of potential lateralization effects).
For channel-wise comparisons, 6 channels showed significant differences in [deoxyHB] between conditions. All 6 channels showed higher activation during city courses than during country courses. Activated structures included left MFG (lMFG, 2 channels), right MFG (rMFG, 3 channels) and right SFG (rMFG, 1 channel). No channel-wise differences between conditions were found in [oxyHB]. See Table 2 for significant channel-wise hemodynamic results and Supplementary Table S2 for full channel-wise hemodynamic results. See Fig. 1C for effect locations.
Discussion
In this study we used fNIRS to examine the frontal hemodynamics associated with different levels of mental workload during simulated driving. To this end, we designed two sets of courses. Both country and city courses required basic driving maneuvers (as accelerating, breaking, steering etc.). However, city courses due to more complex surroundings additionally required more attention and stimulus processing, as well as the regular updating of conceived action plans. Consequently, perceived mental workload, as measured by both the ISA12 and the NASA-TLX11, was substantially higher during city than during country courses. On a neural level, this effect was reflected by higher activity in all four examined ROIs as indicated by a greater decline in deoxyHB concentrations during city than during country courses in these regions. This finding mirrors the results of previous studies that found a relation between mental workload and prefrontal activity in a range of different driving scenarios like different levels of automated driving44,45,46,48, single driving maneuvers41 and longer courses47. To gain more insight into the spatial structure of the activation, we conducted a single channel analysis. Channel-wise analysis of neural data offers a significantly higher spatial resolution of neural activation patterns compared to the analysis of larger ROIs. The downside of this detailed analysis is that each result is based on less aggregated data and thus is more susceptible to the influence of artifacts and suffers from a worse signal-to-noise ratio. Nonetheless our analysis revealed six significant effects. Again, all effects indicated higher activity during city compared to country courses and solely presented in deoxyHB. A reason for this could be that fNIRS measurements of prefrontal oxyHB activity (in contrast to deoxyHB activity) are susceptible to systemic artifacts49, a problem that might be exacerbated by the motion heavy environment of simulated driving.
Effects were found in three underlying structures, the rMFG the lMFG and the rSFG. Most prior research to (simulated) driving did not engage in channel-wise analysis and thus no comparisons can be drawn. The finding does however coincides with fNIRS23 and fMRI50,51 laboratory findings regarding spatial working memory, which also most strongly activates rMFG/ rDLPFC regions. It stands to reason that spatial working memory plays an important role in maneuvering complex driving scenarios because drivers have to be aware of and integrate a multitude of fix and moving parts to derive operating action plans.
Implications for research and application
With modern fNIRS systems, it is easy to measure neural activity over large areas of the skull for research purposes. Using such large-area optode mountings, several brain computer interfacing studies have achieved promising classification accuracies37,48. However, future practical application will most likely have to make due with much more limited mountings as to not compromise driver comfort. While driving is a complex tasks, whereby many different brain regions work in accord24,25,27,28,52, it is vital to identify those brain regions most sensitive to changes in workload, to maximize the efficiency of brain computer interfaces. We argue that especially the rostral part of the rMFG might be a promising candidate to monitor mental workload during driving. In this region, we found three adjacent channels (AF8—FP2, AF8—F6 and F4—F6) that showed significant differences in activation related to track difficulty. Conceptually it makes sense that the rMFG is especially sensitive to high mental workload in driving. As argued above, the rMFG is closely related to spatial working memory51, which in turn is essential in successfully maneuvering difficult driving situations. To our knowledge, only two studies have explicitly examined a potential link between spatial working memory and driving related processes53,54. While in a study by Morris et al.53 verbal and spatial working memory load did not differentially influence driving performance, Gugerty54 found a link between spatial working memory load and attentional processes in driving. Future research should explore the link between mental workload in driving and spatial working memory load using neural imaging. A potential gain from this is twofold. Firstly, should a link between active neural processes during driving and spatial memory processing be established, spatial working memory paradigms could be used to economically train and test machine learning algorithms, before running elaborate field experiments. Secondly, if spatial working memory load indeed constitutes the main component of mental workload during driving, neural research could help in the development of adaptable visual guiding systems.
Beyond this, to further develop and successfully implement assisting systems that adapt to a drivers current needs, we generally have to develop a better understanding of the specific neural processes that underlie driving and how they are strained by environmental demands. While this study provides some preliminary results in this regard, further research is certainly necessary confirm and expand the presented results.
Conclusion
We used fNIRS to examine the neuro hemodynamic correlates of mental workload during simulated driving. In accordance to previous research, we found that prefrontal activation rose with workload. Beyond previous research, we determined the rostral part of the rMFG as a potential center of workload related activity. We propose that the rMFG is essential in maneuvering complex-driving scenarios in part, because it upholds and integrated spatial information of the environment.
Methods
Participants
Twenty-four participants completed the experiment (several more participated but had to abort the experiment early due to extensive simulator sickness). Two of these participants had to be excluded because of technical problems with the neuronal recoding. The final sample consisted of 22 participants (12 female, median age = 22 years, mean age = 22.23 years with a range of 19–31 years and a standard deviation of 2.72 years). All participants had a valid driver’s license for automobile, stated normal or corrected-to-normal vision and no participant stated any history of neurological disease or predisposition for motion or simulator sickness. Participants gave written informed consent to participation as well as publication of anonymized data before examination and received course credit for their participation. Additionally, the participant shown in Fig. 2A gave written informed consent for publication of an identifying image in an (online) open-access publication. The study was conducted in accordance with the Declaration of Helsinki. Furthermore, the local ethical review committee at the University of Trier evaluated and approved the study.
Design
The study had a one-factorial (track-type: country route, city route) within subject design.
Material
The dynamic driving simulator (FaSiMo) of Trier University of Applied Sciences was used to conduct the study (see Fig. 2A). The moving base provides six degrees of freedom (6-DOF), with system performance values as shown in Table 3.
The visual system comprises a 210° cylindrical projection screen, and three displays serving as rear view, and side mirrors, respectively. The cockpit provides a force feedback steering wheel, a driver seat (both electrically adjustable), a three-point seat belt, a digital display as dashboard, controls for headlights and blinkers, shift paddles, accelerator, and brake pedal. The audio system provides a 3D sound model with 5 satellite speakers plus a subwoofer based on dolby-digital 5.1. The control software stack comprises three logical layers: (a) traffic simulation, (b) driving dynamics, and (c) platform/mockup control, as shown in Fig. 2B.
Only adaptations and additions to the upper layer (SILAB) were necessary to facilitate the FaSiMo driving simulator for the conducted study. Besides the actual experimental tracks (see below), a coupling of the SILAB Software with the fNIRS system was developed, to guarantee a time synchronous logging of measured fNIRS data with waypoints. SILAB provides the possibility of street events ("Hedgehogs"), which can be placed at arbitrary points within lanes of simulated streets. Whenever the Ego Vehicle passes over these points, the associated event is triggered. Students of Trier University of Applied Sciences programmed an addition to SILAB to transmit the occurrence of designated "Hedgehogs" via an Arduino Board to the Trigger Input Interface of the used fNIRS System. Thereby, the fNIRS data records contain automatically inserted markers showing when these "Hedgehogs" were "killed".
To get used to the simulator environment and reduce the occurrence of simulator sickness, the participants had to absolve a set of three familiarization courses (provided by WIVW GmbH with the SILAB Software) before the actual experiments. The first course was a straight road, designed to test basic driving maneuvers like accelerating, breaking and swerving about. The second course comprised two sections. The first section prompted emergency breaking at 100 and 120 km/h. The second section prompted double lane changes to evade pylons at 30 and 50 km/h. The third course was a heavily frequented city route including several everyday traffic events. Further, 25 experimental tracks divided into three track types were constructed for/with SILAB (by WIVW GmbH, used Version was SILAB 5.1). These were 12 simple country courses with low traffic volume and without any scripted events, a straight transition track without traffic and a 50 km/h speed limit and 12 city courses with medium to high traffic volume, sidewalks populated with pedestrians and several every day traffic situations. These every day traffic situations included right of way situations, changing traffic lights, pedestrians at crosswalks (either crossing or standing indecisively at the roadside), a traffic circle, waiting school busses (with and without hazard lights), roadworks narrowing the street, a play street with a ball bouncing on the street, cars pulling out of a parking space right ahead of the driver, a motorcycle overtaking the driver, police cars with blue light at a crossing and a police car checking a vehicle at the roadside.
Two common workload measures, the ISA12 and the NASA RTLX11 were used to capture subjective workload during city and country courses. For the ISA participants were instructed to indicate subjective task-related workload on a scale from 1 to 5 (underutilized, relaxed, comfortable busy pace, high, excessive) using hand gestures (holding up one, two, three, four, five fingers). The ratings were later averaged for all courses of each track type. For the NASA RTLX, task related workload ratings regarding five dimensions (mental effort, physical effort, time pressure, performance, fatigue, frustration) on scales from 1 (minor) to 20 (high, inverted for performance) were averaged for each track type.
Procedure
After arrival, participants were informed about the general procedure and potential risks of the experiment, filled out the biographic questionnaire and signed consent. Participants then got an introduction regarding the operation of the driving simulator and relevant safety measures in the surroundings of the driver’s cabin. Subsequently they were seated in the simulator and drove the three familiarization courses.
After familiarization, participants were prepared for the fNIRS measurement and got instructions for the experimental courses. During the experiment, participants alternatingly drove country and city courses always divided by the transition track. They were instructed to give ISA difficulty-ratings for the prior city or country course during each transition track.
After the experimental courses, participants left the simulator and the fNIRS cap was removed. Then participants gave overall NASA RTLX ratings for both city and country courses and finally received course credit.
fNIRS measurement
Hemodynamic changes were recorded with an eight source, eight detector, portable, time-multiplexed, two wavelengths NIRSport (NIRx Medical Technologies LLC, USA) fNIRS device. Optodes were fixed in a standard 10–10 NIRScaps (NIRx Medical Technologies LLC, USA). The placement of fNIRS sources and detectors was chosen utilizing fNIRS Optodes' Location Decider (fOLDv2.2)55. fOLD is a Matlab (MathsWorks, USA) based toolbox which computes optimal optode placement in the 10–10 system in regards of covering specific brain areas. For optimal coverage of the MFG AF3, AF4, AF7, AF8, F3, F4, FC1 and FC2 were computed as source positions and FP1, FP2, F1, F2, F5, F6, FC3 and FC4 were computed as detector positions. This resulted in eighteen different channels fourteen of which most likely recorded the MFG while the remaining four most likely recorded the SFG (Fig. 2C depicts the fNIRS Montage, see Supplementary Table S3 for all channel positions). Signals were recorded with NIRStar (NIRx Medical Technologies LLC, USA) recording software with a frequency of 7.81 Hz.
fNIRS data preprocessing and analysis
NIRS Brain AnalyzIR Toolbox56 was used to preprocess and analyze neuro hemodynamic data. For preprocessing, raw voltage data was transformed into light-intensity data and subsequently used to calculate the relative concentration of oxygenated and deoxygenated hemoglobin via Beer–Lambert-Law57.Finally, to remove low-frequency characteristics and outliers, a wavelet-filter58 was applied. Preprocessed data was then entered into a two-level general linear model (GLM). The first level analysis included four predictors and was conducted for each subject separately. Two predictors coded city and country routes two additional predictors coded intermission tracks and driving mistakes. All predictors were derived from triggers set during the experiment. While triggers for city, country and intermission tracks were automatically set when participants crossed specific waypoints at the beginning of each course, triggers for driving mistakes were manually set by the experimenters. GLM predictors were generated by convolving each event with the canonical hemodynamic response function (HRF). To adapt modeling for individual differences in onset and dispersion of HRF we included the first and second temporal derivative of each prediction term. We corrected for serially autocorrelated errors as well as artifacts induced by systemic physiology and motion with a prewhitening algorithm (AR-IRLS59). The predictors for intermission tracks and driving mistakes were excluded from second level analysis, the beta values obtained for city and country courses for each subject were entered into a weighted mixed effects model estimating a fixed intercept for each experimental condition and a random intercept for each subject to best fit the overall data. Betas for each condition were compared for each channel via t-contrasts. Additionally we built four greater regions of interest (as described in56), laDLPFC (AF7–FP1, AF7–F5, AF3–FP1, AF3–F5), lpDLPFC (F3–F5, F3–F1, F3–FC3, FC1–F1, FC1–FC3), raDLPFC (AF8–FP2, AF8–F6, AF4–FP2, AF4–F6), rpDLPFC (F4–F6, F4–F2, F4–FC4, FC2–F2, FC2–FC4, see Fig. 2C). To account for alpha inflation due to multiple comparisons p values were corrected applying positive false discovery rate (FDR60). Only contrasts that yielded corrected p < 0.05 were regarded as statistically significant.
Data availability
Our data is publicly available via PsychArchives (https://doi.org/10.23668/psycharchives.4423).
Code availability
Our data is publicly available via PsychArchives (https://doi.org/10.23668/psycharchives.4424).
References
World Health Organization. Global status Report on Road Safety 2018 (World Health Organization, Geneva, 2018).
Ansari, S., Akhdar, F., Mandoorah, M. & Moutaery, K. Causes and effects of road traffic accidents in Saudi Arabia. Public Health 114(1), 37–39. https://doi.org/10.1038/sj.ph.1900610 (2000).
Sabey, B., & Staughton, E. (1975). Interacting roles of road environment, vehicle and road user in accidents. In 5th International Conference of the International Association for Accident and Traffic Medicine.
Treat, J. R. et al. Tri-Level Study of Causes of Traffic Accidents: Executive Summary (Institute For Research in Public Safety, Bloomington, 1979).
Pereira da Silva, F. Mental workload, task demand and driving performance: What relation?. Proced. Soc. Behav. Sci. 162, 310–319. https://doi.org/10.1016/j.sbspro.2014.12.212 (2014).
Young, M. S. & Stanton, N. A. Attention and automation: New perspectives on mental underload and performance. Theoret. Issues Ergonom. Sci. 3(2), 178–194. https://doi.org/10.1080/14639220210123789 (2002).
Young, M. S. & Stanton, N. A. Malleable attentional resources theory: A new explanation for the effects of mental underload on performance. Hum. Factors 44(3), 365–375. https://doi.org/10.1518/0018720024497709 (2002).
Young, M. S. & Stanton, N. A. What’s skill got to do with it? Vehicle automation and driver mental workload. Ergonomics 50(8), 1324–1339. https://doi.org/10.1080/00140130701318855 (2007).
Eggemeier, F. T., Wilson, G. F., Kramer, A. F. & Damos, D. L. Workload assessment in multi-task environments. In Multiple Task Performance (ed. Damos, D. L.) 207–216 (Taylor & Francis, London, 1991).
Kantowitz. , Defining and measuring pilot mental workload. In NASA Conference Publication: Vol 2504. Mental-State Estimation: Proceedings of a Workshop Sponsored by the National Aeronautics and Space Administration (ed. Comstock, J. R.) 179–188 (National Aeonautics and Space Administration, Scientific and Technical Information Division, Washington, 1987).
Hart, S. G. & Staveland, L. E. Development of NASA-TLX (task load index): Results of empirical and theoretical research. In Human Mental Workload Vol. 52 (eds Hancock, P. A. & Meshkati, N.) 139–183 (North Holland Press, Amsterdam, 1988). https://doi.org/10.1016/S0166-4115(08)62386-9.
Hulbert, T. A comparison of the “NASA-TLX” and “ISA” Subjective Workload Rating Techniques: Internal Report (Civil Aviation Authority Air Traffic Control Evaluation Unit, Bournemouth, 1989).
O'Donnell, R. D., & Eggemeier, F. T. (1986). Workload assessment methodology. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of Perception and Human Performance (Vol. 2, pp. 1–49).
Lean, Y. & Shan, F. Brief review on physiological and biochemical evaluations of human mental workload. Human Fact. Ergonom. Manuf. Serv. Ind. 22(3), 177–187. https://doi.org/10.1002/hfm.20269 (2012).
Kecklund, G. & Akerstedt, T. Sleepiness in long distance truck driving: An ambulatory EEG study of night driving. Ergonomics 36(9), 1007–1017. https://doi.org/10.1080/00140139308967973 (1993).
Lal, S. K. L. & Craig, A. Driver fatigue: Electroencephalography and psychological assessment. Psychophysiology 39(3), 313–321. https://doi.org/10.1017/s0048577201393095 (2002).
Wascher, E. et al. Evaluating mental load during realistic driving simulations by means of round the ear electrodes. Front. Neurosci. 13, 940. https://doi.org/10.3389/fnins.2019.00940 (2019).
Calhoun, V. D. et al. Different activation dynamics in multiple neural systems during simulated driving. Hum. Brain Mapp. 16(3), 158–167. https://doi.org/10.1002/hbm.10032 (2002).
Calhoun, V. D., Pekar, J. J. & Pearlson, G. D. Alcohol intoxication effects on simulated driving: Exploring alcohol-dose effects on brain activation using functional MRI. Neuropsychopharmacology 29(11), 2097–2107. https://doi.org/10.1038/sj.npp.1300543 (2004).
Fraser, S. A., Dupuy, O., Pouliot, P., Lesage, F. & Bherer, L. Comparable cerebral oxygenation patterns in younger and older adults during dual-task walking with increasing load. Front. Aging Neurosci. 8, 1–9. https://doi.org/10.3389/fnagi.2016.00240 (2016).
Koike, S. et al. Reduced but broader prefrontal activity in patients with schizophrenia during n-back working memory tasks: A multi-channel near-infrared spectroscopy study. J. Psychiatr. Res. 47(9), 1240–1246. https://doi.org/10.1016/j.jpsychires.2013.05.009 (2013).
Fishburn, F. A., Norr, M. E., Medvedev, A. V. & Vaidya, C. J. Sensitivity of fNIRS to cognitive state and load. Front. Human Neurosci. 8, 1–11. https://doi.org/10.3389/fnhum.2014.00076 (2014).
Geissler, C. F., Domes, G. & Frings, C. Shedding light on the frontal hemodynamics of spatial working memory using functional near-infrared spectroscopy. Neuropsychologia 146, 107570. https://doi.org/10.1016/j.neuropsychologia.2020.107570 (2020).
Nosrati, R., Vesely, K., Schweizer, T. A. & Toronov, V. Event-related changes of the prefrontal cortex oxygen delivery and metabolism during driving measured by hyperspectral fNIRS. Biomed. Opt. Express 7(4), 1323–1335. https://doi.org/10.1364/BOE.7.001323 (2016).
Oka, N. et al. Greater activity in the frontal cortex on left curves: A vector-based fNIRS study of left and right curve driving. PLoS One 10(5), 1–21. https://doi.org/10.1371/journal.pone.0127594 (2015).
Shimizu, S., Inoue, H., Nara, H., Takahashi, N., Miwakeichi, F., Hirai, N., Kikuchi, S., Watanabe, E., & Kato, S. (2013). Basic study for new assistive system based on brain activity during car driving. In L. Miller (Ed.), Achi 2013: The Sixth International Conference on Advances in Computer-Human Interations: February 24-March 1, 2013, Nice, France (pp. 466–471). IARIA.
Yoshino, K., Oka, N., Yamamoto, K., Takahashi, H. & Kato, T. Correlation of pre-frontal cortical activation with changing vehicle speeds in actual driving: A vector-based functional near-infrared spectroscopy study. Front. Human Neurosci. 7, 1–19. https://doi.org/10.3389/fnhum.2013.00895 (2013).
Yoshino, K., Oka, N., Yamamoto, K., Takahashi, H. & Kato, T. Functional brain imaging using near-infrared spectroscopy during actual driving on an expressway. Front. Human Neurosci. 7, 1–16. https://doi.org/10.3389/fnhum.2013.00882 (2013).
Ahn, S., Nguyen, T., Jang, H., Kim, J. G. & Jun, S. C. Exploring neuro-physiological correlates of drivers’ mental fatigue caused by sleep deprivation using simultaneous EEG, ECG, and fNIRS data. Front. Human Neurosci. 10, 1–14. https://doi.org/10.3389/fnhum.2016.00219 (2016).
Gao, Y., Pan, B., Li, K., & Li, T. (2016). Shed a light in fatigue detection with near-infrared spectroscopy during long-lasting driving. In S. J. Madsen, V. X. D. Yang, E. D. Jansen, Q. Luo, S. K. Mohanty, & N. V. Thakor (Eds.), SPIE Proceedings, Clinical and Translational Neurophotonics; Neural Imaging and Sensing; and Optogenetics and Optical Manipulation (96901T). SPIE. https://doi.org/10.1117/12.2210846.
Jawad Khan, M., Hong, K.‑S., Naseer, N., & Bhutta, M. R. (2015). Drowsiness detection in dorsolateral-prefrontal cortex using fNIRS for a passive-BCI. In Iccas 2015: 2015 15th International Conference on Control, Automation and Systems: October 13–16, 2015, BEXCO, Busan, Korea: Proceedings (pp. 1811–1816). IEEE. https://doi.org/10.1109/ICCAS.2015.7364653.
Khan, M. J., Liu, X., Bhutta, M. R., & Hong, K.‑S. (2016). Drowsiness detection using fNIRS in different time windows for a passive BCI. In 6th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), June 26–29, 2016, UTown, Singapore (pp. 227–231). IEEE. https://doi.org/10.1109/BIOROB.2016.7523628.
Khan, M. J. & Hong, K.-S. Passive BCI based on drowsiness detection: An fNIRS study. Biomed. Opt. Express 6(10), 4063–4078. https://doi.org/10.1364/BOE.6.004063 (2015).
Li, T., Lin, Y., Gao, Y. & Zhong, F. Longtime driving induced cerebral hemodynamic elevation and behavior degradation as assessed by functional near-infrared spectroscopy and a voluntary attention test. J. Biophoton. 11(12), 1–9. https://doi.org/10.1002/jbio.201800160 (2018).
Li, Z. et al. Assessment of cerebral oxygenation during prolonged simulated driving using near infrared spectroscopy: Its implications for fatigue development. Eur. J. Appl. Physiol. 107(3), 281–287. https://doi.org/10.1007/s00421-009-1122-6 (2009).
Nguyen, T., Ahn, S., Jang, H., Jun, S. C. & Kim, J. G. Utilization of a combined EEG/NIRS system to predict driver drowsiness. Sci. Rep. 7(43933), 1–10. https://doi.org/10.1038/srep43933 (2017).
Tanveer, M. A., Khan, M. J., Qureshi, M. J., Naseer, N. & Hong, K.-S. Enhanced drowsiness detection using deep learning: An fNIRS study. IEEE Access 7, 137920–137929. https://doi.org/10.1109/ACCESS.2019.2942838 (2019).
Xu, L. et al. Functional connectivity analysis using fNIRS in healthy subjects during prolonged simulated driving. Neurosci. Lett. 640, 21–28. https://doi.org/10.1016/j.neulet.2017.01.018 (2017).
Balters, S., Sibi, S., Johns, M., Steinert, M., & Ju, W. (2017). Learning-by-doing. In I. Politis (Ed.), Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 134–143). ACM. https://doi.org/10.1145/3122986.3123006.
Ihme, K., Unni, A., Zhang, M., Rieger, J. W. & Jipp, M. Recognizing frustration of drivers from face video recordings and brain activation measurements with functional near-infrared spectroscopy. Front. Human Neurosci. 12, 1–15. https://doi.org/10.3389/fnhum.2018.00327 (2018).
Shimizu, T. et al. Measurement of frontal cortex brain activity attributable to the driving workload and Increased Attention. SAE Int. J. Passenger Cars Mech. Syst. 2(1), 736–744. https://doi.org/10.4271/2009-01-0545 (2009).
Scheunemann, J., Unni, A., Ihme, K., Jipp, M. & Rieger, J. W. Demonstrating brain-level interactions between visuospatial attentional demands and working memory load while driving using functional near-infrared spectroscopy. Front. Human Neurosci. 12, 1–17. https://doi.org/10.3389/fnhum.2018.00542 (2018).
Unni, A., Ihme, K., Jipp, M. & Rieger, J. W. Assessing the driver’s current level of working memory load with high density functional near-infrared spectroscopy: A realistic driving simulator study. Front. Human Neurosci. 11, 1–14. https://doi.org/10.3389/fnhum.2017.00167 (2017).
Sibi, S., Ayaz, H., Kuhns, D. P., Sirkin, D. M., & Ju, W. (2016). Monitoring driver cognitive load using functional near infrared spectroscopy in partially autonomous cars. In 2016 IEEE Intelligent Vehicles Symposium (IV): 19–22 June 2016 (pp. 419–425). IEEE. https://doi.org/10.1109/IVS.2016.7535420.
Sibi, S., Baiters, S., Mok, B., Steiner, M., & Ju, W. (2017). Assessing driver cortical activity un-der varying levels of automation with functional near infrared spectroscopy. In 28th IEEE Intelligent Vehicles Symposium: June 11–14, 2017, Redondo Beach, California, USA (pp. 1509–1516). IEEE. https://doi.org/10.1109/IVS.2017.7995923.
Tsunashima, H. & Yanagisawa, K. Measurement of brain function of car driver using functional near-infrared spectroscopy (fNIRS). Comput. Intell. Neurosci. https://doi.org/10.1155/2009/164958 (2009).
Foy, H. J. & Chapman, P. Mental workload is reflected in driver behaviour, physiology, eye movements and prefrontal cortex activation. Appl. Ergonom. 73, 90–99. https://doi.org/10.1016/j.apergo.2018.06.006 (2018).
Huve, G., Takahashi, K., & Hashimoto, M. (2019). Online recognition of the mental states of drivers with an fNIRS-based brain-computer interface using deep neural network. In Proceedings, 2019 IEEE International Conference on Mechatronics (ICM): Humboldt Building, TU Ilmenau, Ilmenau, Germany, 18–20 March, 2019 (pp. 238–242). IEEE. https://doi.org/10.1109/ICMECH.2019.8722936.
Kirilina, E., Jelzow, A., Heine, A., Niessing, M., Wabnitz, H., Brühl, R. & Tachtsidis, I. The physiological origin of task-evoked systemic artefacts in functional near infrared spectroscopy. NeuroImage 61(1), 70–81. https://doi.org/10.1016/j.neuroimage.2012.02.074 (2012).
Callicott, J. H. et al. Physiological characteristics of capacity constraints in working memory as revealed by functional MRI. Cereb. Cortex 9(1), 20–26. https://doi.org/10.1093/cercor/9.1.20 (1999).
Owen, A. M., McMillan, K. M., Laird, A. R. & Bullmore, E. N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies. Hum. Brain Mapp. 25(1), 46–59. https://doi.org/10.1002/hbm.20131 (2005).
Shimizu, T., Nanbu, T. & Sunda, T. An exploratory study of the driver workload assessment by brain functional imaging using onboard fNIRS. SAE Techn. Pap. Ser. 2011, 1–11. https://doi.org/10.4271/2011-01-0592 (2011).
Morris, N., Phillips, C., Thibault, K. & Chaparro, A. Sources of secondary task interference with driving: Executive processes or verbal and visuospatial rehearsal processes?. Proc. Human Fact. Ergonom. Soc. Annu. Meet. 52(19), 1556–1559. https://doi.org/10.1177/154193120805201953 (2008).
Gugerty, L. J. Situation awareness during driving: Explicit and implicit knowledge in dynamic spatial memory. J. Exp. Psychol. Appl. 3(1), 42–66. https://doi.org/10.1037/1076-898X.3.1.42 (1997).
Zimeo Morais, G. A., Balardin, J. B. & Sato, J. R. Fnirs Optodes’ Location Decider (fOLD): A toolbox for probe arrangement guided by brain regions-of-interest. Sci. Rep. 8(1), 1–11. https://doi.org/10.1038/s41598-018-21716-z (2018).
Santosa, H., Zhai, X., Fishburn, F. & Huppert, T. The NIRS brain AnalyzIR toolbox. Algorithms 11(5), 1–33. https://doi.org/10.3390/a11050073 (2018).
Jacques, S. L. Optical properties of biological tissues: A review. Phys. Med. Biol. 58(11), 5007–5008 (2013).
Molavi, B. & Guy, G. A. Wavelet-based motion artifact removal for functional near-infrared spectroscopy. Physiol. Meas. 33, 259–270 (2012).
Barker, J. W., Aarabi, A. & Huppert, T. J. Autoregressive model based algorithm for correcting motion and serially correlated errors in fNIRS. Biomed. Opt. Express. 4(8), 1366–1379. https://doi.org/10.1364/BOE.4.001366 (2013).
Benjamini, Y. & Hochberg, Y. Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B (Methodol.) 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x (1995).
Morey, R. D. Confidence intervals from normalized data: A correction to Cousineau (2005). Tutor. Quant. Methods Psychol. 4(2), 61–64 (2008).
Maldjian, J. A., Laurienti, P. J., Kraft, R. A. & Burdette, J. H. An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets. NeuroImage 19(3), 1233–1239. https://doi.org/10.1016/S1053-8119(03)00169-1 (2003).
Funding
Open Access funding enabled and organized by Projekt DEAL. The driving Simulator was funded by Carl-Zeiss-Invest a program of the Carl-Zeiss-Foundation. No further founding was received.
Author information
Authors and Affiliations
Contributions
C.F. and C.G. conceptualized the experiment. C.G. handled data curation and formal analysis. C.F. and J.S. acquired funding. C.G. conducted literature research. All authors contributed to the methodology. C.G. handled project administration. C.F. and J.S. provided resources. J.S. handled the software. C.G. supervised the project. C.F. and C.G. validated the results. All authors contributed to visualization. All authors contributed to writing the original article and its revisions.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Geissler, C.F., Schneider, J. & Frings, C. Shedding light on the prefrontal correlates of mental workload in simulated driving: a functional near-infrared spectroscopy study. Sci Rep 11, 705 (2021). https://doi.org/10.1038/s41598-020-80477-w
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-020-80477-w
- Springer Nature Limited
This article is cited by
-
Prefrontal Correlates of Passengers’ Mental Activity Based on fNIRS for High-Level Automated Vehicles
Automotive Innovation (2024)
-
Assessing the development of mental fatigue during simulated flights with concurrent EEG-fNIRS measurement
Scientific Reports (2023)
-
A functional near-infrared spectroscopy study on the prefrontal correlates of cognitive offloading via a personal knowledge assistant
Scientific Reports (2023)
-
Driving risk cognition of passengers in highly automated driving based on the prefrontal cortex activity via fNIRS
Scientific Reports (2023)
-
Investigating mental workload-induced changes in cortical oxygenation and frontal theta activity during simulated flights
Scientific Reports (2022)