Cognition and Operator Performance

Abstract

Developing systems that foster situation awareness in operators requires that stakeholders can make informed decisions about the design. These decisions must account for the operator’s underlying cognitive processes based on perception, comprehension, and projection of the system state. This chapter reviews the core cognitive processes responsible for monitoring and responding to changes in system state. Operators must perceive information before they can act in response, and the interface design affects operator accuracy and speed via known mechanisms (i.e., effects of color on visual search time). Perception of key information also relies on how the operator thinks during tasks, and certain design choices can support better attention control and detection of signals. After perceiving the information, operators also must comprehend and interpret the information. Design guidance and factors related to supporting comprehension are presented alongside explanations of how cognitive load and working memory affect the operator’s ability to develop and maintain a useful mental model of the system. This review of cognitive mechanisms gives designers a strong foundation to make informed decisions ranging from choosing an alarm color to assessing how much information should be on screen at once.

3.1 Introduction

This chapter explains in more detail the primary cognitive mechanisms used by operators to perform their tasks. This chapter should help designers have a better mental model of operators. These details should help a designer understand how an operator does their tasks and thus support the operator better.

In this approach, based on the cognitive architecture shown in Fig. 2.4, cognition can be described as an emergent phenomenon arising from a collection of mechanisms. The mechanisms are components of an information processing system in the same way that a computer has components. The component mechanisms can be described in isolation (e.g., visual processing of an object) with a great degree of useful truth. However, it is important to understand that this is a practical consideration. In truth, cognition relies on an extremely complex, highly interconnected neural system.

This chapter explains these mechanisms in detail to help a designer . The mechanisms discussed here include visual perception, attention (which is perhaps emergent from other system interactions), memory, and learning. In each section, we note design principles to summarize the results and aid design.

3.2 Visual Perception

The most basic level of cognition for operators is the perception of stimuli. Whereas we may be able to receive signals from a variety of sources, visual stimuli provide the proportional supermajority of signals. Auditory comes in second, followed in a distant third by tactile (which does not appear to be used nor needed currently in most control rooms). We will follow this natural system order in our analysis. Thus, we will primarily focus our discussion on visual perception.

3.2.1 Visual Processing

Understanding the nuances of visual processing enables system designers to build their interface around the natural capabilities and limitations of the operators. At a basic level, visual processing is the process of capturing light on some visual sensor and transmitting this information to the processing system. For many robotic systems, this is a relatively straightforward process where information only flows in one direction. In contrast, human processing is a bidirectional process including feature detection, goal-directed attention, pre-attentive assessment of stimuli, and active interpretation of the signals. This complex system allows us to make a sensible, coherent world out of small snapshots of information without the need for detailed processing. While humans may excel at particular tasks like pattern detection, we also can be easily tricked by unconscious misapplication of visual processing heuristics (e.g., visual illusions, misrecognition, not seeing target objects). While some sources of errorful behavior can be inhibited or corrected through conscious effort, others are essentially reflexive actions without any reasonable method for self-regulation.

A classic example of our failure to inhibit automatic processing is the Stroop task (Stroop 1935). The task is simple. A subject is presented with a color word (e.g., red, blue, yellow) written in one of those same colors. The task is to name the color of the ink. The experiment has two conditions, congruous and incongruous. When congruous, the ink color and word will match (e.g., “red” written in red ink). When incongruous, the ink color and word will not match (e.g., “red” written in yellow ink). This task seems simple in the congruous condition, but when the incongruous condition is tested, and the word and its color differ, the subject will typically stumble through responses, be significantly slower, and make many more mistakes. Once we learn how to read, we simply cannot inhibit the natural response to read text. The mechanistic explanation is that the reading skill is practiced so much more than the naming skill; thus, the reading skill must be suppressed to name the color. Unless some cognitive effort is used to direct attention, the “over-practiced” reading skill will force out the less-practiced skill when both use the same mechanisms.

A more comprehensive overview of low-level visual processing as well as additional resources can be found in the chapter “Behavior: Basic Psychology of the User ” (Ritter et al. 2014, Chapter 4).

3.2.2 Color Blindness

Color blindness is a particularly salient concern for designers due to its prevalence among the population. For the Western population, about 8% of men and 0.5% of women have some form of red–green color blindness. This causes affected individuals to have difficulty differentiating red from green. Individuals may also have blue–yellow color blindness, or even total color blindness, but these are significantly more rare than red–green color blindness (Ritter et al. 2014).

There are many several forms of color blindness, based on the specific deficiency in the visual system, but the general design recommendations that alleviate their effects are the same. Good design will avoid using only color as a signal for an operator . Instead, the design should incorporate multiple signals into a cohesive message for the operator . For example, an important alarm could flash bolded text information, have red coloring, and use textual indicators like exclamation marks to ensure that the message is clear.

Thus, better designs will dual-code results. That is, meaning will not just be encoded by color but color and font , or line thickness and name, or line type and texture. Dual-coding stimuli makes them faster to be recognized and discriminated (Garner 1974). It may be useful to check designs for adherence to color blindness design standards. There are tools online to show how color-blind individuals perceive images and interfaces.¹ They typically take a URL or image file and show how color-blind individuals would see it. Given the prominence of color blindness among the general population, dual-coding signals and ensuring color-blind compliance would be well-advised for any system that requires human operators.

3.2.3 Visual Search

The visual system can be broadly broken up into two subsystems based on their role. The eye handles stimulus detection, and the brain (in specialized regions) handles stimulus interpretation. Stimulus detection occurs within the eye, but the process itself is driven by a combination of goal-directed attention from the mind (top-down) and automatic processing of salient features (bottom-up). Top-down and bottom-up directives guide the visual processing and integration of the environment that occur during visual search. This conflict between top-down and bottom-up visual processing means that designers should consider how their design interacts our natural visual mechanisms.

Visual search of the information displayed on an interface is a core activity for operators, regardless of the task . As their attention is oriented to the task at hand, the operator will need to comprehend the information presented on any given interface . Visual processing is an intermittent process in which our eyes are constantly alternating between saccades (rapid eye movements to some feature) and fixations (resting moments of information intake). What we perceive as a continuous visual experience is actually an intermittent series of fixations that are unconsciously aggregated into a coherent, though not necessarily accurate, mental model of our surroundings (Irwin et al. 1988). During fixations, feature detection relies on distinguishing target features from distracter features through pre-attentive visual processing (Healey and Enns 2012). This summary of vision as being active can be contrasted with folk psychology and early understanding of vision where humans were understood to see and understand the whole display at once. We now know that the eye must search for information actively on the display and often refresh what it sees (Findlay and Gilchrist 2003).

During complex tasks that require visual search, both bottom-up feature recognition and top-down goal-oriented activity influence the performance of the operator at finding that information. While top-down directives lead visual search towards a certain set of features, our eyes are unable to fully inhibit the bottom-up feature detection. Given the effects that distracting features can present for operators, designers should understand what types of visual features draw people’s attention and the role of higher-level graphical organization. The best systems engineers and designers will have a theory of how users will scan displays, find the salient information, and understand it.

3.2.4 Pre-attentive Visual Processing

Once an operator perceives the signals presented by an interface , the visual processing system immediately begins working to form a coherent mental model of the scene. Cognitive limitations on information processing prevent humans from scanning, processing, and understanding every individual signal within the visual field. Instead, we have developed a complex pattern-matching system that reduces workload without (usually) negatively impacting comprehension.

There are two main processes that occur during the early stages of visual search. The first is pre-attentive visual processing based on relatively simple features of the objects. Figure 3.1 shows examples of the types of features that are easily and immediately detected during visual search. The common element across these examples is the contrast between features. When objects vary in orientation, length, or size (compared to other objects in their environment), they are identified and distinguished much more quickly than other objects. Easily distinguished visual features are more salient to the operator , particularly when the operator is distracted or overworked.

Fig. 3.1

Examples of pre-attentive visual features . (Adapted from Healey and Enns 2012, p. 1172)

The contrasting features shown in Fig. 3.1 vary in their salience. Just by glancing across the examples, we can notice a difference in how rapidly we acquire the target stimulus among the distracters. The target feature for orientation is easily discerned, while the target features for lighting direction and length take slightly longer to be recognized. Designers must consider the salience of the signals they will present to the operator and allocate the most salient cues to the most important differences.

The second major process of early visual processing is the grouping of individual features into shared, higher-order visual structures. This is known as Gestalt grouping or Gestalt theory (Chang et al. 2002; Moore and Egeth 1997). Just as particular features are distinguished individually, sets of features are organized into visual structures to be further processed by the viewer. This organization in the scene enables the viewer to maintain a mental representation of a coherent set of distinct objects drawn from the information-dense world. Just like the processing of pre-attentive visual features, Gestalt grouping is an involuntary processing step that shapes how a person perceives the world around them (Moore and Egeth 1997).

Gestalt theory encompasses a family of related psychological principles of perceptual organization used to describe common instances of visual integration. The literature on this subject is varied, and as such, the specific principles can often be described in multiple ways depending on the situation or researcher. Though not exhaustive, Fig. 3.2 shows seven of the most common examples of Gestalt principles affecting how we aggregate component pieces of a visual image. These principles can be used by a designer to group information together or separate different subgroups appropriately.

Fig. 3.2

Common examples of Gestalt principles affecting image perception . (Revised from Ritter et al. 2014; Figure 4-15)

Even without other factors affecting visual processing, Gestalt theory can serve as a useful framework for analyzing and improving the design of an interface . Chang et al. (2002) demonstrate how Gestalt theory can be used to guide the redesign of an electronic learning tool. During their background research, the authors identified a subset of the many Gestalt “laws” from prior research and used these as the basis for their redesign process. The redesign process described by Chang and colleagues provides a useful exemplar of the methodology; however, they did not collect the empirical data necessary to provide a detailed analysis of how their redesign affected interface performance.

3.2.5 Summary of Visual Perception and Principles

Nearly everything on the interface is a signal or feature. Designers should assess the importance of each signal as well as the salience associated with it. The theories in this section provide ways to make the combined operator-interface system work more reliably and, thus, reduce the risk of total system failure.

To make signals recognizable, designers can change the hue, make it flash, increase the size, or use the pre-attentive visual features shown in Fig. 3.1 to modify the salience of the information. The inverse is also true. For irrelevant features (at least for the current task ), ensure their salience is appropriate by modifying their visual representation to make them less apparent.

If an operator does not perceive an alarm or signal directed their way, they have no way of knowing there is an issue, or even that they missed an alarm at all! Creating a mental model requires unconscious assumptions about the world. Do not assume that the operator will eventually realize that they must attend to a minor signal or remember to look at something; help them.

It may be appropriate to test the interface for color blindness compatibility. Where colors cannot be changed, one could test the users to support reconsidering changing colors, or to find other ways to support color-blind users.

Gestalt principles give engineers the ability to predict, and thus improve, how operators will perceive the interface and its functionality. Designing the system layout around these principles can ensure that the engineer’s intentions are clearly conveyed to the operator .

To summarize how to use results from visual perception in design, we present a few design principles related to vision.

• Principle 3.1 Designing to accommodate color blindness will solve multiple problems at once

The prevalence of color blindness among the general population means that accommodating color blindness should be the default plan for high-stakes systems. Presenting information with multiple signals and modes can help ensure the message is clearly received regardless of the operator ’s color perception, and it will lead to faster detection of key signals.

• Principle 3.2 Colors must be used sparingly, used consistently, and should be reserved for critical information

Color can be recognized and interpreted much more quickly than a complex signal, but overuse reduces the effectiveness. If possible, follow these rules: use no more than four different colors, adopt a dull screen as background, and reserve specific colors for specific signals.

Thus, ensure that color provides a valuable signal to the operator through purposeful use of specific colors to emphasize critical information on an otherwise dull interface . Often, color can be a distracter just as easily as a signal if the colors are overused or misused. Three specific examples are shown in Figs. 3.3, 3.4, and 3.5.

Fig. 3.3

Labeled example of interface with dull color overall, allowing the green “active pump” and red “critical event” signals to stand out. Figure redrawn by authors and modified from Ulrich and Boring (2013)

Fig. 3.4

Examples of muted interface with dulled colors, dedicated alarm colors, and merged information for easier perception. Images (a) and (b) show an initial and revised pressure gauge . Figure redrawn by authors and modified from Ulrich and Boring’s (2013) guidelines

Fig. 3.5

Incremental improvement of power level indicator. Final product can be quickly referenced for general status and examined more closely for detailed information like voltage and time remaining

Designers must consider how each color used in the system will be interpreted by operators. Figure 3.3 shows a relatively dull interface that can be quickly scanned to identify which system processes are active without any distracting signals. Connecting lines between components (light yellow) are easily distinguished, but the reduced saturation demotes their importance during typical use.

Color is often a major factor used within an interface to encode signals with meaning. Color use will usually use pairs or sets of colors to provide a categorical piece of information for the operator . Green, yellow, and red can indicate the system status on a range from healthy to critical failure. Blue can represent active pumps for a liquid, while gray shows inactive. Color is a valuable signaling method for typical operators, but designers should ensure that their design has multiple signals indicating critical information.

Figure 3.4 shows an example of how color can be used to highlight critical information (Ulrich and Boring 2013). The use of color within an interface should be considered as a scarce resource. On a completely plain background, one color can be extremely visible, but each new color and new use of a color reduces the salience of that signal. The information in Fig. 3.4a uses a blue line to indicate the current level, which is then compared to “safe” levels on the right side (red and green lines).

For example, the gauges shown in Fig. 3.4a may be unable to provide color-blind operators with enough information to ensure system success. Figure 3.4b shows a revised interface that would be better suited for all users. Though the second gauge sacrifices some contrast between the safe and dangerous system states, the thick black line and arrow indicating the current level reduce the risk of color blindness leading to operator , and thus system, failure.

• Principle 3.3 Make text with readable fonts, use no more Than three font types, use fonts of proper sizes, and use simple, short text strings

Reading from screens tends to be slower and more difficult than print-based reading. This may be due to the difference between projective and reflective light or due to pixel density. Researchers have studied the effects of screen-based reading quite extensively. They consistently find that reading from screens takes about 10–30% longer, leads to increased errors, and fatigues the user more quickly than print reading (Ritter et al. 2014, pp. 208–210). Many operators will not be trained to differentiate font types, so use different fonts sparingly and be cautious about using font type as an important signal. Improve readability and comprehension by using readable, simple fonts. Ensure font size is appropriate for the expected viewing distance. Concise text, accompanied by a symbol or icon , will be faster than a description and more easily interpreted than an icon alone.

Designers should thus avoid using unnecessarily “fancy” fonts and settle on simple, effective presentation of the key information. In general, long strings of text should be avoided. They can be replaced with symbols and bullet points or, at the very least, augmented with emphasized words to make scanning easier. Figure 3.5 shows an example of improvement.

• Principle 3.4 Ensure signals indicating missing information are clear and obvious

Operators rely on gathering and interpreting information to make key decisions. Uncertain or missing information can affect performance through incorrect assumptions by operators.

Missing information from a sensor or system can be a signal to the operator about the situation, but this is only possible if the operator is aware that the information is missing. When operators do not realize that some information is missing, they may rely on their base assumption of normal operating conditions. This is called the normalcy bias and can lead to potential disaster .

For example, a pilot operating a plane in cloud cover with malfunctioning terrain sensors can respond differently if aware of the missing information. If aware of the issue, they could climb to a safe altitude regardless of any “true” obstacle. If unaware, they may crash after assuming they were on a safe trajectory. This type of catastrophic failure is so common that it has its own name, CFIT, or controlled flight into terrain.

As an example for the WDS , signals indicating success for a repeating procedure could be represented as a simple binary response: success or failure (1a and 3 from Fig. 3.6). The interface design in Fig. 3.6 may allow operators to quickly see when the last test occurred and provides an intermediate signal for a missing self-test. If the update schedule is known to vary by 30 min, this could lead to many false alarms if a missing self-test at the exact due time qualifies as a critical failure. These additional states added to the design give operators a signal to be in a “ready” state to respond to a critical failure.

Fig. 3.6

The role of color to represent missing and aging information

• Principle 3.5 Arrangement of screen components should be useful, consistent, and close

Whether designing the full system interface with multiple objects or creating the objects themselves, limit the distance between signals that are commonly used together. This means having a theory of how the interface will be used and using the task analysis, operator knowledge, and characteristics to design the interface such that the information and signals used for the same tasks are near each other. This principle is implied by the Gestalt principles.

As an operator scans the system interface during typical monitoring tasks, they will be generally searching for alarms, alerts, or any sign indicating a potentially risky situation. The task analysis should provide a summary of the tasks, their importance, and their frequency. Checking systems with distant components (measured as travel time through the interface ) requires more time and effort to perform well. Additionally, upon identifying an alarm , operators often will search for signals that confirm the veracity of the alarm . Grouping related components together makes this easier, reduces strain, and increases their ability to search for information.

Grouping and arrangement should also attempt to follow consistent patterns both visually and semantically across multiple displays. The design guidelines in Appendix 3 (specifically in A3.3: Visual Feature Index) provide guidance about the terminology, significance, and heuristics that designers should use when building these systems.

3.3 Attention

Visual perception is broadly described as the integration of information through the field of vision. However, this does not account for how useful signals are isolated from the noisy environment around them. Attention is the “spotlight” that makes a set of stimuli more active or relevant than the rest of the display. As operators are presented with a constant array of information, an executive control system in the mind is directing attention towards features or items in that set of information. A crucial feature of attention is enhanced acuity for the target of interest at the expense of awareness of peripheral stimuli (Ritter et al. 2014, p. 139). The shift in focus from one target to another can occur due to the salience of certain features, perceived relevance to a particular goal , or an active process of cognitive control.

In this section, we will first discuss the basics of the underlying mechanisms of attention and how task-switching affects operator performance. Next, we will describe the causes and implications of limited attentional resources and the attrition of attention.

Attention plays a crucial role in visual perception by providing a mechanism for isolating specific features of interest. Visual perception involves making sense of a world with too much information present; attention is the tool for “working around” this natural limitation. Attention provides guidance for, though not total control of, the sequence of eye saccades and fixations during goal-directed search for visual features. The interaction between visual perception and attention is moderated by cognitive control (e.g., goal-directed behavior) and aspects of features in the visual field (e.g., salience). The interaction between these two systems can affect performance by altering the usage of “cognitive resources” during a particular task . For example, inhibiting a response to look at a flashing light requires active control of visual search, and thus attention. The skill with which a user can inhibit these responses is governed, at least in part, by their working memory capacity (Unsworth et al. 2004). The inverse is true as well: an extremely salient signal will require fewer cognitive resources to detect.

3.3.1 Attentional Vigilance

The role that attention plays in cognitive tasks cannot be overstated. Although we have primarily been describing the role of attention on visual processes, attention plays a central role in both internal (e.g., problem-solving, goal sustenance) and external cognitive mechanisms (e.g., visual search). The act of maintaining attention on a task is called attentional vigilance, or just vigilance. Tasks that require vigilance are characterized by the need to maintain attention over an extended period while attempting to detect target stimuli without responding to neutral or distracting stimuli. Performance loss is often ascribed to a vigilance decrement, or the performance decline that occurs over a period of active monitoring. Tasks that require vigilance are extremely common for operators during their work in op centers.

Sustained attention on a task can be impaired by several factors. First, the salience of the goal signals directly affects the decay rate of operator performance due to the vigilance decrement (Helton and Warm 2008). Increased working memory load leads to worse performance on vigilance tasks. If an operator needs to remember other tasks or keep other information in working memory, they will have a higher cognitive load (Helton and Russell 2011). Depending on the type of information being remembered, the impact on performance may be reduced. For example, listening to a supervisor speak (verbal) while monitoring trends on a graphical display (visual) is easier than listening while reading text (both verbal) (Epling et al. 2016).

The ability to maintain attention over minutes or hours is also affected by the time of day and the natural circadian rhythm that is driving the operator ’s sleep schedule. The impact of sleep and restfulness on performance varies by the task characteristics. Discrete, active motor control tasks (e.g., tilting a platform to roll a ball towards a hole) seem to be less affected by sustained time awake (Bolkhovsky et al. 2018) . However, the biggest concerns should be for monitoring tasks that require focus over minutes or hours to catch infrequent events. Sustained alertness tasks with reaction time-dependent performance show increased reaction times, error rates, and instances of “sleep attacks,” an event where attention lapses for tens of seconds mid-task causing a signal to be missed (Gunzelmann et al. 2009). If sustained attention is a major component for tasks on an interface , designers should consider the attentional requirements of the task and take advantage of tools like FAST (Fatigue Avoidance Scheduling Tool ; Eddy and Hursh 2006) to plan work schedules that are compatible with the sleep patterns of the operators. For further information on sleep and circadian rhythms, it can be found in Wide Awake at 3:00 A.M.: By Choice Or By Chance? (1986) by R.M. Coleman.

3.3.2 Resuming Attention: Interruptions and Task-Switching

Interruptions provide a major risk in disrupting the ability of operators to maintain their attention on a given task . Unanticipated breaks during the completion of a task have been shown to increase subjective workload and error rates, even for experienced professionals (e.g., Campoe and Giuliano 2017; DeMarco and Lister 1999). Campoe and Giuliano (2017) found that the errors when programming medical pumps occurred 7% more often when more than two interruptions occurred during the ≈5-min task . Designers should be aware of how interruptions, even when planned, can impair performance of operators.

The overall framework for understanding task interruption can be divided into several phases. First, the worker will be completing some primary task. At some point prior to completing the primary task, the worker is exposed to a distraction signaling the need to complete a secondary task. The time between receiving the signal and initiating the secondary task is called the interruption lag. Next, the worker begins the secondary task. The time to complete the secondary task is called the interruption length. Upon concluding the secondary task, a period called the resumption lag occurs until the worker is able to resume the primary task (Trafton et al. 2013). This process can occur multiple times throughout the completion of a primary task.

Distractions force the operator to lose their attention on one task , begin attending to a different task , and then transition back into attending to the original task . Each time the operator transfers their focus (in both directions), there will be a necessary “activation period” where the operator is working through the stages of situational awareness: perceiving the task features, forming a mental model of the situation, and finally extending their mental model into likely future scenarios to guide action. This process takes time and leads to performance impairment. It is also a source of errors. Well-designed systems should attempt to alleviate the risks associated with interruptions to primary tasks.

Systems engineers and designers can exhibit significant control over the design of the associated tasks. Although designers may be able to influence operator training, it is more practical to design the system and tasks around a range of skill levels (when possible). The first method for reducing the effects of interruptions on performance is simply removing them from the possible task structure. Even among experienced professionals working in high-stakes situations, the number of interruptions is directly correlated with an increased error rate, cognitive workload, and stress level (Campoe and Giuliano 2017).

If interruptions cannot be limited, there are several ways to alleviate the performance impairment. First, designers can provide a preliminary warning signal that indicates an interruption is imminent (within the next 10 s). This allows operators to begin preparing to switch tasks (e.g., mentally noting a suitable stopping point) without the need to fully place their focus on the new task just yet. Trafton et al. (2003) informally describe the process that occurs after the warning signal as the operator answering two questions and storing the response in memory: “Now what was I doing?” “Now what am I about to do?” The answer to the first question helps the operator identify the point from which to resume the primary task, thus reducing the resumption lag. The answer to the second question prompts the user to gradually begin attending to the interruption task , thus reducing the interruption lag. The same study demonstrated that providing a warning signal with 10-s notice for a distraction reduced the resumption lag by nearly 50% (8 s without warning vs. 4 s with a warning ) for an unpracticed task . Although this effect diminished with repeated practice, this design guideline is particularly useful for infrequent tasks that may be minimally practiced.

Besides offering a warning , designers can design interruptions that minimize the performance impairment. First, interruption length is a large predictor of the resumption lag. Working memory plays a significant role in managing attention. Long interruptions impair the ability to rehearse the previous task state and lead to an operator forgetting their place in the task . Designers can account for this by reducing the length of interruptions and preventing interruptions during high-stakes tasks (Campoe and Giuliano 2017). Interruptions that force the operator to change contexts also impair performance. Context change is a broad descriptor that may include changing locations, unexpected transitions from visual processing to verbal processing (e.g., talking to a coworker), or generally unexpected shifts in cognitive requirements (Marsh et al. 2006) . So, when possible, allow the operator to finish their current primary task step. This reduces the resumption lag for computer-based work, though this benefit appears to disappear for manual work (Campoe and Giuliano 2017).

3.3.3 Signal Thresholds and Habituation

Visual input is naturally limited by the minimum stimulus strength that is detectable by the structures in the eye. The threshold that separates undetectable and detectable stimuli is called a detection threshold. For visual signals in the human eye, the threshold for light detection is approximately 100 quanta. The threshold corresponds to being able to detect a candle flame from 50 km on a clear dark night (Galanter 1962).

The amount of change necessary to create detectable differences between stimuli is called a just noticeable difference (JND). We use JND to generally refer to a detectable difference as measured by the appropriate scale for the metric (e.g., decibels for sound). For example, let’s say we ask a person to select the darker shade of orange between two similar, but different, orange color swatches. If the difference between the two is less than a single JND for the human visual system, then the person will perform no better than chance, even though a computer can instantly recognize a difference. A change in the interface display with less than one JND will have signals that are physiologically impossible to detect for the user . Thus, the signals and stimuli directed to the operator must be sufficiently clear and distinct to be detected, and designers should avoid implementing visual features that communicate important changes through subtle differences.

Although human vision can be very sensitive during the initial presentation of a stimulus , there is also a natural process of habituation that occurs during persistent detection of certain stimuli. As an operator becomes accustomed to a predictable, persistent visual stimulus , they lose the ability to perceive it without conscious effort; the stimulus becomes background to them. For example, people living next to train tracks stop noticing the trains. Though it is more common with simple stimuli, habituation can also occur with complex stimuli that require action (e.g., clicking a “confirm action” box for every action; Ritter et al. 2014).

System designers already will be taking some steps towards accounting for these low-level issues during the design process. For example, system designers will often use particular visual characteristics such as flickering or flashing lights, changes in color, or motion to indicate that an operator ’s attention is needed. However, designers should use caution when deciding when to use alerting signals. When a system is working as intended, the designer should be aiming for signals that facilitate habituation, that is, the changes appear normal and do not call attention to themselves. However, once the system detects an alert of some kind, the design principles become inverted. Rather than facilitating habituation, designers should actively attempt to prevent habituation.

3.3.4 Speed-Accuracy Trade-off (Or How to Design for Acceptable Errors)

There is a constant in human behavior represented by Fig. 3.7. This graph shows that behavior can be slow and careful with low errors, or rather fast and with higher errors. Operators will vary in what their curve looks like. Similar operators may be at different points on the same curve as well. To avoid the extremes, psychology studies often instruct subjects “to work as quickly and accurately as possible” to attempt to put subjects at some ideal center point along this curve. The center point allows fair comparisons between conditions in a study , but, typically, users will move along the curve to suit the task and situation.

Fig. 3.7

The speed-accuracy trade-off curve. (Reprinted with minor changes from Ritter et al. 2014; Fig. 6-12)

We note this speed-accuracy trade-off to designers so that when they are observing users, they realize that operators may be working at different points in the curve. For example, when typing drafts, we type quickly and use spell correction to clean up. When entering passwords, we type slowly because errors take time and force us to redo the whole task .

3.3.5 Summary of Attention

Attention can be seen as the tasks and information that the operator is attending to or working with. There are consistencies and effects that arise from this process. To the extent that designers can understand the operator and their tasks, they have a role to facilitate the allocation of attention and to support its use.

To summarize how designers can support operators’ attention, we present a few design principles related to attention.

• Principle 3.6 Present information needed for comprehension directly

Attention and working memory are limited; information shown to the operator should be processed and integrated as much as possible to reduce operator workload and support the system goals.

Avoid giving operators extra work, particularly for tasks that can be automated or otherwise more effectively handled by the system. Methods for implementing this can range in complexity, but beneficial design choices will be structured around eliminating extraneous work for the operator . Simple examples might include reducing unnecessary mental math or just moving related information closer together. Eye movements take time, as do mouse movements. Making an interface easier to use with many small changes is important: milliseconds matter (Gray and Boehm-Davis 2000). Complex examples include totally redesigning a complicated display around a relatable design metaphor with a unified representation of the information, as shown in Figs. 3.8 and 3.9.

Fig. 3.8

The interfaces for two different altimeters. The generic digital altimeter (a) requires the pilot to mentally compare their altitude to the set value while accounting for variables affecting the instrument accuracy. (b) The Garmin G500 simplifies this by including a spatial comparison between accurate barometric altitudes and clear representation of current altitude and ground level. (Used with permission, www.garmin.com)

Fig. 3.9

The OZ display compared to a traditional cockpit. The traditional display (a) is an emulated display, and (b) shows the plane metaphor (top) used to develop the functional OZ cockpit display (bottom). Used with permission from Temme et al. 2003, pp. 75–77

For example, consider a simple altimeter design. Pilots are often skilled operators with a lot of experience in their primary tasks. However, the human limits on attention and memory are always a factor. Designing to improve comprehension will reduce mental strain for experienced and inexperienced pilots alike.

A pilot need not calculate the difference between assigned altitude and present altitude. Technology has advanced so that this can be calculated and displayed better than the initial dials. Simplify the task and use each system’s strengths. The computer can handle simple mathematical calculations and could show the values using two lines separated by the deviation. The pilot can then identify any issues with altitude much more quickly with the visual process.

Compare the two altimeters in Fig. 3.8. On Fig. 3.8a, the pilot must personally compute the difference, and direction of difference, between the present and assigned altitudes before responding accordingly. However, on Fig. 3.8b, the altitude difference is interpreted visually and is a much faster and less error-prone task .

As another example that is more complex, consider Fig. 3.9 which shows the OZ display. It provides a redesign of an airplane’s control panel around a direct implementation of an airplane metaphor. Flying with traditional airplane displays requires the pilot to mentally calculate their current flight relative to the limits based on the flight envelope (i.e., stable flight based on related parameters like airspeed, altitude, and orientation). This mental calculation is difficult and cognitively taxing, particularly during times of high workload from adverse conditions such as fog or turbulence.

When vision is impaired, pilots rely solely on instrument flight (IF) with no visual reference frame. This risky situation led Temme et al. (2003) to propose an interface titled “OZ” that portrays the key information as an integrated display built around a digital plane, shown in Fig. 3.9 (b, top). This display presents exactly what the pilot needs to know for the task : current aircraft performance compared to aircraft limits and optimal performance values. A comparison between old and new displays is shown in Fig. 3.9 (a and b, bottom).

Although the OZ display in Fig. 3.9(b, bottom) appears complex to novice or unfamiliar users, it was designed to support common tasks that are familiar to pilots and is derived from the mental model used by the pilot during flight. The improvements from the new design were confirmed via tests showing that novice pilots using the OZ interface performed significantly better than novice pilots with the conventional display. With the OZ display, subjects with no flight experience immediately showed greater flight precision (for orientation and altitude) and reduced performance loss from turbulence than when using the typical display. After about 80 h of flight time with both displays, subjects attempted to perform a reading task while operating the plane. This task was essentially impossible with the conventional display, but subjects saw almost no loss in performance when using OZ. Similar designs could be created for control rooms, perhaps as a summary supporting task performance while retaining the raw data visible behind the summary display.

• Principle 3.7 Provide support for operators that may deal with interruptions.

To summarize, to support operators so they can deal with interruptions:

• 1. High-stakes work should be distraction-free.

• 2. Warn operators that an interruption is imminent when possible, that is, allow operators to prepare for task-switching.

• 3. Promote completion of primary task steps before beginning secondary tasks. Simplify the process for resuming a postponed task . This can be done by suspending the secondary task, autocompleting the primary task, or providing note-taking tools for recording the status of the primary task.

• 4. If interruptions are necessary, reduce the distance and difference between the primary and secondary tasks as measured semantically or syntactically.

• Principle 3.8 Consider the risks of stimulus habituation appropriately

Even highly salient signals will become habituated with repeated presentation. Constant presentation of a signal leads to habituation, and thus reduced detection and attention by operators. Designers should create a hierarchy of signal salience to ensure the right signals get through to the operator .

3.4 Working Memory and Cognition

Following the perception of information from the environment, the operator needs to use that information to make decisions and complete their work. Task-related information must be analyzed, manipulated, and transformed into useful information that can guide the actions taken by the operator . The operator must integrate their knowledge of the state of the world with their mental model of the task . For example, an operator sees that the temperature of some module is above the safe threshold and the battery is running low. The operator stores these facts in their working memory and then consults their long-term memory on how to respond to the issue. The response is then also added to working memory alongside the facts about the world state. The operator responds with the appropriate actions in the system, ensures the problem is fixed, and then discards the old information before moving onto their next task .

Variations of this process occur many times throughout an operator ’s shift. These human memories do not work as well (at least under conventional views) as computer memory, so designers familiar with computers should be aware of the differences. Designers should particularly be aware of the differences because their own mental models of their own memories are likely to be particularly incorrect—if your memory fails, you are unlikely to be able to notice this! This section will describe how working memory and long-term memory affect operator performance.

3.4.1 Working Memory

Often, the work performed in op centers requires operators to integrate snippets of information from various sources to come to a decision or understand the situation. This process of storing and manipulating that information occurs within the working memory of the operator . Working memory stores and manipulates information for near-term use (Ricker et al. 2010). Some tasks require multiple pieces of information to be analyzed and processed near-simultaneously; working memory enables people to handle this by offering a “scratch pad” for relevant information. Though particularly relevant during the performance of complex tasks, working memory is a foundational mediator for how each person interacts with the world. Working memory acts as a store for both internal events (i.e., recalling long-term memories) and external events (i.e., perceiving visual signals). In many ways, working memory is often analogized to be comparable to the RAM of a computer system, whereas long-term memory is like the ROM. The RAM, or working memory, allows rapid data access, efficient manipulation, and quick turnover between processes. The ROM, or long-term memory, provides a slower, semipermanent location for information storage and retrieval.

The RAM–ROM analogy also applies to the limitations of working memory. While long-term memory does not appear to have a clear storage limit in humans, working memory is constrained by a capacity of only a few items—the most common general storage limit is about seven items plus or minus two items (Miller 1956). The seven-item limit is overly simplistic but provides a useful anchor for working memory capacity. Working memory capacity also varies across the population with greater working memory capacity being associated with better performance on cognitive tasks (Just and Carpenter 1992). The levels of abstraction and familiarity with the relevant concepts also have an effect; less abstract and more practiced tasks are easier to remember and use (Ritter et al. 2014, Ch. 5).

The approximate limit for working memory capacity becomes even more complex due to processes such as chunking. Chunking refers to a mental process for grouping sets of individual information pieces into easily recognizable sets. For example, it will be easier to remember a sequence of items like “N S A F B I” (chunked as NSA, FBI) than “Q G Z T Y V” (not “chunkable” by most ; Chalmers 2003; Ellis 1996). Chunking mechanisms can be leveraged by system designers to increase the practical working memory capacity of the users.

Modern theories of memory suggest that working memory is built from specialized subsystems that differ based on their input: the “visuospatial sketch pad” for visual spatial information and the “phonological loop” for verbal information (Baddeley 2000). This distinction between verbal and visual working memory stores is important because these two systems can perform semi-independently without much interference (i.e., loss of performance) between them. When implemented successfully, this can allow someone to drive a car while listening to an audiobook with almost no loss of performance for the primary task (Granados et al. 2018). However, implementing this concept is not necessarily foolproof. When the secondary task requires too much mental effort (i.e., maintaining a conversation vs. passive listening), driving performance tends to be degraded to a noticeable degree (Strayer et al. 2003). Although multitasking is best avoided, making attempts to isolate the tasks to distinct working memory stores can provide some measure of risk reduction when it is impossible to eliminate the need for multiple tasks.

For the designer , there are a few takeaway implications for design:

• (a) Working memory has limitations on capacity and performance. Don’t use it up asking the user to remember items the system can remember for them.

• (b) Chunking of items can increase the functional working memory capacity. Support chunking when you can by putting items in a canonical order, spacing items to support chunking (e.g., FBI vs. F____BI), and understanding the patterns that operators know and choose, or even teaching them new acronyms.

• (c) Working memory has a time-based decay. Maintenance requires rehearsal at some cost to the operator ’s cognitive resources. Ensure users are not required to independently store and remember lots of information for minutes at a time.

3.4.2 Cognitive Load

Cognitive work is inherently taxing on our mental resources. We have previously discussed the impairment of cognition as it relates to attention, but higher-order processes are also affected. Throughout the performance of cognitive work within an op center, operators are presented with information that must be monitored and assessed and may need to be compared across time. These types of work are inherently difficult, particularly when during long periods of performing the tasks. Cognitive load theory (CLT) describes how the various factors such as working memory load, personal stress, and task difficulty can provide an overall decrement on performance of cognitive work (Sweller 1988). Cognitive load theory provides a way to compare task difficulty (relative to the expertise of the user ) across different task environments. Reducing cognitive load provides a broadly effective way to improve performance by freeing up working memory capacity for more important tasks like integrating information and learning. CLT currently lacks units and an objective way to measure it; however, we find CLT to be useful nonetheless because it provides a framework for comparing system design choices.

A review of cognitive load’s role in human–computer interaction design is provided by Hollender et al. (2010). Their review integrates CLT research into a useful framework for systems engineers. They posit three main types of cognitive load: intrinsic, extrinsic, and germane. Intrinsic cognitive load refers to the inherent complexity of the information being processed by the user . Comparing intrinsic load can only really be done by comparing two tasks rather than by providing a stand-alone value. For example, driving on an empty highway would likely provide less inherent complexity compared to driving on a busy city street.

Extrinsic cognitive load refers to environmental and context-dependent factors that provide unnecessary contributions to task difficulty. Integrating spatially distant information from displays that are on opposite ends of the room will be inherently more difficult than if the displays were side by side due to the required storage of the information in working memory between task steps.

Finally, germane cognitive load refers to the beneficial cognitive work that improves task performance. Learning and practice of the skills and schema required to perform a task also require cognitive resources, in contrast to unhelpful portions of the overall cognitive load. All three types of load contribute to the overall working memory needs of any given task , and the ideal task will reduce the intrinsic and extrinsic load to provide more resources for the beneficial mechanisms that occur from germane cognitive load.

Reducing the cognitive load of extraneous tasks can provide a consistently useful method for improving the performance of operators. A simple method for reducing cognitive load is by enforcing consistency across the layout, color scheme, and overall information presentation style for components of an individual system and across multiple systems (Chalmers 2003). Even experienced users that may switch between a Windows OS and Mac OS will know the feeling of attempting to use a Mac-only shortcut on a Windows machine (or vice versa).

Many of the recommendations for reducing cognitive load can be succinctly described as follows: when possible, reduce the space and distance between codependent pieces of information. In some cases, it’s a relatively simple process to find multiple solutions. Disparate information sources could be split across multiple displays to maximize information presentation, or alternatively, a single display could be trimmed of unnecessary information to bring the most important features onto a single, more efficient display (Brown et al. 2013). Other cases provide less clarity in determining the best practices for a given context. Providing redundancy in feature presentation can help reinforce certain information, but the additional features inherently increase the intrinsic cognitive load during interaction with the system (Grunwald and Corsbie-Massay 2006).

Engineers and other stakeholders must use the risk-driven approach to make informed decisions; competing design recommendations are rarely weighted on easily comparable scales. Krug’s (2005) approach provides further suggestions to reduce cognitive load that center around the titular message of the book: Don’t Make Me Think. Krug argues that small design flaws like unclear labels, confusing buttons, and unclear feedback introduce minor inconveniences that can add up and lead to a noticeable drop in overall system performance.

Further ways to support operators and reduce cognitive load can involve shifting cognitively taxing tasks and information onto the system. This includes (a) reminding operators when tasks should begin; (b) reducing load by simplifying the number, length, and complexity of actions; and (c) automating tasks that can be automated, like how automobile turn signals automatically shut off after the steering wheel rotates back to straight.

3.4.3 Summary of Working Memory and Cognition

Operators will be using their working memory on every task , but there are inherent limitations to capacity and processing power that need to be considered when designing the interface . Off-loading information to the system (when possible) reduces strain on working memory, as does simplifying or optimizing how information is displayed to leverage mechanisms like chunking to increase functional working memory capacity. By understanding the tasks and operators for their system, designers can identify ways to support operator performance through design choices.

• Principle 3.9 Reduce the cognitive resources used during multi-step tasks

Operators’ cognitive resources, including working memory and attention, are limited, and these limitations are made worse by fatigue , stress, and task difficulty. Simplifying the work will reduce workload and make errors less likely to occur.

Simplifying tasks can be done in many ways depending on the specific scenario. The common factor for all successful implementations of this guideline is a reduction in the amount of working memory, attention, or other cognitive resources needed to perform the task .

For example, if an operator is alerted for a task that needs to be done in 30 min, the system should provide an additional reminder at the appropriate time rather than relying on the operator ’s memory.

If a common task requires several steps to complete, provide an interactive task checklist that indicates the current state of the procedure—checklists are very helpful to support complex tasks. A simpler solution could be incorporating a window showing all inputs and outputs for the system with associated timestamps.

3.5 Summary

The mechanisms that operators use while performing their work influence how the work gets done, what errors are likely to occur, and how to design to support system success. This concept is common across other engineering fields. For an electrical engineer, the components that comprise electrical circuits influence how circuits produce their outputs, what errors are likely to occur within the circuit, and how to design effective systems that require electrical circuits.

The most salient mechanisms of operators that are relevant to improving the design of op centers are perception, attention, and working memory. These mechanisms interact, and good design will be based on a theory of how they are used by operators to perform their tasks based on the information presented to them in the interface .

We include design principles to help with design. When these principles contradict themselves, which design principles and guidelines will inevitably do, the designers will have to resort to analysis of the tasks and their procedures, importance, and frequency to resolve the design trade-offs.

There are also other mechanisms of operators, shown in Fig. 2.4, that will influence performance in op centers. These mechanisms include motor output and other forms of perception. An overview of these mechanisms is available in Ritter et al. (2014).