Keywords

1 Introduction

Tactile graphics (TGs) are embossed outline drawings which provide a means for acquiring pictorial information through touch particularly by persons with visual impairment and blindness (BVI) [1]. Researchers have discussed the advantages of use of tactile graphics in education over verbal or textual descriptions of graphics as they allow users to actively explore the diagrams [2]. Since, a graphic is a layout of symbols on a two-dimensional (2D) surface, the study of tactile perception of 2D spaces (both exploration and retention strategies) by persons with blindness is necessary for effective TG design. Additionally, with the rise in need for advanced computer interfaces, new opportunities to develop more complex multi-sensory displays have emerged. When designing such displays it is necessary to consider human perceptual capabilities and understand how people find patterns, recognize forms and organize individual elements into structures and groups [3]. The inferences of this work can contribute to the development of design guidelines for designing intuitive multi-modal interfaces as well as TGs.

Despite the fact that every TG is essential an arrangement of tactile information organised in a 2D field, there has been very little research into the “compositional semantics”, i.e. the perceptual salience and potential semantic significance of various positions. For example, it is unknown if BVI students perceive TG shapes positioned in the centre of a framed field as semantically central to the overall meaning of the theme of that field. One study [4] found that BVI subjects using vibro-tactile touchscreens remember information better when it was associated with a specific position in document layout. Auditory feedback cued positions on a touch screen can also help BVI subjects interpret graphics [5]. It is interesting to note that many blind individuals play chess (personal observation, school of the National Association of the Blind, New Delhi) which requires spatial awareness and spatial learning strategies such as chunking [6].

This current work investigates if participants with blindness can explore and remember sequential or framed arrangements of tactile shapes and asks whether the recreation of such arrangements is facilitated by semantic memory (remembering where a particular symbol was positioned); and/or by positional memory (remembering specific positions as occupied by specific shape). A set of experiments were conducted to determine if certain positions in an ordered arrangement or framed field are perceived and remembered better than other positions and thus may be considered “positional primitives”. It is also noteworthy that a majority of earlier work in this area have been conducted with sighted (sometimes blindfolded) participants and the conclusions cannot be extrapolated for blindness. Therefore, the studies in this work were conducted with BVI subjects.

2 Related Work

Limited works investigate the retention of tactually acquired information from tactile-stimuli (in contrast to the large number of studies that have addressed retention of visual stimuli or visuo-tactile stimuli). Even fewer have addressed the exploration and retention of meaningful tactile representations (i.e. tactile graphics with some information) and have rather used pictures of common day to day objects or abstract shapes in experiments. Some works study the “compositional semantics”, i.e. the perceptual salience and potential semantic significance of various positions.

Significant work by Berla & Butterfield (1977) demonstrated that BVI children learn and remember tactile geometric maps better if they use certain learning strategies: a) systematic tactual/haptic exploration within a framed field; b) methodical tracing of each contour; and c) “distinctive feature analysis”, i.e. mentally noting when a contour has unique graphic features that other shapes lack. Two other beneficial strategies for remembering positions of multiple TG shapes: d) noting shape-to-shape relationship, e.g. this shape is next to or on top of another; and b) noting a shape’s position relative to the top, bottom and sides of its field [8]. The subject’s orientation to the frame becomes another relevant factor in tactuo-spatial learning. In experiments reported by Newell et al. blindfolded subjects learned an arrangement of seven distinct haptic objects in a frame, then were challenged to recognize which two objects had been repositioned by researchers [9]. Performance decreased significantly when the frame was rotated 60° between learning and testing. This finding resonates with another work [10].

Although definite conclusions may be elusive, these issues suggest an opportunity for designing innovative strategies for tactile compositions in framed fields. The objective of this work is to investigate the tactile exploration and retention strategies for learning sequences and layouts of small tactile shapes by persons with BVI and identify if there are any positional primitives (or positions of salience that facilitate information acquisition and encoding). The aim of this work is to explore information organization strategies for tactile perception and utilize that knowledge into making effective tactile graphics and better information organization for BVI.

3 Experiments

Two experiments were conducted to test memorability of sequences and layouts of tiles with abstract TG shapes (see Fig. 1). The goal was to identify positional primitives i.e. sequence or grid positions with greater perceptual salience that facilitate information acquisition and its retention.

To achieve the said goal, participants were asked to explore and recreate sequences and layouts. Students from Indiana School for the Blind and Visually Impaired, Indianapolis, USA (ISBVI) and National Association for the Blind, New Delhi, India (NAB) participated in the experiments. All participants had blindness with no cognitive impairments. All participants had prior experience with TGs as part of their school education. Ethical clearance was obtained from ethical committee at IIT Delhi. Informed consent was obtained from participants and school authorities.

A set of 30 abstract TG shapes (see Fig. 1) were prepared on 5 cm- × -5 cm plastic tiles with a single symbol on each tile. Tiles were placed in a sequence of 4 to 8 tiles on a table-top for Exp.1 and in a 5 × 5 gridded board for Exp.2 (see Fig. 2).

Fig. 1.
figure 1

Set of abstract tactile shapes used in experiments 1 & 2

Full size image
Fig. 2.
figure 2

(left) Sequence of tiles on table-top for Exp.1; (right) Tactile tiles in a grid for Exp. 2

Full size image
Fig. 3.
figure 3

(left) A bar chart showing mean (±SEM) percentage accuracy of responses for Exp. 1, (right) A scatter plot showing the frequency of accurate responses for each position for 4, 5, 6 and 7 tile sequences

Full size image

3.1 Experiment 1

The objective of this experiment was to investigate if participants can explore, learn, and recreate sequences of tiles from memory and which positions facilitate acquisition and retention. This experiment was conducted at ISBVI in April 2018 and at NAB in October 2018. Seven participants (four males, three females) aged 15–18 years (M = 15.57, SD = 1.13) participated in this experiment at ISBVI in April 2018 and 14 (9 males, 5 females) students aged 12–17 years (M = 14.78, SD = 1.25) participated at NAB in October 2018.

Sequences of randomly selected tiles were presented to participants who were asked to freely explore. Then a jumbled set of the same tiles was presented to them and they were asked to recreate the sequence from memory. The experiment was repeated with four different sequences of 4, 5, 6 and 7 random tiles. The responses of the participants were visually observed and noted by the research team and number of accurate placement (correct tile in correct position) was counted for each recreation.

Results.

Figure 3 shows mean percentage accuracy of correct placement of tiles in a sequence (mean includes all participants from both locations). The results show a degradation in performance with the increase in number of sequenced tiles. However, a one-way ANOVA indicated that the difference between the accuracy for sequences with different number of tiles was not significant (F(3,80) = 0.788, p = 0.504).

The “primacy effect” was observed: Subjects remembered the correct placement of tiles that were at the beginning of the sequence better than tiles in middle, no matter how long the sequence. The same can be seen in Fig. 3. However, further empirical and statistical validation is needed to strengthen this observation.

3.2 Experiment 2

This experiment aimed to evaluate subjects’ memory of different layouts of tiles in gridded board, to determine if: (1) some positions in the grid are remembered better than others (e.g. corners vs center); (2) a type of layout is remembered better than others (e.g. structured vs randomized); (3) some structures are remembered better than other (e.g. row vs column, chunks vs line, cross vs chunks or lines). This used 6 structured layouts and 1 random layout (See Fig. 5). The experiment tested 21 participants from NAB and ISBVI during Oct-Nov 2018. Seven students (four females, three males) aged 15–18 years (M = 16.29, SD = 0.951) participated at ISBVI and 14 students (5 females, 9 males) aged 12–17 years (M = 14.78, SD = 1.25) participated at NAB. The participants were presented with test layouts, were asked to explore it for 5 min and then recreate the layout on an empty grid from memory (called ‘response layouts’ in following text).

Results.

Figure 4 shows the mean percentage accuracy of the subjects’ (including data from both test locations) ability to recall: (1) ‘only positions’ from test layouts (i.e. placing incorrect tile in correct position); and (2) the correct target tile and its correct positions (referred to as ‘identity+position’ in chart). A one-way ANOVA for ‘only position’ data indicated a significant difference between the performance for these layouts (F(6,140) = 13.520, p < 0.001). A subsequent Tukey comparison showed that performance for a 4-tile layout was significantly better than that for 9-tile layout (p < 0.05). Additionally, performance for a random 7-tile layout was found to be significantly worse than other structured layouts (p < 0.001 for all pairs).

Fig. 4.
figure 4

A bar chart showing mean (±SEM) percentage accuracy of responses for tiles in correct positions and correct tiles in correct in correct positions for the various layouts in Exp. 2

Full size image
Fig. 5.
figure 5

Test layouts and heat maps of responses in Exp. 2

Full size image

As can be seen in Fig. 4, the participants remembered the position of tiles better that the identity of the tiles in those positions. A one-way ANOVA for ‘identity+position’ data indicated a significant difference in accurate tiles placement in layouts with different numbers of tiles (F(6,140) = 5.773, p < 0.001). The analysis was followed by a Tukey comparison of means which indicated that performance for 4-tile layouts was significantly better than for 8-tile (p < 0.01) and 9-tile layouts (p < 0.05). Additionally, the performance for randomized 7-tile layout was significantly worse than for structured layouts with 4, 5, 6 and 7 tiles (p < 0.05 in all cases). A heat diagram presented in Fig. 5 shows the test layouts versus the cumulative placements of tiles by participants in response. This indicates overlap in test and response particularly for layouts with adjacency to edges and corners (e.g. 4, 6 and 7 tile layouts). Further investigation is required to understand how various kinds of “structures” are perceived and retained.

4 Discussion

This work investigates perception and retention of sets of 5-cm-sq. tiles embossed with tactile shapes, arranged both in simple horizontal sequences and layouts on gridded boards. The aim was to determine if some positions are more perceptually salient (more readily perceived during exploration and remembered) than others. If identified, particularly salient positions could be considered “positional primitives” to guide designers of tactile graphics, particularly when presenting information without pictorial sources, an important consideration in advanced grade levels when classroom concepts become increasingly abstract or intangible. Our findings suggest positional primitives exist, and that their use as semantic drivers needs a deeper understanding for effective classroom applications and pedagogical strategies.

4.1 Salience of Boundary and Structure

In Exp. 1, it was observed that the tiles in the beginning and end of the sequence (mostly the participants started from left although they were not directed to do so) was remembered better than the ones in the middle. This demonstrates the presence of primacy and recency effects in serial memory of tactile stimuli, a well-known phenomenon called “serial position effect” [11] (recency effect was seen as well, though not very strongly). This was observed in participants from both US and India.

In Experiment 2, tiles placed in corners or along board edges of a tactile layout (composition of tiles on a 5 × 5 gridded board) were more easily remembered for ‘identity+position’ (i.e. provides salience for both positional and semantic memory). This suggests that boundaries of any perceptual field offer a high level of positional/identity salience for BVI subjects and subject can most easily locate a tactile landmark positioned in a corner or along the edge of a framed field. It is noteworthy that this observation contrasts with common understanding visual perception, which suggests that centre of a visual composition has higher salience and hence attracts more attention [12]. Additional empirical validation is required to confirm this phenomenon. This salience persists in semantic memory, i.e. subjects can identify a tile removed from the grid and easily remember its former position (however, this was not formally tested).

It was seen that “structured” layouts were easier to remember than randomized layouts (focussed empirical investigation is needed to understand the role of different kinds of structures in aiding memory). This finding resonates with the concept of finding of ‘figural goodness’ being favorable to memory [13]. This insight resonates with the concept that coherently configured tile sets offer “chunks” of positional information. Chunking is a well-studied mental technique for remembering series of numbers or other data sets [14]. Interestingly, chunking also functions in playing chess [6] where practiced players cognitively chunk piece positions and plan moves. However, not surprisingly, even with chunking and corner/edge adjacency, overall retention of tile position/identity degrades as the number of distinct tiles in the layout increases. This suggests that compositional and juxta positional simplicity is the most effective design strategy. We speculate there may be an optimum number of positioned elements that subjects can easily learn in a framed field using the very simple target layouts (as seen in Experiment 2). Notably, this number matches well-known estimates of working memory capacity as 7 elements, plus/minus 2 [15]. These inferences pose interesting research questions and need further empirical validation.

4.2 Exploration and Retention Strategies

In post-experiment discussions, subjects revealed self-generated tactual-semantic strategies to encode the position/identity of tiles. These strategies included: (1) chunking, i.e. dividing the presented tiles into self-defined sets of 3–4 items even when tiles were not adjacent; (2) noting adjacency of tiles to board corners/edges; (3) using basic internal, directional self-instructions such as, “this tile was on the left side of the board”; (4) naming, i.e. picking a word to associate with the tactile symbol in order to remember it easily; (5) self-narrating a simple story to provide positional context for the tiles, e.g. one participant imagined the upper half of the grid board as sky and the lower half as ground, then imagined this tile “flying in air” and that tile “lying on the ground”.

We also observed that participants used different techniques of manual exploration. Some began their exploration in the centre of the grid while others started at the top left corner (like reading). These differences were not factored into our analysis; it is unknown how this observation impacted retention and requires further investigation.

5 Conclusion

This work demonstrates that 1) compositions of abstract tactile tiles in a framed field can be learned and remembered by BVI subjects using tactual exploration; 2) tactile elements adjacent to field corners and edges were retained better; 3) coherent or structured configurations of adjacent tiles; 4) in linear sequences of tiles, the serial position effect was observed. These compositional principles comprise “positional primitives” for TG design. Additionally, retention quality degrades as the number of tiles increases in both sequences and layouts. Though, these insights offer an understanding into tactile perception of sequences and layouts, there are several parameters whose role remains to be understood such as complexity of the tactile images used, discriminability of these images and similarity or categorization of the tile images. The role of factors such as orientation, movement and experiment surrounding and alignment with the experimental setup pose additional intriguing research questions.

These findings can inform the design of more effective tactile learning resources for persons with BVI. Further investigation is needed to understand how tactile compositions facilitate semantic relationships and its pedagogical validation. This work can also find application in the design of tactile/multi-modal interfaces.