Importance of outline information in a relief representation for tactile graphic display

Yutaka SHIMIZU*, Masami SHINOHARA**, Hideji NAGAOKA*** and Yasushige YONEZAWA****
*Department of Information Science, Tsukuba College of Technology, 4-3-15, Amakubo, Tsukuba, Ibaraki 305-0005 Japan
**National Institute of Advanced Industrial Science and Technology, 1-4, Higashi, Tsukuba, Ibaraki 305-8566 Japan
***Research Center on Educational Media, Tsukuba College of Technology, 4-12-7, Kasuga, Tsukuba, Ibaraki 305-0821 Japan
****Department of Computer Science, Tsukuba College of Technology, 4-12-7, Kasuga, Tsukuba, Ibaraki 305-0821 Japan

Abstract: Although the speech synthesizer is an effective device to receive textual information, it is difficult to transmit non-linguistic information such as graphics, charts and images. Tactile displays promise to overcome this difficulty. Several presenting modes for tactile displays have been examined. The relief mode analogous to the original object in a tactile graphic display is known as an effective presenting mode. Two presentations for the relief mode were examined for a computer-driven tactile display. One was a clearoutline relief that emphasized the contour proportion of an object in the tactile depiction. The other was an unclear-outline relief that was a normal depiction of simple encoding information by two camera signals. All visually impaired participants identified the clear-relief mode significantly easier than the unclear-outline mode. Results indicated that the information in an object is an important factor for haptic pattern recognition. Therefore results meant that the contour portion of a visual object should make clear in encoding for a tactile graphic image.
Key Words: tactile graphic display, haptic recognition, presenting mode

1. Introduction
 The recent spread in the speech synthesizer mounted on a PC (personal computer) is making it easier to receive textual information for the blind. Although the speech synthesizer is an effective device to assist in transferring linguistic information, it is difficult to transmit non-linguistic information such as graphics, charts and images. The tactile display promises to overcome this difficulty. At present, two types of tactile display have been devised. One is a matrix of vibrators that are selectively activated to form a pattern1). The other is a matrix of tactor-pins that produces raised-dot drawings or solid forms2).
 To use the former type, users have to fix their palm or fingertip against the display to perceive the object being presented. On the contrary, the latter type depends on haptic search by the blind users. The former type, which presents patterns passively, is useful for alphanumeric letter identification. Furthermore, it is effective for mobile use because of its size. However, it seems difficult to transmit complicated patterns because of the limited reception of information. The haptic search is one of the most important factors for tactile recognition3). Therefore, to develop the latter type of tactile display seems useful for a PC terminal.
 Many psychological studies have been performed on tactile pattern recognition. In these experiments, raised points or lines produced on a paper or a plastic film were used for tactile stimuli. Most of these recognized the importance of haptic ability.4-9) Shimizu et al10) introduced a relief mode which consisted of a three-dimensional presentation wherein the tactor-pins formed a surface analogous to the original object. In addition to the relief mode, the outline mode in which an object was presented with the outline pins raised to an equal height, the plane mode in which the surface of the object was elevated uniformly, and the hollowed-plane mode which represents the surface of the object is a reverse of that in the plane mode, have been examined. The superiority of the relief mode has been shown.11)
 An actual computer-driven system that can present the relief mode has been devised.12) This system presents a three-dimensional tactile image by a matrix of pins. Since the spatial proportion of the entity can be given by adding height information, this display seems to be superior to the outline or the planar presentation. However, the display does not present information such as texture, temperature, and volume, which may be necessary for identifying an object. Therefore it is necessary to further research the encoding technique in the relief mode.
 The objective of the present study was to examine the effectiveness of two relief modes psychologically. One is an unclear-outline relief that is easily realized from two camera signals.12) Unfortunately, the contour portion of visual image is not clearly depicted in the tactile graphic display by this mode. The other is a clear-outline relief that needs to emphasize the contour portion of a tactile object. Therefore this study is concerned with whether the outline information is effective in the relief mode for haptic pattern recognition using the tactile graphic display.

 This work was supported by the Ministry of Education, Sports, Culture, Science and Technology (Grant-in-Aid for scientific research #12480102).

2. Experiment
2.1 Method
Apparatus
 The tactile graphic display used in this experiment consisted of 1,827 pins arranged with 3-mm interspacing. In order to produce a smooth depiction, the tactile surface consisted of a hexagonal matrix of pins. The diameter of the tactor-pin was 2.75mm, which was made of resin. The haptic area for exploration was 120 X 120mm. Each pin was adjustable to 30mm in height. A detailed description of the system is available in previous studies.9-11) A schematic illustration of the unclear-outline relief is shown in Fig. 1(a). The maximum pin-height of the depiction analogous to the original object was 10mm. The clear-outline relief is illustrated in Fig. 1(b). Each activated pin has been elevated 1mm from the base surface of the tactile display, while the maximum pin-height was the same as the unclear-outline relief. Thus the contour proportion of a tactile entity was depicted more clearly than the unclear-relief mode.
Stimuli
 Fifteen shapes of objects with which people who are visually impaired are normally familiar were selected as stimuli. They were an eggplant, a cucumber, a banana, an apple, cherries, a bunch of grapes, a seashell, a spoon, a pan, a bottle opener, a teacup, a scissors, a plug, a telephone, and a left hand. Fig. 2 shows an example of an eggplant represented in two presentation modes.
Participants
 Seven visually impaired subjects (three men and four women), aged 19, 25, 20, 20, 19, 27, 20 years old participated. They were visually impaired students at the Tsukuba College of Technology. Three participants (SB5, SB6, SB7) were blindfolded because they have the sense of light.
Procedure
 Prior to the stimuli presentation, the participants were told that the stimuli to be presented were all familiar objects, such as plants, household items, body parts, although the sizes would be smaller than in reality. They were also told that the orientations that would be depicted would not always be the same. Furthermore, they were told that the same object might reappear but the frequency might not always be the same. The participants were then asked to respond by naming the object as quickly and accurately as possible. A total of 2 (modes) X 15 (objects) were presented to each participant in random order. Therefore each subject experienced the same objects but in a different presenting mode. There was no limit on haptic exploration to identify the stimulus, and the maximum exploration time was unlimited. No feedback was given for correct answers at each presentation. Participants' responses and recognition (or reaction) times (RTs) were recorded. The time count was started when each participant touched the tactile display for haptic exploration and stopped when each participant voiced a response to each stimulus. In addition, each participant was asked to report the rating for the haptic legibility after stimulus identification. Haptic legibility was a rating made by the participant as to whether the stimulus was a good or bad depiction of the object. The participant assigned a number to each stimulus, from 1 (excellent) to 5 (worst).

2.2 Results
 Table 1 shows recognition accuracy (% correct), mean and SD of RTs, and those of haptic legibility to each participant. In addition, values by a paired t-test comparing two modes is shown. The tendency that the clear-relief is superior to the unclear-relief in correct answers, mean RTs, and mean haptic legibility was observed. Correct answers demonstrated that the clear-relief mode was significantly greater than the unclear-relief mode [t(6) = 7.14, p<.01]. Furthermore, a significant difference in haptic legibility was found by the paired t-test (df=14, p<.01). However, no significant difference was found between RTs except with participants SB4 and SB7.
 ANOVA were performed in the participants' RTs and haptic legibility. Significant differences in haptic legibility were found [F(1,196) = 59.9, p<.01 between the presentation modes, and F(6,196) = 6.19, p<.01 between participants]. Although in RTs a significant difference between the participants was found [F(6,196) = 9.79, p<.01], the difference did not hold for the presentation modes [F(1, 196) = 3.14, p>.05].

Fig. 1 Schematic illustration of the unclear-relief presentation (a) and the clear-relief presentation (b).
Fig. 2 Illustration of an eggplant in the unclear-relief mode (a) and the clear-relief mode (b).

3. Discussion
 All participants identified the clear-relief mode significantly easier than the unclear-relief mode on the basis of the recognition accuracy and the haptic legibility. However, no clear tendency was observed in mean RTs except with two participants (SB4 and SB7). One factor might explain this insignificance. Since the participants were not instructed as to the method of haptic exploration, we observed peculiar movements in touch. Although they mostly moved their fingertips and palms widely and smoothly, they sometimes left their fingertips for a long time at a few selected proportions, such as an edge or endpoint of an object. Therefore, a difference in haptic method might affect RTs.
 A significant difference by ANOVA was indicated between the participants in the percentage of correct response and in the haptic legibility. The age and the mental ability were similar because they were students at the Tsukuba College of Technology. Therefore the familiarity of the stimuli and the difference in haptic strategy might have influenced the results. Several studies have explained that difference may exist between participants who are congenitally and adventitiously blind.4,7,9) In this study SB1 and SB4 were congenitally blind participants. Further experimentation is needed to explore this explanation. However, a limitation is that the population of people who are congenitally blind in Japan is small.
 The relief mode has an advantage in identifying a tactile object because such information as height, tilt, and curvature can be represented in the tactile graphic display. Although both modes examined in this experiment have the same advantage, the clear-outline mode was shown to be superior to the unclear-outline mode. Therefore, this result indicates that the contour information was a dominant factor for haptic recognition. Kennedy13) explained the possibility for picture recognition from a line drawing pattern. His indication was not a normal outline representation such as a raised line, but a line representation in which the blind could receive information about such features as occlusion, foreground and background, flat and curved surfaces, and vantage points. Our relief mode has more possibility than the line drawing or planar representation because it provides information about depth dimension, which can depict Kennedy's indicated features directly.

Table 1 Percentage of correct responses, mean recognition time (in seconds), standard deviation of recognition time (in seconds), mean haptic legibility, and standard deviation of haptic legibility, by presenting modes and participants.

4. Conclusion
 In summary, we confirmed that the preferable presenting mode for a tactile graphic display was the clear-relief mode. The tactile graphic system using this presentation mode will be available for people who are blind in order to use computerrelated data and to receive such information as guide maps and scientific illustrations.

References
1) Saida S, Shimizu Y, Wake T. Computer controlled TVSS and some characteristics of vibro-tactile letter recognition. Perceptual and Motor Skills 1982; 55: 651-653.
2) Shimizu Y. Tactile display terminal for the visually handicapped. Displays 1986; 7: 116-120.
3) Gibson JJ. Observation on active touch. Psychological Review 1966; 69: 477-491.
4) Heller M.A, Calcaterra JA, Burson LL, Tyler LA. Tactual picture identification by blind and sighted people: Effects of providing categorical information. Perception & Psychophysics 1996; 58: 310-323.
5) Klatzky RL, Lederman SJ. Toward a computational model of constraint-driven exploration and haptic object identification. Perception 1993; 22: 597-621.
6) Lederman SJ, Klatzky RL, Chataway C, Summers CD. Visual mediation and the haptic recognition of two-dimensional pictures of common objects. Perception & Psychophysics 1990; 47: 54-64.
7) Heller MA. Picture and pattern perception in the sighted and blind: The advantage of the late blind. Perception 1989; 18: 379-389.
8) Lederman SJ, Campbel JI. Tangible graphics for the blind. Human Factors 1982; 24: 85-100.
9) Shimizu Y, Saida S, Shimura H. Tactile pattern recognition by graphic display:Importance of 3-D information for haptic perception of familiar objects. Perception & Psychophysics 1993; 53: 43-48.
10) Shimizu Y, Shinohara M, Nagaoka H. Recognition of tactile patterns in a graphic display: Evaluation of presenting modes. Journal of Visual Impairment & Blindness 2000; July,: 456-461.
11) Shinohara M, Shimizu Y, Mochizuki A. Three-dimensional tactile display for the blind. IEEE Trans. Rehabilitation Engineering 1998; 6: 249-256.
12) Shinohara M, Saida S, Shimizu Y, Mochizuki A, Sorimachi K. Three-dimensional tactile display by multi-stage actuator. Journal of Robotics and Mechatronics 1992; 4: 231-236.
13) Kennedy JM. How the blind draw. Scientific American 1997; January: 76-81.