Singing Accuracy of Hearing Persons Using a Tactile Voice Pitch Feedback System

SAKAJIRI Masatsugu1), MIYOSHI Shigeki2), ONISHI Junji1), ONO Tsukasa1), IFUKUBE Tohru3) 1)Department of Information Technology, Faculty of Health Sciences,
Tsukuba University of Technology,
Kasuga 4-12-7, Tsukuba City, Ibaraki 305-8521, Japan E-mail: sakajiri@cs.k.tsukuba-tech.ac.jp
2)Division of Research on Support for the Hearing and Visually Impaired,
Research and Support Center on Higher Education for the Hearing and Visually Impaired, Tsukuba University of Technology
3)Institute of Gerontology,The University of Tokyo

Abstract: Those who are deafblind and hearing impaired cannot perceive their own voice pitch, and thus have difficulty controlling it. In particular, while singing, it is very difficult for them to control their voice pitch because they need to maintain a stable tone. Hence, we developed a tactile voice pitch control system using a two-dimensional tactile display to assist such people with singing. In a previous study, two deafblind participants used the proposed system to control their voice pitch with accuracy that was comparable to that of children who can hear. While auditory and proprioceptive feedback are significant cues for controlling pitch, tactile and proprioceptive feedback were used to control voice pitch. In the present study, we investigate the proprioceptive pitch control and effect of the proposed voice pitch control system on normal-hearing people under conditions of added noise. We describe the outline of these results.


Keywords: Voice pitch, Deafblind, Hearing impaired, Tactile display, Singing


1.	Introduction
Those who are deafblind and hearing impaired cannot perceive their own voice pitch, and thus have difficulty controlling it. While singing, it is very difficult for them to control their voice pitch because they need to maintain a stable tone. Hence, we have developed a tactile voice pitch control system to assist such people in singing [1]. This previous research showed that two deafblind participants who used this system could control their voice pitch with as much accuracy as children who could hear; this was achieved while singing the song “Frog Round” (“Kaeru-no-uta” in Japanese) [1]. We also investigated the pitch control accuracy of deafblind people while singing “Frog Round” and another song in which successive notes are separated by wider musical intervals [2]. The results showed that the proposed voice pitch control system allows for effective control of voice pitch not only for songs with adjacent notes but also for songs with separated notes. While auditory and proprioceptive feedback are significant cues for controlling pitch, we used tactile and proprioceptive feedback to control voice pitch. In this study, we investigate proprioceptive pitch control and the influence of the proposed voice pitch control system on the singing ability of normal-hearing people in a noise-added condition. We describe the outline of these results.


2.	Voice Pitch Control System
The voice pitch control system comprises a personal computer (PC) and the main unit of the tactile display. The participant places the center of the first joint of his or her right index finger on the tactile display to perceive the stimulus from the display.
 
A schematic view of the voice pitch control system is shown in Figure 1. In this system, the target musical scale is first set using a PC. Next, a deafblind person vocalizes so that this input produces the same musical scale as the target. The voice is input from a microphone connected to the PC. The musical note corresponding to the calculated voice pitch frequency is displayed on the PC. Similarly, a vibrotactile stimulus is presented corresponding to the position of the musical note on the tactile display. The tactile display   shown   in   Figure   1   comprises   64 piezoelectric vibrators in four rows (16 × 4 rows).
 

Figure 1. Schematic diagram of voice pitch control system.


 
Figure 2. Musical notation for “Frog Round” (“Kaeru-no-uta” in Japanese).
 
The two rows on the left side of the tactile display present the target musical scale, whereas the two rows on the right side present the musical scale corresponding to the voice pitch frequency of the user. When the target musical interval (left side) and the musical interval of the user’s voice (right side) are in the same position, the user’s pitch is 
on-key. One stimulus pin corresponds to a semitone with an equal temperament of 12 degrees. The male voice ranges from C4# to A2#, while the female voice ranges from C5# to A3#. A detailed description of the voice pitch control system appears in our previous study [1].


3.	Method
3.1.	Participants
Four females and two males who were normal-hearing university students participated. Their ages ranged from 20 to 24 years (M =
21.5 years).


3.2.	Procedure
The experiments used the voice-pitch control system shown in Figure 1. During the experiments, all participants wore sound-isolating headphones and used a head-mounted microphone positioned 1 cm in front of the left corner of the mouth. The sound pressure level of the noise was 100 dB. The participants were instructed to use the sound-level meter to maintain the intensity of their voices in the range of 55 to 85 dB.
Trials were implemented in two sessions. The first session (session 1) was without tactile feedback, whereas the second session (session 2) used tactile feedback. The term “without tactile feedback” means that the participants sang using only proprioceptive feedback (auditory feedback was masked). With tactile feedback, each participant sang using tactile feedback and proprioceptive feedback (auditory feedback was masked in an analogous way). In session 1, the participants sang Frog Round (Figure 2) five times without tactile feedback under the conditions described above. Before starting session 2, each participant practiced using the voice-pitch control system with tactile feedback for one hour. In the practice session, the participants first sang the notes do (C3), re, mi, fa, so, la, si, do (C4) in sequence to match the musical scale (pitch frequency) presented. Next, the participants repeated the process, but for descending musical notes [do (C4), si, la, so, fa, mi, re, do (C3)]. After this practice, session 2 started and the participants sang Frog Round five times with tactile feedback.
 

4.	Results and Discussion
Figures 3–8 show the average tone interval for participants A to F in session 1 (without tactile feedback) and session 2 (with tactile feedback). The horizontal axes are analogues in the musical scale corresponding to the notations shown in Figure 2. The vertical axis gives the musical interval with a cent value that is based on C3 (do); here, 1 semitone is 100 cents. C3 (do) is 0 cents, D3 (re) is 200 cents, E3 (mi) is 400 cents, G3 (so) is 700 cents, and A3 (la) is 900 cents. The horizontal red dashes show the target musical interval, the black squares show the average voice pitch from session 1, and the gray triangles show the average voice pitch from session 2. The error bars give the standard deviation. Without tactile feedback, all participants except participant B sang approximately 300 to 500 cents below the target pitch. However, with tactile feedback, all participants sang relatively accurately compared to the case without tactile feedback. Without tactile feedback, all participants (except B) sang with essentially the same deviation from the target pitch.
These results show that with noise added, those who can hear sing using proprioceptive feedback to control voice pitch. However,because there is no absolute standard against which to adjust voice pitch, very few participants could accurately control their pitch. In contrast, with tactile feedback, all were able to use the voice-pitch control system to adjust their voice pitch and sing at the absolute standard. In other words, normal-hearing people sing not only using their already acquired proprioceptive feedback ability but also using the absolute standard mean to adjust their voice pitch using our system.

Figure 3. Average for tone interval with participant A
Figure 4. Average for tone interval with participant B
Figure 5. Average for tone interval with participant C
Figure 6. Average for tone interval with participant D
Figure 7. Average for tone interval with participant E
Figure 8. Average for tone interval with participant F


5. Acknowledgements
This research was partially supported by the Ministry of Education, Science, Sports and Culture Grant-in-Aid for Scientific Research
(C) 15K01015, (B) 26285210, (B) 26280070 and Special Research Funds from the Tsukuba University of Technology. This paper is an excerpt from a paper presented at the 2014 IEEE Symposium Series on Computational Intelligence in Robotic Rehabilitation and Assistive Technologies (IEEE SSCI2014) in Orlando, Florida, United States of America [3].


References
[1] Sakajiri M, Miyoshi S, Nakamura K, Fukushima S, Ifukube T. Voice Pitch Control Using Tactile Feedback for the Deafblind or Hearing Impaired Persons to Assist Their Singing. IEEE International Conference on Systems, Man, and Cybernetics; 2010: pp. 1483-1487.
[2] Sakajiri M, Miyoshi S, Nakamura K, Fukushima S, Ifukube T. Accuracy of Voice Pitch Control in Singing Using Tactile Voice Pitch Feedback Display. IEEE International Conference on Systems, Man, and Cybernetics; 2013: pp. 4201-4206.
[3] Sakajiri M, Miyoshi S, Onishi J, Ono T, Ifukube T. Tactile Pitch Feedback System for Deafblind or Hearing Impaired Persons. Singing Accuracy of Hearing Persons Under Conditions of Added Noise. IEEE Symposium Series on Computational Intelligence in Robotic Rehabilitation and Assistive Technologies; 2014: pp. 31-35.