On the Recognition of Timbre HIRAGA Rumi Tsukuba University of Technology 4-3-15 Amakubo, Tsukuba, 305-8520 Japan E-mail: rhiraga@a.tsukuba-tech.ac.jp, phone/fax: +81-29-858-9384 Abstract: Many hearing-impaired college students have interests in music, and some of them listen to music every day. What do hearing-impaired people enjoy in music? Among the several elements in music, focus was placed on timbre in an experiment. Timbre differs if a musician performs a score with different musical instruments. Several sets of musical data were prepared from commercial CDs; the same phrases were played with a variety of musical instruments, and subjects were asked to listen and compare the timbres. Subject groups consisted of hearing-impaired people, hearing individuals with little or no experience in playing music, and hearing people who regularly play music. Significant differences were found in the ability to differentiate timbre among the three groups. Findings showed that hearing-impaired people effectively noted sets in which all timbres were similar, but they encountered difficulties in differentiating timbres in other cases. The brightness curve was identified as a possibility for explaining such difficulties. Keywords: Hearing-impaired people, Music activities, Recognition, Timbre 1. Introduction Timbre is modeled by integrating the concepts of color and sound texture, where color is an instantaneous spectral envelope and texture is the temporal nature of the sound; thus, sequential changes in color occur according to an arbitrary time scale. Timbre is also considered subjectively as a multidimensional attribute. In psychological experiments, subjects have been presented with a set of semantic scales for rating activities. For example, scales may assess gradations of light/dark, sharp/dull, and cold/warm. Hearing people enjoy listening to several different arrangements of musical performances. Is this way of enjoying music common to hearing-impaired people? To find out, we conducted an experiment regarding differentiating timbre for the same phrase from two musical pieces. This research is related to the area of music recognition in terms of timbre perception and the music recognition ability of hearing-impaired people. The results showed that there were significant differences in the ability to differentiate timbre between the hearing-impaired and the hearing subject groups. Hearing-impaired people effectively noted sets in which all timbres were similar. In other cases, however, they experienced difficulties in differentiation of timbres. A brightness curve was identified as a possibility for explaining the reason for such difficulty. 2. Experiment The purpose of our experiment on timbre was to ascertain what hearing-impaired people enjoy when listening to music and to identify whether they could recognize timbre in doing so. Subjects listened to several sets of musical fragments, and they were asked to judge whether they could perceive similar timbres within a set. 2.1. Materials We used commercial music CDs to prepare several sets of music fragments from the same phrases. We used the musical phrases from the beginning four measures of Claude Debussy’s “Claire de Lune” and the chorus from Joe Hisaishi’s theme song for the animation movie “My Neighbor Totoro.” For each musical piece, we prepared six musical fragments played with different instruments. Each set consisted of three identical musical phrases (i.e., “musical fragments”). We used three types of fragment sets: in one, all three musical fragments were similar in terms of timbre; in another, all three musical fragments were different in terms of timbre; and in the other, two of the three musical fragments were similar. We created eight fragment sets for “Claire de Lune” and nine sets for the “My Neighbor Totoro” theme. 2.2. Subjects The three subject groups described below participated in the experiment. 1. Hearing-impaired people included eight males and one female ranging in age from 20 to 23 years. All of them wore hearing aids, but none of them made use of cochlear implants. 2. Hearing people who did not regularly play or listen to music included three males and five females ranging in age from 35 to 54 years. 3. Hearing people who regularly played or listened to music included two males and five females ranging in age from 43 to 45 years. 2.3. Procedure Musical fragment sets were provided to subjects in the form of a Microsoft Excel file. Subjects listened to three musical fragments in each set and answered the following two questions. 1. How difficult was it for you to recognize timbre similarity? Choose one from “Difficult,” “Neither difficult nor easy,” and “Easy.” 2. How similar did you feel the musical fragments were in terms of timbre for the melody? Choose one from the following: 1) All three musical fragments were similar. 2) All three musical fragments were different. 3) Two of the three musical fragments were similar. If the answer was 3), then subjects were asked to name the two musical fragments that they felt were similar. 3. Results 3.1. Differences between subject groups One-way Analysis of Variance (ANOVA) showed that there were significant differences between subject groups. The average correct rates for hearing-impaired people (HI), hearing people with little music activity experience (HL), and hearing people with music activity experience (HE) were, respectively, 0.68, 0.93, and 0.97. The p-value of 4.60e-07 showed that there were significant differences between HI, HL, and HE. Multiple comparisons showed significant differences between HI and HL as well as HI and HE. 3.2. Differences between musical phrases There were no differences between the two musical phrases (p-value=0.31). 3.3. Differences between musical fragment sets Claire de Lune The p-value obtained by calculating ANOVA was 0.15, which means that there were no significant differences between fragment sets. On the other hand, the p-value was 0.021 when ANOVA was calculated for hearing-impaired subjects only. My Neighbor Totoro The p-value was 0.28 for all subjects but 0.25 for hearing-impaired subjects only. For the “My Neighbor Totoro” phrase, there were no significant differences between fragment sets. 4. Discussion 4.1. Recognizing similarities in timbres For both musical phrases, the correct answer rates were the highest if a set consisted of a unique timbre. This indicates that for hearing-impaired subjects, similarity is perceived more easily when the number of similar timbres is large. 4.2. Confusing timbres and their attributes There were sets for which less than half of the hearing-impaired subjects answered correctly. They had difficulties in differentiating timbres of the flute and a full orchestra. Among the timbre attributes we calculated with MIRToolbox on Matlab, the brightness curve was a candidate for explaining the confusion between timbres of the flute and orchestra. Brightness shows sound energy at a higher frequency. When a frequency value is assigned, the ratio of energy above the frequency is returned by a function of MIRToolbox. The brightness curves show how energy decreases according to different frequency values. The higher energy and higher frequency values for timbres of the flute and orchestra as compared to other timbres may create confusion. 4.3. Self-assessment The results of self-assessment regarding the difficulty in differentiating timbres for the two hearing subject groups were similar, but hearing-impaired subjects used a different standard to make their assessments. We did not find any relationships between self-assessment and correct answers for all subject groups, though. Acknowledgment This research was supported by Special Research Funds from the Tsukuba University of Technology. This paper is an excerpt from a paper presented at the IEEE SMC 2012 Conference held in Seoul [1]. References [1] R. Hiraga and K. Otsuka, “On the recognition of timbre, a first step toward understanding how hearing-impaired people perceive timbre,” IEEE SMC 2012, 2012: 2013-2018.