Image Properties and Emotion Elicited

Rumi HIRAGA

Faculty of Industrial Technology, Tsukuba University of Technology
4-3-15 Amakubo, Tsukuba, 305-8520 Japan
email: rhiraga@a.tsukuba-tech.ac.jp,phone/fax: +81-29-858-9384

Abstract: We investigated the relationship between image properties and emotion elicited. Using close-shaped drawings that are intending to express a specific emotion, we grouped them by k-means clustering on the principal component analysis space, asked subjects to categorize them into groups depending on how they felt when they view them, and then compared the results by the clustering and categorization. Two principal components are good to group images into four clusters by emotions. The results showed that the viewers made their judgment base on the specific meaning of the drawing rather than the shape, if the shape of drawings was not abstract. Our future work includes to find out another set of properties for clustering and to use various types of images for analysis.
Keywords: images, emotion, categorization, clustering

1. Introduction
 We have the experience of getting the positive response of the students with hearing impairment to music that encourages us to develop a musical performance assistance system with which deaf and hard-of-hearing people play percussion instruments. With our objective of enhancing the use of media augmentation to increase emotional understanding to music communication and having completed the first step of conducting cognitive experiments, we proceeded to the next step: identifying the relationship between the properties of an image and the emotion it elicits.

2. Related Works
 Research by Pavlova et al. [1], Oyama et al. [2], and Yamaguchi and Shiina [3] focused on the causal relationship between emotions and images with an abstract shape. Their material drawings coincidentally resemble that of ours; they were close and abstract shaped. Since their explanations of the relationship are subjective, it is hard for them to reconstruct images with image features. We used the image properties to elicit a causal relationship between the image and an emotion to explain the relationship quantitatively.

3. Image Analysis
 The purpose of the image analysis is to identify the relationships between image properties and the emotions of joy, fear, anger, and sadness. The results are useful to determine the emotion represented by an image without an intended emotion. This will enable us to add new images to our performance assistance system.
 We prepared drawings as shown in Table 1. The images were made by students who study design using a 24 cm long white braid. We asked them to make the drawings in an abstract shape as best they can. The shapes were glued to black Kent sheets, then scanned and converted into image files, which were black lined shapes on a white background.
3.1 Categorization
 We asked 20 participants to group in four emotions depending on how they felt when they view the drawings. If an emotion chosen by a participant was the same as the emotion that was originally intended to express with the image, then the emotion of the image was regarded to be categorized.
3.2 Clustering
 Using the property values obtained by the function of Matlab’s Image Processing Toolbox, we calculated the principal components and classified the images into four groups by k-means clustering on the principal components space.
 We set a threshold to determine whether a clustering is successfully done. A clustering is successful only if the three conditions are satisfied; 1) the number of elements in all clusters should be over 7 and below 12, 2) a cluster is uniquely labeled depending on majority emotions of the elements, and 3) the labels are unique to four clusters.
 When an image is clustered to a group whose label is the same as the emotion that was originally intended to express with the image, then the emotion of the image was regarded to be clustered.

4. Results
 Table 2 shows the clustering rates and categorization rates. The categorization rate of the emotion of fear is significantly different from other rates, while there were no significant differences among the rates in clustering.
 Among the 100,000 times of clustering, about 10% was regarded to be successful. Figure 1 shows the cluster centroids of all the successful clustering where the x-axis represents the first principal component and the y-axis represents the second principal component. The sum of the first and second contribution ratios was 76%. Red diamonds show the centroids of clusters labeled joy, blue circles show that labeled fear, green stars show that labeled anger, and light blue squares show that labeled sadness. We can see the first principal component discriminates fear from joy and the second principal component discriminates anger from sadness.

Table 1 Drawings used for recognition and analysis
Table 2 Average rates

5. Discussion
 By comparing the two rates on each drawing, we found that J8 and J9 illustrate a typical difference between the two methods; the viewers had a high recognition rate while clustering failed completely.
 This implies that if an image has a specific meaning, viewers judge the emotion by the meaning but not by the image properties.
 Three images, J3, J6, and S3 are reliable because they have high rates in both clustering and categorization. Thus, in spite of the two metrics we obtained, namely the two principal components, that are able to cluster images into four groups, the clustering does not work to be equivalent to the categorization. Our future work will include to investigate the use of additional image properties in our analysis. Candidate properties are the coefficients of elliptic Fourier transformation and the line length distribution. In this experiment, we analyzed simple images with little variety; close-shaped, fixed line width, and monocolor. Analysis of images with color, texture, and various line types should lead to interesting results for the relationship between the combination of image properties and emotion.
 Then we will proceed to enhance our musical performance assistance system to include anonymous images and video sequences.

Figure 1 Cluster centroids

Acknowledgment
 This research was supported by the Kayamori Foundation of Information Science Advancement and by Special Research Funds from the Tsukuba University of Technology. This paper is the excerpt from a paper presented at 2010 IEEE International Conference on Systems, Man, and Cybernetics in Istanbul, Turkey[4].

References
[1] M. Pavlova, A. Sokolov, and A. Sokolov, “Perceived dynamics of static images enables emotional attribution,” Perception, 34, pp. 1107-1116, 2005.
[2] T. Oyama, H. Miyano, and H. Yamada, “Multidimensional scaling of computer-generated abstract forms,” Proc. of the Int’l Meeting of the Psychometric Society IMPS 2001, p. 551, 2001.
[3] Y. Yamaguchi and K. Shiina, “Communication of affective state through the hand-drawn line-figures,” Library, information and media studeis, 3(2), pp. 15-23, 2005.
[4] R. Hiraga and N. Matsuda, “Emotion-intending Drawings, relationship between image properties and emotion elicited,” IEEE SMC 2010, pp. 2163-2168, 2010.