Analysis of Emotions Using Multimodal Data: A Case Study
Analysis of Emotions Using Multimodal Data: A Case Study作者机构：Graduate School of Health Sciences Tokushima University Tokushima Japan Department of Psychiatric Nursing Nursing Course of Kochi Medical School Kochi University Kochi Japan Department of Nursing Art and Sciences Tokushima University Tokushima Japan Department of Physical Therapy Hiroshima Cosmopolitan University Hiroshima Japan Department of Nursing Outcomes Management Institute of Biomedical Sciences Tokushima University Tokushima Japan
出 版 物：《Journal of Biosciences and Medicines》 (生物科学与医学（英文）)
年 卷 期：2023年第11卷第12期
页 面：54-68页
主 题：Humanoid Robots Multimodal Data Emotion Analysis
摘 要：In this case study, we hypothesized that sympathetic nerve activity would be higher during conversation with PALRO robot, and that conversation would result in an increase in cerebral blood flow near the Broca’s area. The facial expressions of a human subject were recorded, and cerebral blood flow and heart rate variability were measured during interactions with the humanoid robot. These multimodal data were time-synchronized to quantitatively verify the change from the resting baseline by testing facial expression analysis, cerebral blood flow, and heart rate variability. In conclusion, this subject indicated that sympathetic nervous activity was dominant, suggesting that the subject may have enjoyed and been excited while talking to the robot (normalized High Frequency normalized Low Frequency: 0.22 ± 0.16 0.78 ± 0.16). Cerebral blood flow values were higher during conversation and in the resting state after the experiment than in the resting state before the experiment. Talking increased cerebral blood flow in the frontal region. As the subject was left-handed, it was confirmed that the right side of the brain, where the Broca’s area is located, was particularly activated (Left right: 0.15 ± 0.21 1.25 ± 0.17). In the sections where a “happy facial emotion was recognized, the examiner-judged “happy faces and the MTCNN “happy results were also generally consistent.
维普期刊数据库