Multi-modal emotion detection using deep learning for interpersonal communication analytics

In recent years, deep learning technologies have been increasingly applied to generate meaningful data for advanced research in humanities and sciences. Interpersonal communication skills are crucial to success in science. Communication skills, either in a small group learning environment or a large group setting, are always useful in any future workplace. In this study, we aim to analyze mutual communication and interactions between speakers/audiences from a broader perspective, including emotional and cognitive interactions, in TED talk or classroom settings. We are mainly interested in the recognition of facial and gesture emotions captured in such contexts. More specifically, we proposed a multi-modal emotion detection approach for facial expression, e.g., facial sentiment, gender, age, ethnicity, hairstyles, as well as gesture expression, e.g., sitting, standing, raising their hands, folded hands and crossed legs. The real-time feedback of the proposed system on individual or group communication can effectively be used for improving their communication skills.

Name thesis:degree_name

M.S. (Master of Science)

Level thesis:degree_level

M.S.

Discipline thesis:degree_discipline

Computer Science (UMKC)

Grantor

University of Missouri--Kansas City

Year dc:date.issued

2019

Author dc:creator

• Gogadi, Sravanthi

Advisor dc:contributor.advisor

• Lee, Yugyung, 1960-

Handle dc:identifier.uri

https://hdl.handle.net/10355/71101

OAI identifier oai:identifier

oai:mospace.umsystem.edu:10355/71101