{"id":{"repo_id":"houston","oai_identifier":"oai:uh-ir.tdl.org:10657/17765"},"canonical_url":"https://search.dev.ndltd.org/etd/houston/oai:uh-ir.tdl.org:10657/17765","repository":{"repo_id":"houston","name":"University of Houston","base_url":"https://uh-ir.tdl.org/server/oai/request"},"display":{"title":"A Mixed Emotions Framework And Its Applications In Affective Computing","abstract":"Traditionally, computer vision has categorized displayed emotions into seven basic categories.: sadness, happiness, anger, fear, disgust, surprise, and neutral. This approach is restrictive because human emotions do not always fall neatly into these seven categories but often cross categorical borders, forming interesting mixtures. To address this problem, we developed a methodology based on two components: 1) CNN for simple emotions: A convolutional neural network (CNN) that identifies the seven emotions. We validated this CNN on RAVDESS, a well-known dataset of facial videos of people expressing distinct emotions while talking. 2) Co-occurrence matrix for mixed emotions: A post-hoc mixed emotion generation method, where the original seven-emotion probabilistic vector output by the CNN is used in an outer-product with itself to produce a co-occurrence matrix. The diagonal of this co-occurrence matrix holds the adjusted probabilities of the seven emotions. In contrast, the upper and lower triangles hold all their pair-wise combinations, which contain the corresponding probabilities of mixed emotions. The methodology takes probabilistic strength away from the original seven-emotion vector and allocates it to mixed emotions. We tested the novel mixed-emotion vs. the conventional seven-emotion methodologies in two naturalistic experiments: First, in an experiment focusing on the emotional effects of email interruptions during cognitive work, and second, in an experiment focusing on the emotional effects of online public speaking. In the email interruption experiment, the mixed-emotion methodology uniquely determined that, unlike knowledge workers who work uninterrupted, knowledge workers who are frequently interrupted by emails tend to display a mixture of sadness and fear on their faces. The latter presumably results from re-occurring negative stimuli in random email arrivals. The mixed-emotion methodology in the online public speaking experiment uniquely determined that more conscientious participants displayed a preponderance of mixed emotions concerning less conscientious participants. Mixed emotions represent a moderation of pure emotions, like angry-neutral vs. totally angry, and constitute a more acceptable visual communication between the speaker and the audience. Altogether, the mixed-emotion methodology significantly enhances analytical insight across different experimental settings.","abstract_html":"Traditionally, computer vision has categorized displayed emotions into seven basic categories.: sadness, happiness, anger, fear, disgust, surprise, and neutral. This approach is restrictive because human emotions do not always fall neatly into these seven categories but often cross categorical borders, forming interesting mixtures. To address this problem, we developed a methodology based on two components: 1) CNN for simple emotions: A convolutional neural network (CNN) that identifies the seven emotions. We validated this CNN on RAVDESS, a well-known dataset of facial videos of people expressing distinct emotions while talking. 2) Co-occurrence matrix for mixed emotions: A post-hoc mixed emotion generation method, where the original seven-emotion probabilistic vector output by the CNN is used in an outer-product with itself to produce a co-occurrence matrix. The diagonal of this co-occurrence matrix holds the adjusted probabilities of the seven emotions. In contrast, the upper and lower triangles hold all their pair-wise combinations, which contain the corresponding probabilities of mixed emotions. The methodology takes probabilistic strength away from the original seven-emotion vector and allocates it to mixed emotions. We tested the novel mixed-emotion vs. the conventional seven-emotion methodologies in two naturalistic experiments: First, in an experiment focusing on the emotional effects of email interruptions during cognitive work, and second, in an experiment focusing on the emotional effects of online public speaking. In the email interruption experiment, the mixed-emotion methodology uniquely determined that, unlike knowledge workers who work uninterrupted, knowledge workers who are frequently interrupted by emails tend to display a mixture of sadness and fear on their faces. The latter presumably results from re-occurring negative stimuli in random email arrivals. The mixed-emotion methodology in the online public speaking experiment uniquely determined that more conscientious participants displayed a preponderance of mixed emotions concerning less conscientious participants. Mixed emotions represent a moderation of pure emotions, like angry-neutral vs. totally angry, and constitute a more acceptable visual communication between the speaker and the audience. Altogether, the mixed-emotion methodology significantly enhances analytical insight across different experimental settings.","abstract_has_math":false,"creators":["Wesley, Amanveer"],"institution":"University of Houston","degree_name":"Doctor of Philosophy","degree_level":"Doctoral","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":[],"advisors":["Pavlidis, Ioannis"],"committee_chairs":[],"committee_members":["Vilalta, Ricardo","Tsekos, Nikolaos","Cirino, Paul"],"year":2024,"date_issued":"2024-04-28","date_published":"2024-04-28","updated_at":"2026-07-24T02:33:01Z","subjects":["Affective Computing","Valence","Mixed Emotions"],"languages":["en"],"rights":[],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/10657/17765","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Pavlidis, Ioannis"]},{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Vilalta, Ricardo","Tsekos, Nikolaos","Cirino, Paul"]},{"key":"dc:creator","label":"Author","values":["Wesley, Amanveer"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2024-07-27T18:32:15Z"]},{"key":"dc:date.issued","label":"Date","values":["2024-04-28"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Doctoral"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Houston"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Affective Computing","Valence","Mixed Emotions"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/10657/17765"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Traditionally, computer vision has categorized displayed emotions into seven basic categories.: sadness, happiness, anger, fear, disgust, surprise, and neutral. This approach is restrictive because human emotions do not always fall neatly into these seven categories but often cross categorical borders, forming interesting mixtures. To address this problem, we developed a methodology based on two components: 1) CNN for simple emotions: A convolutional neural network (CNN) that identifies the seven emotions. We validated this CNN on RAVDESS, a well-known dataset of facial videos of people expressing distinct emotions while talking. 2) Co-occurrence matrix for mixed emotions: A post-hoc mixed emotion generation method, where the original seven-emotion probabilistic vector output by the CNN is used in an outer-product with itself to produce a co-occurrence matrix. The diagonal of this co-occurrence matrix holds the adjusted probabilities of the seven emotions. In contrast, the upper and lower triangles hold all their pair-wise combinations, which contain the corresponding probabilities of mixed emotions. The methodology takes probabilistic strength away from the original seven-emotion vector and allocates it to mixed emotions. We tested the novel mixed-emotion vs. the conventional seven-emotion methodologies in two naturalistic experiments: First, in an experiment focusing on the emotional effects of email interruptions during cognitive work, and second, in an experiment focusing on the emotional effects of online public speaking. In the email interruption experiment, the mixed-emotion methodology uniquely determined that, unlike knowledge workers who work uninterrupted, knowledge workers who are frequently interrupted by emails tend to display a mixture of sadness and fear on their faces. The latter presumably results from re-occurring negative stimuli in random email arrivals. The mixed-emotion methodology in the online public speaking experiment uniquely determined that more conscientious participants displayed a preponderance of mixed emotions concerning less conscientious participants. Mixed emotions represent a moderation of pure emotions, like angry-neutral vs. totally angry, and constitute a more acceptable visual communication between the speaker and the audience. Altogether, the mixed-emotion methodology significantly enhances analytical insight across different experimental settings."]},{"key":"dc:format.mimetype","label":"Dc Format Mimetype","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["A Mixed Emotions Framework And Its Applications In Affective Computing"]}]}],"canonical_facts":{"dc:contributor.advisor":["Pavlidis, Ioannis"],"dc:contributor.committeemember":["Vilalta, Ricardo","Tsekos, Nikolaos","Cirino, Paul"],"dc:creator":["Wesley, Amanveer"],"dc:date.accessioned":["2024-07-27T18:32:15Z"],"dc:date.issued":["2024-04-28"],"dc:description.abstract":["Traditionally, computer vision has categorized displayed emotions into seven basic categories.: sadness, happiness, anger, fear, disgust, surprise, and neutral. This approach is restrictive because human emotions do not always fall neatly into these seven categories but often cross categorical borders, forming interesting mixtures. To address this problem, we developed a methodology based on two components: 1) CNN for simple emotions: A convolutional neural network (CNN) that identifies the seven emotions. We validated this CNN on RAVDESS, a well-known dataset of facial videos of people expressing distinct emotions while talking. 2) Co-occurrence matrix for mixed emotions: A post-hoc mixed emotion generation method, where the original seven-emotion probabilistic vector output by the CNN is used in an outer-product with itself to produce a co-occurrence matrix. The diagonal of this co-occurrence matrix holds the adjusted probabilities of the seven emotions. In contrast, the upper and lower triangles hold all their pair-wise combinations, which contain the corresponding probabilities of mixed emotions. The methodology takes probabilistic strength away from the original seven-emotion vector and allocates it to mixed emotions. We tested the novel mixed-emotion vs. the conventional seven-emotion methodologies in two naturalistic experiments: First, in an experiment focusing on the emotional effects of email interruptions during cognitive work, and second, in an experiment focusing on the emotional effects of online public speaking. In the email interruption experiment, the mixed-emotion methodology uniquely determined that, unlike knowledge workers who work uninterrupted, knowledge workers who are frequently interrupted by emails tend to display a mixture of sadness and fear on their faces. The latter presumably results from re-occurring negative stimuli in random email arrivals. The mixed-emotion methodology in the online public speaking experiment uniquely determined that more conscientious participants displayed a preponderance of mixed emotions concerning less conscientious participants. Mixed emotions represent a moderation of pure emotions, like angry-neutral vs. totally angry, and constitute a more acceptable visual communication between the speaker and the audience. Altogether, the mixed-emotion methodology significantly enhances analytical insight across different experimental settings."],"dc:format.mimetype":["application/pdf"],"dc:identifier.uri":["https://hdl.handle.net/10657/17765"],"dc:language.iso":["en"],"dc:subject":["Affective Computing","Valence","Mixed Emotions"],"dc:title":["A Mixed Emotions Framework And Its Applications In Affective Computing"],"dc:type":["Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Doctoral"],"thesis:degree_name":["Doctor of Philosophy"],"thesis:institution_name":["University of Houston"]},"updated_at":"2026-07-24T02:33:01Z"}