{"id":{"repo_id":"abertay","oai_identifier":"oai:rke.abertay.ac.uk:studenttheses/8ce7da5a-d034-4d9d-800f-f0e4d0e13c0a"},"canonical_url":"https://search.dev.ndltd.org/etd/abertay/oai:rke.abertay.ac.uk:studenttheses/8ce7da5a-d034-4d9d-800f-f0e4d0e13c0a","repository":{"repo_id":"abertay","name":"Abertay University","base_url":"https://rke.abertay.ac.uk/ws/oai"},"display":{"title":"Using affective computing to enhance the believability of virtual agents and the social interaction with users","abstract":"The drive to create believable virtual agents capable of engaging in natural social interactions with humans has fuelled extensive research efforts in affective computing, the field dedicated to developing systems that recognize and adapt to human emotions. Despite advancements in recent decades, the field continues to face challenges related to (1) responding accurately to human emotions and fostering a continuous affective loop, (2) finding reliable stimuli to elicit a wide range of affective states, (3) processing affective cues appropriately and (4) generalizing affect recognition across contexts. In an attempt to address these issues, this work proposed identifying general affective states that include thematically similar emotions. Equipping agents with a model capable of detecting such sets would enable them to provide thematically close responses, enhancing their emotional coherence.
To assess the potential of the approach, the first phase of the project used the eight subspaces of the Pleasure-Arousal-Dominance (PAD) model to define the general affective states of the user and to design agent responses. An experiment was conducted and included interactions with two agents that dynamically adjusted facial expressions and text in response to the eight general states. The second phase of the project explored horror games as a medium to elicit anxiety-related emotions within the ”Anxious” PAD subspace. A new tool was designed to help collect affect annotation over time. These annotations, along with physiological measures generated a new dataset. Using this dataset, classifiers were built to identify subclasses within the P, A and D components of the selected subspace, while investigating how varying the size and offset of time windows for processing physiological signals affected the classifiers' performance. The final phase of the project explored routes for context generalization using the proposed adaptive windowing technique.
Results indicated a positive overall perception of the agent’s believability when responding to eight general affective states, with further improvement observed when changes in the user’s physiological arousal were included. Adjusting the window settings for processing physiological signals led to improved prediction performance. Moreover, the implementation of the adaptive windowing technique demonstrated the potential to improve the model's ability to better characterize affective states across diverse datasets.","abstract_html":"The drive to create believable virtual agents capable of engaging in natural social interactions with humans has fuelled extensive research efforts in affective computing, the field dedicated to developing systems that recognize and adapt to human emotions. Despite advancements in recent decades, the field continues to face challenges related to (1) responding accurately to human emotions and fostering a continuous affective loop, (2) finding reliable stimuli to elicit a wide range of affective states, (3) processing affective cues appropriately and (4) generalizing affect recognition across contexts. In an attempt to address these issues, this work proposed identifying general affective states that include thematically similar emotions. Equipping agents with a model capable of detecting such sets would enable them to provide thematically close responses, enhancing their emotional coherence. <br/><br/>To assess the potential of the approach, the first phase of the project used the eight subspaces of the Pleasure-Arousal-Dominance (PAD) model to define the general affective states of the user and to design agent responses. An experiment was conducted and included interactions with two agents that dynamically adjusted facial expressions and text in response to the eight general states. The second phase of the project explored horror games as a medium to elicit anxiety-related emotions within the ”Anxious” PAD subspace. A new tool was designed to help collect affect annotation over time. These annotations, along with physiological measures generated a new dataset. Using this dataset, classifiers were built to identify subclasses within the P, A and D components of the selected subspace, while investigating how varying the size and offset of time windows for processing physiological signals affected the classifiers' performance. The final phase of the project explored routes for context generalization using the proposed adaptive windowing technique.<br/><br/>Results indicated a positive overall perception of the agent’s believability when responding to eight general affective states, with further improvement observed when changes in the user’s physiological arousal were included. Adjusting the window settings for processing physiological signals led to improved prediction performance. Moreover, the implementation of the adaptive windowing technique demonstrated the potential to improve the model's ability to better characterize affective states across diverse datasets.","abstract_has_math":false,"creators":["Graja, Sarra"],"institution":"Abertay University","degree_name":"PhD","degree_level":"Doctoral Thesis","degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":["Lovell, Paul","Fee, Kenneth"],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-2-25","date_published":"2025-2-25","updated_at":"2026-07-24T00:50:29Z","subjects":["Affective Computing","Virtual agents","HCI","Human-computer interaction","Emotions","Multimodal dataset","Affective feedback","Annotation tools","AffectRankTrace","Pleasure Arousal Dominance","PAD","Anxiety","Physiology","Heart Rate Variability","Respiration","Galvanic Skin Response","Electrocardiography","Anxiety-related emotions","General affective state","Emotion subspaces","Thematically related emotions","Agent behaviour","Affect aware agent","Believability","Emotion adaptation","Affective loop","Signal processing","Classification","Machine learning","Time windows segmentation","Emotion recognition","Anxious"],"languages":["eng"],"rights":[],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["oai:rke.abertay.ac.uk:studenttheses/8ce7da5a-d034-4d9d-800f-f0e4d0e13c0a"],"render_values":[{"text":"oai:rke.abertay.ac.uk:studenttheses/8ce7da5a-d034-4d9d-800f-f0e4d0e13c0a","href":null,"code":true}]}]},"links":{"outbound_url":"https://rke.abertay.ac.uk/en/studentTheses/8ce7da5a-d034-4d9d-800f-f0e4d0e13c0a","outbound_label":"Repository record","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Lovell, Paul","Fee, Kenneth"]},{"key":"dc:creator","label":"Author","values":["Graja, Sarra"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-2-25"]},{"key":"dc:date.issued","label":"Date","values":["2025-2-25"]},{"key":"dc:publisher.department","label":"Dc Publisher Department","values":["Faculty of Design, Informatics and Business"]},{"key":"dc:publisher.institution","label":"Dc Publisher Institution","values":["Abertay University"]},{"key":"dc:relation.isreferencedby","label":"Dc Relation Isreferencedby","values":["https://rke.abertay.ac.uk/en/studentTheses/8ce7da5a-d034-4d9d-800f-f0e4d0e13c0a"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"dc:type.qualificationlevel","label":"Dc Type Qualificationlevel","values":["Doctoral Thesis"]},{"key":"dc:type.qualificationname","label":"Dc Type Qualificationname","values":["PhD"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Affective Computing","Virtual agents","HCI","Human-computer interaction","Emotions","Multimodal dataset","Affective feedback","Annotation tools","AffectRankTrace","Pleasure Arousal Dominance","PAD","Anxiety","Physiology","Heart Rate Variability","Respiration","Galvanic Skin Response","Electrocardiography","Anxiety-related emotions","General affective state","Emotion subspaces","Thematically related emotions","Agent behaviour","Affect aware agent","Believability","Emotion adaptation","Affective loop","Signal processing","Classification","Machine learning","Time windows segmentation","Emotion recognition","Anxious"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["eng"]},{"key":"dc:rights.embargodate","label":"Dc Rights Embargodate","values":["2027-06-30"]},{"key":"dc:rights.embargoreason","label":"Dc Rights Embargoreason","values":["/dk/atira/pure/core/document/studentthesisembargoreason/publicationissues"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["oai:rke.abertay.ac.uk:studenttheses/8ce7da5a-d034-4d9d-800f-f0e4d0e13c0a","https://rke.abertay.ac.uk/en/studentTheses/8ce7da5a-d034-4d9d-800f-f0e4d0e13c0a"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["The drive to create believable virtual agents capable of engaging in natural social interactions with humans has fuelled extensive research efforts in affective computing, the field dedicated to developing systems that recognize and adapt to human emotions. Despite advancements in recent decades, the field continues to face challenges related to (1) responding accurately to human emotions and fostering a continuous affective loop, (2) finding reliable stimuli to elicit a wide range of affective states, (3) processing affective cues appropriately and (4) generalizing affect recognition across contexts. In an attempt to address these issues, this work proposed identifying general affective states that include thematically similar emotions. Equipping agents with a model capable of detecting such sets would enable them to provide thematically close responses, enhancing their emotional coherence.
To assess the potential of the approach, the first phase of the project used the eight subspaces of the Pleasure-Arousal-Dominance (PAD) model to define the general affective states of the user and to design agent responses. An experiment was conducted and included interactions with two agents that dynamically adjusted facial expressions and text in response to the eight general states. The second phase of the project explored horror games as a medium to elicit anxiety-related emotions within the ”Anxious” PAD subspace. A new tool was designed to help collect affect annotation over time. These annotations, along with physiological measures generated a new dataset. Using this dataset, classifiers were built to identify subclasses within the P, A and D components of the selected subspace, while investigating how varying the size and offset of time windows for processing physiological signals affected the classifiers' performance. The final phase of the project explored routes for context generalization using the proposed adaptive windowing technique.
Results indicated a positive overall perception of the agent’s believability when responding to eight general affective states, with further improvement observed when changes in the user’s physiological arousal were included. Adjusting the window settings for processing physiological signals led to improved prediction performance. Moreover, the implementation of the adaptive windowing technique demonstrated the potential to improve the model's ability to better characterize affective states across diverse datasets."]},{"key":"dc:title","label":"Title","values":["Using affective computing to enhance the believability of virtual agents and the social interaction with users"]}]}],"canonical_facts":{"dc:contributor.advisor":["Lovell, Paul","Fee, Kenneth"],"dc:creator":["Graja, Sarra"],"dc:date":["2025-2-25"],"dc:date.issued":["2025-2-25"],"dc:description.abstract":["The drive to create believable virtual agents capable of engaging in natural social interactions with humans has fuelled extensive research efforts in affective computing, the field dedicated to developing systems that recognize and adapt to human emotions. Despite advancements in recent decades, the field continues to face challenges related to (1) responding accurately to human emotions and fostering a continuous affective loop, (2) finding reliable stimuli to elicit a wide range of affective states, (3) processing affective cues appropriately and (4) generalizing affect recognition across contexts. In an attempt to address these issues, this work proposed identifying general affective states that include thematically similar emotions. Equipping agents with a model capable of detecting such sets would enable them to provide thematically close responses, enhancing their emotional coherence.
To assess the potential of the approach, the first phase of the project used the eight subspaces of the Pleasure-Arousal-Dominance (PAD) model to define the general affective states of the user and to design agent responses. An experiment was conducted and included interactions with two agents that dynamically adjusted facial expressions and text in response to the eight general states. The second phase of the project explored horror games as a medium to elicit anxiety-related emotions within the ”Anxious” PAD subspace. A new tool was designed to help collect affect annotation over time. These annotations, along with physiological measures generated a new dataset. Using this dataset, classifiers were built to identify subclasses within the P, A and D components of the selected subspace, while investigating how varying the size and offset of time windows for processing physiological signals affected the classifiers' performance. The final phase of the project explored routes for context generalization using the proposed adaptive windowing technique.
Results indicated a positive overall perception of the agent’s believability when responding to eight general affective states, with further improvement observed when changes in the user’s physiological arousal were included. Adjusting the window settings for processing physiological signals led to improved prediction performance. Moreover, the implementation of the adaptive windowing technique demonstrated the potential to improve the model's ability to better characterize affective states across diverse datasets."],"dc:identifier":["oai:rke.abertay.ac.uk:studenttheses/8ce7da5a-d034-4d9d-800f-f0e4d0e13c0a","https://rke.abertay.ac.uk/en/studentTheses/8ce7da5a-d034-4d9d-800f-f0e4d0e13c0a"],"dc:language":["eng"],"dc:publisher.department":["Faculty of Design, Informatics and Business"],"dc:publisher.institution":["Abertay University"],"dc:relation.isreferencedby":["https://rke.abertay.ac.uk/en/studentTheses/8ce7da5a-d034-4d9d-800f-f0e4d0e13c0a"],"dc:rights.embargodate":["2027-06-30"],"dc:rights.embargoreason":["/dk/atira/pure/core/document/studentthesisembargoreason/publicationissues"],"dc:subject":["Affective Computing","Virtual agents","HCI","Human-computer interaction","Emotions","Multimodal dataset","Affective feedback","Annotation tools","AffectRankTrace","Pleasure Arousal Dominance","PAD","Anxiety","Physiology","Heart Rate Variability","Respiration","Galvanic Skin Response","Electrocardiography","Anxiety-related emotions","General affective state","Emotion subspaces","Thematically related emotions","Agent behaviour","Affect aware agent","Believability","Emotion adaptation","Affective loop","Signal processing","Classification","Machine learning","Time windows segmentation","Emotion recognition","Anxious"],"dc:title":["Using affective computing to enhance the believability of virtual agents and the social interaction with users"],"dc:type":["Thesis"],"dc:type.qualificationlevel":["Doctoral Thesis"],"dc:type.qualificationname":["PhD"]},"updated_at":"2026-07-24T00:50:29Z"}