{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/129636"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/129636","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Beyond words: Understanding emotional shifts in maternal vocalizations through speech emotion recognition models","abstract":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-05-01","abstract_html":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-05-01","abstract_has_math":false,"creators":["Tin, Alara"],"institution":"University of Illinois Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Hasegawa-Johnson, Mark A."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-05-07","date_published":"2025-05-07","updated_at":"2026-07-22T22:25:05Z","subjects":["Speech Emotion Recognition","Cnn-bilstm","Acoustic Features","Domain Adaptation","Mother-infant Interaction"],"languages":["en","eng"],"rights":["Copyright 2025 Alara Tin"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/129636","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Hasegawa-Johnson, Mark A."]},{"key":"dc:creator","label":"Author","values":["Tin, Alara"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-05-07","2025-05"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Speech Emotion Recognition","Cnn-bilstm","Acoustic Features","Domain Adaptation","Mother-infant Interaction"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2025 Alara Tin"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/129636"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-05-01","The student, Alara Tin, accepted the attached license on 2025-05-06 at 13:04.","The student, Alara Tin, submitted this Thesis for approval on 2025-05-06 at 13:22.","This Thesis was approved for publication on 2025-05-07 at 15:17.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22232 on 2025-10-19 at 19:17:09","Automatic Speech Emotion Recognition (SER) has significant potential to provide insights into our understanding of dyadic communications. This study focuses on maternal vocalizations within mother-infant dyadic interactions, examining how mothers’ happy and neutral emotional tones shift in response to varying infant stress levels. To achieve this, we employ multiple models: our hybrid CNN-BiLSTM architecture, alongside pre-trained transformer-based models such as wav2vec 2.0 and HuBERT. Our evaluation demonstrates that the hybrid model outperforms these transformer-based approaches after fine-tuning, achieving a minimum improvement of 3.94 percentage points in the test accuracy and 11 percentage points in the weighted average of F1 scores in the IDP dataset. Using our fine-tuned model, we analyze maternal vocalizations in different age groups of infants (3, 6, and 9 months) and classify infants into low-, mid-, and high-stress categories based on the Root Mean Square (RMS) energy features of their vocalizations during stress-inducing events. Our findings reveal a moderate effect size (Cohen’s d) of associations between high stress levels and pronounced vocalization changes in mothers of 3-month-olds, more nuanced responses in mothers of 9-month-olds, and a balanced distribution of vocalization shifts in mothers of 6-month-olds. The novel application of SER in mother-infant studies underscores emotional adaptation in maternal vocalizations and its potential to expand analyses to bidirectional influences, providing deeper insights into emotional communication dynamics."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Beyond words: Understanding emotional shifts in maternal vocalizations through speech emotion recognition models"]}]}],"canonical_facts":{"dc:contributor":["Hasegawa-Johnson, Mark A."],"dc:creator":["Tin, Alara"],"dc:date":["2025-05-07","2025-05"],"dc:description":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-05-01","The student, Alara Tin, accepted the attached license on 2025-05-06 at 13:04.","The student, Alara Tin, submitted this Thesis for approval on 2025-05-06 at 13:22.","This Thesis was approved for publication on 2025-05-07 at 15:17.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22232 on 2025-10-19 at 19:17:09","Automatic Speech Emotion Recognition (SER) has significant potential to provide insights into our understanding of dyadic communications. This study focuses on maternal vocalizations within mother-infant dyadic interactions, examining how mothers’ happy and neutral emotional tones shift in response to varying infant stress levels. To achieve this, we employ multiple models: our hybrid CNN-BiLSTM architecture, alongside pre-trained transformer-based models such as wav2vec 2.0 and HuBERT. Our evaluation demonstrates that the hybrid model outperforms these transformer-based approaches after fine-tuning, achieving a minimum improvement of 3.94 percentage points in the test accuracy and 11 percentage points in the weighted average of F1 scores in the IDP dataset. Using our fine-tuned model, we analyze maternal vocalizations in different age groups of infants (3, 6, and 9 months) and classify infants into low-, mid-, and high-stress categories based on the Root Mean Square (RMS) energy features of their vocalizations during stress-inducing events. Our findings reveal a moderate effect size (Cohen’s d) of associations between high stress levels and pronounced vocalization changes in mothers of 3-month-olds, more nuanced responses in mothers of 9-month-olds, and a balanced distribution of vocalization shifts in mothers of 6-month-olds. The novel application of SER in mother-infant studies underscores emotional adaptation in maternal vocalizations and its potential to expand analyses to bidirectional influences, providing deeper insights into emotional communication dynamics."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/129636"],"dc:language":["en","eng"],"dc:rights":["Copyright 2025 Alara Tin"],"dc:subject":["Speech Emotion Recognition","Cnn-bilstm","Acoustic Features","Domain Adaptation","Mother-infant Interaction"],"dc:title":["Beyond words: Understanding emotional shifts in maternal vocalizations through speech emotion recognition models"],"dc:type":["text"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:05Z"}