{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/115694"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/115694","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Decision-making for a regenerative society: voice-based emotion AI","abstract":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-05-01","abstract_html":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-05-01","abstract_has_math":false,"creators":["Porteanu, Monica"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Informatics","degree_department":null,"school":null,"contributors":["Hamilton, Kevin","Karahalios, Kyratso","Grosser, Benjamin","Araujo de Aguiar, Carlos"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-05","date_published":"2022-05","updated_at":"2026-07-22T22:24:55Z","subjects":["data ethnography","algorithm ethnography","interface ethnography","audible design","design with sound","voice-based design","voice-based AI","voice-based emotion AI","algorithmic hearing","tone analysis","wearable computing","affective computing","voice sentiment analysis"],"languages":["en","eng"],"rights":["Copyright 2022 Monica Porteanu"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/115694","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Hamilton, Kevin","Karahalios, Kyratso","Grosser, Benjamin","Araujo de Aguiar, Carlos"]},{"key":"dc:creator","label":"Author","values":["Porteanu, Monica"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-05","2022-04-15"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Informatics"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["data ethnography","algorithm ethnography","interface ethnography","audible design","design with sound","voice-based design","voice-based AI","voice-based emotion AI","algorithmic hearing","tone analysis","wearable computing","affective computing","voice sentiment analysis"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2022 Monica Porteanu"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/115694"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-05-01","The student, Monica Porteanu, accepted the attached license on 2022-04-12 at 14:50.","The student, Monica Porteanu, submitted this Dissertation for approval on 2022-04-12 at 14:54.","This Dissertation was approved for publication on 2022-04-15 at 15:23.","DSpace SAF Submission Ingestion Package generated from Vireo submission #17643 on 2022-11-11 at 12:19:10","Voice-based user interfaces are increasingly present in our daily lives. These systems rely on voice as a key identifier and biometric, sometimes drawing from aggregated data sets to facilitate decision-making based on interpretations of our emotional state, gender, ethnicity and more. However, voice as an indicator of identity and intention is already a fraught part of human interaction in light of bias and difference. The starting hypothesis is that AI-mediated voice-based decision-making and the technology’s assumptions about a user are subjective. Our voices operating in today’s complex technology ecosystem have new effects of which we are not fully aware. This research aims to understand a user’s voice journey through technology and its affective interpretations as it relates to voice-based decision-making that affects a user’s contextual identity and privacy. The dissertation starts with a new media speculative study that explores the use of sound in design as an inclusive and revelatory process of finding ways of living that rely primarily on the perception of sound. Next, the research investigates how users understand the ways in which a wearable device equipped with a voice analyzer “hears.” The study builds on precedents in affective computing, algorithm auditing frameworks, folk theories of algorithmic understanding, and human-centric AI. The outcome points out competencies involved in understanding objects that function using sound-based artificial intelligence. The research continues with a virtual “thing” ethnography for the three components of a voice-based emotion AI: interface, data, and algorithm. Using the five-emotion framework and digital research techniques (e.g., media archaeology, performative inquiry) and following AI best practices, it iterates through three script performances, analyzing the state of the field in terms of technological imaginaries in data classification, voice emotion modelling, voice emotion model evaluation, data orchestration, and agency over voice emotion in a big-data world) and its maturity today. The “thing” ethnography, coupled with the research with human participants study, point to the limitations of using standard datasets and the opportunity for creating performant voice-based emotion AI with real people’s data. The benefit of AI transfer learning, coupled with personal storytelling for data orchestration and a speculative, phenomenological exploration with small data, represents the opportunity to regain agency over voice emotion in a big-data world."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Decision-making for a regenerative society: voice-based emotion AI"]}]}],"canonical_facts":{"dc:contributor":["Hamilton, Kevin","Karahalios, Kyratso","Grosser, Benjamin","Araujo de Aguiar, Carlos"],"dc:creator":["Porteanu, Monica"],"dc:date":["2022-05","2022-04-15"],"dc:description":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-05-01","The student, Monica Porteanu, accepted the attached license on 2022-04-12 at 14:50.","The student, Monica Porteanu, submitted this Dissertation for approval on 2022-04-12 at 14:54.","This Dissertation was approved for publication on 2022-04-15 at 15:23.","DSpace SAF Submission Ingestion Package generated from Vireo submission #17643 on 2022-11-11 at 12:19:10","Voice-based user interfaces are increasingly present in our daily lives. These systems rely on voice as a key identifier and biometric, sometimes drawing from aggregated data sets to facilitate decision-making based on interpretations of our emotional state, gender, ethnicity and more. However, voice as an indicator of identity and intention is already a fraught part of human interaction in light of bias and difference. The starting hypothesis is that AI-mediated voice-based decision-making and the technology’s assumptions about a user are subjective. Our voices operating in today’s complex technology ecosystem have new effects of which we are not fully aware. This research aims to understand a user’s voice journey through technology and its affective interpretations as it relates to voice-based decision-making that affects a user’s contextual identity and privacy. The dissertation starts with a new media speculative study that explores the use of sound in design as an inclusive and revelatory process of finding ways of living that rely primarily on the perception of sound. Next, the research investigates how users understand the ways in which a wearable device equipped with a voice analyzer “hears.” The study builds on precedents in affective computing, algorithm auditing frameworks, folk theories of algorithmic understanding, and human-centric AI. The outcome points out competencies involved in understanding objects that function using sound-based artificial intelligence. The research continues with a virtual “thing” ethnography for the three components of a voice-based emotion AI: interface, data, and algorithm. Using the five-emotion framework and digital research techniques (e.g., media archaeology, performative inquiry) and following AI best practices, it iterates through three script performances, analyzing the state of the field in terms of technological imaginaries in data classification, voice emotion modelling, voice emotion model evaluation, data orchestration, and agency over voice emotion in a big-data world) and its maturity today. The “thing” ethnography, coupled with the research with human participants study, point to the limitations of using standard datasets and the opportunity for creating performant voice-based emotion AI with real people’s data. The benefit of AI transfer learning, coupled with personal storytelling for data orchestration and a speculative, phenomenological exploration with small data, represents the opportunity to regain agency over voice emotion in a big-data world."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/115694"],"dc:language":["en","eng"],"dc:rights":["Copyright 2022 Monica Porteanu"],"dc:subject":["data ethnography","algorithm ethnography","interface ethnography","audible design","design with sound","voice-based design","voice-based AI","voice-based emotion AI","algorithmic hearing","tone analysis","wearable computing","affective computing","voice sentiment analysis"],"dc:title":["Decision-making for a regenerative society: voice-based emotion AI"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Informatics"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:55Z"}