{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/125698"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/125698","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Using wearable IMUs for multi-modal denoising and tracking","abstract":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2026-08-01","abstract_html":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2026-08-01","abstract_has_math":false,"creators":["Wei, Yu-Lin"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Roy Choudhury, Romit","Al-Hassanieh, Haitham","Smaragdis, Paris","Srikant, Rayadurgam","Sabharwal, Ashutosh"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-07-08","date_published":"2024-07-08","updated_at":"2026-07-22T22:25:02Z","subjects":["Multimodal Denoising","Imu","Indoor Localization","Wearable"],"languages":["en","eng"],"rights":["Copyright 2024 Yu-Lin Wei"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/125698","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Roy Choudhury, Romit","Al-Hassanieh, Haitham","Smaragdis, Paris","Srikant, Rayadurgam","Sabharwal, Ashutosh"]},{"key":"dc:creator","label":"Author","values":["Wei, Yu-Lin"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2024-07-08","2024-08"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Multimodal Denoising","Imu","Indoor Localization","Wearable"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2024 Yu-Lin Wei"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/125698"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2026-08-01","The student, Yu-Lin Wei, accepted the attached license on 2024-07-05 at 14:54.","The student, Yu-Lin Wei, submitted this Dissertation for approval on 2024-07-05 at 15:07.","This Dissertation was approved for publication on 2024-07-08 at 16:12.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20963 on 2025-02-04 at 21:16:28","Modern earphones are equipped with microphones and inertial measurement units (IMUs). IMUs are motion sensors used to deduce human activities, such as jogging, falling, and device rotation. These sensors are integrated into many wearable devices, including smartphones, watches, and earphones. While IMUs are well-established sensors, they reveal new potential when utilized on earphone platforms, particularly on the user’s face. Earphone IMUs offer two primary advantages: (a) they can capture jaw vibrations during speech, and (b) they provide smoother and cleaner head motion data compared to lower-body movements. Exploring these opportunities, we propose two applications for earphones. First, by detecting jaw vibrations, IMUs can support the microphone in performing multimodal self-supervised speech denoising. During activities such as phone calls or voice assistant interactions, vibrations from the throat travel through the jawbone and skull, inducing a voltage in the IMU. Although this IMU data is lower resolution and more distorted than microphone recordings, it is unaffected by ambient sounds, providing a unique advantage for multi-modal speech enhancement. Specifically, we explore whether the uninterfered, yet distorted, IMU signal can aid in enhancing speech when the microphone’s signal is compromised by non-stationary ambient noise. Secondly, using the clean acceleration data from the IMU, we propose an infrastructure-free indoor dead reckoning algorithm. By leveraging the inertial sensors in both earphones and smartphones, we can estimate a user’s indoor location and gazing orientation. Additionally, by playing 3D sounds through the earphones and monitoring the user’s responses, we can recalibrate errors in location and orientation estimation. We believe this innovative combination of IMU and acoustics could mark a significant advancement towards indoor Acoustic Augmented Reality (AAR)."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Using wearable IMUs for multi-modal denoising and tracking"]}]}],"canonical_facts":{"dc:contributor":["Roy Choudhury, Romit","Al-Hassanieh, Haitham","Smaragdis, Paris","Srikant, Rayadurgam","Sabharwal, Ashutosh"],"dc:creator":["Wei, Yu-Lin"],"dc:date":["2024-07-08","2024-08"],"dc:description":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2026-08-01","The student, Yu-Lin Wei, accepted the attached license on 2024-07-05 at 14:54.","The student, Yu-Lin Wei, submitted this Dissertation for approval on 2024-07-05 at 15:07.","This Dissertation was approved for publication on 2024-07-08 at 16:12.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20963 on 2025-02-04 at 21:16:28","Modern earphones are equipped with microphones and inertial measurement units (IMUs). IMUs are motion sensors used to deduce human activities, such as jogging, falling, and device rotation. These sensors are integrated into many wearable devices, including smartphones, watches, and earphones. While IMUs are well-established sensors, they reveal new potential when utilized on earphone platforms, particularly on the user’s face. Earphone IMUs offer two primary advantages: (a) they can capture jaw vibrations during speech, and (b) they provide smoother and cleaner head motion data compared to lower-body movements. Exploring these opportunities, we propose two applications for earphones. First, by detecting jaw vibrations, IMUs can support the microphone in performing multimodal self-supervised speech denoising. During activities such as phone calls or voice assistant interactions, vibrations from the throat travel through the jawbone and skull, inducing a voltage in the IMU. Although this IMU data is lower resolution and more distorted than microphone recordings, it is unaffected by ambient sounds, providing a unique advantage for multi-modal speech enhancement. Specifically, we explore whether the uninterfered, yet distorted, IMU signal can aid in enhancing speech when the microphone’s signal is compromised by non-stationary ambient noise. Secondly, using the clean acceleration data from the IMU, we propose an infrastructure-free indoor dead reckoning algorithm. By leveraging the inertial sensors in both earphones and smartphones, we can estimate a user’s indoor location and gazing orientation. Additionally, by playing 3D sounds through the earphones and monitoring the user’s responses, we can recalibrate errors in location and orientation estimation. We believe this innovative combination of IMU and acoustics could mark a significant advancement towards indoor Acoustic Augmented Reality (AAR)."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/125698"],"dc:language":["en","eng"],"dc:rights":["Copyright 2024 Yu-Lin Wei"],"dc:subject":["Multimodal Denoising","Imu","Indoor Localization","Wearable"],"dc:title":["Using wearable IMUs for multi-modal denoising and tracking"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:02Z"}