Breaking down barriers: advancing interdisciplinary speech applications in early children’s development

This thesis aims to develop interdisciplinary speech applications using machine learning algorithms to identify children with developmental disorders or speech and language delays early. Specifically, we build machine learning models to capture critical adult-child interactions under different social contexts, including turn-taking vocalizations between parents and infants (under 14 months old) at home or joint attention between clinicians and toddlers (1-2 years old) at clinics. Turn-taking vocalizations are considered as coordinated interactions; no response and co-vocalizations are considered as uncoordinated interactions. Previous research has shown that daily repeated and reinforced uncoordinated interactions may contribute to mental health problems in children in the long run. In autism screening, detecting whether clinicians and children establish joint attention during semi-structured assessments is crucial, as this is considered a key factor for diagnosing autism. To achieve this goal, we focused on two speech-processing tasks: speaker diarization (identify who spoke when) and vocalization classifications (identify the type of vocalization given a speaker). Because annotating audio is a labor- and time-consuming task, the thesis addresses the technical difficulties in improving the performance of speech-processing models given a limited amount of labeled audio. We explore several transfer learning techniques within supervised learning as well as leverage self-supervised learning for enhancing child audio analysis tasks. With the self-supervised learning scheme, we show that the performance of proposed interdisciplinary speech applications achieved significant advancement in child audio analysis tasks. This thesis expands the application of traditional speech technology like speech-to-text and text-to-speech, exploring its potential in other disciplines such as psychology and healthcare.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Li, Jialu

Contributors dc:contributor

• Hasegawa-Johnson, Mark
• McElwain, Nancy L
• Bhat, Suma
• Varshney, Lav R

• Child Vocalizations Classifications
• Child-adult Speaker Diarization
• Autism Diagnosis
• Family Audio Analysis
• Self-supervised Learning
• Transfer Learning
• Interdisciplinary Speech Applications

Statement dc:rights

• Copyright 2024 Jialu Li

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/124412

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F124412