Spoken Language Processing and Modeling for Aviation Communications

<p>With recent advances in machine learning and deep learning technologies and the creation of larger aviation-specific corpora, applying natural language processing technologies, especially those based on transformer neural networks, to aviation communications is becoming increasingly feasible. Previous work has focused on machine learning applications to natural language processing, such as N-grams and word lattices. This thesis experiments with a process for pretraining transformer-based language models on aviation English corpora and compare the effectiveness and performance of language models transfer learned from pretrained checkpoints and those trained from their base weight initializations (trained from scratch). The results suggest that transformer language models trained from scratch outperform models fine-tuned from pretrained checkpoints. The work concludes by recommending future work to improve pretraining performance and suggestions for downstream, in-domain tasks such as semantic extraction, named entity recognition (callsign identification), speaker role identification, and speech recognition.</p>

Name thesis:degree_name

Master of Science in Electrical & Computer Engineering

Level thesis:degree_level

Thesis - Open Access

Discipline thesis:degree_discipline

Electrical Engineering and Computer Science

Year

2023

Author dc:creator

• Van De Brook, Aaron

• language modeling
• automatic speech recognition
• corpus processing
• aviation communications
• deep learning
• machine learning
• Artificial Intelligence and Robotics
• Data Science
• Signal Processing
• Systems and Communications

Repository record dc:identifier

https://commons.erau.edu/edt/788

OAI identifier oai:identifier

oai:commons.erau.edu:edt-1813