Advancing Language Equity and Sample Efficiency in Task-Oriented Dialogue Systems

Task-oriented dialogue (TOD) systems provide a natural way for humans to interact with machines to accomplish their everyday tasks. While this widely useful technology should benefit everyone equally, it has been mostly developed for English and several high-resource languages, resulting in a significant disparity between the speakers of the majority and minority languages. Equitable multilingual TOD systems should not only be accessible (able to converse with the users in their language) but also useful (addressing specific needs of diverse linguistic communities). This thesis aims to promote language equity in dialogue systems through language and domain transfer. The lack of datasets reflecting real-life interactions in most languages hinders TOD system development. To address this, we present two datasets: MULTI3NLU++ and COD. MULTI3NLU++, collected via human translation, enables systematic evaluation of cross-lingual and cross-domain transfer abilities. COD is collected via a new outline-based generation protocol aimed to mitigate translation-induced unnaturalness in language and content. The analyses and experiments on both highlight the disparity between high- and low-resource languages and emphasise the importance of culturally-aware datasets for the realistic evaluation of multilingual TOD systems. Due to language- and domain-specificity, equitable multilingual TOD systems often need to be developed in extremely data scarce setups. We propose to resolve this through sample-efficient methods. For cross-lingual transfer, we harness Web-scale data to augment limited in-domain in-language resources and unlock full potential of multilingual pretrained models through layer aggregation and contrastive learning. For cross-domain transfer, we propose a question-answering-based supervised instruction tuning approach, with questions capturing the semantics of seen and unseen classes. Proposed approaches were empirically validated across diverse (dialogue and non-dialogue) natural language understanding tasks and languages, outperforming competitive baselines in performance and robustness. Our analysis of state-of-the-art sample-efficient methods revealed the importance of balancing performance with computational and financial costs in real-world TOD systems, while demonstrating the effectiveness of supervised instruction tuning in complex ‘double’ cross-domain cross-lingual transfer. Overall, the work in this thesis shows that sample-efficient methods for cross-lingual and cross-domain transfer hold promise to make TOD systems both accessible and relevant to people worldwide.

Name dc:type.qualificationname

Doctor of Philosophy (PhD)

Level dc:type.qualificationlevel

Doctoral

Grantor dc:publisher.institution

University of Cambridge

Year dc:date.issued

2024

Author dc:creator

• Razumovskaia, Evgeniia

Advisors dc:contributor.advisor

• Korhonen, Anna
• Vulic, Ivan

• Dialogue Systems
• Few-Shot Learning
• Multilingual Natural Language Processing
• Natural Language Processing
• Natural Language Understanding
• Resource-Efficient NLP
• Sample Efficiency,

Licence

• https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/cf75f721-d234-4fdc-9af7-10a2693ff29a/download
• https://creativecommons.org/licenses/by/4.0/

Language dc:language

eng

DOI dc:identifier.doi

https://doi.org/10.17863/CAM.117073

OAI identifier oai:identifier

oai:www.repository.cam.ac.uk:1810/382089