Learning to Adapt to Diverse Data and Systems Heterogeneity

Deep learning has revolutionized the field of artificial intelligence (AI), leading to significant advancements in various industries and seamlessly integrating into our everyday lives. From playing a crucial role in autonomous vehicles to aiding in disease diagnosis, deep learning has enabled machines to perform tasks that were once thought to be exclusive to human intelligence. By leveraging complex models to continuously learn from a huge corpus of data, deep learning approaches have been pushing boundaries of what is possible in this realm of AI. These approaches can be used for a wide range of tasks but are often sub-optimal when making accurate predictions tailored for a specific task due to 1) data scarcity, 2) the differences in the types and quantity of data used for different tasks, and 3) the amount of resources available for adapting to the target task. In addition, for privacy-sensitive tasks where data is held on device, on-device training is required, exacerbating existing challenges. Although current knowledge transfer and efficient deep learning approaches can in a vast number of situations effectively mitigate these challenges, most works focus on a narrow range of challenges, limiting their applicability in real-world scenarios. The research presented in this thesis centers on addressing the difficulties posed by real-life scenarios, where various aforementioned challenges can happen concurrently. Through a unified meta-learning framework, we design effective and high-performing knowledge transfer methods that can handle various levels and types of task heterogeneity simultaneously. Specifically, we present novel task objectives, demonstrate a suitable meta-optimizer, and learn useful meta-representations across tasks. Our results show the effectiveness of our proposed methods at adapting to both discriminative and restorative tasks under realistic data and resource constraints, outperforming existing state-of-the-art in each scenario by a significant margin in both performance and efficacy. Most importantly, our work highlights the strengths of crafting specialized meta-learning approaches to enable high-performing deep learning solutions that can tailor to a wide range of statistical data variations and resource limitations.

Name dc:type.qualificationname

Doctor of Philosophy (PhD)

Level dc:type.qualificationlevel

Doctoral

Grantor dc:publisher.institution

University of Cambridge

Year dc:date.issued

2024

Author dc:creator

• Lee, De Sheng Royson

Advisors dc:contributor.advisor

• Lane, Nicholas
• Huszar, Ferenc

• federated learning
• super-resolution
• meta-learning

Licence

• https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/cdb6dba4-f073-411c-8047-298b93a36e8d/download
• https://www.rioxx.net/licenses/all-rights-reserved/

Language dc:language

eng

DOI dc:identifier.doi

https://doi.org/10.17863/CAM.113921

OAI identifier oai:identifier

oai:www.repository.cam.ac.uk:1810/376864