Advances in Few-Shot Learning for Image Classification and Tabular Data

Modern machine learning systems frequently operate in dynamic, data-scarce environments where assembling large labelled datasets is infeasible or impractical. Key examples include personalisation, federated learning, and privacy-sensitive applications -- each requiring robust learning from minimal data. These constraints underscore the need for few-shot learning methods, enabling models to adapt from just a handful of examples. In addition, many of these applications require updating only a small fraction of model parameters for adaptation -- a crucial form of parameter efficiency -- to enable low-cost personalisation, enhance communication-efficiency in federated learning, and potentially mitigate catastrophic forgetting in adapted systems. In this thesis, we address these challenges by proposing high-performing, parameter-efficient few-shot learning methods for both: i) images, focusing on few-shot classification, i.e. distinguishing multiple object classes from limited labelled images, and ii) tabular data, focusing on producing probabilistic predictions for multiple target variables with heterogeneous data, given only a few examples. By emphasising parameter efficiency without compromising accuracy, our contributions aim to advance few-shot learning toward real-world deployment. We start by proposing FiLM Transfer (FiT), an approach that combines transfer learning and meta-learning to produce parameter-efficient models with superior image classification accuracy in data limited settings. FiT achieves strong few-shot performance by updating fewer than 1% of the total model parameters. Our experiments confirm FiT’s superiority over competitive contemporaneous methods on the VTAB-1k benchmark. We further demonstrate FiT’s parameter efficiency and high accuracy in distributed low-shot settings, including model personalisation and federated learning, where the size of model updates is a critical performance factor. Next, we investigate privacy in the few-shot setting, a critical aspect for real-world applications involving sensitive data. Specifically, we conduct a comprehensive set of experiments to identify the conditions under which few-shot differential privacy (DP) remains effective. Our analysis reveals how the accuracy and the vulnerability to attack of few-shot DP image classification models are affected as the number of shots per class, privacy level, model architecture, downstream dataset, and subset of learnable parameters in the model vary. We also demonstrate that learning parameter-efficient FiLM adapters under DP is competitive with training only the final classifier layer or training all network parameters. Lastly, we evaluate DP federated learning systems and establish state-of-the-art performance on the FLAIR benchmark. Finally, we move beyond image classification by introducing JoLT (Joint LLM Process for Tabular Data), harnessing the in-context learning capabilities of Large Language Models (LLMs) to define joint distributions over heterogeneous tabular data. JoLT operates without any data conversion, data preprocessing, special handling of missing data, or model training, making it accessible and efficient for practitioners. Our experiments demonstrate that JoLT outperforms competitive methods on low-shot single-target and multi-target tabular classification and regression tasks. Furthermore, we show that JoLT can automatically handle missing data and perform data imputation by leveraging textual side information. By advancing parameter-efficient few-shot learning and investigating the feasibility of privacy-aware adaptation, this thesis provides insights into the practical challenges of learning from limited data, setting the stage for future research on enhancing model adaptability and efficiency, refining privacy-utility trade-offs, and extending few-shot learning to multimodal and continually evolving environments.

Name dc:type.qualificationname

Doctor of Philosophy (PhD)

Level dc:type.qualificationlevel

Doctoral

Grantor dc:publisher.institution

University of Cambridge

Year dc:date.issued

2025

Author dc:creator

• Shysheya, Aliaksandra

Advisor dc:contributor.advisor

• Turner, Richard

• Computer Vision
• Few-Shot Learning
• Tabular Data

Licence

• https://www.repository.cam.ac.uk/bitstreams/9299f570-ee25-4ab6-8232-384fe6f76d4c/download
• http://purl.org/NET/rdflicense/allrightsreserved

Language dc:language

eng

DOI dc:identifier.doi

https://doi.org/10.17863/CAM.125054

OAI identifier oai:identifier

oai:www.repository.cam.ac.uk:1810/395609