Probabilistic Modelling in Function Space

Gaussian processes have established themselves as powerful tools for inferring functions from data. They provide a flexible framework for defining distributions over functions, enabling closed form solutions and principled handling of uncertainty. However, their application is often hindered by the difficulty of formulating suitable priors through analytical covariance functions and ensuring computational scalability. In contrast, deep generative models, while not offering closed form solutions, excel at learning data-driven priors and inherently scale to large datasets using neural networks. This thesis aims to bridge the gap between the principled but computationally expensive Gaussian process approaches and the more flexible, yet less transparent, deep generative model approaches. The first contribution is an in-depth analysis of Gaussian processes on the hypersphere, an idea inspired by infinite-width neural networks. This leads to a class of kernels, known as zonal kernels, and a linear-time Gaussian process method that address the scalability concerns. Building on this, the second contribution introduces a compositional Gaussian process model that mirrors the structure of deep neural networks. This endows Gaussian processes with the ability to learn features from data, akin to neural networks, while retaining its principled uncertainty estimates. The second part of the thesis shifts focus to diffusion models, a leading technique in deep generative modelling. The main contribution in this section is the extension of diffusion models to function spaces. We demonstrate the potential of this approach as a viable alternative to Gaussian processes for complex data, especially when ample data is available to learn suitable priors. In the final chapter, we introduce geometric structure into these functional diffusion models. We show that this is crucial for tackling real-world scientific problems, particularly those dealing with non-Euclidean spaces and symmetries, which we exemplify by scenarios such as tracking cyclones on the Earth's surface.

Name dc:type.qualificationname

Doctor of Philosophy (PhD)

Level dc:type.qualificationlevel

Doctoral

Grantor dc:publisher.institution

University of Cambridge

Year dc:date.issued

2024

Author dc:creator

• Dutordoir, Vincent

Advisors dc:contributor.advisor

• Ghahramani, Zoubin
• Ek, Carl

• Artificial Intelligence
• Diffusion Models
• Gaussian Processes
• Generative Modelling
• Machine Learning

Licence

• https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/72d5f369-2242-463d-bcdc-748a231d4785/download
• https://www.rioxx.net/licenses/all-rights-reserved/

Language dc:language

eng

DOI dc:identifier.doi

https://doi.org/10.17863/CAM.112783

OAI identifier oai:identifier

oai:www.repository.cam.ac.uk:1810/374898