Improved Gaussian process approximations for spatial and flow fields

Many modelling problems in scientific and engineering applications require quantitative models of uncertainty, for example to appropriately weight different sources of information, or to make risk-aware decisions. A natural way to formalise this is as a probabilistic model with a Bayesian point of view. When the main object of interest is a function - such as a spatial or flow field - Gaussian processes (GPs) are suitable probabilistic models. These can incorporate expert knowledge (priors), and data can be used to learn the model parameters. The cost of learning is high, motivating approximations, most notably variational approximations. This thesis improves on variational GP approximations in different settings. Firstly, I consider approximate learning of low dimensional spatial fields. In Chapter 2, I argue for precomputable variational approximations, which only access the training data once during learning. These lead to significant computational savings, but are limited to an excessively narrow class of priors. In Chapter 3, I develop a class of precomputable approximations which first replace the prior with a periodic approximation. These approximate Fourier series methods outperform existing methods, have compelling theoretical guarantees, and are applicable to a broad class of priors - stationary priors whose covariance functions have well-defined spectral densities. Since many spatial fields are highly non-stationary, in Chapter 4 I construct a new class of non-stationary priors based on multiresolution (discrete wavelet) approximations, with a compatible precomputable approximation. In Chapter 5 I turn to modelling flow fields for system identification problems. Here I propose a new approach - approximating the state posterior with Kalman smoothing - leading to faster and less biased learning, while also applicable to either discrete or continuous time. Overall, this thesis presents improved GP variational approximations to make learning cheaper, with applicability to a broader class of priors, supported by a mix of theoretical guarantees and empirical evaluation.

Name dc:type.qualificationname

Doctor of Philosophy (PhD)

Level dc:type.qualificationlevel

Doctoral

Grantor dc:publisher.institution

University of Cambridge

Year dc:date.issued

2024

Author dc:creator

• Cheema, Talay

Advisor dc:contributor.advisor

• Rasmussen, Carl

• Dynamical systems
• Gaussian processes
• Machine learning

Licence

• https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/032e81e5-4a04-4ccb-aa9d-6820fbb6b54e/download
• https://www.rioxx.net/licenses/all-rights-reserved/

Language dc:language

eng

DOI dc:identifier.doi

https://doi.org/10.17863/CAM.112770

OAI identifier oai:identifier

oai:www.repository.cam.ac.uk:1810/374873