Statistical Surrogate Models for Robust Design Optimisation in Reduced Dimension

The efficient and accurate emulation of the response of complex engineering products is crucial for multi-query problems such as design optimisation. However, the design of these products depends on complex and often deterministic computational models that may be expensive-to-evaluate. Consequently, it is expedient to consider these models as black-box functions, such that they are interrogated by observing the output for prescribed input variables. The inputs of computational models are usually high-dimensional and uncertain. This complexity is managed using surrogates to approximate the computational model underlying the black-box function. The system input variables may consist of immutable variables such as PDE source terms and design variables that can be varied to optimise the system. Traditional methods use surrogates that consider only a single output variable, where complex engineering products may have multiple objective or constraint functions requiring consideration. Multi-view learning (MVL) is proposed for the discovery of a low-dimensional linear subspace that is embedded in a Gaussian process (GP) surrogate for increased scalability when emulating multiple objective and constraint functions. Although conventional MVL methods such as partial least squares (PLS) are effective when embedded within GPs, they provide no error estimate when reconstructing from a low-rank approximation. This is addressed by probabilistic partial least squares (PPLS) which treats latent variables sampled from the linear subspace as random, enabling a design variable probability density to be learned. When combined with adaptive sampling methodologies such as Bayesian optimisation (BO), solutions that are more robust to an incorrect specification of the linear subspace dimension are proposed; due to the solutions being proposed from the design variable probability density that accurately estimates reconstruction error. The partial least squares Bayesian optimisation (PLS-BO) and probabilistic partial least squares Bayesian optimisation (PPLS-BO) algorithms are introduced and shown to cause significant improvement in convergence rates when compared to classical BO. For complex engineering products, PPLS-BO is capable of estimating reconstruction error while achieving similar results to its deterministic counterpart PLS-BO; but in some cases, with a linear subspace consisting of several fewer dimensions. Although effective, neither the PLS-BO or PPLS-BO algorithms are considered for problems where uncertainties are present in the system input variables, making them unsuitable for robust design optimisation (RDO) applications. Consequently, Bayesian inference for the construction of novel probabilistic surrogates with input uncertainties and intrinsic dimensionality reduction is considered, and the novel reduced dimension variational Gaussian process (RDVGP) surrogate is introduced. The surrogate is trained by fitting to a finite collection of observations from the deterministic computational model. A GP prior probability density is assumed and used to determine the posterior probability density and parameters of the posited statistical model using variational Bayes (VB). The non-Gaussian posterior probability density is approximated with a simplified Gaussian trial density with free variational parameters, and the discrepancy between the densities is measured using the Kullback-Leibler (KL) divergence. Stochastic gradient descent is used to minimise the KL divergence to jointly learn the variational parameters and other parameters of the statistical model. The accuracy, efficiency, and versatility of the RDVGP surrogate is demonstrated on illustrative and RDO examples. Instead of optimising the RDVGP surrogate directly using metaheuristic methods, adaptive sampling can be used to reduce the quantity of training data required. The novel reduced dimension variational Gaussian process Bayesian optimisation (RDVGP-BO) algorithm that combines the RDVGP surrogate with an adaptive sampling procedure based on BO is presented. Optimisation using the RDVGP-BO algorithm is shown to achieve a more optimum result than optimising the RDVGP surrogate directly using metaheuristic methods. A more optimum robust solution is obtained when compared to traditional approaches that neglect the uncertain input variables and apply conservative safety factors, or require the constraints to be relaxed to yield a solution.

Name dc:type.qualificationname

Doctor of Philosophy (PhD)

Level dc:type.qualificationlevel

Doctoral

Grantor dc:publisher.institution

University of Cambridge

Year dc:date.issued

2024

Author dc:creator

• Archbold, Thomas

Advisor dc:contributor.advisor

• Cirak, Fehmi

• Bayesian inference
• Bayesian optimisation
• Gaussian processes
• Multi-view learning
• Robust optimisation
• Surrogate modelling

Licence

• https://www.repository.cam.ac.uk/bitstreams/5d21c50d-7e26-481d-a0ec-17a6e75be72d/download
• https://www.rioxx.net/licenses/all-rights-reserved/

Language dc:language

eng

DOI dc:identifier.doi

https://doi.org/10.17863/CAM.112609

OAI identifier oai:identifier

oai:www.repository.cam.ac.uk:1810/374593