{"id":{"repo_id":"cambridge","oai_identifier":"oai:www.repository.cam.ac.uk:1810/374593"},"canonical_url":"https://search.dev.ndltd.org/etd/cambridge/oai:www.repository.cam.ac.uk:1810/374593","repository":{"repo_id":"cambridge","name":"Cambridge University","base_url":"https://api.repository.cam.ac.uk/server/oai/request"},"display":{"title":"Statistical Surrogate Models for Robust Design Optimisation in Reduced Dimension","abstract":"The efficient and accurate emulation of the response of complex engineering products is crucial for multi-query problems such as design optimisation. However, the design of these products depends on complex and often deterministic computational models that may be expensive-to-evaluate. Consequently, it is expedient to consider these models as black-box functions, such that they are interrogated by observing the output for prescribed input variables. The inputs of computational models are usually high-dimensional and uncertain. This complexity is managed using surrogates to approximate the computational model underlying the black-box function. The system input variables may consist of immutable variables such as PDE source terms and design variables that can be varied to optimise the system. Traditional methods use surrogates that consider only a single output variable, where complex engineering products may have multiple objective or constraint functions requiring consideration. Multi-view learning (MVL) is proposed for the discovery of a low-dimensional linear subspace that is embedded in a Gaussian process (GP) surrogate for increased scalability when emulating multiple objective and constraint functions. Although conventional MVL methods such as partial least squares (PLS) are effective when embedded within GPs, they provide no error estimate when reconstructing from a low-rank approximation. This is addressed by probabilistic partial least squares (PPLS) which treats latent variables sampled from the linear subspace as random, enabling a design variable probability density to be learned. When combined with adaptive sampling methodologies such as Bayesian optimisation (BO), solutions that are more robust to an incorrect specification of the linear subspace dimension are proposed; due to the solutions being proposed from the design variable probability density that accurately estimates reconstruction error. The partial least squares Bayesian optimisation (PLS-BO) and probabilistic partial least squares Bayesian optimisation (PPLS-BO) algorithms are introduced and shown to cause significant improvement in convergence rates when compared to classical BO. For complex engineering products, PPLS-BO is capable of estimating reconstruction error while achieving similar results to its deterministic counterpart PLS-BO; but in some cases, with a linear subspace consisting of several fewer dimensions. Although effective, neither the PLS-BO or PPLS-BO algorithms are considered for problems where uncertainties are present in the system input variables, making them unsuitable for robust design optimisation (RDO) applications. Consequently, Bayesian inference for the construction of novel probabilistic surrogates with input uncertainties and intrinsic dimensionality reduction is considered, and the novel reduced dimension variational Gaussian process (RDVGP) surrogate is introduced. The surrogate is trained by fitting to a finite collection of observations from the deterministic computational model. A GP prior probability density is assumed and used to determine the posterior probability density and parameters of the posited statistical model using variational Bayes (VB). The non-Gaussian posterior probability density is approximated with a simplified Gaussian trial density with free variational parameters, and the discrepancy between the densities is measured using the Kullback-Leibler (KL) divergence. Stochastic gradient descent is used to minimise the KL divergence to jointly learn the variational parameters and other parameters of the statistical model. The accuracy, efficiency, and versatility of the RDVGP surrogate is demonstrated on illustrative and RDO examples. Instead of optimising the RDVGP surrogate directly using metaheuristic methods, adaptive sampling can be used to reduce the quantity of training data required. The novel reduced dimension variational Gaussian process Bayesian optimisation (RDVGP-BO) algorithm that combines the RDVGP surrogate with an adaptive sampling procedure based on BO is presented. Optimisation using the RDVGP-BO algorithm is shown to achieve a more optimum result than optimising the RDVGP surrogate directly using metaheuristic methods. A more optimum robust solution is obtained when compared to traditional approaches that neglect the uncertain input variables and apply conservative safety factors, or require the constraints to be relaxed to yield a solution.","abstract_html":"The efficient and accurate emulation of the response of complex engineering products is crucial for multi-query problems such as design optimisation. However, the design of these products depends on complex and often deterministic computational models that may be expensive-to-evaluate. Consequently, it is expedient to consider these models as black-box functions, such that they are interrogated by observing the output for prescribed input variables. The inputs of computational models are usually high-dimensional and uncertain. This complexity is managed using surrogates to approximate the computational model underlying the black-box function. The system input variables may consist of immutable variables such as PDE source terms and design variables that can be varied to optimise the system. Traditional methods use surrogates that consider only a single output variable, where complex engineering products may have multiple objective or constraint functions requiring consideration. Multi-view learning (MVL) is proposed for the discovery of a low-dimensional linear subspace that is embedded in a Gaussian process (GP) surrogate for increased scalability when emulating multiple objective and constraint functions. Although conventional MVL methods such as partial least squares (PLS) are effective when embedded within GPs, they provide no error estimate when reconstructing from a low-rank approximation. This is addressed by probabilistic partial least squares (PPLS) which treats latent variables sampled from the linear subspace as random, enabling a design variable probability density to be learned. When combined with adaptive sampling methodologies such as Bayesian optimisation (BO), solutions that are more robust to an incorrect specification of the linear subspace dimension are proposed; due to the solutions being proposed from the design variable probability density that accurately estimates reconstruction error. The partial least squares Bayesian optimisation (PLS-BO) and probabilistic partial least squares Bayesian optimisation (PPLS-BO) algorithms are introduced and shown to cause significant improvement in convergence rates when compared to classical BO. For complex engineering products, PPLS-BO is capable of estimating reconstruction error while achieving similar results to its deterministic counterpart PLS-BO; but in some cases, with a linear subspace consisting of several fewer dimensions. Although effective, neither the PLS-BO or PPLS-BO algorithms are considered for problems where uncertainties are present in the system input variables, making them unsuitable for robust design optimisation (RDO) applications. Consequently, Bayesian inference for the construction of novel probabilistic surrogates with input uncertainties and intrinsic dimensionality reduction is considered, and the novel reduced dimension variational Gaussian process (RDVGP) surrogate is introduced. The surrogate is trained by fitting to a finite collection of observations from the deterministic computational model. A GP prior probability density is assumed and used to determine the posterior probability density and parameters of the posited statistical model using variational Bayes (VB). The non-Gaussian posterior probability density is approximated with a simplified Gaussian trial density with free variational parameters, and the discrepancy between the densities is measured using the Kullback-Leibler (KL) divergence. Stochastic gradient descent is used to minimise the KL divergence to jointly learn the variational parameters and other parameters of the statistical model. The accuracy, efficiency, and versatility of the RDVGP surrogate is demonstrated on illustrative and RDO examples. Instead of optimising the RDVGP surrogate directly using metaheuristic methods, adaptive sampling can be used to reduce the quantity of training data required. The novel reduced dimension variational Gaussian process Bayesian optimisation (RDVGP-BO) algorithm that combines the RDVGP surrogate with an adaptive sampling procedure based on BO is presented. Optimisation using the RDVGP-BO algorithm is shown to achieve a more optimum result than optimising the RDVGP surrogate directly using metaheuristic methods. A more optimum robust solution is obtained when compared to traditional approaches that neglect the uncertain input variables and apply conservative safety factors, or require the constraints to be relaxed to yield a solution.","abstract_has_math":false,"creators":["Archbold, Thomas"],"institution":"University of Cambridge","degree_name":"Doctor of Philosophy (PhD)","degree_level":"Doctoral","degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":["Cirak, Fehmi"],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-04-30","date_published":"2024-04-30","updated_at":"2026-07-24T01:33:18Z","subjects":["Bayesian inference","Bayesian optimisation","Gaussian processes","Multi-view learning","Robust optimisation","Surrogate modelling"],"languages":["eng"],"rights":[],"rights_urls":["https://www.repository.cam.ac.uk/bitstreams/5d21c50d-7e26-481d-a0ec-17a6e75be72d/download","https://www.rioxx.net/licenses/all-rights-reserved/"],"identifier_entries":[]},"links":{"outbound_url":"https://doi.org/10.17863/CAM.112609","outbound_label":"DOI","outbound_source":"dc:identifier.doi"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Cirak, Fehmi"]},{"key":"dc:contributor.sponsor","label":"Sponsor","values":["The author's research has been funded by the Woolf Fisher Trust and Cambridge Trust."]},{"key":"dc:creator","label":"Author","values":["Archbold, Thomas"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.issued","label":"Date","values":["2024-04-30"]},{"key":"dc:publisher.institution","label":"Dc Publisher Institution","values":["University of Cambridge"]},{"key":"dc:relation.isreferencedby.uri","label":"Dc Relation Isreferencedby URI","values":["https://www.repository.cam.ac.uk/handle/1810/374593"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"dc:type.qualificationlevel","label":"Dc Type Qualificationlevel","values":["Doctoral"]},{"key":"dc:type.qualificationname","label":"Dc Type Qualificationname","values":["Doctor of Philosophy (PhD)"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Bayesian inference","Bayesian optimisation","Gaussian processes","Multi-view learning","Robust optimisation","Surrogate modelling"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["https://www.repository.cam.ac.uk/bitstreams/5d21c50d-7e26-481d-a0ec-17a6e75be72d/download","https://www.rioxx.net/licenses/all-rights-reserved/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.doi","label":"DOI","values":["https://doi.org/10.17863/CAM.112609"]},{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://www.repository.cam.ac.uk/bitstreams/21c584f7-2c00-48a1-b54b-a5b73c5fe277/download"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["The efficient and accurate emulation of the response of complex engineering products is crucial for multi-query problems such as design optimisation. However, the design of these products depends on complex and often deterministic computational models that may be expensive-to-evaluate. Consequently, it is expedient to consider these models as black-box functions, such that they are interrogated by observing the output for prescribed input variables. The inputs of computational models are usually high-dimensional and uncertain. This complexity is managed using surrogates to approximate the computational model underlying the black-box function. The system input variables may consist of immutable variables such as PDE source terms and design variables that can be varied to optimise the system. Traditional methods use surrogates that consider only a single output variable, where complex engineering products may have multiple objective or constraint functions requiring consideration. Multi-view learning (MVL) is proposed for the discovery of a low-dimensional linear subspace that is embedded in a Gaussian process (GP) surrogate for increased scalability when emulating multiple objective and constraint functions. Although conventional MVL methods such as partial least squares (PLS) are effective when embedded within GPs, they provide no error estimate when reconstructing from a low-rank approximation. This is addressed by probabilistic partial least squares (PPLS) which treats latent variables sampled from the linear subspace as random, enabling a design variable probability density to be learned. When combined with adaptive sampling methodologies such as Bayesian optimisation (BO), solutions that are more robust to an incorrect specification of the linear subspace dimension are proposed; due to the solutions being proposed from the design variable probability density that accurately estimates reconstruction error. The partial least squares Bayesian optimisation (PLS-BO) and probabilistic partial least squares Bayesian optimisation (PPLS-BO) algorithms are introduced and shown to cause significant improvement in convergence rates when compared to classical BO. For complex engineering products, PPLS-BO is capable of estimating reconstruction error while achieving similar results to its deterministic counterpart PLS-BO; but in some cases, with a linear subspace consisting of several fewer dimensions. Although effective, neither the PLS-BO or PPLS-BO algorithms are considered for problems where uncertainties are present in the system input variables, making them unsuitable for robust design optimisation (RDO) applications. Consequently, Bayesian inference for the construction of novel probabilistic surrogates with input uncertainties and intrinsic dimensionality reduction is considered, and the novel reduced dimension variational Gaussian process (RDVGP) surrogate is introduced. The surrogate is trained by fitting to a finite collection of observations from the deterministic computational model. A GP prior probability density is assumed and used to determine the posterior probability density and parameters of the posited statistical model using variational Bayes (VB). The non-Gaussian posterior probability density is approximated with a simplified Gaussian trial density with free variational parameters, and the discrepancy between the densities is measured using the Kullback-Leibler (KL) divergence. Stochastic gradient descent is used to minimise the KL divergence to jointly learn the variational parameters and other parameters of the statistical model. The accuracy, efficiency, and versatility of the RDVGP surrogate is demonstrated on illustrative and RDO examples. Instead of optimising the RDVGP surrogate directly using metaheuristic methods, adaptive sampling can be used to reduce the quantity of training data required. The novel reduced dimension variational Gaussian process Bayesian optimisation (RDVGP-BO) algorithm that combines the RDVGP surrogate with an adaptive sampling procedure based on BO is presented. Optimisation using the RDVGP-BO algorithm is shown to achieve a more optimum result than optimising the RDVGP surrogate directly using metaheuristic methods. A more optimum robust solution is obtained when compared to traditional approaches that neglect the uncertain input variables and apply conservative safety factors, or require the constraints to be relaxed to yield a solution."]},{"key":"dc:format.checksum.md5","label":"Dc Format Checksum Md5","values":["6a15d725e1fd3203decf1376de018316","87eda9de84448d1f82354d60eee3eb5f"]},{"key":"dc:title","label":"Title","values":["Statistical Surrogate Models for Robust Design Optimisation in Reduced Dimension"]}]}],"canonical_facts":{"dc:contributor.advisor":["Cirak, Fehmi"],"dc:contributor.sponsor":["The author's research has been funded by the Woolf Fisher Trust and Cambridge Trust."],"dc:creator":["Archbold, Thomas"],"dc:date.issued":["2024-04-30"],"dc:description.abstract":["The efficient and accurate emulation of the response of complex engineering products is crucial for multi-query problems such as design optimisation. However, the design of these products depends on complex and often deterministic computational models that may be expensive-to-evaluate. Consequently, it is expedient to consider these models as black-box functions, such that they are interrogated by observing the output for prescribed input variables. The inputs of computational models are usually high-dimensional and uncertain. This complexity is managed using surrogates to approximate the computational model underlying the black-box function. The system input variables may consist of immutable variables such as PDE source terms and design variables that can be varied to optimise the system. Traditional methods use surrogates that consider only a single output variable, where complex engineering products may have multiple objective or constraint functions requiring consideration. Multi-view learning (MVL) is proposed for the discovery of a low-dimensional linear subspace that is embedded in a Gaussian process (GP) surrogate for increased scalability when emulating multiple objective and constraint functions. Although conventional MVL methods such as partial least squares (PLS) are effective when embedded within GPs, they provide no error estimate when reconstructing from a low-rank approximation. This is addressed by probabilistic partial least squares (PPLS) which treats latent variables sampled from the linear subspace as random, enabling a design variable probability density to be learned. When combined with adaptive sampling methodologies such as Bayesian optimisation (BO), solutions that are more robust to an incorrect specification of the linear subspace dimension are proposed; due to the solutions being proposed from the design variable probability density that accurately estimates reconstruction error. The partial least squares Bayesian optimisation (PLS-BO) and probabilistic partial least squares Bayesian optimisation (PPLS-BO) algorithms are introduced and shown to cause significant improvement in convergence rates when compared to classical BO. For complex engineering products, PPLS-BO is capable of estimating reconstruction error while achieving similar results to its deterministic counterpart PLS-BO; but in some cases, with a linear subspace consisting of several fewer dimensions. Although effective, neither the PLS-BO or PPLS-BO algorithms are considered for problems where uncertainties are present in the system input variables, making them unsuitable for robust design optimisation (RDO) applications. Consequently, Bayesian inference for the construction of novel probabilistic surrogates with input uncertainties and intrinsic dimensionality reduction is considered, and the novel reduced dimension variational Gaussian process (RDVGP) surrogate is introduced. The surrogate is trained by fitting to a finite collection of observations from the deterministic computational model. A GP prior probability density is assumed and used to determine the posterior probability density and parameters of the posited statistical model using variational Bayes (VB). The non-Gaussian posterior probability density is approximated with a simplified Gaussian trial density with free variational parameters, and the discrepancy between the densities is measured using the Kullback-Leibler (KL) divergence. Stochastic gradient descent is used to minimise the KL divergence to jointly learn the variational parameters and other parameters of the statistical model. The accuracy, efficiency, and versatility of the RDVGP surrogate is demonstrated on illustrative and RDO examples. Instead of optimising the RDVGP surrogate directly using metaheuristic methods, adaptive sampling can be used to reduce the quantity of training data required. The novel reduced dimension variational Gaussian process Bayesian optimisation (RDVGP-BO) algorithm that combines the RDVGP surrogate with an adaptive sampling procedure based on BO is presented. Optimisation using the RDVGP-BO algorithm is shown to achieve a more optimum result than optimising the RDVGP surrogate directly using metaheuristic methods. A more optimum robust solution is obtained when compared to traditional approaches that neglect the uncertain input variables and apply conservative safety factors, or require the constraints to be relaxed to yield a solution."],"dc:format.checksum.md5":["6a15d725e1fd3203decf1376de018316","87eda9de84448d1f82354d60eee3eb5f"],"dc:identifier.doi":["https://doi.org/10.17863/CAM.112609"],"dc:identifier.uri":["https://www.repository.cam.ac.uk/bitstreams/21c584f7-2c00-48a1-b54b-a5b73c5fe277/download"],"dc:language":["eng"],"dc:publisher.institution":["University of Cambridge"],"dc:relation.isreferencedby.uri":["https://www.repository.cam.ac.uk/handle/1810/374593"],"dc:rights":["https://www.repository.cam.ac.uk/bitstreams/5d21c50d-7e26-481d-a0ec-17a6e75be72d/download","https://www.rioxx.net/licenses/all-rights-reserved/"],"dc:subject":["Bayesian inference","Bayesian optimisation","Gaussian processes","Multi-view learning","Robust optimisation","Surrogate modelling"],"dc:title":["Statistical Surrogate Models for Robust Design Optimisation in Reduced Dimension"],"dc:type":["Thesis"],"dc:type.qualificationlevel":["Doctoral"],"dc:type.qualificationname":["Doctor of Philosophy (PhD)"]},"updated_at":"2026-07-24T01:33:18Z"}