{"id":{"repo_id":"toronto-retro","oai_identifier":"oai:utoronto.scholaris.ca:1807/125369"},"canonical_url":"https://search.dev.ndltd.org/etd/toronto-retro/oai:utoronto.scholaris.ca:1807/125369","repository":{"repo_id":"toronto-retro","name":"University of Toronto","base_url":"https://utoronto.scholaris.ca/server/oai/request"},"display":{"title":"Function-Space Bayesian Learning: from Gaussian Processes to Bayesian Deep Learning","abstract":"Bayesian methods provide a general and principled framework to quantify and update beliefs based on prior knowledge and observed evidence. This thesis presents my works about function-space Bayesian learning, whose priors and posteriors are specified and computed over the function-space. This thesis first focuses on Gaussian processes (GPs), a family of traditional function-space Bayesian models. To learn a suitable kernel for the GP prior, I propose a neural network structure that computes expressive kernel compositions. Because this structure is fully differentiable, we can learn the prior kernel from data with end-to-end gradient-based optimization. Moreover, I propose a new variational GP approximation to address computational bottlenecks in variational Gaussian processes. This variational GP achieves substantial improvement in computational complexity and approximates the true posterior with high fidelity. In the next part, I propose a general framework for function-space Bayesian learning that goes beyond GPs. I apply this framework in Bayesian deep learning and demonstrate its ability to train Bayesian neural networks with interpretable function-space priors. Furthermore, by exploiting a function-space perspective, I show that the hidden units in neural networks can be viewed as inter-domain inducing points so that I can draw an equivalence between finite neural networks and sparse Gaussian processes. In the last part, I employ Bayesian models for online memory selection and improve the robustness against imbalanced data streams in continual learning. Finally, I summarize the thesis and discuss potential future directions.","abstract_html":"Bayesian methods provide a general and principled framework to quantify and update beliefs based on prior knowledge and observed evidence. This thesis presents my works about function-space Bayesian learning, whose priors and posteriors are specified and computed over the function-space. This thesis first focuses on Gaussian processes (GPs), a family of traditional function-space Bayesian models. To learn a suitable kernel for the GP prior, I propose a neural network structure that computes expressive kernel compositions. Because this structure is fully differentiable, we can learn the prior kernel from data with end-to-end gradient-based optimization. Moreover, I propose a new variational GP approximation to address computational bottlenecks in variational Gaussian processes. This variational GP achieves substantial improvement in computational complexity and approximates the true posterior with high fidelity. In the next part, I propose a general framework for function-space Bayesian learning that goes beyond GPs. I apply this framework in Bayesian deep learning and demonstrate its ability to train Bayesian neural networks with interpretable function-space priors. Furthermore, by exploiting a function-space perspective, I show that the hidden units in neural networks can be viewed as inter-domain inducing points so that I can draw an equivalence between finite neural networks and sparse Gaussian processes. In the last part, I employ Bayesian models for online memory selection and improve the robustness against imbalanced data streams in continual learning. Finally, I summarize the thesis and discuss potential future directions.","abstract_has_math":false,"creators":["Sun, Shengyang"],"institution":null,"degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":"Computer Science","school":null,"contributors":[],"advisors":["Grosse, Roger"],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-11","date_published":"2022-11","updated_at":"2026-07-27T21:28:02Z","subjects":["Bayesian Deep Learning","Bayesian Methods","Gaussian Process","Machine Learning","Uncertainty Estimation"],"languages":[],"rights":["Attribution 4.0 International"],"rights_urls":["http://creativecommons.org/licenses/by/4.0/"],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/1807/125369","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Grosse, Roger"]},{"key":"dc:contributor.department","label":"Department","values":["Computer Science"]},{"key":"dc:creator","label":"Author","values":["Sun, Shengyang"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-11"]},{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2022-11-11T17:56:20Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2022-11-11T17:56:20Z"]},{"key":"dc:date.issued","label":"Date","values":["2022-11"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Bayesian Deep Learning","Bayesian Methods","Gaussian Process","Machine Learning","Uncertainty Estimation"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["Attribution 4.0 International"]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://creativecommons.org/licenses/by/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["http://hdl.handle.net/1807/125369"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Bayesian methods provide a general and principled framework to quantify and update beliefs based on prior knowledge and observed evidence. This thesis presents my works about function-space Bayesian learning, whose priors and posteriors are specified and computed over the function-space. This thesis first focuses on Gaussian processes (GPs), a family of traditional function-space Bayesian models. To learn a suitable kernel for the GP prior, I propose a neural network structure that computes expressive kernel compositions. Because this structure is fully differentiable, we can learn the prior kernel from data with end-to-end gradient-based optimization. Moreover, I propose a new variational GP approximation to address computational bottlenecks in variational Gaussian processes. This variational GP achieves substantial improvement in computational complexity and approximates the true posterior with high fidelity. In the next part, I propose a general framework for function-space Bayesian learning that goes beyond GPs. I apply this framework in Bayesian deep learning and demonstrate its ability to train Bayesian neural networks with interpretable function-space priors. Furthermore, by exploiting a function-space perspective, I show that the hidden units in neural networks can be viewed as inter-domain inducing points so that I can draw an equivalence between finite neural networks and sparse Gaussian processes. In the last part, I employ Bayesian models for online memory selection and improve the robustness against imbalanced data streams in continual learning. Finally, I summarize the thesis and discuss potential future directions."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["Ph.D."]},{"key":"dc:title","label":"Title","values":["Function-Space Bayesian Learning: from Gaussian Processes to Bayesian Deep Learning"]}]}],"canonical_facts":{"dc:contributor.advisor":["Grosse, Roger"],"dc:contributor.department":["Computer Science"],"dc:creator":["Sun, Shengyang"],"dc:date":["2022-11"],"dc:date.accessioned":["2022-11-11T17:56:20Z"],"dc:date.available":["2022-11-11T17:56:20Z"],"dc:date.issued":["2022-11"],"dc:description.abstract":["Bayesian methods provide a general and principled framework to quantify and update beliefs based on prior knowledge and observed evidence. This thesis presents my works about function-space Bayesian learning, whose priors and posteriors are specified and computed over the function-space. This thesis first focuses on Gaussian processes (GPs), a family of traditional function-space Bayesian models. To learn a suitable kernel for the GP prior, I propose a neural network structure that computes expressive kernel compositions. Because this structure is fully differentiable, we can learn the prior kernel from data with end-to-end gradient-based optimization. Moreover, I propose a new variational GP approximation to address computational bottlenecks in variational Gaussian processes. This variational GP achieves substantial improvement in computational complexity and approximates the true posterior with high fidelity. In the next part, I propose a general framework for function-space Bayesian learning that goes beyond GPs. I apply this framework in Bayesian deep learning and demonstrate its ability to train Bayesian neural networks with interpretable function-space priors. Furthermore, by exploiting a function-space perspective, I show that the hidden units in neural networks can be viewed as inter-domain inducing points so that I can draw an equivalence between finite neural networks and sparse Gaussian processes. In the last part, I employ Bayesian models for online memory selection and improve the robustness against imbalanced data streams in continual learning. Finally, I summarize the thesis and discuss potential future directions."],"dc:description.degree":["Ph.D."],"dc:identifier.uri":["http://hdl.handle.net/1807/125369"],"dc:rights":["Attribution 4.0 International"],"dc:rights.uri":["http://creativecommons.org/licenses/by/4.0/"],"dc:subject":["Bayesian Deep Learning","Bayesian Methods","Gaussian Process","Machine Learning","Uncertainty Estimation"],"dc:title":["Function-Space Bayesian Learning: from Gaussian Processes to Bayesian Deep Learning"],"dc:type":["Thesis"]},"updated_at":"2026-07-27T21:28:02Z"}