{"id":{"repo_id":"cambridge","oai_identifier":"oai:www.repository.cam.ac.uk:1810/343542"},"canonical_url":"https://search.dev.ndltd.org/etd/cambridge/oai:www.repository.cam.ac.uk:1810/343542","repository":{"repo_id":"cambridge","name":"Cambridge University","base_url":"https://api.repository.cam.ac.uk/server/oai/request"},"display":{"title":"Relaxing assumptions in deep probabilistic modelling","abstract":"The current generation of deep neural network-based models demonstrate tremendous capacity to learn distributions at scale. Given this success, deep learning and deep generative modelling have progressively been applied across a broader range of increasingly demanding applications, as well as in safety-critical domains such as healthcare. However, existing models are reliant upon restrictive theoretical assumptions, deriving from longstanding distributions and divergences at their core, which inhibit their continued advance. By leveraging wider distribution and divergence families, transferring broader parametric assumptions to deep generative models increases the scope of the functions they can approximate. In particular, Kullback-Leibler divergence and the Gaussian distribution are assumed at the heart of variational autoencoders and score-based models, and are central to their limitations. This thesis argues that both assumptions can be viewed through wider lenses—the skew-geometric Jensen-Shannon divergence family and the generalised normal distribution family respectively. Several contributions are made to both the theory of deep learning, specifically deep generative modelling, and its application to electronic health records (EHRs). Firstly, a new type of variational autoencoder is introduced, capitalising on the flexibility of the skew-geometric Jensen-Shannon divergence, to overcome the prior theoretical shortcomings and lack of interpretability of latent space constraints. JSGα-VAEs lead to better reconstruction and generation when compared to baseline VAEs and utilise a single hyperparameter which can be easily interpreted in latent space. Secondly, heavy-tailed denoising score matching (HTDSM) is proposed, motivated by superior concentration of measure for the noising distribution in high-dimensional space. HTDSM offers improved score estimation, controllable sampling convergence, and more class-balanced unconditional generative performance. Finally, several results which indicate that the generalisation of EHR pipelines and models leads to increased flexibility with greater utility in the clinic are presented. Specifically, restrictions on data collation, curation, and chronology are removed while maintaining competitive performance on clinical objectives such as mortality prediction.","abstract_html":"The current generation of deep neural network-based models demonstrate tremendous capacity to learn distributions at scale. Given this success, deep learning and deep generative modelling have progressively been applied across a broader range of increasingly demanding applications, as well as in safety-critical domains such as healthcare. However, existing models are reliant upon restrictive theoretical assumptions, deriving from longstanding distributions and divergences at their core, which inhibit their continued advance. By leveraging wider distribution and divergence families, transferring broader parametric assumptions to deep generative models increases the scope of the functions they can approximate. In particular, Kullback-Leibler divergence and the Gaussian distribution are assumed at the heart of variational autoencoders and score-based models, and are central to their limitations. This thesis argues that both assumptions can be viewed through wider lenses—the skew-geometric Jensen-Shannon divergence family and the generalised normal distribution family respectively. Several contributions are made to both the theory of deep learning, specifically deep generative modelling, and its application to electronic health records (EHRs). Firstly, a new type of variational autoencoder is introduced, capitalising on the flexibility of the skew-geometric Jensen-Shannon divergence, to overcome the prior theoretical shortcomings and lack of interpretability of latent space constraints. JSGα-VAEs lead to better reconstruction and generation when compared to baseline VAEs and utilise a single hyperparameter which can be easily interpreted in latent space. Secondly, heavy-tailed denoising score matching (HTDSM) is proposed, motivated by superior concentration of measure for the noising distribution in high-dimensional space. HTDSM offers improved score estimation, controllable sampling convergence, and more class-balanced unconditional generative performance. Finally, several results which indicate that the generalisation of EHR pipelines and models leads to increased flexibility with greater utility in the clinic are presented. Specifically, restrictions on data collation, curation, and chronology are removed while maintaining competitive performance on clinical objectives such as mortality prediction.","abstract_has_math":false,"creators":["Deasy, Jacob"],"institution":"University of Cambridge","degree_name":"Doctor of Philosophy (PhD)","degree_level":"Doctoral","degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":["Lio, Pietro","Ercole, Ari"],"committee_chairs":[],"committee_members":[],"year":2021,"date_issued":"2021-12-01","date_published":"2021-12-01","updated_at":"2026-07-22T22:23:56Z","subjects":["deep learning","machine learning","statistics","VAEs","SBMs"],"languages":["eng"],"rights":[],"rights_urls":["https://www.rioxx.net/licenses/all-rights-reserved/"],"identifier_entries":[]},"links":{"outbound_url":"https://doi.org/10.17863/CAM.90966","outbound_label":"DOI","outbound_source":"dc:identifier.doi"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Lio, Pietro","Ercole, Ari"]},{"key":"dc:creator","label":"Author","values":["Deasy, Jacob"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.issued","label":"Date","values":["2021-12-01"]},{"key":"dc:publisher.institution","label":"Dc Publisher Institution","values":["University of Cambridge"]},{"key":"dc:relation.isreferencedby.uri","label":"Dc Relation Isreferencedby URI","values":["https://www.repository.cam.ac.uk/handle/1810/343542"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"dc:type.qualificationlevel","label":"Dc Type Qualificationlevel","values":["Doctoral"]},{"key":"dc:type.qualificationname","label":"Dc Type Qualificationname","values":["Doctor of Philosophy (PhD)"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["deep learning","machine learning","statistics","VAEs","SBMs"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["https://www.rioxx.net/licenses/all-rights-reserved/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.doi","label":"DOI","values":["10.17863/CAM.90966"]},{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://www.repository.cam.ac.uk/bitstreams/c8fcf273-0a8e-4d49-b64f-7cda0d28ab5b/download"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["The current generation of deep neural network-based models demonstrate tremendous capacity to learn distributions at scale. Given this success, deep learning and deep generative modelling have progressively been applied across a broader range of increasingly demanding applications, as well as in safety-critical domains such as healthcare. However, existing models are reliant upon restrictive theoretical assumptions, deriving from longstanding distributions and divergences at their core, which inhibit their continued advance. By leveraging wider distribution and divergence families, transferring broader parametric assumptions to deep generative models increases the scope of the functions they can approximate. In particular, Kullback-Leibler divergence and the Gaussian distribution are assumed at the heart of variational autoencoders and score-based models, and are central to their limitations. This thesis argues that both assumptions can be viewed through wider lenses—the skew-geometric Jensen-Shannon divergence family and the generalised normal distribution family respectively. Several contributions are made to both the theory of deep learning, specifically deep generative modelling, and its application to electronic health records (EHRs). Firstly, a new type of variational autoencoder is introduced, capitalising on the flexibility of the skew-geometric Jensen-Shannon divergence, to overcome the prior theoretical shortcomings and lack of interpretability of latent space constraints. JSGα-VAEs lead to better reconstruction and generation when compared to baseline VAEs and utilise a single hyperparameter which can be easily interpreted in latent space. Secondly, heavy-tailed denoising score matching (HTDSM) is proposed, motivated by superior concentration of measure for the noising distribution in high-dimensional space. HTDSM offers improved score estimation, controllable sampling convergence, and more class-balanced unconditional generative performance. Finally, several results which indicate that the generalisation of EHR pipelines and models leads to increased flexibility with greater utility in the clinic are presented. Specifically, restrictions on data collation, curation, and chronology are removed while maintaining competitive performance on clinical objectives such as mortality prediction."]},{"key":"dc:format.checksum.md5","label":"Dc Format Checksum Md5","values":["149f383c5a9ea92d64450e2d07192dc4"]},{"key":"dc:title","label":"Title","values":["Relaxing assumptions in deep probabilistic modelling"]}]}],"canonical_facts":{"dc:contributor.advisor":["Lio, Pietro","Ercole, Ari"],"dc:creator":["Deasy, Jacob"],"dc:date.issued":["2021-12-01"],"dc:description.abstract":["The current generation of deep neural network-based models demonstrate tremendous capacity to learn distributions at scale. Given this success, deep learning and deep generative modelling have progressively been applied across a broader range of increasingly demanding applications, as well as in safety-critical domains such as healthcare. However, existing models are reliant upon restrictive theoretical assumptions, deriving from longstanding distributions and divergences at their core, which inhibit their continued advance. By leveraging wider distribution and divergence families, transferring broader parametric assumptions to deep generative models increases the scope of the functions they can approximate. In particular, Kullback-Leibler divergence and the Gaussian distribution are assumed at the heart of variational autoencoders and score-based models, and are central to their limitations. This thesis argues that both assumptions can be viewed through wider lenses—the skew-geometric Jensen-Shannon divergence family and the generalised normal distribution family respectively. Several contributions are made to both the theory of deep learning, specifically deep generative modelling, and its application to electronic health records (EHRs). Firstly, a new type of variational autoencoder is introduced, capitalising on the flexibility of the skew-geometric Jensen-Shannon divergence, to overcome the prior theoretical shortcomings and lack of interpretability of latent space constraints. JSGα-VAEs lead to better reconstruction and generation when compared to baseline VAEs and utilise a single hyperparameter which can be easily interpreted in latent space. Secondly, heavy-tailed denoising score matching (HTDSM) is proposed, motivated by superior concentration of measure for the noising distribution in high-dimensional space. HTDSM offers improved score estimation, controllable sampling convergence, and more class-balanced unconditional generative performance. Finally, several results which indicate that the generalisation of EHR pipelines and models leads to increased flexibility with greater utility in the clinic are presented. Specifically, restrictions on data collation, curation, and chronology are removed while maintaining competitive performance on clinical objectives such as mortality prediction."],"dc:format.checksum.md5":["149f383c5a9ea92d64450e2d07192dc4"],"dc:identifier.doi":["10.17863/CAM.90966"],"dc:identifier.uri":["https://www.repository.cam.ac.uk/bitstreams/c8fcf273-0a8e-4d49-b64f-7cda0d28ab5b/download"],"dc:language":["eng"],"dc:publisher.institution":["University of Cambridge"],"dc:relation.isreferencedby.uri":["https://www.repository.cam.ac.uk/handle/1810/343542"],"dc:rights":["https://www.rioxx.net/licenses/all-rights-reserved/"],"dc:subject":["deep learning","machine learning","statistics","VAEs","SBMs"],"dc:title":["Relaxing assumptions in deep probabilistic modelling"],"dc:type":["Thesis"],"dc:type.qualificationlevel":["Doctoral"],"dc:type.qualificationname":["Doctor of Philosophy (PhD)"]},"updated_at":"2026-07-22T22:23:56Z"}