{"id":{"repo_id":"cambridge","oai_identifier":"oai:www.repository.cam.ac.uk:1810/334899"},"canonical_url":"https://search.dev.ndltd.org/etd/cambridge/oai:www.repository.cam.ac.uk:1810/334899","repository":{"repo_id":"cambridge","name":"Cambridge University","base_url":"https://api.repository.cam.ac.uk/server/oai/request"},"display":{"title":"Design of Deep Neural Networks Formulated as Optimisation Problems","abstract":"The design of deep neural networks (DNNs) involves the explicit definition of network architecture as well as the training of the network weights. Each process can be formulated into an optimisation algorithm and can be investigated with regard to optimisation performance. The training of the network weights is defined as a minimisation of the objective function with regard to network parameters. The architecture search is an optimisation of the objective function with regard to the presence/absence of layers or neurons. I draw similarity between the two scenarios, and propose frameworks that define either the training or the architectural optimisation of DNNs, or a combination of both. The contribution of the thesis is six-fold, in which I propose: 1) a quasi-Newton training algorithm based on Truncated Newton and Gradient Flow methods, 2) a lifting scheme to allow network sparsification, 3) a lifting framework to automatically evolve neural architectures, 4) a multi-scale hierarchical search framework involving sensitivity analysis suitable for the training of neural networks, 5) a heuristic search algorithm for architectural optimisation of a dynamic model, and 6) a dynamic cascade learning model solved in the context of de novo drug design. In each contribution, I define the optimisation problem and solve the optimisation problem under different frameworks. The ultimate aim of this research is to facilitate the democratisation of AI, enabling people with less domain expertise to participate in the design of a deep neural network under a guided framework.","abstract_html":"The design of deep neural networks (DNNs) involves the explicit definition of network architecture as well as the training of the network weights. Each process can be formulated into an optimisation algorithm and can be investigated with regard to optimisation performance. The training of the network weights is defined as a minimisation of the objective function with regard to network parameters. The architecture search is an optimisation of the objective function with regard to the presence/absence of layers or neurons. I draw similarity between the two scenarios, and propose frameworks that define either the training or the architectural optimisation of DNNs, or a combination of both. The contribution of the thesis is six-fold, in which I propose: 1) a quasi-Newton training algorithm based on Truncated Newton and Gradient Flow methods, 2) a lifting scheme to allow network sparsification, 3) a lifting framework to automatically evolve neural architectures, 4) a multi-scale hierarchical search framework involving sensitivity analysis suitable for the training of neural networks, 5) a heuristic search algorithm for architectural optimisation of a dynamic model, and 6) a dynamic cascade learning model solved in the context of de novo drug design. In each contribution, I define the optimisation problem and solve the optimisation problem under different frameworks. The ultimate aim of this research is to facilitate the democratisation of AI, enabling people with less domain expertise to participate in the design of a deep neural network under a guided framework.","abstract_has_math":false,"creators":["Zhang, Sushen"],"institution":"University of Cambridge","degree_name":"Doctor of Philosophy (PhD)","degree_level":"Doctoral","degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":["Lapkin, Alexei","Vassiliadis, Vassilios"],"committee_chairs":[],"committee_members":[],"year":2021,"date_issued":"2021-10-01","date_published":"2021-10-01","updated_at":"2026-07-22T22:24:04Z","subjects":["Neural Network","Optimisation","Architecture Search"],"languages":["eng"],"rights":[],"rights_urls":["https://www.rioxx.net/licenses/all-rights-reserved/"],"identifier_entries":[{"key":"dc:creator.authoridentifier","label":"Author Identifier","values":["0000000176210889"],"render_values":[{"text":"0000-0001-7621-0889","href":"https://orcid.org/0000-0001-7621-0889","code":true}]}]},"links":{"outbound_url":"https://doi.org/10.17863/CAM.82337","outbound_label":"DOI","outbound_source":"dc:identifier.doi"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Lapkin, Alexei","Vassiliadis, Vassilios"]},{"key":"dc:contributor.sponsor","label":"Sponsor","values":["Cambridge Overseas Trust and China Scholarship Council"]},{"key":"dc:creator","label":"Author","values":["Zhang, Sushen"]},{"key":"dc:creator.authoridentifier","label":"Author Identifier","values":["0000000176210889"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.issued","label":"Date","values":["2021-10-01"]},{"key":"dc:publisher.institution","label":"Dc Publisher Institution","values":["University of Cambridge"]},{"key":"dc:relation.isreferencedby.uri","label":"Dc Relation Isreferencedby URI","values":["https://www.repository.cam.ac.uk/handle/1810/334899"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"dc:type.qualificationlevel","label":"Dc Type Qualificationlevel","values":["Doctoral"]},{"key":"dc:type.qualificationname","label":"Dc Type Qualificationname","values":["Doctor of Philosophy (PhD)"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Neural Network","Optimisation","Architecture Search"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["https://www.rioxx.net/licenses/all-rights-reserved/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.doi","label":"DOI","values":["10.17863/CAM.82337"]},{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/a6a39126-2730-4f8b-945b-1b72ac43c0f0/download"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["The design of deep neural networks (DNNs) involves the explicit definition of network architecture as well as the training of the network weights. Each process can be formulated into an optimisation algorithm and can be investigated with regard to optimisation performance. The training of the network weights is defined as a minimisation of the objective function with regard to network parameters. The architecture search is an optimisation of the objective function with regard to the presence/absence of layers or neurons. I draw similarity between the two scenarios, and propose frameworks that define either the training or the architectural optimisation of DNNs, or a combination of both. The contribution of the thesis is six-fold, in which I propose: 1) a quasi-Newton training algorithm based on Truncated Newton and Gradient Flow methods, 2) a lifting scheme to allow network sparsification, 3) a lifting framework to automatically evolve neural architectures, 4) a multi-scale hierarchical search framework involving sensitivity analysis suitable for the training of neural networks, 5) a heuristic search algorithm for architectural optimisation of a dynamic model, and 6) a dynamic cascade learning model solved in the context of de novo drug design. In each contribution, I define the optimisation problem and solve the optimisation problem under different frameworks. The ultimate aim of this research is to facilitate the democratisation of AI, enabling people with less domain expertise to participate in the design of a deep neural network under a guided framework."]},{"key":"dc:format.checksum.md5","label":"Dc Format Checksum Md5","values":["9dd173424c1790227ea7cd7172786477"]},{"key":"dc:title","label":"Title","values":["Design of Deep Neural Networks Formulated as Optimisation Problems"]}]}],"canonical_facts":{"dc:contributor.advisor":["Lapkin, Alexei","Vassiliadis, Vassilios"],"dc:contributor.sponsor":["Cambridge Overseas Trust and China Scholarship Council"],"dc:creator":["Zhang, Sushen"],"dc:creator.authoridentifier":["0000000176210889"],"dc:date.issued":["2021-10-01"],"dc:description.abstract":["The design of deep neural networks (DNNs) involves the explicit definition of network architecture as well as the training of the network weights. Each process can be formulated into an optimisation algorithm and can be investigated with regard to optimisation performance. The training of the network weights is defined as a minimisation of the objective function with regard to network parameters. The architecture search is an optimisation of the objective function with regard to the presence/absence of layers or neurons. I draw similarity between the two scenarios, and propose frameworks that define either the training or the architectural optimisation of DNNs, or a combination of both. The contribution of the thesis is six-fold, in which I propose: 1) a quasi-Newton training algorithm based on Truncated Newton and Gradient Flow methods, 2) a lifting scheme to allow network sparsification, 3) a lifting framework to automatically evolve neural architectures, 4) a multi-scale hierarchical search framework involving sensitivity analysis suitable for the training of neural networks, 5) a heuristic search algorithm for architectural optimisation of a dynamic model, and 6) a dynamic cascade learning model solved in the context of de novo drug design. In each contribution, I define the optimisation problem and solve the optimisation problem under different frameworks. The ultimate aim of this research is to facilitate the democratisation of AI, enabling people with less domain expertise to participate in the design of a deep neural network under a guided framework."],"dc:format.checksum.md5":["9dd173424c1790227ea7cd7172786477"],"dc:identifier.doi":["10.17863/CAM.82337"],"dc:identifier.uri":["https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/a6a39126-2730-4f8b-945b-1b72ac43c0f0/download"],"dc:language":["eng"],"dc:publisher.institution":["University of Cambridge"],"dc:relation.isreferencedby.uri":["https://www.repository.cam.ac.uk/handle/1810/334899"],"dc:rights":["https://www.rioxx.net/licenses/all-rights-reserved/"],"dc:subject":["Neural Network","Optimisation","Architecture Search"],"dc:title":["Design of Deep Neural Networks Formulated as Optimisation Problems"],"dc:type":["Thesis"],"dc:type.qualificationlevel":["Doctoral"],"dc:type.qualificationname":["Doctor of Philosophy (PhD)"]},"updated_at":"2026-07-22T22:24:04Z"}