Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 6 of 6 for “"loss landscapes"”.
-
On Machine Learning Loss Landscapes
Loss functions are pivotal to the training of every machine learning model. When evaluating the loss function over a large range of parameters, a loss landscape is obtained. In this thesis, loss landscapes for various classes of machine learning models are explored. Geometric properties of the loss …
-
Understanding The Effects of Incorporating Scientific Knowledge on Neural Network Outputs and Loss Landscapes
… to experts. Second, I use and further develop loss landscape visualization tools to better understand ML model optimization at the network parameter level. Such an understanding has proven to be effective at evaluating and diagnosing different model architectures and loss functions in the field …
-
An optimization approach to relate neural circuit architecture, loss landscapes and learning performance in static and dynamic tasks
… architecture. We link the geometry of the loss landscape to the difficulty of a task and demonstrate how network expansions modify the loss landscape. For dynamic tasks, we consider the cerebellum, which is involved in motor control and has a unique architecture. Cerebellar mossy fibre …
-
On Bilevel Optimization without Full Unrolls: Methods and Applications
… the inner optimization can lead to chaotic meta-loss landscapes; short unrolls can lead to truncation bias. To address both of these issues, we introduce Persistent Evolution Strategies (PES), an approach for computing unbiased gradient estimates of parameters that govern a dynamical system, …
-
The Trainability and Expressivity of Quantum Machine Learning Models
… as optimization problems with quantum-evaluated loss functions. We show that when the operations implemented on the quantum device are drawn from a certain problem-independent distribution, the loss landscapes (in expectation) exhibit a phase transition in trainability. We argue that the …
-
Advances in Meta-Learning, Robustness, and Second-Order Optimisation in Deep Learning
… as a minimisation problem of the training loss with respect to the model’s parameters — though in practice we also require our algorithms to generalise which is not a concern of optimisation more broadly. The chosen optimisation strategy affects the speed at which algorithms learn and the …