Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 5 of 5 for “"Second-Order Optimization"”.
-
Investigating a Second-Order Optimization Strategy for Neural Networks
… of the conjugate gradient method CG for the optimization of artificial neural networks (NNs) and compares this method with common first-order optimization methods, especially the stochastic gradient descent (SGD). The presented research results show that CG can effectively optimize both small …
-
Regulating Orthogonality Of Feature Functions For Highly Compressed Deep Neural Networks
… of the superior convergence properties of the second-order optimization, without directly computing the Hessian matrix.
-
Faster algorithms for matrix scaling and balancing via convex optimization
… doubly-stochastic variant of matrix scaling. In order to establish these results, we develop a new second-order optimization framework that enables us to treat both problems in a unified and principled manner. This framework identifies a certain generalization of linear system solving which we …
-
EFFECTIVE TRAINING OF NEURAL NETWORKS FOR BETTER GENERALIZATION
… and generalization-aware, addressing from optimization and data perspectives. From the optimization side, we develop practical algorithms that improve convergence speed and stability. We propose DRAG, a dimension-reduced adaptive gradient method that unifies the benefits of SGD and Adam, …
-
Three essays on nonparametric estimation
… treatment, Peyré and Cuturi (2019) state that second order methods are not an “applicable” approach for optimization in this setting. This is because of the large scale of many applications of interest, as well as because the Hessian is dense, poorly scaled, and not expressible in closed form. …