Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 228 for “"gradient descent"”.
-
Fracture optimization based on gradient descent methodology
… (NPV) is selected as the target function and the gradient descent methodology is adopted as the optimizer. Relying on the gradient information, gradient descent searches for the optimal solution along the steepest decent direction. A comprehensive reservoir simulator, MSFLOW, with gas …
-
GDSVD: Scalable k-SVD via Gradient Descent
We show that a gradient-descent with a simple, universal rule for step-size selection provably finds k-SVD, i.e., the k ≥ 1 largest singular values and corresponding vectors, of any matrix, despite nonconvexity. There has been substantial progress towards this in the past few years where existing …
-
Hybrid Distributed Stochastic Gradient Descent for Federated Learning
… we propose the Hybrid Distributed Stochastic Gradient Descent (Hybrid DSGD), a training scheme for federated learning which utilizes the advantages of both digital and analog transmissions to reduce communication overhead and latency. We demonstrate why the conventional analog-based …
-
Semiparametric Approaches for Dimension Reduction Through Gradient Descent on Manifold
… function. CS-OPG, a modified outer product gradient (OPG) method for CS, is developed as an initial estimator for CS-gemDR. The basis B of the CMS or CS is estimated by a gradient descent algorithm. An update scheme on a Grassmann manifold is to preserve the orthogonality constraint on the …
-
A hardware acceleration technique for gradient descent and conjugate gradient
Gradient descent, conjugate gradient, and other iterative algorithms are a powerful class of algorithms; however, they can take a long time for conver- gence. Baseline accelerator designs feature insu cient coverage of operations and do not work well on the problems we target. In this thesis we …
-
Inductive logic programming with gradient descent for supervised binary classification
… by Evans and Grefenstette [3] to develop an gradient descent-based inductive logic programming technique for supervised binary classification. We start by developing our methodology for binary input data, and then extend the approach to numerical data using a threshold-gate based binarization …
-
A waypoint-driven gradient descent solution for a parallel robot
… for an expandable segment, utilizes a gradient ascent algorithm to determine the three-dimensional coordinate that would allow for that desired orientation, and then creates waypoints across the path in order to ensure minimal displacement error and reduce chances of damage to the …
-
The implicit bias of gradient descent: from linear classifiers to deep networks
Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2022-11-11 without embargo terms
-
One-Pass Learning via Bridging Orthogonal Gradient Descent and Recursive Least-Squares
… least-squares (RLS) algorithm and orthogonal gradient descent (OGD). Our algorithm uses the memory efficiently by exploiting the structure of the streaming data via an incremental principal component analysis (IPCA). Further, we show that, for overparameterized linear models, the parameter …
-
Convergence Rates of Gradient Descent-ascent Dynamics under Computation Constraints in Solving Min-max Optimization
… a new analysis of the convergence rates of the Gradient Descent-ascent (GDA) method for solving the Min-max optimization (MMO) problems under computation constraints. In particular, we focus our study on two main classes of MMO: a continuous-time variant of the centralized Min-max problem where …
-
Understanding Deep Learning via Analyzing Training Dynamics
… theoretical understanding lags behind. How does gradient descent successfully optimize the highly non-convex training objective, and how does it find a solution that also generalizes well to unseen data despite the model being over-parameterized? Answering these questions requires a …
-
Implementation of Adaptive Filter Algorithms for the Suppression of Thermoacoustic Instabilities
… were incorporated, which are divided into gradient descent algorithms and pattern searches. The algorithms were tested on three separate platforms. The first was an analog electronic simulator, which uses a second order acoustics model and a first order low pass filter to simulate the flame …
-
Dynamical systems perspectives in machine learning
… we look at training the neural network with the gradient descent algorithm on the squared loss function and discuss the generalization properties of the output of the gradient descent algorithm on an unseen data point. We look at two problems in this setting. First, we discuss the effect of l2 …
-
Completion of hinge loss has an implicit bias
… for the underlying dynamical system of the gradient descent iterates for two-layer linear networks on the inner product loss function is derived. Using the derived dynamical system, a precise explicit algorithm is developed which when implemented reproduces the gradient descent iterates of …
-
Smoothness and Adaptivity in Nonlinear Optimization for Machine Learning Applications
… well explain why adaptive methods outperform gradient descent in certain machine learning tasks like training transformers. In this thesis, to bridge this gap, we propose more general smoothness conditions that are closer to machine learning practice and study the convergence of popular …
-
Phase Retrieval Under a Generative Prior
… landscape exhibits favorable global geometry for gradient descent with information theoretically optimal sample complexity. Based on this geometric result, we introduce a gradient descent algorithm to converge to the true solution. We corroborate these results with experiments showing that …
-
On the Complexity of Nonconvex-Strongly-Concave Smooth Minimax Optimization Using First-Order Methods
… we study the specific first-order algorithm, gradient descent-ascent (GDA). We show that for quadratic or nearly quadratic nonconvex-strongly-concave functions under our assumptions, two-time-scale GDA with appropriate stepsizes achieves a linear convergence rate. Then we also extend our …
-
Optimization and Generalization of Minimax Algorithms
… analysis of widely used algorithms such as Extra-Gradient (EG) and Optimistic Gradient Descent Ascent (OGDA), whose convergence behavior was not systematically understood. We derive convergence rates for these algorithms in the convex-concave setting. We show that these algorithms work effectively …
-
Foundations of Multiple-Time-Scale Stochastic Approximation for Fast and Resilient Distributed Optimization Algorithms
… key aspects. First, we study consensus-based gradient methods over networks with a clustered structure, characterized by dense intra-cluster and sparse inter-cluster communication. This setting naturally gives rise to textit{two-time-scale} dynamics. We develop a framework to analyze these …
-
Motion planning for multi-link robots with artificial potential fields and modified simulated annealing
… To improve the performance of the algorithm, the gradient descent method is replaced by Newton’s method which helps in reducing the zigzagging phenomenon in gradient descent method while the robot moves in the vicinity of an obstacle.
Page 1 of 12