Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 68 for “"Stochastic Gradient Descent"”.
-
Hybrid Distributed Stochastic Gradient Descent for Federated Learning
… this thesis, we propose the Hybrid Distributed Stochastic Gradient Descent (Hybrid DSGD), a training scheme for federated learning which utilizes the advantages of both digital and analog transmissions to reduce communication overhead and latency. We demonstrate why the conventional analog-based …
-
Distributed Singular Value Decomposition Through Least Squares
… in nature. We show the efficacy of a distributed stochastic gradient descent algorithm by implementing parallelized alternating least squares and prove theoretical guarantees for its convergence and empirical results, which allow for the development of a simple framework for solving SVD in a …
-
Speeding Up Coded Distributed Machine Learning
… such as coded matrix multiplication and gradient coding, and proposed numerous coding techniques to alleviate the stragglers. However, since many codings are designed based on theory only, there are still many challenges in practice. In this dissertation, we study such practical …
-
Training hierarchical networks for function approximation
… the difficulty of training RBF networks with stochastic gradient descent (SGD) and hierarchical RBF. We discovered that training singled layered RBF networks can be quite simple with a good initialization and good choice of standard deviation for the Gaussian. Training hierarchical RBFs …
-
Investigating a Second-Order Optimization Strategy for Neural Networks
… investigates the application of the conjugate gradient method CG for the optimization of artificial neural networks (NNs) and compares this method with common first-order optimization methods, especially the stochastic gradient descent (SGD). The presented research results show that CG can …
-
Testing, Learning, and Optimization in High Dimensions
… We show that despite nonconvexity, a variant of Stochastic gradient descent (SGD) converges to a good solution for which we prove a novel generalization bound that is proportional to our complexity measure.
-
Understanding generalization
… Hardt et al [HRS16]. We apply this technique to stochastic gradient descent with momentum and investigate the resulting stability bounds under some assumptions. In the second direction, we explore the effectiveness of stability in obtaining generalization bounds under the violation of some model …
-
Stochastic optimization application in molecular electronics
… and an analysis of a class of algorithms, called stochastic gradient descent, that may be useful in the post-fabrication programming phase.
-
Revisiting Generalization for Deep Learning: PAC-Bayes, Flat Minima, and Generative Models
… nonvacuous. We begin with an analysis of stochastic gradient descent (SGD) in supervised learning. By formalizing the notion of flat minima using PAC-Bayes generalization bounds, we obtain nonvacuous generalization bounds for stochastic classifiers based on SGD solutions. Despite strong …
-
On the Complexity of Nonconvex-Strongly-Concave Smooth Minimax Optimization Using First-Order Methods
… et al., 2020b) up to logarithmic factors. For stochastic oracles, we provide a lower bound of Ω (︀√ 𝜅𝜖⁻² + 𝜅 ¹/³ 𝜖 ⁻⁴)︀ . Second, we study the specific first-order algorithm, gradient descent-ascent (GDA). We show that for quadratic or nearly quadratic nonconvex-strongly-concave functions under …
-
Building a question answering system for the introduction to statistics course using supervised learning techniques
… Logistic Regression, Support Vector Machines, Stochastic Gradient Descent and Random Forests were compared to see which one provides the best results for the categorisation of a new question. The cosine similarity method was used to find the most similar past question. The Round-Trip …
-
Implementing a RESTful Software Architecture to Coordinate Heterogeneous Networked Embedded Devices
… micro-benchmarks, implementation of distributed stochastic gradient descent, and application to the design of versatile stateless services for vehicle-to-vehicle communication and military Joint All-Domain Command and Control (JDAC). From this evaluation, it was determined that CLES meets the …
-
Understanding neural network sample complexity and interpretable convergence-guaranteed deep learning with polynomial regression
… we empirically show that the widely-used Stochastic Gradient Descent algorithm makes the weights of the trained neural networks converge to the optimal polynomial regression weights.
-
Local differential privacy in decentralized optimization
… Method of Multipliers (ADMM), and decentralized (stochastic) gradient descent(D(S)GD) as two concrete examples to propose a framework of first-order based optimization with random local aggregators. We prove such local randomization lead to the same utility guarantee but amplify average LDP by a …
-
Topics in non-convex optimization and learning
… curvature, iteration complexity of Riemannian (stochastic) gradient descent methods is derived. I also show that some fast first-order methods in Euclidean space, such as Nesterov's accelerated gradient descent (AGD) and stochastic variance reduced gradient (SVRG), have Riemannian counterparts …
-
Leveraging Basis Alignment to create a Generalized Multi-Relational Graph Convolution Network in the Federated Setting
… for deep learning, Differentially Private Stochastic Gradient Descent (DP-SGD) [2], is not viable in this domain due to the nature of graph data and the internal framework of Graph Convolutional Networks.
-
Hyperparameters and neural architectures in differentially private deep learning
… Using optimization algorithms such as the DP stochastic gradient descent (DP-SGD), one can train deep learning models under DP guarantees. This thesis analyzes the impact of changes to the hyperparameters and the neural architecture on the utility/privacy tradeoff, the main tradeoff in DP, for …
-
Algorithms for structural learning with decompositions
… into learning, we use a dual projected subgradient ascent algorithm which naturally decomposes the task into simpler components. In the discriminative latent variable scenario, we present a supervised latent variable model for clustering called the Latent Left-Linking Model (L3M) that can …
-
Latent tree structure learning for cross-document coreference resolution
… parameter estimation algorithm based on existing stochastic gradient-descent based algorithms and show how to further tune regularization parameters. The latent tree structure is then learned using MCMC inference. We show how structural regularization plays a critical role in the inference …
-
Integral Quadratic Constraints and Safety Certificates for Uncertainty Characterization and Control Safety-Aware Filtering of Proximity Operations Between Satellites
… functions (CLFs/CBFs) using neural networks and stochastic gradient descent to provide safety-aware filtering for the fuel-optimal control policies. A linear quadratic regulator controller for a servicer satellite (Servicer) is analyzed via the dissipativity inequality principle and quadratic …
Page 1 of 4