{"id":{"repo_id":"rice","oai_identifier":"oai:repository.rice.edu:1911/113496"},"canonical_url":"https://search.dev.ndltd.org/etd/rice/oai:repository.rice.edu:1911/113496","repository":{"repo_id":"rice","name":"Rice University","base_url":"https://repository.rice.edu/server/oai/request"},"display":{"title":"Multilevel-in-Time Methods for Optimal Control of PDEs and Training of Recurrent Neural Networks","abstract":"This thesis develops and analyzes “time”-parallel methods for two problem classes: Optimality systems arising in discretized linear-quadratic Partial Differential Equation (PDE)-constrained optimization problems and Recurrent Neural Network (RNN) training problems. While these two problem classes seem different, they share a time-like structure. The proposed methods exploit decompositions in time of the optimization problem or of some of its components to introduce parallelism into the solution process, to reduce the global memory requirements of the solution algorithm, or to mitigate numerical issues due to stability of the underlying equations. For the solution of PDE-constrained problems, the proposed approach builds on a time-domain decomposition of the optimality system into coupled time-subdomain optimality systems and an extension of the Parareal algorithm to this system. The resulting two-grid iteration is interpreted as a Schur complement iteration. This allows its extension to a multigrid iteration and it also reveals how to derive a preconditioner for use in a Krylov subspace methods. The performance of the multigrid iterations as well as of flexible GMRES with the multigrid preconditioner are illustrated for an optimal control problem governed by an advection-diffusion PDE. The optimization problem arising in the training of RNNs is interpreted as a discrete-time optimal control problem. This interpretation is used to extend methods originally designed for optimal control of (partial) differential equations to the training of RNNs. Specifically, this thesis formulates a Gauss-Newton-type method and shows that the computation of the Gauss-Newton step can be formulated as the solution of a discrete-time linear quadratic optimal control problem. This formulation provides a connection with the multigrid-in-time methods developed for PDE-constrained problems. This thesis also develops a convergence analysis for inexact stochastic gradient methods. These methods allow the use of small, randomly selected batches of training data during the optimization, and they allow biases in the gradient approximations. The analysis does not require convexity of the problem and extends recent convergence results for a biased stochastic first-order (SFO) method developed for (strongly) convex PDE-constrained optimization problems under uncertainty. Moreover, the convergence analysis developed in this thesis is applied to the recently proposed Multigrid Reduction in Time (MGRIT) training of implicit Gated Recurrent Units.","abstract_html":"This thesis develops and analyzes “time”-parallel methods for two problem classes: Optimality systems arising in discretized linear-quadratic Partial Differential Equation (PDE)-constrained optimization problems and Recurrent Neural Network (RNN) training problems. While these two problem classes seem different, they share a time-like structure. The proposed methods exploit decompositions in time of the optimization problem or of some of its components to introduce parallelism into the solution process, to reduce the global memory requirements of the solution algorithm, or to mitigate numerical issues due to stability of the underlying equations. For the solution of PDE-constrained problems, the proposed approach builds on a time-domain decomposition of the optimality system into coupled time-subdomain optimality systems and an extension of the Parareal algorithm to this system. The resulting two-grid iteration is interpreted as a Schur complement iteration. This allows its extension to a multigrid iteration and it also reveals how to derive a preconditioner for use in a Krylov subspace methods. The performance of the multigrid iterations as well as of flexible GMRES with the multigrid preconditioner are illustrated for an optimal control problem governed by an advection-diffusion PDE. The optimization problem arising in the training of RNNs is interpreted as a discrete-time optimal control problem. This interpretation is used to extend methods originally designed for optimal control of (partial) differential equations to the training of RNNs. Specifically, this thesis formulates a Gauss-Newton-type method and shows that the computation of the Gauss-Newton step can be formulated as the solution of a discrete-time linear quadratic optimal control problem. This formulation provides a connection with the multigrid-in-time methods developed for PDE-constrained problems. This thesis also develops a convergence analysis for inexact stochastic gradient methods. These methods allow the use of small, randomly selected batches of training data during the optimization, and they allow biases in the gradient approximations. The analysis does not require convexity of the problem and extends recent convergence results for a biased stochastic first-order (SFO) method developed for (strongly) convex PDE-constrained optimization problems under uncertainty. Moreover, the convergence analysis developed in this thesis is applied to the recently proposed Multigrid Reduction in Time (MGRIT) training of implicit Gated Recurrent Units.","abstract_has_math":false,"creators":["Lin, Shengchao"],"institution":"Rice University","degree_name":"Doctor of Philosophy","degree_level":"Doctoral","degree_discipline":"Engineering","degree_department":null,"school":null,"contributors":[],"advisors":["Heinkenschloss, Matthias"],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-04-22","date_published":"2022-04-22","updated_at":"2026-07-24T04:10:21Z","subjects":["Linear quadratic optimal control problems","Multigrid","Recurrent neural network"],"languages":["eng"],"rights":["Copyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder."],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/1911/113496","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Heinkenschloss, Matthias"]},{"key":"dc:creator","label":"Author","values":["Lin, Shengchao"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2022-10-05T18:27:11Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2022-10-05T18:27:11Z"]},{"key":"dc:date.issued","label":"Date","values":["2022-04-22"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Doctoral"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["Rice University"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Linear quadratic optimal control problems","Multigrid","Recurrent neural network"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder."]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1911/113496"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["This thesis develops and analyzes “time”-parallel methods for two problem classes: Optimality systems arising in discretized linear-quadratic Partial Differential Equation (PDE)-constrained optimization problems and Recurrent Neural Network (RNN) training problems. While these two problem classes seem different, they share a time-like structure. The proposed methods exploit decompositions in time of the optimization problem or of some of its components to introduce parallelism into the solution process, to reduce the global memory requirements of the solution algorithm, or to mitigate numerical issues due to stability of the underlying equations. For the solution of PDE-constrained problems, the proposed approach builds on a time-domain decomposition of the optimality system into coupled time-subdomain optimality systems and an extension of the Parareal algorithm to this system. The resulting two-grid iteration is interpreted as a Schur complement iteration. This allows its extension to a multigrid iteration and it also reveals how to derive a preconditioner for use in a Krylov subspace methods. The performance of the multigrid iterations as well as of flexible GMRES with the multigrid preconditioner are illustrated for an optimal control problem governed by an advection-diffusion PDE. The optimization problem arising in the training of RNNs is interpreted as a discrete-time optimal control problem. This interpretation is used to extend methods originally designed for optimal control of (partial) differential equations to the training of RNNs. Specifically, this thesis formulates a Gauss-Newton-type method and shows that the computation of the Gauss-Newton step can be formulated as the solution of a discrete-time linear quadratic optimal control problem. This formulation provides a connection with the multigrid-in-time methods developed for PDE-constrained problems. This thesis also develops a convergence analysis for inexact stochastic gradient methods. These methods allow the use of small, randomly selected batches of training data during the optimization, and they allow biases in the gradient approximations. The analysis does not require convexity of the problem and extends recent convergence results for a biased stochastic first-order (SFO) method developed for (strongly) convex PDE-constrained optimization problems under uncertainty. Moreover, the convergence analysis developed in this thesis is applied to the recently proposed Multigrid Reduction in Time (MGRIT) training of implicit Gated Recurrent Units."]},{"key":"dc:format.mimetype","label":"Dc Format Mimetype","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Multilevel-in-Time Methods for Optimal Control of PDEs and Training of Recurrent Neural Networks"]}]}],"canonical_facts":{"dc:contributor.advisor":["Heinkenschloss, Matthias"],"dc:creator":["Lin, Shengchao"],"dc:date.accessioned":["2022-10-05T18:27:11Z"],"dc:date.available":["2022-10-05T18:27:11Z"],"dc:date.issued":["2022-04-22"],"dc:description.abstract":["This thesis develops and analyzes “time”-parallel methods for two problem classes: Optimality systems arising in discretized linear-quadratic Partial Differential Equation (PDE)-constrained optimization problems and Recurrent Neural Network (RNN) training problems. While these two problem classes seem different, they share a time-like structure. The proposed methods exploit decompositions in time of the optimization problem or of some of its components to introduce parallelism into the solution process, to reduce the global memory requirements of the solution algorithm, or to mitigate numerical issues due to stability of the underlying equations. For the solution of PDE-constrained problems, the proposed approach builds on a time-domain decomposition of the optimality system into coupled time-subdomain optimality systems and an extension of the Parareal algorithm to this system. The resulting two-grid iteration is interpreted as a Schur complement iteration. This allows its extension to a multigrid iteration and it also reveals how to derive a preconditioner for use in a Krylov subspace methods. The performance of the multigrid iterations as well as of flexible GMRES with the multigrid preconditioner are illustrated for an optimal control problem governed by an advection-diffusion PDE. The optimization problem arising in the training of RNNs is interpreted as a discrete-time optimal control problem. This interpretation is used to extend methods originally designed for optimal control of (partial) differential equations to the training of RNNs. Specifically, this thesis formulates a Gauss-Newton-type method and shows that the computation of the Gauss-Newton step can be formulated as the solution of a discrete-time linear quadratic optimal control problem. This formulation provides a connection with the multigrid-in-time methods developed for PDE-constrained problems. This thesis also develops a convergence analysis for inexact stochastic gradient methods. These methods allow the use of small, randomly selected batches of training data during the optimization, and they allow biases in the gradient approximations. The analysis does not require convexity of the problem and extends recent convergence results for a biased stochastic first-order (SFO) method developed for (strongly) convex PDE-constrained optimization problems under uncertainty. Moreover, the convergence analysis developed in this thesis is applied to the recently proposed Multigrid Reduction in Time (MGRIT) training of implicit Gated Recurrent Units."],"dc:format.mimetype":["application/pdf"],"dc:identifier.uri":["https://hdl.handle.net/1911/113496"],"dc:language.iso":["eng"],"dc:rights":["Copyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder."],"dc:subject":["Linear quadratic optimal control problems","Multigrid","Recurrent neural network"],"dc:title":["Multilevel-in-Time Methods for Optimal Control of PDEs and Training of Recurrent Neural Networks"],"dc:type":["Thesis"],"thesis:degree_discipline":["Engineering"],"thesis:degree_level":["Doctoral"],"thesis:degree_name":["Doctor of Philosophy"],"thesis:institution_name":["Rice University"]},"updated_at":"2026-07-24T04:10:21Z"}