{"id":{"repo_id":"gatech","oai_identifier":"oai:repository.gatech.edu:1853/66058"},"canonical_url":"https://search.dev.ndltd.org/etd/gatech/oai:repository.gatech.edu:1853/66058","repository":{"repo_id":"gatech","name":"Georgia Tech","base_url":"https://repository.gatech.edu/server/oai/request"},"display":{"title":"Feynman-Kac Numerical Techniques for Stochastic Optimal Control","abstract":"Three significant advancements are proposed for improving numerical methods in the solution of forward-backward stochastic differential equations (FBSDEs) appearing in the Feynman-Kac representation of the value function in stochastic optimal control (SOC) problems. First, we propose a novel characterization of FBSDE estimators as either on-policy or off-policy, highlighting the intuition for these techniques that the distribution over which value functions are approximated should, to some extent, match the distribution the policies generate. Second, two novel numerical estimators are proposed for improving the accuracy of single-timestep updates. In the case of LQR problems, we demonstrate both in theory and in numerical simulation that our estimators result in near machine-precision level accuracy, in contrast to previously proposed methods that can potentially diverge on the same problems. Third, we propose a new method for accelerating the global convergence of FBSDE methods. By the repeated use of the Girsanov change of probability measures, it is demonstrated how a McKean-Markov branched sampling method can be utilized for the forward integration pass, as long as the controlled drift terms are appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of a space-filling tree.","abstract_html":"Three significant advancements are proposed for improving numerical methods in the solution of forward-backward stochastic differential equations (FBSDEs) appearing in the Feynman-Kac representation of the value function in stochastic optimal control (SOC) problems. First, we propose a novel characterization of FBSDE estimators as either on-policy or off-policy, highlighting the intuition for these techniques that the distribution over which value functions are approximated should, to some extent, match the distribution the policies generate. Second, two novel numerical estimators are proposed for improving the accuracy of single-timestep updates. In the case of LQR problems, we demonstrate both in theory and in numerical simulation that our estimators result in near machine-precision level accuracy, in contrast to previously proposed methods that can potentially diverge on the same problems. Third, we propose a new method for accelerating the global convergence of FBSDE methods. By the repeated use of the Girsanov change of probability measures, it is demonstrated how a McKean-Markov branched sampling method can be utilized for the forward integration pass, as long as the controlled drift terms are appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of a space-filling tree.","abstract_has_math":false,"creators":["Hawkins, Kelsey Pal"],"institution":"Georgia Institute of Technology","degree_name":null,"degree_level":"Doctoral","degree_discipline":null,"degree_department":"Interactive Computing","school":null,"contributors":[],"advisors":["Tsiotras, Panagiotis"],"committee_chairs":[],"committee_members":["Berenson, Dmitry","Coogan, Samuel","Theodorou, Evangelos","Vamvoudakis, Kyriakos"],"year":2021,"date_issued":"2021-08-25","date_published":"2021-08-25","updated_at":"2026-07-27T19:48:57Z","subjects":["Stochastic optimal control","Forward-backward stochastic differential equations","Robotics","Partial differential equations","Motion planning","Rapidly exploring random trees"],"languages":["en_US"],"rights":[],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/1853/66058","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Tsiotras, Panagiotis"]},{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Berenson, Dmitry","Coogan, Samuel","Theodorou, Evangelos","Vamvoudakis, Kyriakos"]},{"key":"dc:contributor.department","label":"Department","values":["Interactive Computing"]},{"key":"dc:creator","label":"Author","values":["Hawkins, Kelsey Pal"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2022-01-14T16:07:08Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2022-01-14T16:07:08Z"]},{"key":"dc:date.issued","label":"Date","values":["2021-08-25"]},{"key":"dc:publisher","label":"Institution","values":["Georgia Institute of Technology"]},{"key":"dc:type","label":"Dc Type","values":["Text"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Doctoral"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Stochastic optimal control","Forward-backward stochastic differential equations","Robotics","Partial differential equations","Motion planning","Rapidly exploring random trees"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en_US"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["http://hdl.handle.net/1853/66058"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Three significant advancements are proposed for improving numerical methods in the solution of forward-backward stochastic differential equations (FBSDEs) appearing in the Feynman-Kac representation of the value function in stochastic optimal control (SOC) problems. First, we propose a novel characterization of FBSDE estimators as either on-policy or off-policy, highlighting the intuition for these techniques that the distribution over which value functions are approximated should, to some extent, match the distribution the policies generate. Second, two novel numerical estimators are proposed for improving the accuracy of single-timestep updates. In the case of LQR problems, we demonstrate both in theory and in numerical simulation that our estimators result in near machine-precision level accuracy, in contrast to previously proposed methods that can potentially diverge on the same problems. Third, we propose a new method for accelerating the global convergence of FBSDE methods. By the repeated use of the Girsanov change of probability measures, it is demonstrated how a McKean-Markov branched sampling method can be utilized for the forward integration pass, as long as the controlled drift terms are appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of a space-filling tree."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["Ph.D."]},{"key":"dc:format.mimetype","label":"Dc Format Mimetype","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Feynman-Kac Numerical Techniques for Stochastic Optimal Control"]}]}],"canonical_facts":{"dc:contributor.advisor":["Tsiotras, Panagiotis"],"dc:contributor.committeemember":["Berenson, Dmitry","Coogan, Samuel","Theodorou, Evangelos","Vamvoudakis, Kyriakos"],"dc:contributor.department":["Interactive Computing"],"dc:creator":["Hawkins, Kelsey Pal"],"dc:date.accessioned":["2022-01-14T16:07:08Z"],"dc:date.available":["2022-01-14T16:07:08Z"],"dc:date.issued":["2021-08-25"],"dc:description.abstract":["Three significant advancements are proposed for improving numerical methods in the solution of forward-backward stochastic differential equations (FBSDEs) appearing in the Feynman-Kac representation of the value function in stochastic optimal control (SOC) problems. First, we propose a novel characterization of FBSDE estimators as either on-policy or off-policy, highlighting the intuition for these techniques that the distribution over which value functions are approximated should, to some extent, match the distribution the policies generate. Second, two novel numerical estimators are proposed for improving the accuracy of single-timestep updates. In the case of LQR problems, we demonstrate both in theory and in numerical simulation that our estimators result in near machine-precision level accuracy, in contrast to previously proposed methods that can potentially diverge on the same problems. Third, we propose a new method for accelerating the global convergence of FBSDE methods. By the repeated use of the Girsanov change of probability measures, it is demonstrated how a McKean-Markov branched sampling method can be utilized for the forward integration pass, as long as the controlled drift terms are appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of a space-filling tree."],"dc:description.degree":["Ph.D."],"dc:format.mimetype":["application/pdf"],"dc:identifier.uri":["http://hdl.handle.net/1853/66058"],"dc:language.iso":["en_US"],"dc:publisher":["Georgia Institute of Technology"],"dc:subject":["Stochastic optimal control","Forward-backward stochastic differential equations","Robotics","Partial differential equations","Motion planning","Rapidly exploring random trees"],"dc:title":["Feynman-Kac Numerical Techniques for Stochastic Optimal Control"],"dc:type":["Text"],"thesis:degree_level":["Doctoral"]},"updated_at":"2026-07-27T19:48:57Z"}