{"id":{"repo_id":"vt","oai_identifier":"oai:vtechworks.lib.vt.edu:10919/140039"},"canonical_url":"https://search.dev.ndltd.org/etd/vt/oai:vtechworks.lib.vt.edu:10919/140039","repository":{"repo_id":"vt","name":"Virginia Tech","base_url":"https://vtechworks.lib.vt.edu/oai/request"},"display":{"title":"Deep Reinforcement Learning for Multirotor Flight Control: A Comparative Study of Sim-to-Real Training and Real-World Performance","abstract":"This dissertation investigates Deep Reinforcement Learning (DRL) for low-level flight control of multirotor unmanned aerial vehicles (UAVs), focusing on factors that most influence sim-to-real policy transfer. Using the Proximal Policy Optimization (PPO) algorithm, extensive ablation studies evaluated the effects of domain randomization, reward weighting, observation representation, and neural network architecture. Policies were trained in a MuJoCo-based simulator and deployed via TensorFlow Lite Micro inference on PX4 flight controllers. Domain randomization of actuator and mass properties yielded the best balance between positional accuracy and attitude stability, achieving reductions of 40–50 % in position error and roll–pitch oscillation. Policies required only the current vehicle state and previous commanded action, maintaining less than 0.1 m steady-state error and less than 5 % overshoot. Among activation functions tested, ReLU outperformed tanh and ELU, lowering steady-state error by up to 36 % and inference time by 26 %. Post-training quantization further reduced inference latency by ≈40 % with negligible performance loss. Incorporating trajectory tracking during training decreased tracking error by ≈75 % and eliminated temporal lag. The optimal training configuration generalized effectively to multiple vehicle morphologies, including quadcopter, hexacopter, and coaxial octocopter platforms. In addition, the process was successfully extended to an omnidirectional multi-rotor vehicle (OMV). For the OMV, a learned Multi-Layer Perceptron (MLP) controller outperformed both adaptive and PID-based baselines when commanded to track a complex reference attitude, demonstrating stable six-degree-of-freedom trajectory tracking in experimental flights. Collectively, these results provide new insight into parameter sensitivities within the DRL training pipeline and establish a reproducible methodology for sim-to-real policy transfer in aerial robotics.","abstract_html":"This dissertation investigates Deep Reinforcement Learning (DRL) for low-level flight control of multirotor unmanned aerial vehicles (UAVs), focusing on factors that most influence sim-to-real policy transfer. Using the Proximal Policy Optimization (PPO) algorithm, extensive ablation studies evaluated the effects of domain randomization, reward weighting, observation representation, and neural network architecture. Policies were trained in a MuJoCo-based simulator and deployed via TensorFlow Lite Micro inference on PX4 flight controllers. Domain randomization of actuator and mass properties yielded the best balance between positional accuracy and attitude stability, achieving reductions of 40–50 % in position error and roll–pitch oscillation. Policies required only the current vehicle state and previous commanded action, maintaining less than 0.1 m steady-state error and less than 5 % overshoot. Among activation functions tested, ReLU outperformed tanh and ELU, lowering steady-state error by up to 36 % and inference time by 26 %. Post-training quantization further reduced inference latency by ≈40 % with negligible performance loss. Incorporating trajectory tracking during training decreased tracking error by ≈75 % and eliminated temporal lag. The optimal training configuration generalized effectively to multiple vehicle morphologies, including quadcopter, hexacopter, and coaxial octocopter platforms. In addition, the process was successfully extended to an omnidirectional multi-rotor vehicle (OMV). For the OMV, a learned Multi-Layer Perceptron (MLP) controller outperformed both adaptive and PID-based baselines when commanded to track a complex reference attitude, demonstrating stable six-degree-of-freedom trajectory tracking in experimental flights. Collectively, these results provide new insight into parameter sensitivities within the DRL training pipeline and establish a reproducible methodology for sim-to-real policy transfer in aerial robotics.","abstract_has_math":false,"creators":["Thomas, Patrick James"],"institution":"Virginia Tech","degree_name":"Doctor of Philosophy","degree_level":"doctoral","degree_discipline":"Aerospace Engineering","degree_department":"Aerospace and Ocean Engineering","school":null,"contributors":[],"advisors":[],"committee_chairs":["Schroeder, Kevin Kent","Black, Jonathan T."],"committee_members":["Woolsey, Craig A.","L'Afflitto, Andrea"],"year":2025,"date_issued":"2025-12-18","date_published":"2025-12-18","updated_at":"2026-07-22T22:20:00Z","subjects":["UAV","Deep Reinforcement Learning","Controls"],"languages":["en"],"rights":["Creative Commons Attribution 4.0 International"],"rights_urls":["http://creativecommons.org/licenses/by/4.0/"],"identifier_entries":[{"key":"dc:identifier.other","label":"Dc Identifier Other","values":["vt_gsexam:45302"],"render_values":[{"text":"vt_gsexam:45302","href":null,"code":true}]}]},"links":{"outbound_url":"https://hdl.handle.net/10919/140039","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.committeechair","label":"Committee Chair","values":["Schroeder, Kevin Kent","Black, Jonathan T."]},{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Woolsey, Craig A.","L'Afflitto, Andrea"]},{"key":"dc:contributor.department","label":"Department","values":["Aerospace and Ocean Engineering"]},{"key":"dc:creator","label":"Author","values":["Thomas, Patrick James"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2025-12-19T09:00:55Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2025-12-19T09:00:55Z"]},{"key":"dc:date.issued","label":"Date","values":["2025-12-18"]},{"key":"dc:publisher","label":"Institution","values":["Virginia Tech"]},{"key":"dc:type","label":"Dc Type","values":["Dissertation"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Aerospace Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["doctoral"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["Virginia Polytechnic Institute and State University"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["UAV","Deep Reinforcement Learning","Controls"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Creative Commons Attribution 4.0 International"]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://creativecommons.org/licenses/by/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.other","label":"Dc Identifier Other","values":["vt_gsexam:45302"]},{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/10919/140039"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["This dissertation investigates Deep Reinforcement Learning (DRL) for low-level flight control of multirotor unmanned aerial vehicles (UAVs), focusing on factors that most influence sim-to-real policy transfer. Using the Proximal Policy Optimization (PPO) algorithm, extensive ablation studies evaluated the effects of domain randomization, reward weighting, observation representation, and neural network architecture. Policies were trained in a MuJoCo-based simulator and deployed via TensorFlow Lite Micro inference on PX4 flight controllers. Domain randomization of actuator and mass properties yielded the best balance between positional accuracy and attitude stability, achieving reductions of 40–50 % in position error and roll–pitch oscillation. Policies required only the current vehicle state and previous commanded action, maintaining less than 0.1 m steady-state error and less than 5 % overshoot. Among activation functions tested, ReLU outperformed tanh and ELU, lowering steady-state error by up to 36 % and inference time by 26 %. Post-training quantization further reduced inference latency by ≈40 % with negligible performance loss. Incorporating trajectory tracking during training decreased tracking error by ≈75 % and eliminated temporal lag. The optimal training configuration generalized effectively to multiple vehicle morphologies, including quadcopter, hexacopter, and coaxial octocopter platforms. In addition, the process was successfully extended to an omnidirectional multi-rotor vehicle (OMV). For the OMV, a learned Multi-Layer Perceptron (MLP) controller outperformed both adaptive and PID-based baselines when commanded to track a complex reference attitude, demonstrating stable six-degree-of-freedom trajectory tracking in experimental flights. Collectively, these results provide new insight into parameter sensitivities within the DRL training pipeline and establish a reproducible methodology for sim-to-real policy transfer in aerial robotics."]},{"key":"dc:description.abstractgeneral","label":"General Abstract","values":["Unmanned aerial vehicles (UAVs)—more commonly known as drones—have become indispensable tools across industries ranging from filmmaking and infrastructure inspection to logistics and defense. However, their ability to operate safely and reliably in the real world depends on how well their onboard flight controllers can adapt to uncertain and changing conditions. Traditional control systems struggle when real-world conditions differ from their mathematical models. This dissertation explores how deep reinforcement learning (DRL)—a branch of machine learning where agents learn through trial and error—can be used to train flight controllers that are more robust and adaptable. Using advanced simulation techniques, this research identifies which aspects of the training process—such as environmental variability, reward functions, and neural network design—most affect real-world performance when controllers are transferred from simulation to physical flight. The work also extends these techniques to an omnidirectional multirotor vehicle, capable of moving and rotating freely in all directions. The findings offer a framework for developing more reliable and generalizable learning-based flight controllers, contributing to the broader goal of making autonomous aerial systems more capable in complex, uncertain environments."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["Doctor of Philosophy"]},{"key":"dc:format.medium","label":"Dc Format Medium","values":["ETD"]},{"key":"dc:title","label":"Title","values":["Deep Reinforcement Learning for Multirotor Flight Control: A Comparative Study of Sim-to-Real Training and Real-World Performance"]}]}],"canonical_facts":{"dc:contributor.committeechair":["Schroeder, Kevin Kent","Black, Jonathan T."],"dc:contributor.committeemember":["Woolsey, Craig A.","L'Afflitto, Andrea"],"dc:contributor.department":["Aerospace and Ocean Engineering"],"dc:creator":["Thomas, Patrick James"],"dc:date.accessioned":["2025-12-19T09:00:55Z"],"dc:date.available":["2025-12-19T09:00:55Z"],"dc:date.issued":["2025-12-18"],"dc:description.abstract":["This dissertation investigates Deep Reinforcement Learning (DRL) for low-level flight control of multirotor unmanned aerial vehicles (UAVs), focusing on factors that most influence sim-to-real policy transfer. Using the Proximal Policy Optimization (PPO) algorithm, extensive ablation studies evaluated the effects of domain randomization, reward weighting, observation representation, and neural network architecture. Policies were trained in a MuJoCo-based simulator and deployed via TensorFlow Lite Micro inference on PX4 flight controllers. Domain randomization of actuator and mass properties yielded the best balance between positional accuracy and attitude stability, achieving reductions of 40–50 % in position error and roll–pitch oscillation. Policies required only the current vehicle state and previous commanded action, maintaining less than 0.1 m steady-state error and less than 5 % overshoot. Among activation functions tested, ReLU outperformed tanh and ELU, lowering steady-state error by up to 36 % and inference time by 26 %. Post-training quantization further reduced inference latency by ≈40 % with negligible performance loss. Incorporating trajectory tracking during training decreased tracking error by ≈75 % and eliminated temporal lag. The optimal training configuration generalized effectively to multiple vehicle morphologies, including quadcopter, hexacopter, and coaxial octocopter platforms. In addition, the process was successfully extended to an omnidirectional multi-rotor vehicle (OMV). For the OMV, a learned Multi-Layer Perceptron (MLP) controller outperformed both adaptive and PID-based baselines when commanded to track a complex reference attitude, demonstrating stable six-degree-of-freedom trajectory tracking in experimental flights. Collectively, these results provide new insight into parameter sensitivities within the DRL training pipeline and establish a reproducible methodology for sim-to-real policy transfer in aerial robotics."],"dc:description.abstractgeneral":["Unmanned aerial vehicles (UAVs)—more commonly known as drones—have become indispensable tools across industries ranging from filmmaking and infrastructure inspection to logistics and defense. However, their ability to operate safely and reliably in the real world depends on how well their onboard flight controllers can adapt to uncertain and changing conditions. Traditional control systems struggle when real-world conditions differ from their mathematical models. This dissertation explores how deep reinforcement learning (DRL)—a branch of machine learning where agents learn through trial and error—can be used to train flight controllers that are more robust and adaptable. Using advanced simulation techniques, this research identifies which aspects of the training process—such as environmental variability, reward functions, and neural network design—most affect real-world performance when controllers are transferred from simulation to physical flight. The work also extends these techniques to an omnidirectional multirotor vehicle, capable of moving and rotating freely in all directions. The findings offer a framework for developing more reliable and generalizable learning-based flight controllers, contributing to the broader goal of making autonomous aerial systems more capable in complex, uncertain environments."],"dc:description.degree":["Doctor of Philosophy"],"dc:format.medium":["ETD"],"dc:identifier.other":["vt_gsexam:45302"],"dc:identifier.uri":["https://hdl.handle.net/10919/140039"],"dc:language.iso":["en"],"dc:publisher":["Virginia Tech"],"dc:rights":["Creative Commons Attribution 4.0 International"],"dc:rights.uri":["http://creativecommons.org/licenses/by/4.0/"],"dc:subject":["UAV","Deep Reinforcement Learning","Controls"],"dc:title":["Deep Reinforcement Learning for Multirotor Flight Control: A Comparative Study of Sim-to-Real Training and Real-World Performance"],"dc:type":["Dissertation"],"thesis:degree_discipline":["Aerospace Engineering"],"thesis:degree_level":["doctoral"],"thesis:degree_name":["Doctor of Philosophy"],"thesis:institution_name":["Virginia Polytechnic Institute and State University"]},"updated_at":"2026-07-22T22:20:00Z"}