{"id":{"repo_id":"toronto-retro","oai_identifier":"oai:utoronto.scholaris.ca:1807/140630"},"canonical_url":"https://search.dev.ndltd.org/etd/toronto-retro/oai:utoronto.scholaris.ca:1807/140630","repository":{"repo_id":"toronto-retro","name":"University of Toronto","base_url":"https://utoronto.scholaris.ca/server/oai/request"},"display":{"title":"Advancing Efficiency and Safety in Autonomous Sequential Decision Making","abstract":"The advent of reinforcement learning (RL) has significantly transformed decision making in autonomous systems. However, its practical deployment faces substantial obstacles, chiefly in achieving sample (data) efficiency and ensuring agent safety in unpredictable, dynamic environments. Additionally, the inherent partial knowledge due to sensory and model limitations complicates agents' functionality in complex scenarios. This thesis aims to enhance sample efficiency and safety by improving adaptability in evolving environments, addressing inaccuracies under uncertainty, and implementing effective risk management strategies. It focuses on boosting RL data efficiency through simultaneous emphasis on epistemic uncertainty and knowledge transfer, and on mitigating risks associated with agent actions through aleatory uncertainty. Initially, the thesis investigates strategies to enhance RL algorithms' adaptability and knowledge transfer across varying scenarios, alongside quantifying model uncertainty—specifically, the inaccuracies in agents' understanding of their operational contexts. It underscores the importance of epistemic uncertainty within agents' approximation models, proposing a unified approach for knowledge transfer and uncertainty management to improve sample efficiency and foster the development of flexible, robust RL applications. Subsequent exploration tackles the challenges posed by partially observable Markov decision processes (POMDPs), which represent an agent's uncertainty about its environmental state. A novel framework that integrates active inference (AIF), based on the free energy principle, with RL in continuous-space POMDPs is introduced. By emphasizing the minimization of expected free energy (EFE), AIF augments RL, enhancing decision making in the face of state uncertainty. This approach notably boosts sample efficiency by systematically reducing state epistemic uncertainty, thus enabling more effective agent interaction in partially known environments. The concluding section revisits risk management, highlighting the interplay between aleatory and epistemic uncertainties in decision making. It introduces a unified uncertainty estimation algorithm that facilitates a risk-sensitive strategy, employing aleatory uncertainty while capitalizing on epistemic uncertainty to augment sample efficiency. This method presents a comprehensive strategy for navigating the complexities of risk management and enhancing sample efficiency in RL applications.","abstract_html":"The advent of reinforcement learning (RL) has significantly transformed decision making in autonomous systems. However, its practical deployment faces substantial obstacles, chiefly in achieving sample (data) efficiency and ensuring agent safety in unpredictable, dynamic environments. Additionally, the inherent partial knowledge due to sensory and model limitations complicates agents' functionality in complex scenarios. This thesis aims to enhance sample efficiency and safety by improving adaptability in evolving environments, addressing inaccuracies under uncertainty, and implementing effective risk management strategies. It focuses on boosting RL data efficiency through simultaneous emphasis on epistemic uncertainty and knowledge transfer, and on mitigating risks associated with agent actions through aleatory uncertainty. Initially, the thesis investigates strategies to enhance RL algorithms' adaptability and knowledge transfer across varying scenarios, alongside quantifying model uncertainty—specifically, the inaccuracies in agents' understanding of their operational contexts. It underscores the importance of epistemic uncertainty within agents' approximation models, proposing a unified approach for knowledge transfer and uncertainty management to improve sample efficiency and foster the development of flexible, robust RL applications. Subsequent exploration tackles the challenges posed by partially observable Markov decision processes (POMDPs), which represent an agent's uncertainty about its environmental state. A novel framework that integrates active inference (AIF), based on the free energy principle, with RL in continuous-space POMDPs is introduced. By emphasizing the minimization of expected free energy (EFE), AIF augments RL, enhancing decision making in the face of state uncertainty. This approach notably boosts sample efficiency by systematically reducing state epistemic uncertainty, thus enabling more effective agent interaction in partially known environments. The concluding section revisits risk management, highlighting the interplay between aleatory and epistemic uncertainties in decision making. It introduces a unified uncertainty estimation algorithm that facilitates a risk-sensitive strategy, employing aleatory uncertainty while capitalizing on epistemic uncertainty to augment sample efficiency. This method presents a comprehensive strategy for navigating the complexities of risk management and enhancing sample efficiency in RL applications.","abstract_has_math":false,"creators":["Malekzadeh, Parvin"],"institution":null,"degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":"Electrical and Computer Engineering","school":null,"contributors":[],"advisors":["Plataniotis, Konstantinos N"],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-11","date_published":"2024-11","updated_at":"2026-07-27T21:28:18Z","subjects":["Active Inference (AIF)","Autonomous Decision Making","Reinforcement Learning (RL)","Risk Safety","Sample Efficiency","Uncertainty-aware Decision Making"],"languages":[],"rights":["Attribution 4.0 International"],"rights_urls":["http://creativecommons.org/licenses/by/4.0/"],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/1807/140630","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Plataniotis, Konstantinos N"]},{"key":"dc:contributor.department","label":"Department","values":["Electrical and Computer Engineering"]},{"key":"dc:creator","label":"Author","values":["Malekzadeh, Parvin"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2024-11"]},{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2024-11-13T16:45:26Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2024-11-13T16:45:26Z"]},{"key":"dc:date.issued","label":"Date","values":["2024-11"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Active Inference (AIF)","Autonomous Decision Making","Reinforcement Learning (RL)","Risk Safety","Sample Efficiency","Uncertainty-aware Decision Making"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["Attribution 4.0 International"]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://creativecommons.org/licenses/by/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["http://hdl.handle.net/1807/140630"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["The advent of reinforcement learning (RL) has significantly transformed decision making in autonomous systems. However, its practical deployment faces substantial obstacles, chiefly in achieving sample (data) efficiency and ensuring agent safety in unpredictable, dynamic environments. Additionally, the inherent partial knowledge due to sensory and model limitations complicates agents' functionality in complex scenarios. This thesis aims to enhance sample efficiency and safety by improving adaptability in evolving environments, addressing inaccuracies under uncertainty, and implementing effective risk management strategies. It focuses on boosting RL data efficiency through simultaneous emphasis on epistemic uncertainty and knowledge transfer, and on mitigating risks associated with agent actions through aleatory uncertainty. Initially, the thesis investigates strategies to enhance RL algorithms' adaptability and knowledge transfer across varying scenarios, alongside quantifying model uncertainty—specifically, the inaccuracies in agents' understanding of their operational contexts. It underscores the importance of epistemic uncertainty within agents' approximation models, proposing a unified approach for knowledge transfer and uncertainty management to improve sample efficiency and foster the development of flexible, robust RL applications. Subsequent exploration tackles the challenges posed by partially observable Markov decision processes (POMDPs), which represent an agent's uncertainty about its environmental state. A novel framework that integrates active inference (AIF), based on the free energy principle, with RL in continuous-space POMDPs is introduced. By emphasizing the minimization of expected free energy (EFE), AIF augments RL, enhancing decision making in the face of state uncertainty. This approach notably boosts sample efficiency by systematically reducing state epistemic uncertainty, thus enabling more effective agent interaction in partially known environments. The concluding section revisits risk management, highlighting the interplay between aleatory and epistemic uncertainties in decision making. It introduces a unified uncertainty estimation algorithm that facilitates a risk-sensitive strategy, employing aleatory uncertainty while capitalizing on epistemic uncertainty to augment sample efficiency. This method presents a comprehensive strategy for navigating the complexities of risk management and enhancing sample efficiency in RL applications."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["Ph.D."]},{"key":"dc:title","label":"Title","values":["Advancing Efficiency and Safety in Autonomous Sequential Decision Making"]}]}],"canonical_facts":{"dc:contributor.advisor":["Plataniotis, Konstantinos N"],"dc:contributor.department":["Electrical and Computer Engineering"],"dc:creator":["Malekzadeh, Parvin"],"dc:date":["2024-11"],"dc:date.accessioned":["2024-11-13T16:45:26Z"],"dc:date.available":["2024-11-13T16:45:26Z"],"dc:date.issued":["2024-11"],"dc:description.abstract":["The advent of reinforcement learning (RL) has significantly transformed decision making in autonomous systems. However, its practical deployment faces substantial obstacles, chiefly in achieving sample (data) efficiency and ensuring agent safety in unpredictable, dynamic environments. Additionally, the inherent partial knowledge due to sensory and model limitations complicates agents' functionality in complex scenarios. This thesis aims to enhance sample efficiency and safety by improving adaptability in evolving environments, addressing inaccuracies under uncertainty, and implementing effective risk management strategies. It focuses on boosting RL data efficiency through simultaneous emphasis on epistemic uncertainty and knowledge transfer, and on mitigating risks associated with agent actions through aleatory uncertainty. Initially, the thesis investigates strategies to enhance RL algorithms' adaptability and knowledge transfer across varying scenarios, alongside quantifying model uncertainty—specifically, the inaccuracies in agents' understanding of their operational contexts. It underscores the importance of epistemic uncertainty within agents' approximation models, proposing a unified approach for knowledge transfer and uncertainty management to improve sample efficiency and foster the development of flexible, robust RL applications. Subsequent exploration tackles the challenges posed by partially observable Markov decision processes (POMDPs), which represent an agent's uncertainty about its environmental state. A novel framework that integrates active inference (AIF), based on the free energy principle, with RL in continuous-space POMDPs is introduced. By emphasizing the minimization of expected free energy (EFE), AIF augments RL, enhancing decision making in the face of state uncertainty. This approach notably boosts sample efficiency by systematically reducing state epistemic uncertainty, thus enabling more effective agent interaction in partially known environments. The concluding section revisits risk management, highlighting the interplay between aleatory and epistemic uncertainties in decision making. It introduces a unified uncertainty estimation algorithm that facilitates a risk-sensitive strategy, employing aleatory uncertainty while capitalizing on epistemic uncertainty to augment sample efficiency. This method presents a comprehensive strategy for navigating the complexities of risk management and enhancing sample efficiency in RL applications."],"dc:description.degree":["Ph.D."],"dc:identifier.uri":["http://hdl.handle.net/1807/140630"],"dc:rights":["Attribution 4.0 International"],"dc:rights.uri":["http://creativecommons.org/licenses/by/4.0/"],"dc:subject":["Active Inference (AIF)","Autonomous Decision Making","Reinforcement Learning (RL)","Risk Safety","Sample Efficiency","Uncertainty-aware Decision Making"],"dc:title":["Advancing Efficiency and Safety in Autonomous Sequential Decision Making"],"dc:type":["Thesis"]},"updated_at":"2026-07-27T21:28:18Z"}