Utilization and Synthesis of Symbolic World Models for Safe, Generalizable, and Efficient Action

Reinforcement learning with neural networks has proven incredibly flexible at learning to act in diverse environments. Model-based RL techniques have helped to ameliorate the dependence on large quantities of data that these models normally have. However, despite their flexibility, neural world models have several drawbacks. Symbolic world models, in comparison, are easier to verify (e.g. for safety concerns), more compatible with domain-independent planning techniques, and able to be learned or adapted with more limited data. In this thesis, I will demonstrate these advantages of symbolic world models in three projects. The first, VSRL, shows how we can use a symbolic world model to ensure that an RL policy is safe during both training and deployment and promote safe exploration. The second, SPARSER, presents a hybrid domain planner which uses world models in a planning domain description language. It showcases how we can exploit the event structure in the world model to enable more efficient planning. In the final project, PWM, I will explore learning a world model directly from observations and actions gathered from interacting with an environment. We combine symbolic and neural synthesis techniques to enable efficient world model synthesis even from visual observations. Together, these projects demonstrate the versatility and value of symbolic world models.

Name thesis:degree_name

Doctoral

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2024

Author dc:creator

• Hunt, Nathan

Advisor dc:contributor.advisor

• Solar-Lezama, Armando

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright retained by author(s)

Licence dc:rights.uri

• https://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/158475

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/158475