Decision making in the presence of complex dynamics from limited, batch data

Robot decision making in real-world domains can be extremely difficult when the robot has to interact with a complex, poorly understood environment. In these environments, a data-driven approach is commonly taken where a model is first learned and then used for decision making since expert knowledge is rarely sucient for specifying the world's dynamics. Unfortunately, learning a model for a complex environment often involves fitting a large number of parameters which can require an unobtainable amount of data. In real-world domains we are also typically confronted with fitting a model that is only an approximation of the true dynamics, causing difficulties for standard learning approaches. In this thesis we explore two core methodologies for learning a model for decision making in the presence of complex dynamics: explicitly selecting the model which achieves the highest estimated performance and allowing the model class to grow as more data is seen. We show that our approach for explicitly selecting the model with the highest estimated performance has desirable theoretical properties and outperforms standard minimum error fitting techniques on benchmark and real-world problems. To grow the size of model class with the amount of data, we first show how this can be accomplished by using Bayesian nonparametric statistics to model the dynamics, which can then be used for planning. We then present an alternative approach which grows the policy class using the principle of structural risk minimization, for which the resulting algorithm has provable performance bounds with weak assumptions on the true world's dynamics.

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Aeronautics and Astronautics

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2014

Author dc:creator

• Joseph, Joshua Mason

Advisor dc:contributor.advisor

• Nicholas Roy.

• Aeronautics and Astronautics.

Statement dc:rights

• M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/90603

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/90603