Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents

Performance of state-of-the art offline and model-based reinforcement learning (RL) algorithms deteriorates significantly when subjected to severe data scarcity and the presence of heterogeneous agents. In this work, we propose a model-based offline RL method to approach this setting. Using all available data from the various agents, we construct personalized simulators for each individual agent, which are then used to train RL policies. We do so by modeling the transition dynamics of the agents as a low rank tensor decomposition of latent factors associated with agents, states, and actions. We perform experiments on various benchmark environments and demonstrate improvement over existing offline approaches in the scarce data regime.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2021

Author dc:creator

• Alumootil, Varkey

Advisor dc:contributor.advisor

• Shah, Devavrat

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright MIT

Licence dc:rights.uri

• http://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/139143

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/139143