Machine social intelligence in Virtualhome

This thesis introduces a Watch-And-Help (WAH) challenge and a multi-agent environment for testing social intelligence in multiple agents. In the challenge, an AI agent needs to help a human-like agent perform a complex household task efficiently. To succeed, the AI agent needs to i) understand the underlying goal of the task by watching a single demonstration of the human-like agent performing the same task (social perception), and ii) coordinate with the human-like agent to solve the task in an unseen environment as fast as possible (human-AI collaboration). For this challenge, we build VirtualHome-Social, a multi-agent household environment, and provide a benchmark including both planning and learning based baselines. Experimental results demonstrate that in order to achieve success in the challenge, an AI agent has to accurately understand and predict the human-like agent's behaviors, and adapt its collaborative plan accordingly in novel environments.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2020

Author dc:creator

• Li, Shuang,S.M.Massachusetts Institute of Technology.

Advisor dc:contributor.advisor

• Antonio Torralba.

• Electrical Engineering and Computer Science.

Statement dc:rights

• MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/129367

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/129367