Environmental curriculum learning for efficiently achieving superhuman play in games

Reinforcement learning has made large strides in training agents to play games, including complex ones such as arcade game Pommerman and real-time strategy game StarCraft II. To allow agents to grasp the many concepts in these games, curriculum learning has been used to teach agents multiple skills over time. We present Environmental Curriculum Learning, a new technique for creating a curriculum of environment versions for an agent to learn in sequence. By adding helpful features to the state and action spaces, and then removing these helpers over the course of training, agents can focus on the fundamentals of a game one at a time. Our experiments in Pommerman illustrate the design principles of ECL, and our experiments in StarCraft II show that ECL produces agents with far better final performance than without it, when using the same training algorithm. Our StarCraft II ECL agent exceeds previous score records in a StarCraft II minigame, including human records, while taking far less training time to do so than previous approaches.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2020

Author dc:creator

• Sun, Ray

Contributors dc:contributor

• Peng, Jian

• reinforcement learning
• curriculum learning
• sample efficiency
• StarCraft II
• Pommerman
• Monte Carlo tree search

Statement dc:rights

• Copyright 2020 Ray Sun

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/108014

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/108014