Optimization as estimation with Gaussian processes in bandit settings

Optimizing an expensive unknown function is an important problem addressed by Bayesian optimization. Motivated by the challenge of parameter tuning in both machine learning and robotic planning problems, we study the maximization of a black-box function with the assumption that the function is drawn from a Gaussian process (GP) with known priors. We propose an optimization strategy that directly uses a maximum a posteriori (MAP) estimate of the argmax of the function. This strategy offers both practical and theoretical advantages: no tradeoff parameter needs to be selected, and, moreover, we establish close connections to the popular GPUCB and GP-PI strategies. GP-UCB and GP-PI may be viewed as special cases of MAP estimation; while, conversely, MAP criterion can be understood as automatically and adaptively trading off exploration and exploitation in GP-UCB and GP-PI. We illustrate the effects of this adaptive tuning both theoretically and empirically. We establish tighter regret bounds than previous methods, as well as an upper bound on the number of steps necessary to achieve a low regret. In our experiments, we show an extensive empirical evaluation on robotics and vision tasks, demonstrating the robustness of this strategy for a range of performance criteria.

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2016

Author dc:creator

• Wang, Zi, Ph.D. Massachusetts Institute of Technology

Advisor dc:contributor.advisor

• Stefanie Jegelka and Leslie Pack Kaelbling.

• Electrical Engineering and Computer Science.

Statement dc:rights

• M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/103668

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/103668