Principled exploration in sequential decision-making

Interactive Machine Learning (IML) possesses the unique capability to harness feedback from interactions, making it indispensable in a wide array of real-world applications. However, a significant challenge, known as the "exploitation and exploration dilemma," prominently arises within the domain of IML. In this context, learners must not only exploit current information but also explore to uncover potential knowledge for long-term gains. Despite decades of research yielding a rich landscape of algorithms, frameworks, and theories for effectively utilizing collected data to train machine learning models, a fundamental question has remained largely unaddressed: How can IML models systematically make principled exploration for long-term benefits, alongside the full exploitation of current data? This thesis will motivate the exploration of IML by human principles in sequential decision-making, and then present our research efforts in developing principled exploration strategies, including adaptive exploration, collaborative exploration, and customized exploration, and the future directions in trustworthy exploration. The content of this thesis will cover the fundamental algorithms and theories in exploration and show how the exploration strategies impact other machine learning problems and real-world applications.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Ban, Yikun

Contributors dc:contributor

• He, Jingrui
• Banerjee, Arindam
• Jiang, Nan
• Xing, Eric P.

• Multi-armed Bandits
• Contextual Bandits
• Neural Networks
• Exploitation And Exploration

Statement dc:rights

• Copyright 2024 Yikun Ban

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/124489

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F124489