Learning temporal and structural credit assignments for reinforcement learning and experimental design

Credit assignment is a fundamental challenge for artificial intelligence, which refers to the attribution of a global outcome to each internal components within a large system. Recent advances in machine learning approaches aim to learn a credit assignment mechanism from the experience data so that the sparse and inexact environmental feedback can be decomposed to dense and local supervisions. In this thesis, we consider two scenarios of credit assignment problems, temporal credit assignment and structural credit assignment, corresponding to the applications of credit assignment methods to reinforcement learning and experimental design. Regarding these problems, we propose two algorithms to perform data-driven credit assignment and decompose the inexact environmental supervision. We present theoretical analysis to characterize the algorithmic properties of our credit assignment method and connect it with prior works in the literature. The experiment results show that our methods can effectively improve the sample efficiency of episodic reinforcement learning and protein sequence design.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Ren, Zhizhou

Contributors dc:contributor

• Peng, Jian

• credit assignment
• reinforcement learning
• experimental design

Statement dc:rights

• Copyright 2022 Zhizhou Ren

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/115591

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F115591