Self-reflective experiential learning for persistent autonomy

Robots have been expected to achieve persistent autonomy for a long time, in which robots are required to safely operate in unknown environments for extended lengths of time while without human interventions. Reinforcement learning holds the promise for persistent autonomy because it can adapt to dynamic and unstructured environments by automatically learning optimal policies from the interactions between robots and environments. However, failures can be unavoidable in the learning process as reinforcement learning can learn the outcome of an action only by executing the action itself. These failures can cause damages to robots or environments in practical applications and hence hinder persistent autonomy. In general, human interventions are usually needed to avoid or resolve the learning failures, but they can be unavailable in practical applications such as space exploration, search and rescue, and underground or underwater construction. Based on a multi-level architecture for persistent autonomy, this dissertation proposes new self-reflective, experiential strategies and methods, aiming at achieving safe adaptions to different environments with minimum human interventions. At the strategic level, while understanding learning is not always necessary and beneficial, this dissertation adds a high-level sophistication of whether and when to learn to reduce failures during learning. At the tactical level, this dissertation proposes a new self-recoverable reinforcement learning algorithm that consists of a multi-state recovery strategy and a failure-prevention strategy. The multi-state recovery strategy improves the learning’s own capability of handling already occurred failures and the failure-prevention strategy learns from failures that are usually ignored and abandoned before to generate a more effective strategy of preventing future similar failures. At the operational level, this dissertation proposes a new multi-objective-optimization-based auto-tuning method to adjust control parameters for robustly achieving the upper-level learning behaviors.

Name thesis:degree_name

Doctor of Philosophy (Ph.D.)

Level thesis:degree_level

Doctoral

Discipline thesis:degree_discipline

Mechanical Engineering

Grantor dc:publisher

Colorado School of Mines. Arthur Lakes Library

Year dc:date.issued

2019

Author dc:creator

• Zhou, Xu

Advisor dc:contributor.advisor

• Zhang, Xiaoli

Committee members dc:contributor.committeemember

• Tang, Gongguo
• Petrella, Anthony J.
• Vincent, Tyrone

• reinforcement learning
• persistent autonomy
• self-reflective learning

Statement dc:rights

• Copyright of the original work is retained by the author.

Language dc:language.iso

eng, English

Identifier

T 8821

OAI identifier oai:identifier

oai:repository.mines.edu:11124/173372