Self-Modeling Neural Systems

Goal-directedness is a fundamental property of all living things, but it is perhaps most easily identified in the movement patterns of animals. Ethologists have divided the basic forms of animal behavior into three categories: reproductive, defensive, and ingestive, all of which depend on the complex orchestration of motor control. In this dissertation, we use the framework of optimal control theory to model goal-directed behavior and repurpose it in new ways. We demonstrate a method for creating a hierarchical control network in which higher levels of the control hierarchy deal with tasks of increased abstractness. In a two-level system, the lower-level deals with short time-scale, low-dimensional motor control, and the higher-level is charged with longer time-scale, higher-dimensional planning. Central to our approach to joining the levels is the construction of a forward model of the behavior of the lower-level by the higher-level. Thus, we extend ideas of optimal control theory from controlling a "plant" to controlling a controller. We apply our method to the example problem of guiding a semi-truck in reverse around a field of obstacles. The lower-level controller drives the truck, and the higher-level detects obstacles and plans routes around them. In other work, we consider whether it is possible for a neural system that obeys certain biological constraints to solve optimal control problems. We exhibit a simple method to train a different kind of internal model, a neural network model of the Jacobian of the plant, and we integrate the internal model in a forward-in-time computation that produces an optimal feedback controller. We apply our method to two well-known model problems in optimal control, the torque-limited pendulum and cart-pole swing-up problems.

Author dc:creator

• Wayne, Gregory D.

• Neurosciences
• Computer science

Language dc:language

English

DOI dc:identifier

https://doi.org/10.7916/D8FX7HVF

OAI identifier oai:identifier

oai:academiccommons.columbia.edu:10.7916/D8FX7HVF