A hardware acceleration technique for gradient descent and conjugate gradient

Gradient descent, conjugate gradient, and other iterative algorithms are a powerful class of algorithms; however, they can take a long time for conver- gence. Baseline accelerator designs feature insu cient coverage of operations and do not work well on the problems we target. In this thesis we present a novel hardware architecture for accelerating gradient descent and other similar algorithms. To support this architecture, we also present a sparse matrix-vector storage format, and software support for utilizing the format, so that it can be e ciently mapped onto hardware which is also well suited for dense operations. We show that the accelerator design outperforms similar designs which target only the most dominant operation of a given algorithm, providing substantial energy and performance bene ts. We further show that the accelerator can be reasonably implemented on a general purpose CPU with small area overhead.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2011

Author dc:creator

• Kesler, David R.

Contributors dc:contributor

• Kumar, Rakesh

• Gradient Descent
• Conjugate Gradient
• Hardware Acceleration
• Matrix Multiplication

Statement dc:rights

• Copyright 2011 David R. Kesler

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/24241

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/24241