Optimized Calibration for Analog Computations Targeting Deep Neural Networks on the Example of BrainScaleS-2

Machine learning is pervasive today, but as more complex models are developed, their application is becoming increasingly costly. This work explores analog computing as a scalable and energy-efficient alternative to the typically used digital computations. Leveraging the BrainScaleS-2 system as an analog matrix multiplication accelerator, we optimize for typical imperfections like noise and saturation effects that measurably degrade accuracy. While training with hardware in the loop can partially recover lost accuracy, the gap can usually not be fully bridged, deterring potential users. Our research aims to narrow this gap through calibration parameter adjustments and algorithmic optimizations. We examine how the translation of operands to the hardware affects training and analyze the effects of calibration parameters to develop an overall better solution that is tuned for neural networks to ultimately enhance their accuracy. As the primary contribution, we find that higher circuit time constants, which lead to a higher average load on the analog amplifiers, allow for decreased amplifier gain at similar output amplitudes while improving noise behavior. By using this custom calibration together with statically scaled input operands, we achieve approximately 41 % accuracy improvement before retraining and 7 % afterward. Notably, we identify that the varying matrix sizes due to the varying number of neurons between layers require gain adjustments, which can potentially further increase accuracy. Our work addresses common challenges in the efficient execution of artificial neural networks on BrainScaleS-2, providing a path toward competitive usage. These findings have significant implications for cost-effective and energy-efficient machine learning model applications.

Level thesis:degree_level

master

Grantor dc:publisher

Universität Heidelberg

Year

2023

Author dc:creator

• Kern, Eric Matthias

Contributors dc:contributor

• Fröning, Holger

Repository record source_url

http://www.ub.uni-heidelberg.de/archiv/38592

OAI identifier oai:identifier

oai:archiv.ub.uni-heidelberg.de:38592