Efficient learning machines

Science is in a constant state of evolution. There is a permanent quest for advancing knowledge in the light of changing capabilities and matters. The field of Machine Learning itself is shaped by the ever-increasing amount of data and computing power, creating new challenges as well as paving the way for new opportunities. This thesis is on adapting learning-based machines to these emerging prospects. In particular, subject of this work are three distinct research topics with the underlying drivers: the wish to reliably predict (a) given a large number of classes, (b) given a large number of samples, and (c) to understand complex algorithms and data models. Each reflects a unique need for an efficient proposition and we contribute by creating approaches located in the intersection of algorithm and software development in order to tackle the following problem statements. The first contribution researches multi-class classification with large label spaces and the effective prediction method support vector machines. Recent work suggests so-called all-in-one support vector machine formulations outperform one-vs.-rest formulations, but it is a challenge to leverage their potential for settings with a large number of classes. We approach this problem by proposing for two all-in-one machines exact optimization algorithms that distribute computation and model parameters evenly on computing instances. This allows us to perform an analysis on text data with a large label spaces and to confirm the favorable performance of all-in-one formulations. Other cornerstones of Machine Learning are kernel methods and neural networks. The ever-growing data collections expose a scaling issue of kernel methods with respect to a large number of data points. The predominant approach to alleviate this is an approximation based on random features. In our second contribution we argue that this randomness renders the method inefficient and dissect the effect of these data- and task-agnostic learning bases by means of an empirical study. Viewing approximated kernel machines as neural networks and a novel, efficient optimization approach enable us to shed light onto the interplay of these two important learning paradigms. Our last contribution aims to facilitate a better understanding of the predictions of deep neural networks. These data models have shown impressive results in a wide range of applications and are an invaluable tool. Yet, compared to many other Machine Learning techniques, their functioning is hard to understand and to retrace. Among many proposed methods, propagation-based prediction analysis has shown convincing results and is a promising candidate to address this shortcoming. A drawback is the lack of efficient software for many methods and emerging network structures. We contribute to this by developing the software library iNNvestigate, whose features are an intuitive interface and a modular design — enabling non-expert users access to these methods as well as accelerating research on complex neural networks.

Author dc:creator

• Alber, Maximilian

Advisor dc:contributor.advisor

• Müller, Klaus-Robert

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Language dc:language.iso

en

Identifier URI

http://dx.doi.org/10.14279/depositonce-8528

OAI identifier oai:identifier

oai:depositonce.tu-berlin.de:11303/9472