Incremental Learning in Regression Contexts

Incremental or Online Learning is a Machine Learning paradigm in which models are updated with each new incoming data sample. In the real world, these models are usually deployed on Data Streams, that are potentially infinite in size and therefore cannot be tackled by common Batch or Offline Learning approaches. Other reasons for using incremental algorithms are continuously evolving Data Streams, Concept Drift or specific objectives like Personalization. Regression is a procedure, that learns the relationship between independent input variables and dependent, continuous, output variables. In that, it stands in contrast to Classification, where the dependent output variable is discrete. While Classification in the Incremental setting has successfully been applied many times in the literature, real world Data Streams often necessitate the deployment of Regression procedures, which has not been investigated in as much detail so far. Therefore, this thesis is about four core topics from the Incremental Regression domain: 1. We investigate, which Incremental Models are best suited for the problem of Data Stream forecasting under Change Point occurrence. Hereby, we evaluate the algorithms on theoretical benchmark data as well as on real world data for the specific use case of Human Gait Prediction. 2. Based on the results of the previous study, we introduce a new Incremental Learning scheme, that can tackle the problem of Incremental Human Gait Prediction without falling prey to Catastrophic Forgetting. Here, the use case we have in mind is Predictive Exoskeleton Control. Therefore, we evaluate our proposed model on a public, multi modal, Human Gait Database and show, that it delivers competitive results with regard to the state of the art, while, in contrast to the state of the art, being able to deal with new Concepts that arise out of the Data Stream. 3. Since, Incremental Prediction schemes naturally encounter new Concepts in Data Streams that have never been seen before, it can be beneficial to enhance them with a Reject Option, or in other words, with the ability to abstain from prediction when the outcome is likely to be inaccurate. Therefore, we explore how such Reject Options can be facilitated in an Incremental Regression setting, which then leads to a systematic evaluation of four rejection possibilities. 4. In the last topic, we switch the application focus from Human Gait Prediction to the Water Distribution Network Monitoring domain. Here, we demonstrate the transferability of a popular Incremental Learning framework to another domain of high relevance, by enhancing it with Metric Learning, in order to get insights into the Feature Relevance for the specific problem of Sensor Fault Detection. Finally, we also provide Open Source implementations of all models that are introduced throughout the four core topics from above.

Level thesis:degree_level

thesis.doctoral

Grantor dc:publisher

Universität Bielefeld

Year

2024

Author dc:creator

• Jakob, Jonathan

Repository record source_url

https://pub.uni-bielefeld.de/record/2990589

OAI identifier oai:identifier

oai:pub.uni-bielefeld.de:2990589