Mathematical analysis of uncertainty in machine learning and deep learning

In this paper, we study uncertainty in machine learning and deep learning from the mathematical point of view. Uncertainty is involved in many real-world situations. The Bayesian modelling can handle such uncertainty in machine learning community. However, the traditional deep learning model fails to show uncertainty for its outputs. Recently, at the intersection of the Bayesian modelling and deep learning, a new framework called the Bayesian deep learning (BDL) has been proposed and studied, which enables us to estimate uncertainty of deep learning models. As an example of it, we can review the results of Yarin Gal, in which the famous dropout method can be seen as a Bayesian modelling. We also see that overfitting problem of the framework due to the property of the KL divergence, and review the modified algorithm using o-divergence which generalizes the KL divergence. We also study a confidence band to assess uncertainty of a kernel ridge regression estimator. We propose the formulation to obtain a confidence band as the convex optimization, which enables us to use existing algorithms such as the primal-dual inner point method. The proposed method acquires a more accurate and fast confidence band than a bootstrap algorithm. We also see the effectiveness of our proposed method both in the case of function approximation and an estimate of an actual dataset.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Engineering Systems Division

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2020

Author dc:creator

• Kashimura, Takuya.

Advisor dc:contributor.advisor

• Bart P.G. Van Parys.

• Engineering Systems Division.
• System Design and Management Program.

Statement dc:rights

• MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/145230

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/145230