Anomalous Inputs in Deep Learning: a Probabilistic Perspective
type: CreativeWork
description: Can neural networks recognize their own limitations? A crucial aspect of robustness is the ability to identify when an input falls outside the scope of one’s knowledge or training --- a task known as out-of-distribution (OOD) detection. For example, a dog breed classifier should ideally recognize a cat image as OOD and refrain from classifying it as a breed of dog. Conversely, can we manipulate neural networks into making confident but incorrect classifications? This task, known as adversarial attack, involves altering an input to be misclassified while preserving its original semantic content. Both of these tasks are concerned with anomalous inputs to a neural network, but they have so far been addressed by two different bodies of literature, using different sets of tools. In this thesis I introduce a probabilistic framework that I call the `three distribution problem', which unifies both tasks. I use this framework to develop new methods for detecting OOD inputs and for creating adversarial attacks. Furthermore, my three-distribution approach gives insight into how `semantics' is operationalized: I demonstrate how we can visualize the semantics implicit in off-the-shelf OOD detection algorithms; and my adversarial attack method allows the attacker to specify explicitly the semantics that are to be preserved by the attack.
abstract: Can neural networks recognize their own limitations? A crucial aspect of robustness is the ability to identify when an input falls outside the scope of one’s knowledge or training --- a task known as out-of-distribution (OOD) detection. For example, a dog breed classifier should ideally recognize a cat image as OOD and refrain from classifying it as a breed of dog. Conversely, can we manipulate neural networks into making confident but incorrect classifications? This task, known as adversarial attack, involves altering an input to be misclassified while preserving its original semantic content. Both of these tasks are concerned with anomalous inputs to a neural network, but they have so far been addressed by two different bodies of literature, using different sets of tools. In this thesis I introduce a probabilistic framework that I call the `three distribution problem', which unifies both tasks. I use this framework to develop new methods for detecting OOD inputs and for creating adversarial attacks. Furthermore, my three-distribution approach gives insight into how `semantics' is operationalized: I demonstrate how we can visualize the semantics implicit in off-the-shelf OOD detection algorithms; and my adversarial attack method allows the attacker to specify explicitly the semantics that are to be preserved by the attack.
author: Zhang, Andi
language: eng
keywords: Probabilistic Generative Models, OOD Detection, Adversarial Examples
license: https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/257007ef-9a44-4be5-b942-c1a7edc602ad/download
license: https://creativecommons.org/licenses/by/4.0/
date published: 2024-07-31
date modified: 2026-07-22T22:24:30Z
provider: Cambridge University
publisher: Networked Digital Library of Theses and Dissertations
type: WebPage
source organization: University of Cambridge

PropertyValue
property ID: DOI
value: 10.17863/CAM.117540