Adversarial Learning through Red Teaming: From Data to Behaviour
type: CreativeWork
description: The concept of an adversary applies to all facets of the human life as they continue to dwell in a hostile world. The knowledge of an adversary's behaviour is therefore of paramount importance to us to make sound decisions and succeed in our lives. Red teaming is an ancient technique where an adversary's role is emulated by playing a devil's advocate role to improve own defences and decisions. The approach involves dealing with two competing entities, namely blue and red agents, and has been widely used in military planning to role-play the enemy; test and evaluate its course of actions or judgement; assess the vulnerabilities of the red team; and learn to understand the dynamics between the red and blue entities. However, the red teaming concept is easily mapped to domains that share similar characteristics to military planning, such as adversarial learning, risk assessment, and behavioural decision making in a competitive environment. Computational red teaming is a recent approach that extends red teaming concept in the cyber space and benefits from replacing the physical red and blue with simulated entities. The focus of this thesis is to study the effect of information on adversarial behaviour. A Computational Red Teaming based framework is developed to analyse four forms of an adversary or a red agent operating in a fixed self or blue agent's environment: a static red having direct access to manipulate randomly the information received by the blue agent; a dynamic red having the ability to learn and evolve to counteract blue's actions; a real human playing a red agent's role; and a red approximating human behaviours. To understand the impact of information, a statistical framework for simulating adversarial attacks is proposed to model and explore the effect of red. The heart of the simulation lies in attacks against representative samples of the training data available to blue, and the generation of attack is based on statistical sampling methods. The underlyi…
abstract: The concept of an adversary applies to all facets of the human life as they continue to dwell in a hostile world. The knowledge of an adversary's behaviour is therefore of paramount importance to us to make sound decisions and succeed in our lives. Red teaming is an ancient technique where an adversary's role is emulated by playing a devil's advocate role to improve own defences and decisions. The approach involves dealing with two competing entities, namely blue and red agents, and has been widely used in military planning to role-play the enemy; test and evaluate its course of actions or judgement; assess the vulnerabilities of the red team; and learn to understand the dynamics between the red and blue entities. However, the red teaming concept is easily mapped to domains that share similar characteristics to military planning, such as adversarial learning, risk assessment, and behavioural decision making in a competitive environment. Computational red teaming is a recent approach that extends red teaming concept in the cyber space and benefits from replacing the physical red and blue with simulated entities. The focus of this thesis is to study the effect of information on adversarial behaviour. A Computational Red Teaming based framework is developed to analyse four forms of an adversary or a red agent operating in a fixed self or blue agent's environment: a static red having direct access to manipulate randomly the information received by the blue agent; a dynamic red having the ability to learn and evolve to counteract blue's actions; a real human playing a red agent's role; and a red approximating human behaviours. To understand the impact of information, a statistical framework for simulating adversarial attacks is proposed to model and explore the effect of red. The heart of the simulation lies in attacks against representative samples of the training data available to blue, and the generation of attack is based on statistical sampling methods. The underlyi…
author: Wang, Shir Li
language: EN
keywords: Behaviour, Adversarial learning, Red teaming, Neuroevolution, Deception, Perception, Neural network, Ensemble
license: https://purl.org/coar/access_right/c_abf2
license: https://creativecommons.org/licenses/by-nc-nd/3.0/au/
date published: 2012
date modified: 2026-07-24T05:33:31Z
provider: University of New South Wales
publisher: Networked Digital Library of Theses and Dissertations
type: WebPage
source organization: UNSW, Sydney

PropertyValue
property ID: Handle
value: http://hdl.handle.net/1959.4/52214

PropertyValue
property ID: DOI
value: 10.26190/unsworks/15773