On Identifying and Mitigating Against Vulnerabilities of Machine Learning Models

During the last decade, machine learning (ML) has achieved tremendous results in many fields, from traditional learning tasks like image recognition to advanced applications such as brain circuit analysis and healthcare analysis. The availability of large and informa- tive datasets contributes to the success of ML because such datasets can provide diverse training examples for the model. Because datasets can contain individuals’ private in- formation e.g., sensitive images, it is essential that ML models should not leak privacy sensitive information about their training data. In this dissertation, we identify and evaluate the privacy vulnerability of ML models using membership inference (MI) attacks to facilitate the design and training of privacy- preserving models. An MI attack is a fundamental attack that has been widely considered the simplest attack to evaluate the privacy of the training data of ML models because it can identify whether or not a particular sample was used to train an ML model. Firstly, to give a comprehensive introduction to the research background, we categorize and provide taxonomies for existing MI attacks and defenses by conducting a comprehensive review of both attacks and defenses on ML models. Secondly, we propose a new MI defense approach enabling us to train a private model that can mitigate various MI attacks. Thirdly, we identify a new threat to federated learning systems. Based on the observation that existing MI attacks fail to identify the source client of a training sample in the context of federated learning, we propose a new and novel inference attack called source inference attack, which can identify the source client of a training sample in federated learning. Last, in addition to exploring MI from the perspective of an attacker who wishes to breach privacy, we study how it can be used to protect the privacy of a data owner. We propose a new MI approach called membership inference via backdooring. The proposed approach overcomes the drawbacks of existing MI approaches, which require a lot of information. Our newly proposed method enables a data owner to conduct MI in a practical setting where only black-box query access to the model is available.

Name thesis:degree_name

PhD

Level thesis:degree_level

Doctoral

Discipline thesis:degree_discipline

Engineering

Grantor dc:publisher

ResearchSpace@Auckland

Year dc:date.issued

2022

Author dc:creator

• Hu, Hongsheng

Advisors dc:contributor.advisor

• Salcic, Zoran
• Dobbie, Gillian
• Zhang, Xuyun

Statement dc:rights

• Items in ResearchSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Licence dc:rights.uri

• https://researchspace.auckland.ac.nz/docs/uoa-docs/rights.htm

Handle dc:identifier.uri

https://hdl.handle.net/2292/64316

OAI identifier oai:identifier

oai:researchspace.auckland.ac.nz:2292/64316