Improve and certify ML robustness by integrating exogenous information

Several studies have shown that deep neural networks (DNNs) are vulnerable to adversarial examples, which are constructed by adding a carefully crafted noise with small magnitude onto natural inputs thus misleading DNNs. A lot of work has addressed the importance of improving the robustness of DNNs and various robustness enhancing methods have been proposed. However, most of recent works focused on improving robustness of model itself, without exploring exogenous information that are critical for model to make the correct prediction under adversarial scenarios. This thesis presents novel methodologies to improve and certify ML robustness by integrating exogenous information and leveraging various theoretical and empirical techniques on several sub-topics. To import external domain knowledge into model inference, we propose sensing-reasoning pipeline to integrate statistical learning with logical reasoning to cooperate such domain knowledge rules and develop the end-to-end certification algorithm on top of that. We also delve into analyzing the cause of adversarial transferability theoretically under teacher-student setting, providing the sufficient condition of the adversarial example’s existence in student model due to low dimensionality and neuron specialization. On the general cases, among pair of ML models, we first theoretically derive the sufficient and necessary condition on adversarial transferability - gradient similarity and model smoothness, and practically develop Transferability Reduced Smooth (TRS) regularized training algorithm on top of ensemble ML models to improve their robustness. Additionally, on certifiable robustness domain, for ensemble model, we first formally defined the robustness of ensemble w.r.t different ensemble protocol, and analyze the key factors within smoothed ensemble verifiable radius. Beyond that we propose Diversity Regularized Training (DRT) method to improve certified robustness of ensemble ML models by encouraging gradient diversity and enlarging the confidence margin. For single model certifiable robustness, we also explore the internal factors – neuron implications, to accelerate ML verification by reducing redundant branches and improving intermediate bounds in Branch-and-Bound (BaB) procedure. We believe these research works could provide key insights on exploring ML robustness from various perspective – How to gather and integrate exogenous information within models and corresponding tasks could be an interesting direction for later adversarial ML research.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Yang, Zhuolin

Contributors dc:contributor

• Gunter, Carl
• Li, Bo
• Wang, Gang
• Zhang, Huan
• Xu, Kaidi

• Adversarial Attacks
• Ml Robustness
• Nn Verification

Statement dc:rights

• Copyright 2025 Zhuolin Yang

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/130072

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F130072