Evaluation and design of AI red-teaming agent for cybersecurity of web applications

Large language models (LLMs) and LLM-based agents have become increasingly sophisticated, especially in the realm of cybersecurity. Recent studies show that LLM agents are increasingly capable of autonomously conducting cyberattacks, particularly against web applications, which are among the most common targets. This emerging risk highlights the urgent need for a real-world red-teaming framework to systematically assess the capabilities and threats of LLMs in exploiting web application vulnerabilities. However, existing work falls short for two key reasons: (1) there is no real-world benchmark for evaluating LLMs on exploiting web application vulnerabilities, and (2) there is no agentic scaffolding that fully unleashes the potential of LLMs in exploiting web application vulnerabilities, especially under zero-day settings. To close this gap, we first introduce CVE-Bench, a real-world web application cybersecurity benchmark based on critical-severity Common Vulnerabilities and Exposures. In CVE-Bench, we design a sandbox framework that enables LLM agents to exploit vulnerable web applications in scenarios that mimic real-world conditions, while also providing effective evaluation of their exploits. Second, we build HPTSA, an agentic framework that coordinates teams of LLM agents to exploit real-world, zero-day vulnerabilities. While prior agents struggle with exploring many different vulnerabilities and long-range planning when used alone, HPTSA uses a planning agent that can launch subagents. The planning agent explores the system and determines which subagents to call, resolving long-term planning issues when trying different vulnerabilities. Empirically, our multi-agent system improves over prior agent frameworks by up to 4.3×. Using CVE-Bench and HPTSA, we evaluated the cybersecurity capability of frontier LLMs. We show that our state-of-the-art agent framework, HPTSA, can exploit up to 13% of the real-world web application vulnerabilities at an average cost of $1.7 per exploit. These findings highlight the realistic dual-use nature of AI agents: they can both support security testing and maintenance and facilitate web application attacks. We hope that this work encourages frontier LLM providers and stakeholders to carefully consider these dual-use implications when designing, deploying, and governing LLM services.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Zhu, Yuxuan

Contributors dc:contributor

• Kang, Daniel

• artificial intelligence
• benchmark
• agent
• cybersecurity

Statement dc:rights

• Copyright 2025 Yuxuan Zhu

Language dc:language

en

Handle dc:identifier

https://hdl.handle.net/2142/132586

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/132586