Adaptive Reinforcement Learning-Based Fuzzer for 5G RRC Security Evaluation

This thesis presents the development of an adaptive fuzzing framework leveraging the UER- ANSIM simulator to assess the security and resilience of 5G networks. The proposed approach specifically targets potential vulnerabilities in message exchange sequences, authentication procedures, and resource allocation mechanisms. It employs Q-learning to optimize fuzzing strategies across multiple protocol stack layers. The framework evaluates the impact of malicious or malformed inputs on a gNB's stability and performance by emulating multiple user devices and systematically altering key parameters in control-plane messages. A major focus of the study is the simulation of large-scale Distributed Denial-of-Service (DDoS) attacks, wherein numerous simulated UEs generate high volumes of fuzzed signaling traffic to stress the system and observe degradation. The reinforcement learning agent dynamically adjusts its attack patterns based on network feedback, tuning the injection of messages to maximize resource exhaustion and identify critical failure points. The effectiveness of the fuzzing campaign is measured through resource utilization metrics, such as CPU load and Thread count collected during experimentation. By combining adversarial testing with intelligent fuzzing techniques, the research provides important insights into the security posture of 5G infrastructure under adversarial conditions. The findings underscore the need for stronger defenses against protocol-level attacks to bolster the resilience of next-generation wireless communication systems.

Name thesis:degree_name

Master of Science

Level thesis:degree_level

masters

Discipline thesis:degree_discipline

Computer Engineering

Department dc:contributor.department

Electrical and Computer Engineering

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2025

Author dc:creator

• Parikh, Dhairya Pranav

Chair dc:contributor.committeechair

• Burger, Eric William

Committee members dc:contributor.committeemember

• Tripathi, Nishithkumar Dhananjay
• Reed, Jeffrey H.

• Fuzzing
• UERANSIM
• RRC
• Reinforcement Learning
• UE
• gNB
• 5G Security
• Q-learning

Statement dc:rights

• In Copyright

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Language dc:language.iso

en

Dc Identifier Other

vt_gsexam:43517

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/134224