Can an Agentic AI System Increase Willingness to Engage with Support Services in Wales?

Accessing support in Wales is often hampered by a lengthy, confusing application process. Many are unsure of their eligibility or are nervous about how to start. This study aims to develop an agentic AI system capable of eligibility assessment, population of application form fields, and complete annotated PDF form generation. The system presents an easy-to-use conversational interface that employs a non-linear workflow for information gathering and follow-up questions. The system uses Lang Graph’s state graph framework, paired with custom tools and structured input and output via Pydantic models. A novel approach to agentic PDF manipulation is proposed, leveraging deterministic tooling and the large language model’s core strength in information extraction. System testing demonstrated high performance with fields consistently populated successfully and documents annotated precisely. Usability testing through a pre-post intervention study demonstrated a large effect size (Cohen’s d = 1.02), suggesting that the system meaningfully increased participants’ willingness to engage. Unfortunately, while the system was effective in eligibility assessment and form generation, critical challenges were identified, such as hallucinations, user interface blindness, and systemic risks. The findings suggest that while the system shows promise in improving accessibility, there are fundamental risks to address before safe and trustworthy deployment is feasible.

Name dc:type.qualificationname

msc

Level dc:type.qualificationlevel

masters

Grantor dc:publisher.institution

University of Wales Trinity Saint David

Year dc:date.issued

2026

Author dc:creator

• Watkins, Bradley

• QA76 Meddalwedd Cyfrifiadurol

Dc Identifier Grantnumber

UWTSD

OAI identifier oai:identifier

oai:repository.uwtsd.ac.uk:4222