Backward Reasoning in LLMs: A Strategy for Identifying Irrelevant Context in Mathematical Word Problems

We investigate backward reasoning in LLMs for identifying irrelevant context in mathematical word problems. Our evaluation across models from 1B to 70B parameters expose a sizeable performance gap between forward and backward reasoning: accuracy on the latter lags by 3-47%, with parameter-constrained models showing the largest drops. To mitigate this gap we propose five structured prompting methods: semantic-role abstraction, timeline reasoning, tree representations, contrastive prompting, and a unified process-supervision prompt. The unified scheme yields the most reliable gains, boosting backward-reasoning accuracy by almost 20%. In addition, simply rephrasing problems in plain language lifts performance by roughly 8 % across the board, underscoring the importance of task formulation. Strikingly, models can identify irrelevant facts with about 75 % accuracy when asked directly, yet fail to exploit this latent knowledge during problem solving, with accuracy plum- meting to 20% once distractors are embedded in the input. Leveraging this insight, we introduce a consistency-based variable-verification framework that activates hidden knowledge through backward reasoning, filters spurious context, and markedly strengthens robustness in realistic settings where misleading details are common.

Name thesis:degree_name

Master of Science

Level thesis:degree_level

masters

Discipline thesis:degree_discipline

Computer Science & Applications

Department dc:contributor.department

Computer Science and#38; Applications

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2025

Author dc:creator

• Kumaran, Aishwarya

Chair dc:contributor.committeechair

• Ramakrishnan, Narendran

Committee members dc:contributor.committeemember

• Wang, Xuan
• Huang, Lifu

• Large Language Model
• Natural Language Processing
• Structured Prompting
• Mathematical Reasoning

Statement dc:rights

• In Copyright

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Language dc:language.iso

en

Dc Identifier Other

vt_gsexam:44237

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/135022