Enhancing diversity in generative commonsense reasoning for explaining relationships between concepts

The ability to reason based on common sense and knowledge of how things work is crucial for machines to navigate the world. However, large language models (LLMs) often lack explicit representations of relationships between concepts and events, making it challenging to interpret their reasoning processes. To overcome these challenges, in this paper, we propose DimonGen task, which aims to generate diverse sentences describing concept relationships in various everyday scenarios. To support this, we also create a new benchmark dataset for this task by extracting the existing ConceptNet and CommonGen dataset. To address the DimonGen task, we propose two complementary methods: MoREE, a two-stage method that utilizes external knowledge to generate diverse relationship sentences, and DC Decoding, a decoding framework that uses a global energy function to diversify the set of generations. Both methods are evaluated on the benchmark dataset and show significant improvements in the quality and diversity of generated sentences. The results suggest that these methods can generate diverse sentences that reflect relationships between concepts from multiple and varied perspectives. Our code and data for the DimmonGen task are available at https://github.com/liuchenzhengyi/DimonGen.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Liu, Chenzhengyi

Contributors dc:contributor

• Chang, Kevin Chen-Chuan

• Natural Language Processing
• Commonsense Reasoning
• Diverse Text Generation
• Large Language Models
• Mixture Of Experts
• Cold Decoding

Statement dc:rights

• Copyright 2023 Chenzhengyi Liu

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/120265

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F120265