HiBi: A hierarchical bigram model for associative learning

There has been a shift of attention in the AI research where people gradually abandon traditional statistical models in favor of deep neural architectures. While effective in learning input-output mappings from two arbitrary distributions, the complex nature of neural models makes them hard to interpret. In this thesis, we introduce a more interpretable hierarchical bigram (HiBi) model, which is extended based on the simple bigram language model. It contains a few components inspired by theories of human cognition, and has been shown through experiments to be effective in learning meaningful representation from sequential inputs without any labeling. We hope that HiBi could be a starting point to develop more complex cognitive models that are both interpretable and effective for representation learning.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2020

Author dc:creator

• Wang, Xiaoyan

Contributors dc:contributor

• Zhai, Chengxiang

• Natural language processing
• Cognitive models

Statement dc:rights

• Copyright 2020 Xiaoyan Wang

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/108022

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/108022