A Bi-Encoder LSTM Model for Learning Unstructured Dialogs

<p>Creating a data-driven model that is trained on a large dataset of unstructured dialogs is a crucial step in developing a Retrieval-based Chatbot systems. This thesis presents a Long Short Term Memory (LSTM) based Recurrent Neural Network architecture that learns unstructured multi-turn dialogs and provides implementation results on the task of selecting the best response from a collection of given responses. Ubuntu Dialog Corpus Version 2 (UDCv2) was used as the corpus for training. Ryan et al. (2015) explored learning models such as TF-IDF (Term Frequency-Inverse Document Frequency), Recurrent Neural Network (RNN) and a Dual Encoder (DE) based on Long Short Term Memory (LSTM) model suitable to learn from the Ubuntu Dialog Corpus Version 1 (UDCv1). We use this same architecture but on UDCv2 as a benchmark and introduce a new LSTM based architecture called the Bi-Encoder LSTM model (BE) that achieves 0.8%, 1.0% and 0.3% higher accuracy for Recall@1, Recall@2 and Recall@5 respectively than the DE model. In contrast to the DE model, the proposed BE model has separate encodings for utterances and responses. The BE model also has a different similarity measure for utterance and response matching than that of the benchmark model. We further explore the BE model by performing various experiments. We also show results on experiments performed by using several similarity functions, model hyper-parameters and word embeddings on the proposed architecture.</p>

Name thesis:degree_name

M.S.

Level thesis:degree_level

Masters Thesis

Year dc:date.available

2018

Author dc:creator

• Shekhar, Diwanshu

Contributors dc:contributor

• Mohammad H. Mahoor, Ph.D.
• Ryan Elmore
• Matthew Rutherford

• Bi-Encoder
• Learning
• LSTM
• Long short term memory
• Machine
• RNN
• Recurrent neural network
• Ubuntu
• Computer Sciences
• Physical Sciences and Mathematics

Statement dc:rights

• <p>Copyright is held by the author. User is responsible for all copyright compliance.</p>

Language dc:language

en

Repository record dc:identifier

https://digitalcommons.du.edu/etd/1508

OAI identifier oai:identifier

oai:digitalcommons.du.edu:etd-2508