Natural Language Processing and Recommendation Engine for Stack Overflow Data

Query intent classification is important for information retrieval and problem solving. We use natural language processing and collaborative filtering algorithms to build a recommendation engine for Stack Overflow tag predictions. Our pipeline consists of document retrieval (TF-IDF and HOTT), text embedding (Sentence BERT), and classification (multi-label and multi-class). We experiment with neural networks and other classifier strategies to identify the most relevant Stack Overflow tags. We then use these tags to implement collaborative filtering and recommend solutions based on similar existing posts in the database. The results displayed in this paper use Stack Overflow’s public dataset (https://www.kaggle. com/stackoverflow/stackoverflow).

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2022

Author dc:creator

• Wang, Julia J.

Advisor dc:contributor.advisor

• Oliva, Aude

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright MIT

Licence dc:rights.uri

• http://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/144879

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/144879