A Study on Leveraging Generative Artificial Intelligence and Text Clustering to Support Vendors

This research is an initiative to discover how generative artificial intelligence (AI) tools can improve Amazon Last Mile’s feedback systems to enhance the Delivery Service Partner experience. Our specific focus is on the effectiveness of clustering algorithms like DBSCAN and K-Means for grouping text feedback based on semantic similarity and on the employment of retrieval augmented generation (RAG) for extracting actionable insights. Our findings indicate a relative effectiveness of K-Means over DBSCAN in clustering feedback, but the overall effectiveness is moderate, which necessitates the need for human verification to counter potential model hallucinations. Additionally the use of RAG with Claude 2.1 demonstrated promise in answering domain-specific questions in spite of limitations related to text-only input. We propose future emphasis on the integration of AI in current listening mechanisms to offer concise, actionable recommendations for program leaders. This research also recommends continued exploration in embedding models and RAG framework to enhance feedback quality and information retrieval. The potential to integrate generative AI tools within Amazon Last Mile represents an underexplored opportunity for significant enhancements in efficiency, accuracy, and overall partnership satisfaction.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2024

Author dc:creator

• Hubbard, Steven

Advisors dc:contributor.advisor

• Boning, Duane S.
• Farias, Vivek

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright retained by author(s)

Licence dc:rights.uri

• https://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/155619

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/155619