Empower learning-to-rank with language models

Pre-trained large language models bring revolutionary chances to solving NLP problems. This thesis tackles how to leverage pre-training language models for information retrieval tasks. On the one hand, searching and ranking is the most well-grounded machine learning sceneario. On the other hand, we find the progress in NLU can be transferred to search problems given its foundation in document understanding. This thesis consists of 3 parts, each part investigates how we can build practical applications based on the recent success of neural language models. The first part discusses how we design a multi-lingual query understanding system using tailored pre-training language models for this task. In the second part, we discussed how we build a vision-language multimodality transformer for fine-grained classification and retrieval tasks. In the third part, we propose a novel transformer model to mitigate the distribution shift between training and serving of ranking systems. In the forth part, we present a way to mitigate the confounding effects in two-tower models.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Zhang, Yunan

Contributors dc:contributor

• Zhai, Chengxiang

• Ranking
• Information Retrieval
• Data Mining

Statement dc:rights

• Copyright 2022 Yunan Zhang

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/117572

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F117572