Toward building more accessible large language models: A preliminary empirical study on data scarcity in knowledge distillation and algorithm complexity in alignment

Developing large language models (LLMs) boosts various downstream tasks such as question answering. However, for various reasons, most people have to use commercial application programming interfaces (APIs) instead of training LLMs themselves. The limited accessibility of LLMs calls for efforts in democratization. This thesis mainly explores two critical technical bottlenecks that limit LLMs' access to a broader community: data scarcity and algorithm complexity. Data scarcity makes it difficult for individuals to train their own models or distill knowledge from LLMs. Therefore, we explore suitable ways to augment text data to distill knowledge from large language models better. Besides, the complex alignment algorithm (i.e., reinforcement learning from human feedback, RLHF for short) requires lots of engineering effort, which hinders individuals from training their own models. Although there are simple substitutional algorithms, they have different drawbacks. This thesis proposes to improve controllable generation, a simple substitutional algorithm of RLHF, to achieve better alignment performance. The results of this thesis can help the community toward a more democratized LLM research environment.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Wang, Ziqi

Contributors dc:contributor

• Ji, Heng

• Large Language Models
• Distillation
• Data Augmentation
• Alignment
• Controllable Generation

Statement dc:rights

• Copyright 2023 Ziqi Wang

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/122036

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F122036