QStore: quantization-aware compressed model storage

Modern applications commonly leverage large, multi-modal foundation models. Such use cases often feature complex workflows that demand the storage and usage of models in multiple precisions. However, to make model development accessible to the average user, model providers opt to maintain separate files for each precision (e.g., INT8, BF16); Hence, naively handling model usage in multi-precision workflows (e.g., downloading and storing each precision separately) can incur prohibitive storage costs. We present QStore, a unified, lossless compression format for simultaneously storing a model in two (high and low) precisions. QStore’s encoding scheme stores a pair of different-precision models with even less storage cost versus storing the high-precision model alone: it compresses the low-precision model, then stores a novel representation of the ’conditional information’ present in the high-precision model but not in the low precision model. Then, for model usage, QStore allows direct access to the low precision model via decoding; if the high precision model is required, QStore recovers it losslessly by applying the stored conditional information onto the decoded low-precision model. We evaluate QStore for compressing multiple precisions of popular foundation models, and show that QStore reduces the overall storage footprint by up to 2.2× (45% of the original size) while enabling up to 1.7× and 1.8× faster model saving and loading versus existing approaches.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Shah, Raunak

Contributors dc:contributor

• Park, Yongjoo

• Compression
• Quantization
• LLMs
• File Formats
• Storage
• Machine Learning
• AI

Statement dc:rights

• Copyright 2025 Raunak Shah

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129293

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129293