National and Kapodistrian University of Athens
On Post-Hoc Dimensionality Reduction of Text Embeddings
Abstract
dc:descriptionText embeddings are a core representation in modern NLP, supporting tasks such as retrieval, clustering, classification, and semantic search. However, embeddings often have hundreds or thousands of dimensions, creating substantial storage and efficiency challenges at scale. In this work, we present a systematic study of post-hoc dimensionality reduction methods across multiple modern embedding backbones, compression ratios, and downstream tasks. As an additional contribution, we introduce GeoPres, a simple and flexible dimensionality reduction method that explicitly preserves spatial geometry in the form of a linear map that we directly train to preserve semantic distances.Our experiments yield both coarse and fine-grained empirical profiles of different post-hoc dimensionality reductions, highlighting their tradeoffs, and shedding light on which tasks are overall more favorable to compressed embeddings.Specifically, we find that our simple method significantly performs strongest, on average, including also in comparisons against more costly and complex methods of the non-post-hoc type. Overall, our study enables us to derive practical recommendations for selecting efficient post-hoc dimensionality reduction techniques in text embedding models.
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- ΚΑΛΥΦΟΜΜΑΤΟΣ ΚΩΝΣΤΑΝΤΙΝΟΣ
- KALYFOMMATOS KONSTANTINOS
Subjects
dc:subject × 2Rights
- Language dc:language
- English
Identifiers
dc:identifier.*- Identifier
- uoadl:5427133