ScPEFT : a parameter-efficient fine-tuning framework for enhancing single-cell large language models in out-of-context

Single-cell large language models (scLLMs) capture essential biological insights from vast single-cell atlases but struggle in out-of-context applications, where zero-shot predictions can be unreliable. To address this, we introduce a single-cell parameter-efficient fine-tuning (scPEFT) framework that integrates learnable, low-dimensional adapters into scLLMs. By freezing the backbone model and updating only the adapter parameters, scPEFT efficiently adapts to specific tasks using limited custom data. This approach mitigates catastrophic forgetting, reduces parameter tuning by over 96%, and decreases GPU memory usage by more than half, significantly enhancing scLLMs's accessibility for resourceconstrained researchers. Validated across diverse datasets, scPEFT outperformed zero-shot models and traditional finetuning in disease-specific, cross-species, and under-characterized cell population tasks. Its attention-mechanism analysis identified COVID-related genes associated with specific cell states and uncovered unique blood cell subpopulations, demonstrating scPEFT's capacity for conditionspecific interpretations. These findings position scPEFT as an efficient solution for improving scLLMs' utilities in general single-cell analyses.

Name thesis:degree_name

Ph. D.

Level thesis:degree_level

Doctoral

Discipline thesis:degree_discipline

Computer Science

Grantor dc:publisher

University of Missouri--Columbia

Year dc:date.issued

2025

Author dc:creator

• He, Fei

Advisor dc:contributor.advisor

• Xu, Dong

Language dc:language.iso

eng, English

DOI

https://doi.org/10.32469/10355/111003

OAI identifier oai:identifier

oai:mospace.umsystem.edu:10355/111003