Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 7 of 7 for “"KV Cache"”.
-
Pushing the limits of long context LLM inference via KV cache compression
Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-19 without embargo terms
-
Ghostdecoding: leveraging random-feature kernels for error-aware and training-free KV cache selection
Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2027-05-01
-
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
Key-value (KV) caching plays an essential role in accelerating decoding for transformer-based autoregressive large language models (LLMs). However, the amount of memory required to store the KV cache can become prohibitive at long sequence lengths and large batch sizes. Since the invention of the …
-
SKYAPI: structural-aware orchestration for LLM-based multi-agent systems
… Time-To-First-Token (TTFT) deferral to maximize KV-cache reuse among collaborative agents. Extensive evaluations on complex benchmarks, including DeepResearch and Gama-Bench, demonstrate that SkyAPI significantly outperforms state-of-the-art baselines, reducing operational costs by up to 3× while …
-
Hosting LLMs on Shared GPUs
… hierarchical paging, a novel key-value (KV) cache management strategy, and we implement it within the vLLM serving engine. Hierarchical paging organizes GPU memory into a two-level hierarchy: large contiguous memory blocks allocated to individual models, which are then subdivided into …
-
On the Resource Efficiency of Language Models
… distributions. LoRC progressively compresses KV cache, substantially reducing GPU memory requirements while preserving performance. These complementary thrusts form a framework for efficient, deployable language models. The findings demonstrate that targeted resource optimization enables …
-
Co-Designing Efficient Systems and Algorithms for Sparse and Quantized Deep Learning Computing
… further improves performance with activation and KV cache quantization, enhancing the throughput of NVIDIA TensorRT-LLM by 1.2-2.4× on A100 GPUs. Finally, we introduce HART, an efficient autoregressive image generation method that achieves 4.5-7.7× higher throughput compared to diffusion models …