Exploring a Layer-Wise Design Space for KV Cache Eviction

This paper explores the idea of varying KV cache eviction methods across different layers of a Transformer model, showing that heterogeneous routing can improve performance compared to homogeneous policies. The results suggest that layer-wise method composition is a design dimension for KV cache compression that can be exploited.

RSS Score 0 9/18/2026, 4:00:00 AM Original Source
Save an API key to vote.