Minima-KV Compresses Long-Context KV Caches
💡See how mixed FP8/TQ3 KV pages deliver 3.5x BF16 compression without evicting live-request state.
⚡ 30-Second TL;DR
What Changed
Mixed-format paged attention keeps every live-request page addressable, avoiding eviction of active KV state.
Why It Matters
If validated across more models and workloads, Minima-KV could substantially increase long-context serving capacity on memory-constrained GPUs while preserving live-request state. Its mixed-format execution may also reduce the need for aggressive context eviction or large dense cache replicas.
What To Do Next
Prototype Minima-KV on your longest Qwen3.6-27B serving workloads and compare memory per token, LongBench-style quality, and decode throughput against BF16 and FP8 baselines.
Key Points
- •Mixed-format paged attention keeps every live-request page addressable, avoiding eviction of active KV state.
- •Deployment accounting reports 18.3 KiB of attention KV per live token: 3.50x compression versus BF16 and 1.75x versus FP8.
- •The quality profile matches dense control on 16K RULER needle-in-a-haystack tasks, with LongBench v2 deltas of -0.80, -0.60, and -0.40 points at 16K, 32K, and 64K.
- •A two-request 59,008-token direct-decode canary delivers 3.625x active-KV compression and 0.9821x control throughput without a dense shadow.
🧠 Deep Insight
Background and context from public sources — not the original article. 9 sources cited.
🔑 Enhanced Key Takeaways
- •Minima-KV utilizes a tiered page categorization system—Recent (R), Anchor (A), and Stale (S)—to ensure all tokens remain addressable without requiring eviction.
- •The system employs custom CUDA kernels that perform attention computations directly on heterogeneous formats, eliminating the need for a memory-intensive dense shadow cache.
- •Mathematical equivalence is maintained during attention computation via an online-softmax recurrence that merges partial outputs from different tiers without full dequantization.
- •Performance testing was conducted on the Qwen3.6-27B model architecture specifically using NVIDIA RTX PRO 6000 Blackwell hardware.
- •Minima-KV is positioned as a direct alternative to layer-discriminative 2-bit quantization methods like MiniKV, prioritizing retention-preservation over aggressive sparsity.
📊 Competitor Analysis▸ Show
| Feature | Minima-KV | MiniKV | Sparse Attention |
|---|---|---|---|
| Compression Strategy | Tiered (FP8/TQ3) | Layer-discriminative 2-bit | Token pruning/sparsity |
| Addressability | Full (Retention-preserving) | Partial (Eviction-based) | Partial (Eviction-based) |
| Implementation | Custom CUDA Kernels | Standard Quantization | Sparse Kernels |
| Throughput | ~0.98x Control | Variable | High (but lossy) |
🛠️ Technical Deep Dive
- Tiered Storage Hierarchy: Recent and Anchor pages are stored in FP8, while Stale pages are packed into TQ3 (3-bit) format.
- Online-Softmax Recurrence: The attention mechanism merges partial outputs from different tiers dynamically, avoiding the overhead of dequantizing stale pages into a dense buffer.
- Hardware Optimization: Designed specifically for Blackwell-architecture GPUs (RTX PRO 6000) to leverage high-bandwidth memory efficiency.
- PagedAttention Integration: Extends the standard PagedAttention paradigm to support mixed-precision storage within the same memory block structure.
🔮 Future ImplicationsAI analysis grounded in cited sources
⏳ Timeline
📎 Sources (9)
Factual claims are grounded in the sources below. Forward-looking analysis is AI-generated interpretation.
Weekly AI Recap
Read this week's curated digest of top AI events →
👉Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: ArXiv AI ↗
This is a summary, not the original. Read the source, or get the weekly briefing.
Weekly AI briefing
One email a week. Unsubscribe anytime.