DeepSeek's DualPath Boosts Agent LLM Inference 1.9x

🔑 Enhanced Key Takeaways

• •DualPath was tested on DeepSeek-V3 (660B scale) and Qwen models in both offline rollout and online service scenarios, significantly reducing Time To First Token (TTFT) under high load while keeping Token-to-Token (TBT) stable[1][2][3].
• •The system uses 80GB DRAM per node for DeepSeek models and implements on an in-house framework with ~5K lines of code modifications, incorporating FlashMLA, DeepGEMM, DeepEP kernels, 3FS storage, and io_uring-like kernel bypass[3].
• •Paper submitted to arXiv on February 25, 2026, by authors from DeepSeek, Tsinghua, and Peking University, targeting disaggregated architectures with production agentic workloads featuring high KV-Cache reuse[2].

🛠️ Technical Deep Dive

• •Implements dual-path KV-Cache loading: traditional storage-to-prefill plus novel storage-to-decode path with RDMA transfer over compute network to avoid congestion and interference[2][3].
• •Global scheduler dynamically balances load across prefill and decode engines; uses adaptive scheduling and strict traffic isolation[1][2].
• •Built on in-house inference stack with FlashMLA (Li and Liu, 2025), DeepGEMM (DeepSeek-AI, 2025b), DeepEP (Zhao et al., 2025b) CUDA kernels; ~5K LOC modifications[3].
• •Storage via 3FS (DeepSeek-AI, 2025a) with io_uring-like interface for kernel bypass; evaluated on three models with long contexts[2][3].
• •For DeepSeek models: 80GB DRAM/node; Qwen 32B uses 1.5TB DRAM/node total in comparisons[3].

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline