Tag: #latency-reduction7 results
DeepSeek released the DSpark technical report and code repository, introducing a speculative decoding module for the V4 model. This update significantly accelerates inference speeds without requiring a new base model.
GPUHedge is a new open-source tool that uses speculative execution to hedge across multiple serverless GPU providers. By launching backup requests, it significantly reduces p95 cold start latency from 117s to 30s.
This research proposes moving memory retrieval inside the agent's reasoning loop to eliminate network latency. By using an in-process store, retrieval time drops from milliseconds to microseconds, enabling agents to maintain a persistent, high-speed working memory.
DeepSeek has launched DSpark, a new solution aimed at optimizing large model inference performance. The update addresses latency issues, ensuring AI responses are generated more fluidly.
NVIDIA introduces DFlash speculative decoding to address latency bottlenecks in autoregressive LLMs. This technique utilizes lightweight models to draft tokens, significantly improving throughput on Blackwell architecture.
Google has introduced Gemini 3.5 Flash, a model optimized for high efficiency and speed. The company positions this update as a critical component for enabling responsive agentic AI applications.
DeepMind launches Gemini 3.1 Flash Live, the latest voice model with improved precision and lower latency. It enables more fluid, natural, and precise voice interactions.