Google 發布 Gemma 4 的 MTP，文字生成速度提升最高 3 倍

增強重點摘要

• •The MTP implementation for Gemma 4 utilizes a lightweight 'drafter' head architecture that is trained concurrently with the main model, specifically designed to predict the next k-tokens simultaneously rather than sequentially.
• •Integration with inference engines like vLLM and TensorRT-LLM is achieved through a standardized API, allowing developers to swap drafter weights without modifying the underlying model architecture.
• •Performance benchmarks indicate that the 3x speedup is most pronounced in high-throughput scenarios where the KV cache memory bandwidth is the primary bottleneck, rather than compute-bound tasks.

競品分析

技術深入

• •Architecture: Employs a shared transformer backbone with a specialized MTP head that outputs multiple tokens in a single forward pass.
• •Training: The MTP head is trained using a multi-token loss function, forcing the model to learn dependencies between consecutive tokens during the pre-training phase.
• •Inference Logic: During speculative decoding, the drafter proposes a sequence of tokens; the main model validates these in parallel, accepting the longest prefix that matches the target distribution.
• •Memory Overhead: The MTP head adds negligible parameter count (typically <5% of total model size), ensuring minimal impact on VRAM footprint.

前景展望基於引用來源的 AI 分析

時間線