🕷️
SetupAI
🦙較早收集於 5h

測試 Qwen 3.6 35B 多標記預測能力

PostLinkedIn
🦙閱讀原文： Reddit r/LocalLLaMA
#local-llm#mtp#coding-agentqwen-3.6qwenllama.cpp

💡MTP 模型的實測顯示，其在本地 LLM 程式設計任務中可提升 1.5 倍速度。

⚡ 30-Second TL;DR

有什麼變化

MTP 模型可提供約 1.5 倍的標記生成速度提升

為什麼重要

MTP 技術已被證明是本地 LLM 可用性的遊戲規則改變者，讓使用者能在家用硬體上進行更快速的程式開發與複雜專案管理。

下一步行動

如果您在本地程式設計代理工作流中遇到延遲瓶頸，請嘗試使用支援 MTP 的 llama.cpp 建置版本。

誰應關注：Developers & AI Engineers

關鍵要點

  • •MTP 模型可提供約 1.5 倍的標記生成速度提升
  • •在 32GB VRAM 上成功測試 300k 上下文視窗
  • •需要特定的 llama.cpp 伺服器原型以支援 MTP
  • •MoE 模型在極端上下文深度下可能面臨穩定性問題

🧠 深度解析

Web-grounded analysis with 20 cited sources.

🔑 增強重點摘要

  • •Qwen 3.6 35B A3B is a sparse Mixture-of-Experts (MoE) model, featuring 35 billion total parameters but activating only 3 billion parameters per token during inference, which contributes to its efficiency for local deployment.
  • •The model is natively multimodal, capable of processing text, image, and video inputs, and incorporates a 'thinking preservation' feature that allows it to retain reasoning context across conversational turns, enhancing its utility for complex agentic tasks.
  • •Qwen 3.6 35B A3B boasts a native context window of 262,144 tokens, which can be extended to over 1 million tokens using YaRN scaling, enabling it to handle repository-scale reasoning and large documents.
  • •Multi-Token Prediction (MTP) in Qwen 3.6 is implemented as a built-in speculative decoding mechanism, utilizing multiple output heads within a single transformer to predict several future tokens simultaneously, thereby eliminating the need for a separate draft model and simplifying local inference setups.
  • •On agentic coding benchmarks, Qwen 3.6 35B A3B demonstrates superior performance, scoring 73.4% on SWE-bench Verified, significantly outperforming competitors like Google's Gemma 4 26B A4B which scored 52.0% on the same benchmark.
📊 競品分析▸ Show
Feature/BenchmarkQwen 3.6 35B A3BGemma 4 26B A4BQwen 3.6 27B (Dense)
ArchitectureSparse MoE (35B total, 3B active parameters)Sparse MoE (26B total, 4B active parameters)Dense (27B total parameters)
Multi-token PredictionBuilt-in MTPSupports MTP (via llama.cpp beta)Supports MTP (via llama.cpp beta)
Context Window262K native, 1M with YaRNNot explicitly stated, but competitive with long context models262K native, 1M with YaRN
ModalityNatively multimodal (text, image, video)Multimodal reasoningMultimodal
Key FocusAgentic coding, repository-level reasoning, tool useGeneral-purpose multimodal reasoning, multilingualAgentic coding, vision, chat tasks
SWE-bench Verified73.4%52.0%75.0% (Qwen3.5-27B, indicative)
Terminal-Bench 2.051.5%34.2% (Gemma4-26BA4B)41.6% (Qwen3.5-27B, indicative)
Token Generation Speed~120+ tokens/sec (RTX 4090, llama.cpp/vLLM), 3.5-4x faster than 27B~300 tok/s (M2 Ultra, Q8), 40% faster with MTP (M5Max)Slower than 35B-A3B (3.5-4x slower)
VRAM (Q4_K_M)~21 GB (fits 24GB GPU)Not explicitly stated for 26B A4B, but 31B BF16 is ~20GB~18 GB (fits 18GB RAM setups)
LicenseApache 2.0Open-source (Apache 2.0 for some variants)Apache 2.0
API Pricing$0.15/1M input, $1/1M output (OpenRouter)Not explicitly stated for 26B A4B, but generally competitiveNot explicitly stated

🛠️ 技術深入

  • Model Architecture: Qwen 3.6 35B A3B employs a hybrid sparse Mixture-of-Experts (MoE) architecture. It features 35 billion total parameters, but only approximately 3 billion are actively engaged per token during inference, significantly enhancing computational efficiency.
  • Attention Mechanism: The architecture integrates Gated DeltaNet linear attention layers with standard Gated Attention layers.
  • Multi-Token Prediction (MTP) Implementation: MTP is built directly into the model, utilizing a shared transformer backbone with multiple output heads. Each head is trained to predict a different future token (e.g., token 2, token 3, token 4) simultaneously, allowing for the drafting of multiple tokens in a single forward pass, which are then verified. This internal mechanism removes the need for a separate draft model.
  • Context Window: The model natively supports a context length of 262,144 tokens and can be extended to 1,010,000 tokens through YaRN (Yet another RoPE extension) scaling.
  • Thinking Preservation: A notable feature is 'thinking preservation,' which enables the model to retain its reasoning traces from previous conversational turns. This is particularly beneficial for multi-step agentic workflows, where maintaining coherent context is crucial.
  • Expert Configuration: The MoE layer consists of 256 experts, with 8 routed experts and 1 shared expert actively selected per token.
  • Tokenizer and Layers: The model uses a tokenizer with a vocabulary size of 248,320 and is composed of 40 layers.

🔮 前景展望AI analysis grounded in cited sources

Local AI development will experience accelerated innovation and adoption.
The ability of models like Qwen 3.6 35B MTP to deliver frontier-level performance and large context windows efficiently on consumer hardware significantly lowers the barrier to entry for developers and researchers, fostering a more dynamic local AI ecosystem.
Agentic AI applications will become more sophisticated and reliable on edge devices.
Features such as 'thinking preservation' and extended context windows directly address critical challenges in maintaining state and coherence for complex, multi-step agentic workflows, enabling more robust local AI agents.
The competitive landscape between open-source and proprietary cloud-based LLMs will intensify.
With open-source models achieving comparable or superior performance to proprietary cloud offerings in specific domains (e.g., agentic coding) while offering local deployment advantages, the market will likely see increased pressure on cloud providers to innovate further or adjust pricing.

⏳ 時間線

2023-04
Alibaba launched Qwen (Tongyi Qianwen) in beta.
2023-09
Qwen opened for public use after regulatory clearance.
2024-06
Qwen2 model series released.
2025-04
Qwen3 model family released, including dense and MoE variants, trained on 36 trillion tokens.
2026-02
Qwen3.5 series released, featuring native multimodal capabilities and Multi-Token Prediction (MTP).
2026-03-31
Qwen 3.6 Plus, a proprietary cloud model with a 1 million token context window, was released.
2026-04-16
Qwen 3.6 35B A3B, an open-source, open-weight MoE model with MTP, 262K native context, and agentic coding focus, was released.
2026-05-04
llama.cpp merged beta support for Multi-Token Prediction (MTP), initially for Qwen3.5 MTP compatible models.
🦙閱讀原文： Reddit r/LocalLLaMA
📰

AI 週報

閱讀本週精選 AI 大事摘要 →

👉相關動態

同主題

探索 #local-llm

同產品

更多關於 qwen-3.6

同來源

最新來自 Reddit r/LocalLLaMA

AI 策展新聞聚合。所有內容版權歸原始發布者所有。
原始來源： Reddit r/LocalLLaMA ↗