🦙Reddit r/LocalLLaMA•較早收集於 13h
修正 Qwen 35B GGUF 解決張量漂移
#quantization-fix#ssm-layers#local-inferenceqwen3.6-35b-a3b-uncensored-wasserstein-ggufqwen-3.6-35bggufwassersteinunsloth
💡新修正讓 35B Qwen 在本地長上下文穩定使用—擊敗 SSM 層量化錯誤（45字元）
⚡ 30-Second TL;DR
有什麼變化
透過 Wasserstein W1 度量修正區塊 36-38 的 ssm_conv1d.weight 張量漂移
為什麼重要
透過解決量化錯誤，讓大型 Qwen 模型在消費級硬體上可靠本地推理，有助開源 LLM 部署。
下一步行動
從 Hugging Face 下載 Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF，並在 LM Studio 以 temp=0.7 測試。
誰應關注：Developers & AI Engineers
關鍵要點
- •透過 Wasserstein W1 度量修正區塊 36-38 的 ssm_conv1d.weight 張量漂移
- •修正後 W1 誤差從 ~0.5-0.7 降至 <0.001
- •基於 HauhauCS 激進量化，完全無審查，推薦 Q4_K_P
- •使用自訂系統提示測試長上下文角色扮演
🧠 深度解析
AI-generated analysis for this event.
🔑 增強重點摘要
- •The Wasserstein-1 (Earth Mover's Distance) metric is being increasingly adopted in quantization research to address non-linear weight distribution shifts that traditional KL divergence fails to capture in State Space Model (SSM) architectures.
- •The 'tensor drift' issue specifically in Qwen-35B's ssm_conv1d layers is linked to the sensitivity of the underlying Mamba-style selective scan mechanism when subjected to low-bit integer quantization.
- •Community-driven 'uncensored' variants of Qwen models often utilize fine-tuning datasets stripped of RLHF-based refusal mechanisms, which can inadvertently degrade the model's adherence to safety-aligned system prompts during complex roleplay.
📊 競品分析▸ Show
| Feature | Qwen-35B (Fixed GGUF) | Llama-3-70B (GGUF) | Mistral-Large-2 (GGUF) |
|---|---|---|---|
| Architecture | Hybrid Transformer/SSM | Dense Transformer | Dense Transformer |
| Quantization Stability | High (W1-optimized) | Standard (KL-optimized) | Standard (KL-optimized) |
| VRAM Efficiency | High (35B parameter) | Moderate (70B parameter) | Moderate (123B parameter) |
| Primary Use Case | Long-context/Roleplay | General Purpose | Reasoning/Coding |
🛠️ 技術深入
- The ssm_conv1d layer in Qwen-35B utilizes a 1D convolution over the channel dimension, which acts as a feature extractor before the selective state space transition.
- Wasserstein distance (W1) optimization minimizes the cost of transforming the distribution of quantized weights to match the original FP16 distribution, preventing the 'drift' that causes numerical instability in the recurrent state.
- The Q4_K_P quantization format employs a specific block-wise scaling strategy that preserves the dynamic range of the convolution weights more effectively than standard Q4_K_M.
🔮 前景展望AI analysis grounded in cited sources
W1-based quantization will become the industry standard for SSM-based LLMs.
Traditional KL divergence metrics are mathematically insufficient for the specific numerical sensitivities found in recurrent state space architectures.
Automated quantization pipelines will integrate Wasserstein metrics by Q4 2026.
The success of manual fixes for tensor drift in Qwen-35B demonstrates a clear performance benefit that can be generalized into automated model conversion tools.
⏳ 時間線
2025-09
Alibaba releases Qwen-3.6 series, introducing hybrid SSM-Transformer architecture.
2026-02
Community reports numerical instability and 'tensor drift' in quantized Qwen-35B SSM layers.
2026-04
Release of W1-optimized GGUF quantization for Qwen-35B.
📰
AI 週報
閱讀本週精選 AI 大事摘要 →
👉相關動態
AI 策展新聞聚合。所有內容版權歸原始發布者所有。
原始來源： Reddit r/LocalLLaMA ↗