Qwen3.8-27B Gets High-Quality 2.5–3 BPW GGUFs
💡Fit Qwen3.8-27B into 8.4–10.1 GB while retaining surprisingly strong benchmark scores.
⚡ 30-Second TL;DR
What Changed
Three GGUF variants are available at 2.50, 2.75, and 3.00 bpw, ranging from 8.4 to 10.1 GB.
Why It Matters
These releases could make a capable 27B model more practical on consumer hardware with limited RAM or VRAM. If the reported results hold across independent evaluations, learned mixed-precision allocation may substantially improve the quality available at 2–3 bit model sizes.
What To Do Next
Download the 2.75 bpw GGUF from Hugging Face and compare its AIME25, GPQA-Diamond, and LiveCodeBench performance with your current Unsloth quantization in llama.cpp.
Key Points
- •Three GGUF variants are available at 2.50, 2.75, and 3.00 bpw, ranging from 8.4 to 10.1 GB.
- •GSQ jointly learns scalar quantization grid assignments and scales to improve low-bit accuracy.
- •RCO allocates tensor-specific quantization types under a strict size budget using task-loss optimization.
- •The 3.00 bpw version reportedly matches BF16 on AIME25, while the 2.75 bpw version exceeds BF16 on the reported zero-shot average.
🧠 Deep Insight
Background and context from public sources — not the original article. 8 sources cited.
🔑 Enhanced Key Takeaways
- •Qwen3.8-27B utilizes a hybrid-attention architecture where only 16 of its 64 layers employ full gated attention, while the remaining 48 utilize Gated DeltaNet for computational efficiency.
- •The model features a native context window of 262,144 tokens, which can be extended to 1,000,000 tokens through RoPE scaling techniques.
- •Qwen3.8-27B includes a default 'thinking mode' that allows users to adjust reasoning depth via a 'reasoning_effort' parameter, impacting both token usage and output quality.
- •The model is a native vision-language architecture capable of processing complex multimodal inputs, including documents, STEM diagrams, and long-form video content.
- •Qwen3.8-27B achieved a score of 61.7 on SWE-bench Pro, marking a significant improvement over the 53.5 score achieved by the previous Qwen3.6-27B iteration.
📊 Competitor Analysis▸ Show
| Feature | Qwen3.8-27B (GSQ/RCO) | Llama 3.1 70B (4-bit) | Mistral Large 2 (4-bit) |
|---|---|---|---|
| Memory Footprint | 8.4–10.1 GB | ~40 GB | ~28 GB |
| Architecture | Hybrid-Attention | Dense Transformer | Dense Transformer |
| Multimodal | Native Vision/Video | Text-only | Text-only |
| License | Apache 2.0 | Llama 3.1 Community | Mistral Research |
🛠️ Technical Deep Dive
- Architecture: Hybrid-attention backbone utilizing 16 gated attention layers and 48 Gated DeltaNet (linear attention) layers.
- Quantization Methodology: GSQ (Grid Scalar Quantization) optimizes grid assignments and scales; RCO (Rate-Constrained Optimization) performs tensor-specific bit-allocation based on task-loss sensitivity.
- Context Handling: Native 262k token window with support for RoPE-based extension to 1M tokens.
- Reasoning: Integrated 'thinking' mechanism with adjustable 'reasoning_effort' parameters (low, medium, xhigh).
🔮 Future ImplicationsAI analysis grounded in cited sources
⏳ Timeline
📎 Sources (8)
Factual claims are grounded in the sources below. Forward-looking analysis is AI-generated interpretation.
Weekly AI Recap
Read this week's curated digest of top AI events →
👉Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/LocalLLaMA ↗
This is a summary, not the original. Read the source, or get the weekly briefing.
Weekly AI briefing
One email a week. Unsubscribe anytime.