Unsloth Qwen3.5 Quants Disable Thinking

🔑 Enhanced Key Takeaways

• •Qwen3 models have thinking enabled by default in transformers and vLLM via enable_thinking=True, generating blocks for planning before answers[1][4].
• •Fine-tuning Qwen3.5 with mixed datasets can impair reasoning unless using tags like /think for reasoning examples and /no_think for non-reasoning ones[2].
• •To preserve reasoning during fine-tuning, datasets should contain at least 75% reasoning-style examples mixed with direct answers[3][4].

🛠️ Technical Deep Dive

• •Thinking mode in Qwen3.5 small models (0.8B-9B) is toggled via --chat-template-kwargs '{"enable_thinking":true}' in llama-server, generating ... blocks prior to final responses[1].
• •Non-thinking mode skips chain-of-thought, providing direct answers without blocks, suitable for general tasks[1][4].
• •Fine-tuning supports Dynamic 4-bit quantization (MXFP4_MOE) for memory efficiency, with MoE kernels enabled by default and router-layer fine-tuning disabled for stability[3].
• •Qwen3.5 supports context lengths up to 40960, but Unsloth recommends 2048 for testing, enabling 8x longer context fine-tuning[4].

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline