Qwen3.5-35B Crashes with Claude Code on llama.cpp

🔑 Enhanced Key Takeaways

• •Qwen3.5 models in llama.cpp force full prompt reprocessing on every generation due to KV cache truncation failures, leading to inefficiency in multi-turn conversations[1][8].
• •Multiple GitHub issues document Qwen3-Coder-Next crashes from empty grammar stacks during long-context (80K+ tokens) tool calls across CUDA, ROCm, and Vulkan backends[1][2].
• •ROCm support for newer Qwen3.5 models is broken in recent llama.cpp builds, causing failures while Vulkan works reliably[9].

🛠️ Technical Deep Dive

• •Qwen3.5moe bug triggers full prompt reprocess after each message due to 'failed to truncate' errors in token cache, confirmed in llama.cpp issue #19858[1][8].
• •Long-prompt crashes (20K+ tokens) with Qwen3.5MoE on multi-GPU setups stem from CUDA graph optimizations; fixed by replacing in-place tensor ops with non-inplace versions[1].
• •Qwen3-Coder-Next GGUF fails on Windows due to unknown architecture errors and crashes on tool calls from invalid JSON with duplicate fields[1].
• •Tool calling requires specific flags like --tool-call-parser qwen3_coder and higher quantization (Q6_K+) for proper schema adherence[4].

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline