Step3.5 Flash Tops OpenClaw for 3 Days

🔑 Enhanced Key Takeaways

• •Step 3.5 Flash is a 196B parameter sparse Mixture-of-Experts (MoE) model with only ~11B active parameters per token, enabling high throughput of 100-300 tokens/sec and peaking at 350 tok/s for coding tasks on NVIDIA Hopper GPUs.[1][2][4]
• •It supports a 256K-262K context window via a hybrid 3:1 Sliding Window Attention (SWA) ratio, reducing compute overhead for long-context tasks like massive datasets and codebases.[2][4]
• •The model excels in agentic benchmarks, scoring 74.4% on SWE-bench Verified, 51.0% on Terminal-Bench 2.0, and 88.2 on τ²-Bench, matching or exceeding top closed-source systems.[2][3][4]

🛠️ Technical Deep Dive

• •Architecture: Transformer-based sparse MoE with 196.81B total parameters (196B backbone + 0.81B MTP head), ~11B active per token; 45 layers, hidden size 4,096, vocabulary 128,896.[4]
• •Experts: 288 routed experts + 1 shared expert (always active), top-8 selection per token.[4]
• •Attention: 3:1 SWA ratio (three sliding-window layers of size 512 per full-attention layer) for efficient 256K context.[2][4]
• •Prediction: MTP-3 (3-way Multi-Token Prediction) used in both training and inference for accelerated generation.[1][2][4]

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline