Workers AI Adds Kimi K2.5 Support

🔑 Enhanced Key Takeaways

• •Kimi K2.5 is an open-source model from Moonshot AI with a 1 trillion parameter Mixture-of-Experts (MoE) architecture, activating only 32 billion parameters per token for efficiency[1][2][3].
• •Features native multimodality with MoonViT vision encoder, supporting image, video, PDF, and text inputs up to a 256K token context window[1][2][5].
• •Introduces Agent Swarm technology for self-directed, coordinated multi-agent execution, decomposing complex tasks into parallel sub-tasks[1][2][3].
• •Achieves top benchmarks including 96.1% on AIME 2025 math, 76.8% on SWE-Bench Verified coding, and 2x inference speed via native INT4 quantization[2][5].

🛠️ Technical Deep Dive

• •Mixture-of-Experts (MoE) architecture: 1T total parameters, 32B activated, 384 experts (8 selected per token), 1 shared expert, 61 layers[1][2].
• •Multi-head Latent Attention (MLA) with 64 heads, 7168 hidden dimension, SwiGLU activation, 160K vocabulary[1][2].
• •MoonViT vision encoder (400M parameters) for native vision-language integration; supports RGB images, videos (experimental), PDFs via spatial-temporal pooling[1][2][3].
• •Quantization-Aware Training (QAT) for INT4 quantization, enabling 2x speed over FP16 without accuracy loss; optimized for NVIDIA GPUs[1][2][3].
• •Hardware requirements: Full model ~630GB, needs 4x H200 GPUs or >240GB unified memory for 10+ tokens/s; B200 achieves >40 tokens/s[6].

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline