Atlas Unlocks 102 tok/s on DGX Spark
๐กNew Rust engine: 2.3x faster than vLLM on DGX Spark for Qwen MoE models.
โก 30-Second TL;DR
What Changed
102 tok/s on Qwen3.5-35B-A3B (127 MTP K=2), 2.3x vLLM
Why It Matters
Revolutionizes DGX Spark usability, enabling fast local inference of 80B+ MoE models on desktop hardware for AI builders.
What To Do Next
Download Atlas from source and benchmark Qwen3.5-35B-A3B on DGX Spark GB10.
Key Points
- โข102 tok/s on Qwen3.5-35B-A3B (127 MTP K=2), 2.3x vLLM
- โข82 tok/s on Qwen3-Next-80B-A3B vs vLLM's 36.4
- โขPure Rust, custom CUTLASS 3.8 kernels, 2GB image, 2-min cold start
- โขNo PyTorch/Docker, optimized NVFP4 for GB10
๐ง Deep Insight
Background and context from public sources โ not the original article. 9 sources cited.
๐ Enhanced Key Takeaways
- โขAtlas uses Kernel Hypercompilation methodology with Rust abstractions, achieving 32/32 benchmark wins against PyTorch baselines including 18x faster RoPE and 3.9x faster MoE W4A16 across 256 experts.[2]
- โขAtlas Inference Server owns the entire stack from custom kernels to HTTP server layer, eliminating Python, PyTorch, and framework dependencies for native operation on DGX Spark.[2]
- โขDevelopers AzeezIsh and a collaborator built Atlas specifically to avoid vLLM's dependency debugging issues, with source build to first token in under 2 minutes versus vLLM's 40+.[2]
๐ Competitor Analysisโธ Show
| Feature/Benchmark | Atlas | vLLM (NVIDIA stock) |
|---|---|---|
| Qwen3-Next-80B-A3B tok/s (GB10 GPU) | 82 (no speculative decoding) | 36.4 (2.8x slower)[2] |
| Qwen3.5-35B-A3B tok/s (127 MTP K=2) | 102 (2.3x faster)[article] | ~44 (inferred)[article][2] |
| Build to first token | <2 minutes | 40+ minutes[2] |
| Dependencies | None (pure Rust, no PyTorch/Docker) | Python/PyTorch/framework heavy[2] |
๐ ๏ธ Technical Deep Dive
- โข20+ custom kernels compiled directly for NVIDIA SM121 architecture (GB10 GPUs in DGX Spark).[2]
- โขEmploys Kernel Hypercompilation philosophy: custom CUTLASS 3.8 kernels optimized with NVFP4 quantization.[2][article]
- โขFull stack control: kernels to HTTP server, no external frameworks; wins include 18x faster RoPE, 8x faster Gated Delta Rule, 3.9x faster MoE W4A16 (256 experts).[2]
๐ฎ Future ImplicationsAI analysis grounded in cited sources
โณ Timeline
๐ Sources (9)
Factual claims are grounded in the sources below. Forward-looking analysis is AI-generated interpretation.
- siliconangle.com โ Atlas Cloud Optimizes AI Inference Service Boost GPU Throughput
- forums.developer.nvidia.com โ 362210
- dev.to โ Qora Native Rust LLM Inference Engine 4n4n
- softlandia.com โ Best Way to Build an AI Engine in 2026
- yottalabs.ai โ Fastest LLM Inference in 2026 GPU Speed Throughput and Cost Compared
- siliconflow.com โ The Best GPU Inference Acceleration Service
- forums.developer.nvidia.com โ 362210
- pub.towardsai.net โ How to Choose the Right Open Source LLM in 2026 F79a199829de
- hackernoon.com โ The Next Trillion Dollar AI Shift Why Openclaw Changes Everything for Llms
Weekly AI Recap
Read this week's curated digest of top AI events โ
๐Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/LocalLLaMA โ
This is a summary, not the original. Read the source, or get the weekly briefing.
Weekly AI briefing
One email a week. Unsubscribe anytime.