Qwen3.5 27B Costs 0.83€/1M Output Tokens

🔑 Enhanced Key Takeaways

• •Qwen3.5 27B utilizes a Mixture-of-Experts (MoE) architecture, which significantly optimizes inference costs by activating only a subset of parameters per token, explaining the high throughput relative to power consumption.
• •The 0.30€/kWh electricity rate used in the analysis is representative of current average industrial/commercial energy costs in parts of the EU, highlighting the economic viability of self-hosting versus cloud API usage for high-volume workloads.
• •The performance disparity between prompt processing (1691 TPS) and generation (53.8 TPS) is characteristic of the memory-bandwidth-bound nature of autoregressive decoding in LLMs, even when utilizing dual-GPU setups.

🛠️ Technical Deep Dive

• •Model Architecture: Qwen3.5 27B employs a sparse Mixture-of-Experts (MoE) design, allowing for lower compute requirements per token compared to dense models of similar total parameter counts.
• •Inference Engine: The user utilized vLLM, which leverages PagedAttention to optimize KV cache memory management, significantly reducing memory fragmentation and increasing throughput.
• •Hardware Configuration: The setup uses a heterogeneous GPU configuration (RTX 3090 + RTX Pro 4000), suggesting the use of model parallelism (likely tensor parallelism) to distribute the model weights across disparate VRAM capacities.
• •Power Profile: The 535W draw represents the combined TDP/actual load of the dual-GPU system, including overhead from the host system, which is a critical factor in calculating the 'cost-per-token' metric.

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline