Mac Mini M4: 34 tok/s on 20B LLM

🔑 Enhanced Key Takeaways

• •Mac Mini M4 base model (10 CPU cores, 10 GPU cores) achieves Geekbench 6 single-core score of 3788 and multi-core of 14696, positioning it strongly among 2024-2025 Macs[3].
• •M4 Neural Engine delivers 38 trillion operations per second, a 3x improvement over M1 and outperforming Intel's 14th-gen i9 in AI tasks[2].
• •Mac Mini M4 shows significant performance degradation with model sizes: 77.1 tok/s on 1B models dropping to 17.7 tok/s on 8B and 9.6 tok/s on 14B models[1].
• •M4 memory bandwidth reaches 120 GB/s, a 20% boost over M2, aiding heavy data applications like video editing and AI[2].

🛠️ Technical Deep Dive

• •M4 chip: 10 CPU cores at 4.4 GHz, 10 GPU cores; Neural Engine at 38 TOPS (trillion operations per second)[2][3].
• •Memory bandwidth: 120 GB/s, 20% higher than M2, optimized for AI and matrix multiplication tasks[2].
• •Geekbench 6 scores: Single-core 3788, Multi-core 14696 for base M4 Mac Mini[3].
• •Performance scales poorly with model size due to unified memory limits in base 32GB config, evident in 77% drop from 1B to 8B models[1].

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline