ANE Reverse-Engineered for MicroGPT Training

🦙Read original on Reddit r/LocalLLaMA

#apple-silicon#npu-training#reverse-engineering#power-efficiencymicrogpt-on-apple-aneapple-anemicrogptm4-mac-miniclaudecoreml

💡Reverse-engineer ANE for 6.6 TFLOPS/W training – game-changer for Apple AI devs

⚡ 30-Second TL;DR

What Changed

Reverse-engineered ANE APIs with Claude, bypassing CoreML

Why It Matters

Unlocks power-efficient NPU training on Apple silicon, potentially scaling to clusters for larger models and reducing energy costs in edge AI.

What To Do Next

Clone the GitHub repo and run ANE benchmarks on your M4 Mac Mini.

Who should care:Researchers & Academics

Key Points

•Reverse-engineered ANE APIs with Claude, bypassing CoreML
•Trained 110M MicroGPT on single M4 chip
•38 TFLOPS INT8 compute at 2.8W, outperforming GPU/H100 efficiency
•Enables LoRA for 3B/7B models on single device

🧠 Deep Insight

Background and context from public sources — not the original article. 6 sources cited.

🔑 Enhanced Key Takeaways

•Prior reverse-engineering efforts on ANE private frameworks date back to geohot's initial work in the tinygrad repository, providing early insights into direct ANE access[3].
•M4 Mac Mini's Neural Engine delivers approximately 35-40 TOPS, a substantial increase enabling advanced local AI processing beyond prior generations[4].
•Open-source MLX framework offers Pythonic API for direct Apple Silicon programming, including ANE, as an alternative to CoreML for custom model compilation[4].

🔮 Future ImplicationsAI analysis grounded in cited sources

Direct ANE access will accelerate open-source ML on Apple hardware

Reverse-engineered APIs combined with MLX enable custom training and inference without proprietary CoreML constraints, expanding developer experimentation[2][3][4].

Efficiency gains will drive edge deployment of larger LLMs

38 TFLOPS INT8 at 2.8W on M4 supports LoRA fine-tuning of 3B/7B models locally, outperforming high-end GPUs in power efficiency for consumer devices[1].

⏳ Timeline

2018-01

Geohot publishes initial ANE reverse-engineering results in tinygrad repo

2020-12

Hollance documents ANE private frameworks and limitations publicly

2024-10

Apple releases M4 chip with enhanced Neural Engine capabilities

2026-02

MLX framework gains traction for direct ANE model compilation

📎 Sources (6)

Factual claims are grounded in the sources below. Forward-looking analysis is AI-generated interpretation.

🦙Read original article on Reddit r/LocalLLaMA

📰

Weekly AI Recap

Read this week's curated digest of top AI events →

👉Related Updates

Same topic

Explore #apple-silicon

Same product

Tuning Qwen3.8 for 16GB VRAM

Reddit r/LocalLLaMA•Aug 22

AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/LocalLLaMA ↗

This is a summary, not the original. Read the source, or get the weekly briefing.

Weekly AI briefing

One email a week. Unsubscribe anytime.