🏠IT之家•Stalecollected in 5h
Developer Unlocks 15.8TFLOPS AI Power on M4 Chip
#apple-silicon#ai-hardware#local-trainingapple-m4-chipapplem4neural-engine
💡Unlock full AI training potential on M4 chips by bypassing Apple's software restrictions for local model development.
⚡ 30-Second TL;DR
What Changed
Bypassed software locks on M4 Neural Engine to enable direct model training.
Why It Matters
This breakthrough could turn Apple Silicon devices into portable AI training workstations, significantly lowering the barrier for local model experimentation.
What To Do Next
Monitor the developer's GitHub repository to evaluate if this custom MIL approach can be adapted for your local fine-tuning workflows on M4 hardware.
Who should care:Developers & AI Engineers
Key Points
- •Bypassed software locks on M4 Neural Engine to enable direct model training.
- •Developed custom MIL to communicate with hardware without using Core ML or Metal.
- •Achieved 15.8TFLOPS performance by running training processes entirely in RAM.
🧠 Deep Insight
Background and context from public sources — not the original article. 18 sources cited.
🔑 Enhanced Key Takeaways
- •The M4 Neural Engine officially boasts 38 Trillion Operations Per Second (TOPS) for INT8 operations, which is significantly higher than the 15.8 TFLOPS reported in the article. The 15.8 TOPS figure is consistent with the performance of the M2 Neural Engine, suggesting the developer's achievement might be specific to a certain precision (e.g., FP16, where M4's peak is around 19 TFLOPS) or a particular workload.
- •Apple's standard developer framework, Core ML, typically operates as a high-level abstraction, automatically routing machine learning tasks to the CPU, GPU, or Neural Engine without providing direct programmatic access for on-device training or gradient computation.
- •The custom Model Intermediate Language (MIL) developed by the bypass leverages Apple's private
_ANEClientand_ANECompilerAPIs, which are typically undocumented, to communicate directly with the Neural Engine hardware. - •The Neural Engine's architecture is optimized for matrix and vector processing, which are fundamental to AI algorithms, and is particularly efficient as a convolution engine.
- •The M4 Neural Engine maintains a 16-core design, consistent with previous M-series chips, but achieves its substantial performance gains through architectural improvements and a more advanced 3-nanometer process.
🛠️ Technical Deep Dive
- The M4 Neural Engine features a 16-core architecture.
- It is officially rated for 38 Trillion Operations Per Second (TOPS) for INT8 precision workloads.
- The true peak performance for FP16 (half-precision floating-point) operations on the M4 Neural Engine is approximately 19 TFLOPS.
- The developer's method involves bypassing Apple's Core ML framework by utilizing private
_ANEClientand_ANECompilerAPIs for direct hardware interaction. - The Neural Engine's native instruction set is compiled from Apple's Model Intermediate Language (MIL).
- Performance analysis indicates an 'SRAM cliff' where throughput drops significantly for working sets larger than the estimated ~32 MB of on-chip SRAM, forcing spills to DRAM.
- The Neural Engine is fundamentally optimized as a convolution engine, with computations expressed as 1x1 convolutions often yielding better throughput than direct matrix multiplications.
- Projects like 'Orion,' which also bypass Core ML, have identified 14 previously undocumented constraints related to MIL IR programs, memory layout, compilation limits, and numerical behavior.
- A key technique for efficient on-device training involves 'delta compilation,' which allows for surgical updates to compiled programs by patching weight files on disk and reloading them, significantly reducing recompilation time per training step.
🔮 Future ImplicationsAI analysis grounded in cited sources
Increased on-device AI training capabilities on Apple Silicon will accelerate the development of privacy-preserving AI applications.
Bypassing software restrictions for direct hardware access allows developers to train models locally, keeping sensitive user data on the device, which aligns with Apple's privacy focus and user demand for data security.
Apple may eventually provide more direct, low-level APIs for the Neural Engine to foster innovation.
The demonstrated ability by developers to bypass current restrictions and achieve higher performance could pressure Apple to open up more direct access, similar to how Metal evolved for graphics and compute, to support advanced AI development.
The performance unlocked by direct Neural Engine access could make Apple Silicon devices more competitive for professional AI development workflows.
Achieving high TFLOPS for on-device training, especially with unified memory, positions these devices as powerful platforms for local AI model development and fine-tuning, potentially reducing reliance on cloud resources for certain tasks.
⏳ Timeline
2017-09
Apple introduces the first Neural Engine in the A11 Bionic chip, capable of 600 billion operations per second, and releases the Core ML framework.
2020-11
Apple introduces the M1 chip, bringing a 16-core Neural Engine to the Mac, capable of 11 trillion operations per second (TOPS).
2022-06
Apple introduces the M2 chip, featuring a 16-core Neural Engine capable of 15.8 trillion operations per second (TOPS).
2023-10
Apple introduces the M3 chip, with a 16-core Neural Engine capable of over 18 trillion operations per second (TOPS).
2024-05
Apple introduces the M4 chip, featuring a 16-core Neural Engine capable of 38 trillion operations per second (TOPS).
2026-03
The 'Orion' project, an open end-to-end system for direct ANE execution and multi-step LLM training, is presented, bypassing Core ML via private APIs.
📎 Sources (18)
Factual claims are grounded in the sources below. Forward-looking analysis is AI-generated interpretation.
📰
Weekly AI Recap
Read this week's curated digest of top AI events →
👉Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: IT之家 ↗
This is a summary, not the original. Read the source, or get the weekly briefing.
Weekly AI briefing
One email a week. Unsubscribe anytime.