๐ผPandailyโขStalecollected in 24m
Model Best Open-Sources BitCPM-CANN for 1.58-bit Training
๐กReduce inference memory usage by 6x with this new open-source 1.58-bit training framework for domestic hardware.
โก 30-Second TL;DR
What Changed
Enables 1.58-bit model training on domestic AI accelerators.
Why It Matters
This framework could drastically reduce infrastructure costs for teams relying on domestic hardware. It enables the deployment of large-scale models on memory-constrained devices.
What To Do Next
Download the BitCPM-CANN repository and benchmark your current model's memory footprint against the 1.58-bit quantized version.
Who should care:Researchers & Academics
Key Points
- โขEnables 1.58-bit model training on domestic AI accelerators.
- โขReduces inference memory requirements by up to 6x compared to full-precision.
- โขOpen-source framework provides a complete training pipeline for low-bit quantization.
๐ง Deep Insight
Web-grounded analysis with 13 cited sources.
๐ Enhanced Key Takeaways
- โขBitCPM-CANN is specifically designed for Huawei's Ascend AI computing platform and its CANN (Compute Architecture for Neural Networks) software stack, making it the first ternary large model fully trained on this domestic Chinese hardware.
- โขThe 1.58-bit quantization method employs a ternary set of {-1, 0, +1} for neural network weights, achieving an information efficiency of approximately 1.58 bits per weight (log2(3) โ 1.58).
- โขThe framework demonstrates significant performance retention, with 1B, 3B, and 8B models maintaining over 95% of their full-precision capabilities, and the 0.5B model retaining 90.1% while running on approximately 200MB of RAM.
- โขThe release of BitCPM-CANN is particularly timely given the sharp surge in High Bandwidth Memory (HBM) prices in 2026, with costs rising over 165% year-over-year, highlighting the economic benefits of memory-efficient training methods.
- โขBitCPM-CANN utilizes a two-stage training pipeline involving ternary quantization-aware training (QAT) to convergence, followed by post-training distillation, and introduces only a 4.5% training throughput overhead.
๐ ๏ธ Technical Deep Dive
- Quantization Scheme: Employs 1.58-bit (ternary) quantization, representing neural network weights with values from the set {-1, 0, +1}. This is based on the information-theoretic efficiency of log2(3) โ 1.58 bits per parameter.
- Hardware Integration: Tightly integrated with Huawei's Ascend NPU hardware and the Compute Architecture for Neural Networks (CANN) software stack, which provides multi-layer programming interfaces for AI applications on Ascend platforms.
- Training Pipeline: Features a complete, end-to-end training pipeline for low-bit quantization, specifically a two-stage process: ternary Quantization-Aware Training (QAT) to achieve convergence, followed by post-training distillation.
- Computational Efficiency: The use of ternary weights simplifies arithmetic operations, primarily involving integer addition, which eliminates floating-point multiplication overhead and contributes to reduced energy consumption.
- Performance Metrics: Achieves up to an 8x weight memory reduction (approximately 6x end-to-end including scaling factors) at inference. For models up to 8B parameters, it retains 95.7%โ97.2% of full-precision performance across various benchmarks, with the 3B variant showing the highest retention.
- Training Overhead: The QAT integration adds a minimal training throughput overhead of only 4.5% (148 vs. 155 TFLOP/s per NPU), making ternary training a viable default configuration.
- Model Sizes: The BitCPM-CANN series includes models of sizes 0.5B, 1B, 3B, and 8B, initialized from corresponding full-precision MiniCPM4 checkpoints.
๐ฎ Future ImplicationsAI analysis grounded in cited sources
The adoption of 1.58-bit quantization on domestic AI accelerators will significantly increase.
Open-sourcing a complete and efficient training framework like BitCPM-CANN for Huawei Ascend lowers the technical barrier for Chinese companies to develop and deploy advanced AI models, especially amid export controls on high-end chips.
AI development for edge devices and resource-constrained environments will accelerate.
The substantial reduction in memory requirements (up to 6x) and high performance retention of 1.58-bit models make sophisticated AI applications more feasible on hardware with limited computational and memory resources.
Further research and industrial efforts into sub-2-bit quantization techniques will intensify.
The demonstrated success of training 1.58-bit models from scratch on a specific domestic hardware platform provides strong empirical evidence for the viability and benefits of extreme low-bit precision, encouraging more innovation in this area.
โณ Timeline
2018-09
Huawei released CANN 1.0 at HUAWEI CONNECT 2018.
2020-08-10
Huawei released CANN 3.0, an upgrade to a unified heterogeneous computing architecture.
2025-06-25
Microsoft and Tsinghua researchers updated their BitNet b1.58 paper, detailing a 1.58-bit AI model using ternary weights {-1, 0, +1}.
2026-05-25
Model Best, in collaboration with OpenBMB, open-sourced BitCPM-CANN, the first ternary large model fully trained on Huawei's Ascend AI platform.
๐ Sources (13)
Factual claims are grounded in the sources below. Forward-looking analysis is AI-generated interpretation.
๐ฐ
Weekly AI Recap
Read this week's curated digest of top AI events โ
๐Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Pandaily โ