SourceReddit r/MachineLearning•Stalecollected in 15h
1B Pure SNN Scales from Scratch
#spiking-networks#neuromorphic#sparsity1.088b-snnsnnspikebert
1B SNN LM converges from scratch w/ 93% sparsity – code + checkpoint free!
30-Second TL;DR
What Changed
93% sparsity, 7% neurons fire per token
Why It Matters
Demonstrates viability of massive SNNs from scratch, unlocking energy-efficient neuromorphic inference for language tasks.
What To Do Next
Download the 12GB SNN checkpoint from GitHub to experiment with sparsity.
Who should care:Researchers & Academics
Key Points
- •93% sparsity, 7% neurons fire per token
- •Cross-lingual Russian text emergence at 25k steps
- •39% activation routing to memory at 1B scale
- •Pure spike-domain training, no ANN conversion
Weekly AI Recap
Read this week's curated digest of top AI events →
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/MachineLearning ↗
This is a summary, not the original. Read the source, or get the weekly briefing.
The weekly digest
One email a week. Unsubscribe anytime.