Search

Few direct matches — filled in with the latest updates.

Tag: #sparsity3 results

🤖

1B Pure SNN Scales from Scratch

18yo dev trains 1.088B parameter pure Spiking Neural Network for language modeling from random init, converging to loss 4.4 after 27k steps. Exhibits 93% sparsity, untargeted Russian emergence, and self-shifting 39% activations to memory. Full code and 12GB checkpoint released on GitHub.

Reddit r/MachineLearningCommunityApr 13#spiking-networks#neuromorphic#sparsity
144M SNN LM Trained from Scratch

144M SNN LM Trained from Scratch

A 144M parameter Spiking Neural Network (SNN) language model, Nord, was trained from scratch on FineWeb-Edu for $10 on an A5000 GPU. It achieves 97-98% inference sparsity, better topic coherence than GPT-2 Small, and visible interpretability via spike rates. Supports online learning with STDP and features an original architecture.

Reddit r/LocalLLaMACommunityFeb 26#snn#neuromorphic#sparsity