
Aurora-80K Packs Language Modeling into 80K Parameters
Aurora-80K is a tiny language model with exactly 80,000 parameters and a factorized 4,096-token vocabulary. Its reported results include 3.2902 Wikitext-2 BPB, 52.31% BLiMP accuracy, and 26.05% on ARC-Easy.


