Evo2 Spots Gene Regulation Beyond Alignment

🔑 Enhanced Key Takeaways

• •Evo 2 features 40 billion parameters and supports a 1 million token context window for single-nucleotide resolution modeling across DNA, RNA, and proteins[1][2].
• •The model accurately predicts functional impacts of genetic variations, including noncoding pathogenic mutations and BRCA1 variants, without fine-tuning[2].
• •Evo 2 uses a StripedHyena2 hybrid architecture combining convolutional and attention layers, enabling in-context learning that outperforms similarly-scaled language models[5][6].
• •Developed through collaboration with NVIDIA, Stanford, UCSF, UC Berkeley, Goodfire, and University of Washington; trained on over 2,000 NVIDIA H100 GPUs[3][4].
• •Fully open-sourced with model weights, training/inference code, and OpenGenome2 dataset; integrated into NVIDIA BioNeMo framework[1][4].

🛠️ Technical Deep Dive

• •40B parameter model with 7B variant; trained on 9.3T nucleotides from eukaryotic/prokaryotic genomes spanning all domains of life[1][2][6].
• •StripedHyena2 architecture: hybrid of convolutional and attention layers for near-linear scaling with context length up to 1M nucleotides[1][5].
• •Next-token prediction on whole-genome sequences; supports prediction (e.g., mutation impacts) and generation (e.g., coherent genome-scale sequences)[2][3].
• •Mechanistic interpretability via SAEs on layers like layer 26 (StripedHyena), revealing features for exon-intron boundaries, TF binding sites, protein structures, prophage[2][6].
• •Trained on NVIDIA DGX Cloud via AWS using >2,000 H100 GPUs for several months[4].

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline