Mythos 架構被 22 歲小伙「逆推」開源了！MoE 和注意力借鑑 DeepSeek

🔑 增強重點摘要

• •The reverse-engineered project, often referred to as 'OpenMythos' or similar community-driven forks, utilizes a distilled knowledge approach to replicate Claude's proprietary MoE routing logic.
• •The developer leveraged specific side-channel timing attacks and latency analysis of the Claude API to infer the underlying sparsity patterns of the Mythos architecture.
• •The open-source implementation has triggered significant debate regarding the legal boundaries of 'reverse engineering' AI model architectures versus weights, highlighting a gray area in current AI safety and IP policies.

🛠️ 技術深入

• •Architecture: Mixture-of-Experts (MoE) with a dynamic routing mechanism that mimics Claude's proprietary 'Mythos' gating function.
• •Attention Mechanism: Implements a modified Multi-Head Latent Attention (MLA) inspired by DeepSeek-V3, optimized for reduced KV cache memory footprint.
• •Training/Inference: The codebase provides scripts for quantization (4-bit/8-bit) to allow local execution on consumer-grade hardware (e.g., dual RTX 4090 setups).
• •Data Handling: Utilizes synthetic data distillation techniques to approximate the output distribution of the original Mythos model.

🔮 前景展望基於引用來源的 AI 分析

⏳ 時間線