Apple 電路放大提升 LLM 數學推理

🔑 增強重點摘要

• •CCA operates in three stages: generating reasoning traces to identify deviation points, pinpointing pivotal tokens and model components, and performing sparse targeted updates to amplify constructive signals.
• •On the GSM-Symbolic benchmark, CCA achieves up to +11.4% accuracy improvements across multiple model families while modifying only 1.59% of components like attention heads and MLP neurons.
• •CCA demonstrates minimal impact on unrelated abilities, with preserved performance on MMLU, TriviaQA, and TruthfulQA benchmarks.

🛠️ 技術深入

• •CCA's first stage generates reasoning traces to detect where the model deviates toward incorrect answers, building on prior circuit discovery work for single-pass tasks.
• •Updates target specific attention heads and MLP neurons responsible for correct reasoning, amplifying signals from components generating constructive responses.
• •Efficiency: Modifies as little as 1.59% of model components for significant gains, avoiding broad fine-tuning.
• •Tested on GSM-Symbolic (Mirzadeh et al., 2025), showing models possess latent math-solving capacity but deviate on certain tasks.

🔮 前景展望AI analysis grounded in cited sources

⏳ 時間線