Apple 剖析 CoT 追蹤動態

🔑 增強重點摘要

• •Apple's study uses controllable puzzle environments like Tower of Hanoi, Checker Jumping, River Crossing, and Blocks World to manipulate compositional complexity and analyze reasoning traces beyond final answers.[1][2][4]
• •LRMs exhibit compute inversion: reasoning effort (thinking tokens) increases with complexity up to a threshold, then declines despite available inference budget, indicating fundamental scaling limits.[1][2][4]
• •Three performance regimes identified: low-complexity where standard LLMs outperform LRMs, medium where LRMs gain from extended thinking, and high where both collapse completely.[2][4]

🛠️ 技術深入

• •Evaluated puzzles include Tower of Hanoi (disk counts requiring hundreds/thousands of moves), Checker Jumping, River Crossing, and Blocks World, allowing precise control of compositional depth and logical structure.[1][2][4]
• •LRMs fail to maintain stable internal state across deep compositional chains, abandon step-by-step reasoning for shortcuts at high complexity, and do not use explicit algorithms or reason consistently across puzzle types.[1][4][6]
• •Analysis reveals opaque CoT traces that may hide flawed logic despite appearing high-quality, with benchmark contamination concerns addressed via novel synthetic environments.[2][4]

🔮 前景展望AI analysis grounded in cited sources

⏳ 時間線