💼VentureBeat•Stalecollected in 15m
GPT-5.5 Outperforms Claude Fable 5 on New ALE Benchmark
#benchmarking#agentic-ai#evaluation-frameworkagents’-last-exam-(ale)openaianthropicgpt-5.5claude fable 5uc berkeley
💡New rigorous benchmark ALE exposes model failures in real-world professional workflows; GPT-5.5 takes the lead.
⚡ 30-Second TL;DR
What Changed
ALE benchmark evaluates AI across five functional layers: Brain, Eyes, Body, Hands, and Feet.
Why It Matters
ALE sets a new standard for evaluating agentic AI, shifting focus from narrow coding tasks to complex, multi-step professional workflows. This will likely force model developers to prioritize robust tool orchestration and visual perception capabilities.
What To Do Next
Review your agent's performance against the ALE framework to identify weaknesses in multi-step tool orchestration and visual desktop navigation.
Who should care:Researchers & Academics
Key Points
- •ALE benchmark evaluates AI across five functional layers: Brain, Eyes, Body, Hands, and Feet.
- •GPT-5.5 secured a 24.0% pass rate, leading the leaderboard over Claude Fable 5's 22.0%.
- •The benchmark minimizes 'LLM-as-a-judge' grading, favoring deterministic, code-based evaluation for professional tasks.
- •Tasks are aligned with U.S. federal occupational taxonomy (O*NET/SOC 2018) across 55 industries.
🧠 Deep Insight
Web-grounded analysis with 19 cited sources.
🔑 Enhanced Key Takeaways
- •The Agents’ Last Exam (ALE) benchmark is a large-scale evaluation co-led by UC Berkeley RDI and over 300 industry experts, featuring 1,490 real-world professional tasks across 55 non-physical industries, designed to measure AI performance in long-term, economically valuable workflows.
- •GPT-5.5, internally codenamed "Spud," represents OpenAI's first fully retrained base model since GPT-4.5, released on April 23, 2026, incorporating a completely new architecture, pretraining corpus, and agent-oriented objectives, rather than being an incremental update.
- •Anthropic's Claude Fable 5, released on June 9, 2026, is a 'Mythos-class' model made generally available with built-in safety safeguards, while its counterpart, Claude Mythos 5, is the same underlying model with lifted safeguards for a vetted group of cyber defenders and critical infrastructure operators.
- •ALE addresses the limitations of prior AI benchmarks, which were often susceptible to exploitation or focused on narrow, isolated tasks, by implementing a Generalist Computer-Use Agent (GCUA) framework that forces models to operate on real machines and evaluates outcomes deterministically through code-based grading.
- •Despite the advanced capabilities of current frontier models, the average complete pass rate on the most difficult 'Last-Exam' tier of the ALE benchmark is only 2.6%, highlighting a significant disparity between current AI performance and the requirements for reliable real-world professional task completion.
📊 Competitor Analysis▸ Show
| Feature/Metric | GPT-5.5 (OpenAI) | Claude Fable 5 (Anthropic) | Claude Mythos 5 (Anthropic) |
|---|---|---|---|
| Release Date | April 23, 2026 | June 9, 2026 | June 9, 2026 |
| Codename | Spud | N/A | N/A |
| Model Class | New foundational model (post GPT-4.5) | Mythos-class | Mythos-class |
| ALE Benchmark Pass Rate | 24.0% | 22.0% | N/A (same underlying model as Fable 5, but specific ALE score not separately reported) |
| Terminal-Bench 2.0 Score | 82.7% | 69.4% (vs Claude Opus 4.7) | N/A |
| Pricing (Input/Output per million tokens) | $5 / $30 (2x GPT-5.4) | $10 / $50 | $10 / $50 |
| Availability | ChatGPT, Codex, API (delayed at launch, available April 24, 2026) | Generally available (Claude API, Amazon Bedrock, Microsoft Foundry) | Restricted access (Project Glasswing, trusted access program) |
| Safety Safeguards | API deployments required "different safeguards" | Yes, routes flagged cyber, biology, chemistry, distillation requests to Claude Opus 4.8 | Lifted for vetted cyber defenders and critical infrastructure operators |
🛠️ Technical Deep Dive
- Agents’ Last Exam (ALE) Benchmark:
- Evaluates AI agents across five functional layers: Brain, Eyes, Body, Hands, and Feet.
- Utilizes a Generalist Computer-Use Agent (GCUA) framework, requiring agents to operate on real machines and score artifacts against verifiable success criteria.
- Tasks are derived from genuine professional projects, converted into code-graded, fully reproducible tests, eliminating human judges and subjective scoring.
- The evaluation framework,
ale_runtoolkit, provisions sandboxes, runs agents, and grades them.
- GPT-5.5 (OpenAI):
- Codenamed "Spud."
- Represents a fundamentally new foundation, with reworked architecture, pretraining corpus, and agent-oriented objectives, unlike previous GPT-5.x iterations which were post-training updates.
- Features a one million token context window, a first for OpenAI's API.
- Available in standard GPT-5.5 and GPT-5.5 Pro, with the Pro variant applying parallel test-time compute for more challenging tasks.
- Claude Fable 5 / Mythos 5 (Anthropic):
- "Mythos-class" models, indicating a tier above the Opus class in capability.
- Designed for long-running, multi-stage, and asynchronous tasks, demonstrating enhanced autonomy in areas like complex code refactoring and deep research synthesis.
- Possesses advanced vision capabilities, allowing it to understand diagrams, charts, and tables embedded in files and PDFs.
- Incorporates proactive self-verification, enabling the model to self-update skills based on learnings and develop its own evaluations.
- Claude Fable 5 includes safeguards that route harmful prompts related to cybersecurity, biology, chemistry, and health to Claude Opus 4.8.
🔮 Future ImplicationsAI analysis grounded in cited sources
The ALE benchmark will accelerate the development of truly agentic AI systems.
By focusing on long-horizon, real-world tasks with objective, verifiable outcomes, ALE provides a clear target for researchers to build more robust and economically valuable AI agents, moving beyond easily exploitable or narrow benchmarks.
AI models will increasingly be released in differentiated versions based on safety and access.
Anthropic's release of Claude Fable 5 (public with safeguards) and Claude Mythos 5 (restricted, safeguards lifted) demonstrates a trend towards segmenting powerful AI capabilities based on user vetting and risk assessment, particularly for sensitive applications like cybersecurity.
The economic impact of AI will become more measurable and significant as benchmarks align with real-world labor.
ALE's alignment with the O*NET/SOC 2018 occupational taxonomy and its focus on economically valuable tasks provide a more direct link between AI performance and labor market impact, potentially leading to more tangible productivity gains beyond software engineering.
⏳ Timeline
2025-11
Center for AI Safety (CAIS) releases 'The Remote Labor Index' benchmark, testing LLMs on paid freelance work.
2026-03-24
OpenAI's GPT-5.5 (codename 'Spud') completes its pre-training phase.
2026-04-07
Anthropic officially announces the Claude Mythos AI model.
2026-04-23
OpenAI releases GPT-5.5, the first fully retrained base model since GPT-4.5.
2026-06-09
Anthropic releases Claude Fable 5 (public) and Claude Mythos 5 (restricted), its most capable models to date.
2026-06-10
UC Berkeley researchers launch the Agents’ Last Exam (ALE) benchmark.
📎 Sources (19)
Factual claims are grounded in the sources below. Forward-looking analysis is AI-generated interpretation.
📰
Weekly AI Recap
Read this week's curated digest of top AI events →
👉Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: VentureBeat ↗