AI代理可靠性科學

🔑 增強重點摘要

• •AI agents show high benchmark scores but fail practically due to overlooked issues like inconsistency across runs, poor perturbation resistance, unpredictable failures, and unbounded error severity[1].
• •The paper introduces 12 metrics across four dimensions—consistency, robustness, predictability, and safety—drawing from safety-critical engineering principles to provide a holistic reliability profile[1].
• •Evaluation of 14 agentic models on two benchmarks demonstrates only marginal reliability improvements despite significant capability advances[1].
• •Related works highlight agent failure diagnosis challenges in probabilistic, long-horizon, multi-agent settings, with benchmarks like AgentRx for localizing critical failures[3].
• •Broader AI safety reports confirm ongoing reliability issues, including hallucinations, flawed code, and misleading outputs in current systems[6].

🛠️ 技術深入

• •Twelve metrics decompose reliability into consistency (e.g., behavior across runs), robustness (withstanding perturbations), predictability (fail patterns), and safety (bounded error severity), complementing single success metrics[1].
• •Evaluated 14 agentic models on two complementary benchmarks, revealing persistent limitations despite capability gains[1].
• •AgentRx framework uses trajectory-level constraints, LLM adjudication, and violation logs for failure localization, achieving 23.6% improvement in pinpointing first unrecoverable failures[3].
• •METR's task-completion time horizons measure reliability by fitting logistic curves to success probability vs. human task duration, e.g., 50%-time horizon where agent succeeds half the time[5].

🔮 前景展望AI analysis grounded in cited sources

⏳ 時間線