Uber’s Agent Factory Scales Requests Without Higher Token Costs

Post LinkedIn

📚Read original on InfoQ中国

#ai-agents#cost-optimization#token-usage#software-factoryuber-ai-software-factoryuber

💡Learn how Uber scaled agent traffic 9.4x without increasing its token bill.

⚡ 30-Second TL;DR

What Changed

Uber’s agent request volume increased 9.4 times.

Why It Matters

The result suggests that agent adoption does not necessarily require proportional growth in inference spending. Uber’s approach may provide a useful reference for teams scaling internal agents while maintaining predictable AI budgets.

What To Do Next

Instrument your agent workflows with OpenTelemetry to track request volume, input tokens, output tokens, and cost per task before adopting Uber’s optimization methods.

Who should care:Developers & AI Engineers

Key Points

•Uber’s agent request volume increased 9.4 times.
•Despite the traffic growth, Uber’s token bill did not increase.
•Uber is sharing cost-saving practices from its AI software factory.

🧠 Deep Insight

Background and context from public sources — not the original article. 12 sources cited.

🔑 Enhanced Key Takeaways

•Uber achieved a 52% reduction in cost per session since June 2026 by implementing aggressive model routing and context management strategies.
•Over 70% of all pull requests at Uber are now generated by AI agents, reflecting a massive shift in engineering workflow automation.
•The company manages a library of over 3,600 specialized 'agent skills' that are executed more than 30,000 times daily.
•Uber implemented an automatic context compaction mechanism that triggers at 400k tokens to prevent excessive costs, even when using models with 1M token windows.
•The 'Software Factory' success relies on six years of prior investment in a monorepo architecture and the Bazel build system, which provided the necessary standardization for agentic tooling.

📊 Competitor Analysis▸ Show

Feature	Uber (AgentFx)	Standard Enterprise AI
Model Routing	Dynamic (Frontier to Small)	Static/Manual
Context Management	Auto-compaction at 400k	Manual/None
Tooling Integration	Native MCP Gateway	Fragmented/Custom
Cost Attribution	Granular via Model Gateway	Per-Project/Department

🛠️ Technical Deep Dive

Model Gateway: Centralized infrastructure for secure, cost-attributed API access across all engineering teams.
MCP Gateway: Unified interface for agent tool access, enabling standardized interaction with internal services.
AgentFx: Proprietary internal SDK designed for building and deploying scalable agentic workflows.
Context Compaction: Automated logic that compresses input data at the 400k token threshold to optimize cache usage and reduce latency.
Tiered Reasoning: Architecture that defaults to medium-effort reasoning models, reserving frontier models only for complex architectural tasks.

🔮 Future ImplicationsAI analysis grounded in cited sources

Enterprise AI budgets will shift from 'per-seat' to 'per-skill' efficiency metrics.

Uber's success in scaling requests without cost increases demonstrates that granular skill-based optimization is more effective than broad seat-based licensing.

Monorepo standardization will become a prerequisite for large-scale agentic adoption.

Uber's reliance on long-term monorepo and Bazel investments suggests that fragmented codebases will struggle to support high-volume automated agent workflows.