📚InfoQ中国•Freshcollected in 0m
DoorDash打造150万RPS高可用代理缓存
💡參考 DoorDash 如何用 Envoy 與 Valkey 支撐 1.5M RPS 及五個九以上可用性。
⚡ 30-Second TL;DR
What Changed
代理缓存吞吐量达到 1.5M RPS
Why It Matters
The architecture provides a relevant reference for AI platforms that need to serve large volumes of repeated or cacheable requests with strict availability targets. It may help teams reduce pressure on upstream services while maintaining predictable latency at scale.
What To Do Next
Benchmark an Envoy-plus-Valkey cache layer against your highest-volume inference or feature-serving endpoint before scaling upstream capacity.
Who should care:Developers & AI Engineers
Key Points
- •代理缓存吞吐量达到 1.5M RPS
- •技术栈采用 Envoy 与 Valkey
- •系统可用性达到 99.99999%
🧠 Deep Insight
AI-generated analysis for this event.
🔑 Enhanced Key Takeaways
- •The architecture utilizes a multi-tier caching strategy where Envoy acts as the L7 proxy layer to handle connection pooling and request routing, significantly reducing the load on backend databases.
- •DoorDash migrated from Redis to Valkey to leverage its improved performance characteristics and open-source governance model, which aligns with their need for high-throughput, low-latency operations.
- •The 99.99999% (seven nines) availability is achieved through a combination of multi-region deployment, automated failover mechanisms, and circuit breaking implemented within the Envoy sidecars.
- •The system employs a 'cache-aside' pattern optimized with consistent hashing to ensure minimal cache misses and balanced distribution across the Valkey cluster nodes.
- •Operational overhead was reduced by automating the lifecycle management of the Valkey clusters, allowing engineers to scale capacity dynamically without manual intervention during peak traffic spikes.
📊 Competitor Analysis▸ Show
| Feature | DoorDash (Envoy/Valkey) | Netflix (EVCache) | Uber (Ringpop/Redis) |
|---|---|---|---|
| Primary Proxy | Envoy | Custom (Memcached-based) | Ringpop/Custom |
| Storage Engine | Valkey | Memcached | Redis |
| Availability Goal | 99.99999% | 99.999% | 99.99% |
| Scaling Strategy | Consistent Hashing | Sharded/Replicated | Consistent Hashing |
🛠️ Technical Deep Dive
- Envoy Configuration: Utilizes Envoy's high-performance C++ core with custom filters to handle request parsing and routing to Valkey nodes.
- Valkey Integration: Deployed in a sharded cluster configuration to distribute the 1.5M RPS load, utilizing Valkey's multi-threaded I/O capabilities.
- Traffic Management: Implements request hedging and retries at the proxy level to mask transient network issues and node failures.
- Observability: Integrated with distributed tracing and real-time metrics collection to monitor latency percentiles (P99/P99.9) across the entire cache path.
- Data Consistency: Employs eventual consistency models for non-critical data while using versioning for critical state synchronization.
🔮 Future ImplicationsAI analysis grounded in cited sources
Valkey adoption will accelerate among high-scale enterprises.
DoorDash's successful migration demonstrates that Valkey is a viable, high-performance alternative to Redis for massive-scale production environments.
Envoy will become the industry standard for L7 caching proxies.
The ability to handle 1.5M RPS while maintaining seven-nines availability validates Envoy's extensibility and performance for non-service-mesh use cases.
⏳ Timeline
2023-05
DoorDash initiates infrastructure modernization to address scaling bottlenecks.
2024-03
Valkey project is announced as an open-source alternative to Redis.
2025-01
DoorDash completes the migration of core caching layers to Valkey.
2026-02
System achieves the 1.5M RPS milestone during peak traffic events.
📰
Weekly AI Recap
Read this week's curated digest of top AI events →
👉Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: InfoQ中国 ↗