Databricks launches LTAP to eliminate AI data pipeline latency

🔑 Enhanced Key Takeaways

•Lakehouse//RT is powered by a new real-time execution engine named Reyden, designed to handle the concurrency and latency demands of modern agentic enterprises.
•LTAP leverages Databricks' existing Lakebase platform, which functions as a fully-managed, serverless PostgreSQL database built on open object storage, specifically tailored for the agent era.
•In preview, Lakehouse//RT demonstrated up to 16x better performance compared to traditional real-time serving layers, achieving response times as low as 10ms on smaller datasets and sub-100ms on larger ones, while sustaining sub-100ms latency at 12,000 queries per second on analytical benchmarks.
•The LTAP architecture supports instant copy-on-write database branching, allowing developers to quickly spin up full-fidelity branches of live production databases for safe debugging and testing of AI agents with tools like GitHub Copilot.
•Databricks positions LTAP as a distinct approach to Hybrid Transactional/Analytical Processing (HTAP), unifying data at the storage layer using open formats (Delta and Iceberg) rather than through engine convergence, aiming to avoid performance compromises and proprietary lock-in associated with traditional HTAP solutions.

📊 Competitor Analysis▸ Show

While Databricks' LTAP and Lakehouse//RT represent a novel approach to unifying transactional and analytical processing for AI agents, several other platforms offer solutions in the broader HTAP and real-time analytics space. Traditional HTAP databases have historically attempted to merge OLTP and OLAP workloads, and major cloud data platforms are also evolving their offerings to support real-time AI.

Feature/Platform	Databricks LTAP / Lakehouse//RT	Traditional HTAP (e.g., SingleStore, MySQL HeatWave, SAP HANA)	Snowflake (with Unistore)	Microsoft Fabric
Approach	Storage-layer unification (Lakebase + Lakehouse on open formats)	Engine convergence (single database engine for both workloads)	Multi-cluster, shared data architecture with transactional tables	Unified SaaS platform with OneLake and Copilot AI
Transactional Data	Serverless Postgres (Lakebase) writes directly to open columnar formats (Delta/Iceberg)	Optimized for high-volume, low-latency OLTP transactions	Hybrid tables for transactional workloads alongside analytical	Integrates various data sources, including transactional
Analytical Data	Millisecond query latency on Delta/Iceberg via Reyden engine	Real-time analytics on operational data	Strong SQL analytics and BI capabilities	Comprehensive analytics, BI, and data science
Data Movement	Zero ETL, zero data copies, single source of truth	Aims to eliminate ETL between OLTP/OLAP	Reduces data movement between transactional and analytical stores	OneLake centralizes data, reducing copies
AI Agent Focus	Designed specifically to accelerate AI agents with real-time, unified data access	Supports real-time feature stores and autonomous decision-making for AI	Snowpark ML for machine learning workflows	Copilot AI embedded across workloads
Governance	Unified governance via Unity Catalog	Varies by vendor, typically database-level	Unified governance and security	Centralized governance with OneLake
Openness	Open formats (Delta, Iceberg), open source components	Often proprietary, though some use open source databases	Supports Iceberg tables, but core platform is proprietary	OneLake uses Delta Parquet format
Performance (Analytical)	Sub-100ms latency at 12,000 QPS (benchmarks)	Low-latency for mixed workloads	Optimized for analytical queries	Varies by workload
Pricing	Existing Lakehouse customers can adopt as drop-in replacement, promotional pricing planned	Varies by vendor, often proprietary	Usage-based pricing, separates storage/compute	Capacity-based pricing

🛠️ Technical Deep Dive

Lakehouse//RT Engine (Reyden): This new compute engine is specifically designed to deliver millisecond query latency at massive scale, supporting tens of thousands of concurrent users and AI agents directly on governed Delta Lake and Apache Iceberg tables.
Direct Querying: Lakehouse//RT queries Delta and Iceberg tables directly, eliminating the need for proprietary formats, data copies, or separate ingestion pipelines.
Intelligent Autoscaling: The system features intelligent autoscaling capabilities, allowing it to rapidly scale up to meet demanding workloads and scale down to zero during idle periods to optimize costs.
LTAP Foundation (Lakebase): LTAP is built on Lakebase, a serverless PostgreSQL database that stores data directly in Unity Catalog using the same open formats (Delta and Iceberg) as the Lakehouse. This ensures a single source of truth for operational, analytical, and streaming data.
PostgreSQL Compatibility: LTAP maintains PostgreSQL compatibility for applications, allowing transactional applications to operate with native PostgreSQL performance while data is instantly written to columnar formats for analytics.
Copy-on-Write Database Branching: Lakebase supports instant copy-on-write database branching, enabling developers to create full-fidelity, temporary branches of live production databases in seconds for safe debugging and development of AI agents.
Unified Governance (Unity Catalog): Both Lakehouse//RT and LTAP operate under the Unity Catalog's governance framework, providing a single identity, permissions, and audit model across all operational, analytical, and streaming data.

🔮 Future ImplicationsAI analysis grounded in cited sources

The development and deployment of production-grade AI agents will significantly accelerate.

By eliminating data pipeline latency and simplifying the data stack, developers can iterate faster and deploy AI agents that rely on real-time, unified data more efficiently, moving AI from experimental pilots to automated workflows.

The Lakehouse architecture will become the dominant paradigm for both operational and analytical workloads.

LTAP's ability to natively handle transactional data alongside analytical data on a single, open-format storage layer removes a major barrier, making the lakehouse a more comprehensive and unified platform for all enterprise data needs.

The competitive landscape for Hybrid Transactional/Analytical Processing (HTAP) solutions will undergo a significant shift.

Databricks' novel approach to HTAP, unifying at the storage layer with open formats, could challenge traditional HTAP vendors who focus on engine convergence and proprietary systems, potentially leading to broader industry adoption of storage-centric HTAP.

⏳ Timeline

2019

Databricks publicly introduced Delta Lake, an open-source storage layer bringing reliability to data lakes.

2020

Databricks coined the term 'Lakehouse' to describe an architecture combining data lake flexibility with data warehouse capabilities.

2021-05

Delta Lake, Hudi, and Iceberg projects enabled building warehousing-like capabilities directly on the data lake, bringing structure, reliability, and performance.

2025-05

Databricks acquired Neon, a PostgreSQL-based OLTP system, as a strategic move to natively run transactional workloads within its Lakehouse.

2025

Databricks introduced synchronization capabilities to move data between operational and analytical environments, a precursor to the more radical LTAP approach.

2026-06-16

Databricks launched LTAP (Lake Transactional/Analytical Processing) and Lakehouse//RT, unifying transactional and analytical data storage for real-time AI agents.

Databricks launches LTAP to eliminate AI data pipeline latency

⚡ 30-Second TL;DR

🧠 Deep Insight

🔑 Enhanced Key Takeaways

🛠️ Technical Deep Dive

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline

📎 Sources (22)

👉Related Updates

Weibo's 3B Model Challenges AI Scaling Laws on Benchmarks

Z.ai releases GLM-5.2: Open-weights coding model beats GPT-5.5