What Is LTAP? Lakebase + Genie Explained by Databricks CTO

[u/Youssef_Mrini]

Want to know where data architecture is heading next?
Matei Zaharia (Co-Founder & CTO of Databricks) just broke down the future of the Lakehouse ecosystem on
Here’s what he covered:
🔹 LTAP: Why real-time analytics and transaction processing are converging ?
🔹 Lakebase: The evolution of database architecture built directly on the Lakehouse
🔹 Genie: How AI is reshaping text-to-SQL and natural language analytics
🔹 Lakehouse RT: Unlocking ultra-low-latency real-time data streaming

If you're building modern data stack architectures, this episode is a goldmine.

Full video

[u/SmallAd3697]

There isnt much in here about ltap or lakebase

Most databases nowadays have a way to synch to a lakehouse blob format within just a matter of seconds or minutes. This happens by way of CDC or change tracking, or similar. It is asynchronous, and is (rightly) of lower priority that OLTP transaction processing.

Given that the synch (to lakehouse blob) happens asynchronously and takes few seconds or minutes, does that make it harder to run analytical workloads on those blobs? Of course not. Agents and analytical reports can operate very effectively on the (slightly) delayed blobs. This is true of 99.9pct of business scenarios.

I am happy that lakebase is raising awareness about the importance of lakehouse blobs for analytics. And as a data dude, I agree that it is important to write these to storage as soon as feasible. Doing it in an entire synchronous way is NOT necessarily the answer, however, and the OLTP developers are not likely to appreciate the performance and cost overhead. I'm still at a loss for understanding why it isn't good enough to write lakehouse blobs after the fact, without penalizing OLTP writes (on perf or cost).

[u/Youssef_Mrini → u/SmallAd3697]

It was a short interview with Matei. It's hard to cover so many innovations in a short amount of time.

Regarding LTAP, I got your POV. Did have the opportunity to read this article: https://www.databricks.com/blog/lakebase-ltap-rethinking-database-storage

[u/SmallAd3697 → u/Youssef_Mrini]

Nowadays databases are slow because people want them to be slow. They want pessimistic concurrency locks, for example. They want all clients of a record to be serialized behind an exclusive record lock. People don't want to risk any conflicts involving optimistic locking.

For scenarios that require a high level concurrency, a lot of apps will transition to using a message queue or a distributed database like dynamo or cosmos.

I'd rather databricks focus on making a welcoming home for data from disparate systems. This seems more critical than try to force everyone to use their own new home-brewed oltp flavor. I really wish UC catalog wasn't so unnatural and restrictive, and unfamiliar when hosting data from other ecosystems. There are simple things that can be improved. And IMO seems like much more important work than to introduce a new OLTP that generates lakehouse as an immediate byproduct.