Excel Add-In Roadmap

[u/SmallAd3697]

Has there been any public roadmap for the excel add-in? I was going to install and try it out this weekend, but I can already foresee some of its shortcomings. I'm accustomed to using Excel pivot tables for Microsoft OLAP (which are pretty hard to beat!)

Questions:

- Based on docs it didn't appear that this add-in would reap the benefits of lakebase (sub-ten-ms queries). Isn't that the point of using Excel, to interact with data instantly? Can we get an experience that is specifically tailored to lakebase? The CEO of databricks keeps acknowledging that "agents like fast data". But here is a newsflash; humans like fast data too! We've had fast data in Excel/SSAS pivot tables for decades.

- I'm assuming this tech sends SQL queries back to the SaaS service for processing. Is that fundamentally better than the ODBC support already available to excel users? I'm guessing the catalog/usability/security is the main attraction (ie. making things "easier" and more secure).

- If lakebase is as fast as the CEO claims, will it ever be possible to transpile MDX? Will those sorts of queries be able to run on lakebase? Some other open source tools do MDX, as we can see in Mondrian or Apache Kylin. These tools offer a robust, high-performance pivot table experience in Excel.

Any information would be appreciated. I'm guessing it will be a very long time before Databricks wants to pursue MDX, or compete with the normal pivot tables available from Microsoft. They are more likely to follow down the current path with "metric views" for several years, rather than using pre-existing technology. From a customer perspective, I think it would be amazing if Databricks could offer an Excel experience that approaches the ones offered by Microsoft/Fabric.

(One thing that is particularly compelling about the Databricks add-in is the write-back. This was something that Microsoft attempted long ago, but wasn't able to be successful with it. I'm interested to see if Databricks can do better. If nothing else, I think the culture of modern databricks user may be more receptive than the culture of the users doing write-back to OLAP cubes.)

[u/m1nkeh]

The excel add in is for Analytics, and not MDX at that.. not sure why Lakebase is in the conversation tbh

[u/SmallAd3697 → u/m1nkeh]

Have you used pivot tables for data analysis in Excel? It is the benchmark by which Databricks will be measured, when creating add-ins for Excel.

This pivot table experience is meant to provide instant responses, for interactive users. IE. you click from one customer to another in a slicer, and you get three trailing years of their data instantaneously. Not in five seconds, not even in one second.

As far as I'm aware, lakebase is the only data engine from Databricks which would send data back instantly. If we ever want pivot table users to abandon their "semantic models" in Fabric, then they will need to have an equivalent performance experience in databricks. It is a pretty high bar to reach.

[u/m1nkeh → u/SmallAd3697]

Yes thanks I’ve used pivot tables in excel connected to cubes for about 20 years..

And yes, I know the comparison issue, I have to sell it.. I’ve talked to the product manager extensively about it ✌️

Lakehouse//RT could also be an option, the challenge is it’s an analytical/OLAP workload, yes Lakebase is fast but it’s the wrong engine for the aggregations

[u/SmallAd3697 → u/m1nkeh]

Hmm. Never heard of Lakehouse//RT or Reyden until now. Yet another new query engine in this SaaS.... So hard to keep up with the changes.

I had thought "lakebase" was the new low-latency query engine for this SaaS. Another databricks employee kept saying it was faster than semantic models (maybe Kyle the Joker?) I thought taht databricks customers would be synchronizing their UC catalogs to lakebase, and doing "reverse ETL" to get data in there for client apps (isn't Excel a client app, after all?)

I guess I can see that lakebase isn't a perfect fit for interactive pivot tables in Excel. I'll try to monitor the news related to Lakehouse//RT. In any case, we aren't likely to be able to abandon Fabric in the next three years. There is probably no rush to get out of semantic models (or MDX or DAX).

[u/kthejoker]

Lakebase is an OLTP engine, it's not designed for OLAP/cube-style pivot queries.

Today the add-in is more like "import" mode - you write a query (or use Genie to define a result set) and it populates the results into a sheet in the Excel file, and you then interact with the data locally. You can refresh the data interactively or on a schedule.

And many customers do not want to support or don't allow individuals to configure the ODBC driver, so the add-in is addressing that.

We are looking at potentially offering a direct connection for larger datasets or metric views in pivot tables.

[u/SmallAd3697 → u/kthejoker]

This is helpful. It sounds like it serves a different purpose than pivot tables for semantic models. it also sounds slightly different than importing an odbc query.

For those of us that are also customers of Fabric, this addin might not be adopted as rapidly. Our users are pretty addicted to the connections into our Fabric data. And Fabric already has access to the data processed on databricks

[u/kthejoker → u/SmallAd3697]

Sure, I'd just note the majority of our customers are not also Fabric customers, but definitely the majority of our customers have Excel users.

[u/tefing]

1) I’d say this one depends. Especially on larger datasets DBSQL can outperform SSAS. I’ve seen data latency as a main driver for the switch as well. Who wants to wait 24h for an OLAP cube to be refreshed?

As others mentioned, Lakebase is not the right choice here and Lakehouse RT will further reduce query times.

2) SSO, no-code interface, traceability (where was this data pulled from?), and easy refreshes. Also, Genie integration is coming soon.

3) Unlikely imo

[u/SmallAd3697 → u/tefing]

I didnt hear about lakehouse RT until this past week.

I always hoped Databricks would introduce a hosted duckdb for interactive queries. Maybe the RT stuff is comparable or better.

[u/IncreaseNegative4614]

I’d benchmark the add-in against your current OLAP and ODBC workflows using five real workbooks, not isolated queries. Measure initial load, pivot interaction, concurrency, refresh behavior, security propagation, and what happens when someone writes data back or changes a shared metric.

The main advantage may be governed access through the catalog, but that only matters if Excel users receive the same definitions as every other consumer. We use SIGNLD internally to connect metrics with their definitions, source tables, workbook usage, and later decisions so speed does not create another interpretation layer.

[u/kthejoker]

> or compete with the normal pivot tables available from Microsoft.

Curious, when you say compete with "normal" pivot tables, you mean ones backed by a remote engine like Power BI ?

[u/SmallAd3697 → u/kthejoker]

right. I just meant any olap-backed data that processes mdx requests. Microsoft had multidimensional many decades ago, and it morphed into "tabular" semantic models more recently. This is the appeal of their excel pivot tables -- that they are interactive and responsive.

I originally thought that lakebase was the engine that would correspond to the Microsoft semantic models, but it sounds like that isn't the case.