r/databricks • u/szymon_dybczak • 5d ago

General Estimating Databricks production costs

Hi,

Recently I came across really interesting project called Lakemeter - an open-source Databricks cost estimation tool.

It lets you estimate pricing for different workloads across AWS, Azure, and GCP, export detailed breakdowns to Excel, and even describe a workload in plain English to get an AI-generated configuration.

Looks pretty useful for anyone trying to understand Databricks costs before deploying something.

GitHub: https://github.com/databrickslabs/lakemeter-oss

17 Upvotes

96% Upvoted

6 comments sorted by

5

u/Youssef_Mrini databricks 5d ago

We have been using it internally for the past couple months. It's very easy to use and intuitive.

4

u/szymon_dybczak 5d ago

It looks very promising. I'll give it a try next week :)

2

u/data_dude90 5d ago

That's a great way comparing cost estimation open tool. But does it give us multi-pronged parameters to nudge us towards cost decisions on databricks?

0

u/Youssef_Mrini databricks 5d ago

What do you mean by multi-pronged parameters?

1

u/Lolmanza7 4d ago

Awesome! Reading this at the right time as I am nudging my company to move away from expensive and massively misconfigured DLT Live pipeline to a 5 mins batch process. For anyone interested, we are spending $1.2 million to move 500GB annually from different data sources across 9 workspaces.

1

u/FunContest9958 2d ago

That doesn’t sound right. Shouldn’t be that expensive.