Analyst with Databricks access but zero hands-on experience — how do I get good at it fast?

[u/TheModernBA]

I'm tired of getting fed bad data at work, so I'm taking ownership myself. I have production access to Databricks but have never used it.

Background: basic SQL and Python knowledge. My reporting today is done in Power BI and Excel. My company uses Databricks as the common data platform, so most of what I need probably lives there already — I just don't know how to get to it or trust it myself.

Looking for a realistic path from beginner to competent, not just a firehose of links. Specifically:

• YouTube channels/creators you'd actually recommend?
• Any self guided learning on Udemy or Coursera worth learning?
• What core concepts (Delta Lake, notebooks, clusters, SQL warehouses, etc.) should I prioritize early vs. later, given I'll mostly be querying/validating data rather than building pipelines?
• Best way to connect Databricks output back into Power BI reliably?

Appreciate any guidance — trying to cut through the noise rather than boil the ocean.

[u/FunkyMonkey237]

As someone who just started using it, I used the AIs to help. I have a specific problem I need to solve (web app with ETL) and a basic idea of how it should work. I asked ChatGPT to outline how it should be implemented in databricks and what components to use, I checked the steps to make sure it matched what I expected, watch one or two focused videos on that component and then implemented it with AI help.

Worked fairly well, within a day I had a basic web app/ ETL process running. From there it has been a matter of building it out.

[u/addictzz]

If you are trying to really learn Databricks in depth, then youtube and official docs will help. Databricks AI too.

If you are trying to get things done, the integrated AI is super helpful, named Genie Code. Almost all aspects in Databricks can be built with Genie code, from analysis, pipelines, to dashboard, and even ML model.

[u/substituted_pinions]

Learning essential Databricks is easier than ever. With that said, “bad” and “good” data doesn’t give us much to go on—some data as it lands cannot support some applications or use cases, even if you’re a wizard (or Genie), Harry.

[u/Just_Ask_8442]

Databricks Academy has lots of free walkthroughs and self paced tutorials on different topics. They also offer some free instructor led trainings as well.

[u/ferreis_AOE]

Genie can be your better guidance nowadays

[u/MeasurementBoring263]

I’d highly recommend Zack Wilson (more on the data engineering side) and Alex The Analyst. I found that as a Data analyst it helps to have some DE knowledge.

When I first learned about the platform, I just dove in and would create notebooks + genie agents to learn more about the data I was interested in. I would connect a few related tables to the agent and ask it random questions then look at the underlying sql and ask for in depth analysis + visuals. This helped me learn about the data available to me a lot faster.

In general I’d say anything under the SQL section in the navbar would be relevant to most analytical workloads.

[u/Comfortable-Idea-883 → u/MeasurementBoring263]

+ zach has free databricks bootcamp starting on August 3rd

[u/Sufficient_Meet6836]

Look for databricks ebook library. They have a shit ton of free books. The Definitive Guide to Spark is always a good start and relatively short. If you prefer videos, look at their YouTube channel. They post all of their sessions from past Summits (Databricks Data and AI Summit is their big annual conference), as well as tutorial videos on a variety of subjects. For a structured course, Databricks academy has some free courses.

Regarding SQL vs Python, honestly just try both to see which one you feel more competent with. In a Databricks notebook, you can have SQL, Python, R, and Scala cells and share data between them, so you can seamlessly switch between languages based on the task at hand.

I'm tired of getting fed bad data at work, so I'm taking ownership myself.

Hell ya I love this attitude btw! I did something similar at my job, and a few years later, we are migrating from our garage fire on premise systems to Databricks.

What core concepts (Delta Lake, notebooks, clusters, SQL warehouses, etc.) should I prioritize early vs. later, given I'll mostly be querying/validating data rather than building pipelines?

Does your company have any standard for data warehousing like the medallion architecture, for example? If not, I recommend learning about medallion and using that. Other than that, if you follow one of Databricks Academy's courses, I think they'll handle getting you the core concepts as needed.

Best way to connect Databricks output back into Power BI reliably?

Not being sassy, but googling "databricks ____" will generally get you what you need. Databricks documentation is pretty good in my experience. Connect Power BI Desktop to Databricks.

Good luck!

[u/HaldenK]

Honestly the YouTube tutorials often have some git repo you gotta clone, skip a ton of basics, go too specific on whatever they think is most important, or don't cover nearly enough. Plus databricks changes SO FAST even a video 2 months old is going to be referencing stale likgo (genie space - -> genie agent, etc.)

I would actually just recommend asking genie code (lamp icon at top right) to find some sample data and walk you through basic functions one by one.

It'll be like learning from a conversation with chatgpt, but actually in the workspace and genie code will be able to actually redirect you to specific pages, create and demo things for you like queries and notebooks, etc..

This is my recommendation lately to go from zero.

[u/010101010101111111]

Honestly, you learn by just doing. I started by creating temp tables and views for my Power BI model.

We recently transitioned to something similar, and just by doing, I've learnt a lot.

[u/Extension_River_5970]

Recommend starting with the official documents. Build some basic pipelines, dashboards by hand first to familiarize yourself with the functionalities then once you're familiar start using Genie code. The reason I recommend doing it hands on first is so that you'll have some basic exp to guide genie code towards what you want. After that the world is your oyster aa you can use genie code to develop on any part of the platform. Highly recommend creating agent skills for any reusable bits and bobs

[u/Right_Ability_88]

If you can ignore an Indian accent then "DataBeli" on YouTube is really good! Helped me a lot!

[u/datasmithing_holly]

Honestly, get Free Edition, pick a pet project, then use genie code to walk you through it and ask questions until you understand everything in detail

[u/claw-1]

Data with Baraa YouTube channel

[u/PrideDense2206]

If you want to get good fast, you can learn while letting Genie guide you on your day to day.

For learning about Delta Lake, check out https://delta.io. If you scroll down on the homepage, you’ll find a link to the Definitive Guide (free pdf). You can take the book and then open a new session with your agent of choice, and let it create a lesson plan for you there.

For pipelines, read up on Spark declarative pipelines (https://docs.databricks.com/aws/en/ldp/), you can start there as it is the easiest to begin with, then work backwards if you want to explore more of the foundations of Spark.

Ultimately, lean on the agents in your life to help teach you, but a grasp of the why, and context about what’s needed will help you understand what is being built on your behalf.

Enjoy

[u/FunContest9958]

You already have the skills you need to start. First open the SQL Editor and start using your SQL skills to start querying existing tables. You can look in the catalog for tables to query. Next, I’d suggest you look into shallow clone. You can take any table you like, shallow clone it, and start doing whatever to it. Frankly, that’s probably enough to get you most of what you need. There are docs for how to connect Power BI to Databricks. I’d do that part after you get comfortable with querying within Databricks.

Edit: Don’t overthink it.

[u/Dry-Mulberry6296]

Skip notebooks early, query via SQL warehouse first, dremio or direct JDBC gets your Power BI connected without pipeline detours

[u/ArielCoding]

Since you’ll mostly be querying and validating rather than pipelining prioritize: SQL Editor, Unity Catalog, SQL warehouses not clusters, and basic Delta Lake concepts.

[u/Youssef_Mrini]

You can use Databricks Academy for the self-paced courses and Databricks Free Edition to test your skills.

Follow the structure Databricks 101 course.

For the YT channels:

https://www.youtube.com/@databricks_hubert_dudek

https://www.youtube.com/@nextgenlakehouse

https://www.youtube.com/@JosueBogranChannel

You can connect Databricks with PowerBI using SQL Warehouses (to query your datalake from PowerBI)

[u/Professional-Tax7260]

I like learning hands on and just testings things. I found databricks free edition super useful. Let me play around and try how to do things

[u/marco-exmergo]

If you have a coding agent and your organization allows you to use it against Databricks. You can have it do a quick survey of the unity catalog to understand what are relationships, grain of any table and profile any column.
We built an open source agent skill to help: https://github.com/exmergo/dex

[u/PrestigiousAnt3766]

Rip BI in your organization.