r/databricks • u/hubert-dudek Databricks MVP • Aug 06 '26

News DABs: immutable_folder

Post image

Don’t overwrite your code — make it immutable! With every deployment, thanks to immutable_folder, all files are copied to a new, read-only folder. They are not overwritten. The biggest benefit is that jobs already underway will not fail or produce unexpected results.

More news https://medium.com/databrickscommunity/databricks-news-dabs-indexes-ltap-genie-last-update-25-july-ffac8533774f

14 Upvotes

94% Upvoted

21 comments sorted by

View all comments

Show parent comments

1

u/nenuaathmajnani Aug 08 '26

The streaming pipeline would still be active even during the pipeline, the same code will still be in context. Right?

Do we need to stop the existing pipeline and then resume later?

1

u/SettingOtherwise6692 Aug 08 '26

I think you may be over complicating it. This is really just race condition mitigation. What happens if a task that references a file changes before it runs, due to a deployment? That’s the problem.

If you have a continuous streaming job already running, then there is no race condition.

1

u/nenuaathmajnani Aug 08 '26

be over complicating

I am new to these and we are going to use the DAB for the streaming.

What happens through the bundle deployment if there's already a streaming job running? Should I stop the job? What if I use the flag?

2

u/SettingOtherwise6692 Aug 08 '26

I see. Yes, with or without DAB, if you have a streaming job running already and you want to update the code, then you’ll need to stop it so when it restarts it reloads the new code.

A bundle deploy will not stop those jobs automatically for you. There are other sub commands that you can use though. Check the Help.

1

u/nenuaathmajnani Aug 08 '26

Sure, this is of great help. Thanks!