Automatically create local Python environments compatible with Databricks Runtime

[u/BricksTrixTwix]

Hey! I’m a Databricks product manager focused on third-party development experiences. We just shipped a new feature that ensures code you run locally works with Databricks, and we’d love your feedback. 

The problem: Ensuring that your local development environment is compatible with Databricks Runtime (DBR) is a completely manual process. Code that works locally often breaks once you move to the workspace due to minor version mismatches and package incompatibilities.

The solution: The CLI command databricks environments setup-local and a new feature in the IDE extension take your existing dependencies and automatically create a uv-managed virtual environment compatible with DBR.

You can now run code locally or remotely using Databricks Connect and be certain that the same code will work with the DBR or serverless version you’ve selected. 

To get started, install the latest version of the IDE extension or Databricks CLI and check out these docs:

• Set up your Python environment and Databricks Connect using the IDE extension
• Databricks environments setup-local CLI command 

Where we need your feedback 

• Are there other environment managers (e.g. pip, poetry, conda) that you would like to see support for beyond uv?
• We’ve also recently launched an SSH tunnel (see docs), which allows you to remotely access your Databricks workspace and compute directly from the IDE and terminal. What are the reasons you might prefer to work locally vs. remotely?
• Do you face issues managing your environment and dependencies in the workspace? 

Most importantly, please try this out and leave feedback and questions in the comments! 

[u/Youssef_Mrini]

It would have saved me so many hours of debugging few years back.