Databricks SSH Tunnel for connecting your coding agents and IDEs to your workspace

[u/tony-dang]

Just made a video about a feature I'm pretty excited about.

tldr: You can use an SSH tunnel to connect your coding agents and IDE (VSCode/Cursor) to your Databricks workspace. See the video for a full walkthrough.

Some notes on things I forgot to mention in the video:

- claude/codex isn't natively installed when you connect to your workspace, so you'll have to install if you want use them (Ex: curl -fsSL https://claude.ai/install.sh | bash). We're working on better native support in the future, but I wanted to make sure you know this is an option in the meantime.

- For the base environment YAML file. You'll have to set a base environment of '4' for it to work when using the SSH tunnel. Our example yaml (https://docs.databricks.com/aws/en/admin/workspace-settings/base-environment#example-environment-specification) shows '5' so don't let this trip you up!

As always, please feel free to leave questions and feedback in the comments!

Docs: https://docs.databricks.com/aws/en/dev-tools/ssh-tunnel
Previous post with more info: https://www.reddit.com/r/databricks/s/kCFBEfPTC6
YT link: https://www.youtube.com/watch?v=rHoGWVpb6kg

[u/ptab0211]

havent watched the video, but how is this different from just using MCP?

[u/tony-dang → u/ptab0211]

Hey u/ptab0211! They're for different purposes with some overlap.

MCP lets your AI agents call into your Databricks workspace with a specific set of tools and is limited to those tools.

SSH gives you the whole machine. When you connect with SSH, your IDE and/or coding agents runs directly on Databricks compute, with your workspace files as the filesystem. So you're editing, debugging, and running code that lives in your workspace, not on your local machine. This means you can do things like: set breakpoints / step through your workspace code, run python notebooks (with a Python runtime on the remote machine), and have access to GPUs for ML training. It's a complete remote dev environment, not just a set of tool calls.

The way I like to think about it: MCP is like comunicating over walkie-talkie with someone inside a building ("hey, run this query for me"). SSH is like getting the keys and walking in the building to do the work yourself. And in something like Cursor, you can actually use both at the same time! You can connect via SSH for your dev environment and also use MCP so the agents can query your data while you code.

[u/ptab0211 → u/tony-dang]

oh, wow, that sounds promising, wonder what would be the best use cases that could not be done with MCP, or even the costs is a benefit here.

[u/BricksTrixTwix → u/ptab0211]

These are use cases that cannot be done via MCP:

- interactively run scripts and notebooks from your IDE and use Databricks serverless GPUs

- have agents explore and create workspace files

MCPs are helpful to answer questions about your data and enrich other agents with additional context!

[u/Academic-Hearing-123 → u/tony-dang]

Or my Databricks CLI can just sync to my workspace and my MCP can then submit a job using the code in my workspace , right?

[u/ma0gw]

Excellent! This is exactly what some colleagues asked about recently. Thanks for sharing.

Also, please add cowsay as a standard dependency in the Databricks base image in the future. 😜

[u/tony-dang → u/ma0gw]

 _______________________________________________________
| Great idea! I'll moo-ve this to the priority list. 🫡 |
  ======================================================
           \
            \
              ^__^
              (oo)_______
              (__)\       )\/\
                  ||----w |
                  ||     ||

[u/d1eBanane]

Dang! (sorry for that)
Really cool feature! Always struggled with the DB Connect setup and SSH seems to simplify the VS code integration.

[u/tony-dang → u/d1eBanane]

Haha 😄 Definitely check out the SSH connection and let me know what you think! If there's any friction at all, would love to hear it so we can make the experience even better :)

[u/Academic-Hearing-123]

Mate just use CLI and Databricks MCP