How to Connect Your Coding Agent and IDE to Databricks Compute | Databricks SSH Tunnel Demo

Summary

• SSH tunnel for IDEs: Connect your local VS Code, Cursor, or coding agents like Claude Code to Databricks workspace via SSH tunnel, enabling remote code execution while maintaining full context of workspace files.
• Automatic sync: Files edited in your local IDE automatically sync with the Databricks workspace, eliminating manual file syncing and allowing coding agents full visibility into workspace resources.
• Compute flexibility: Configure default serverless or GPU compute, connect to existing clusters, track costs with usage policy IDs, and set up base environments to avoid reinstalling dependencies on each connection.

How to Connect Your Coding Agent and IDE to Databricks Compute | Databricks SSH Tunnel Demo

Connect your local IDE or terminal coding agent to your Databricks workspace with an SSH tunnel — write code in VS Code, Cursor, or an agent like Claude Code, and run it on Databricks compute. This demo walks through connecting with the Databricks CLI and the Databricks extension, so your coding agents get full context of your workspace files.
No more switching back and forth or syncing files by hand: your local editor stays in sync with your workspace automatically. You'll also see the advanced options — picking default serverless or GPU compute, connecting to an existing cluster, tracking costs with a usage policy ID, and setting up a base environment so you don't reinstall dependencies every session.
Try it on a free Databricks workspace: https://www.databricks.com/learn/free-edition
Read the docs: https://docs.databricks.com/aws/en/dev-tools/ssh-tunnel
Read the docs: https://docs.databricks.com/aws/en/dev-tools/ssh-tunnel

FAQs

How does SSH tunnel enable local editing?

The SSH tunnel creates a secure connection between your IDE and Databricks. Write code locally but execute on Databricks compute. Files automatically sync bidirectionally without manual uploads.

What coding agents can connect?

VS Code, Cursor, and coding agents like Claude Code can all connect via SSH. These agents get full context of workspace files and manage code within a single environment.

How do you connect your IDE?

Use Databricks CLI (version 1.5.0+) to run 'Databricks SSH connect' with optional IDE flags. The CLI handles authentication and establishes the tunnel automatically.

What compute options are available?

Set default serverless or GPU compute, connect to existing clusters, configure usage policy IDs for cost tracking, and define base environments with pre-installed dependencies.

Full transcript

The Databricks workspace is an amazing environment for data analysis and data coding agent like Cloud Code or Codex. But, what that means is you end up switching back and forth between your Databricks workspace and your IDE constantly. So, this is why I'm really excited about a new Databricks feature that allows you to connect to your workspace using an SSH tunnel. This allows you to use your local IDE and coding agents, while also accessing your workspace at the same time.
This means you no longer have to worry about syncing files back and forth, and your coding agents will have full context of your workspace files. Let me show you how it works. First off, a few prerequisites. If you want to follow along, you need to have a Databricks workspace and Once you have a workspace, the next step is to make sure you have a compatible CLI, which is version 1. 5. 0 or above. You can check this by using the version flag.
We'll also want to make sure we're logged in. We can do this by running Databricks auth login. >> To connect to your workspace using SSH is a simple terminal command, Databricks SSH connect. You can optionally pass in a named profile if you're not using the default. In our case, we are, so we're going to leave this off. Once connected, we can list the files in our workspace. As you can see here, we have a file. txt file.
If we go over to our workspace on the left-hand side here, you see that file also appears in our UI. Currently, the contents of the file is blank. Over in our terminal session, we can edit that file. And we'll see that the contents also update in our workspace. Of course, editing the file using them in your terminal probably isn't the experience you're looking for. So, next up, let's take a look at how we can connect to our workspace using an IDE.
There are two methods for connecting your IDE to your workspace. You can either use the Databricks CLI or the Databricks extension in your IDE. Let's first start with the CLI. When connecting to your workspace using SSH, we can optionally pass in an IDE flag, which can either be VS Code or Cursor. As you can see, we have the same files we had before in our workspace. Let's create a new Python file named hello. py. In this file, we'll write a simple hello world script.
The important thing to note here is that although we're making edits in our local editor, the files are remote in our Databricks workspace. And running this our IDE, the agents available in our IDE have full visibility of our workspace files. For example, this means that we can tell the agent to list files in our workspace. And of course, the agent can also make edits to our files. So, let's ask the agent to change our hello world script to say hello from IDE instead.
As you can see, connecting our IDE to our workspace is very powerful because we get the full genetic power of our IDE, including changing between all the various LM models that we have available. And remember, all the files we created are automatically in sync with our workspace. So, in the demo I just showed, we were connecting to our workspace by launching the IDE with a terminal command. But, as I mentioned earlier, there's another convenient method for connecting to your workspace directly from your IDE using the Databricks extension.
Let me show you. In both Cursor and VS Code, the Databricks extension is available in the extensions marketplace. This extension will add a Databricks sidebar, which includes a button to start an SSH tunnel. Clicking on this button will prompt us to select or one with a GPU for heavier AI and machine learning workloads. For this demo, we'll stick with the default. Once again, we're in the same workspace with the files we created previously.
And if we go back to our Databricks extension, we can see that we now have a browsable view of our Unity Catalog. So, we've covered the basics of how to connect your workspace using either a terminal command or through an IDE. Now, I wanted to cover some advanced options you have available when connecting to your workspace. When connecting using the IDE earlier, we saw that we had the option to choose between a default serverless machine or one with a GPU.
This can also be done by passing an accelerator flag and specifying pass in your cluster ID. And as with many Databricks features, we've built in a way to track costs. For SSH, we have a usage policy ID flag, which you can pass to your session, so you can see who this compute should be billed to. More details on all these flags and more can be found by running the help flag. And just one last very useful feature I want to mention is how to set up a base environment when connecting via SSH.
This means that you end up reinstalling dependencies every single time you connect to your workspace. Setting up a base environment will solve this problem. Let me show you an example. In the same Databricks workspace, I've created a Python notebook that imports the calce library and we haven't installed the calce library yet. And this is expected because we don't have the library as part of our workspace environment.
To fix this, we want to create a base environment. This is as simple as creating a YAML file where we list the dependencies we want to have in our environment. Then in our workspace, we can go to settings, compute, and choose the option to manage our workspace base environments. Here we have the option to create a new base Once the environment has been created, we can go back to our terminal. Here we can use the base environment flag When we run the Python notebook we should see that everything succeeds because we have the correct dependencies in our environment.
And that's the end of this demo. I hope this video has shown you how useful it can be to connect to your Databricks workspace using an SSH tunnel. If you would like to learn more, please check out our official documentation, which will be linked to this video. Thank you.

Learn more about the Databricks Data and AI platform.