What's New in Databricks Apps: AppKit, Git Deploys, OpenTelemetry and Scaling

Summary

Fully managed serverless runtime: Databricks Apps provides internal data and AI app development on governed data with support for major JavaScript and Python frameworks.
AppKit SDK: Purpose-built with plugin architecture, type-checked SQL, caching and retry logic enabling custom application development.
Production features: Git-based deployment from any branch, OpenTelemetry observability, agent apps with MLflow, horizontal scaling, and zero-downtime deployments for business-critical applications.

What's New in Databricks Apps: AppKit, Git Deploys, OpenTelemetry and Scaling

Talk by Evan Pandya, Product Marketing Manager, Databricks
Databricks Apps is a fully managed, serverless runtime for building internal data and AI apps directly on governed data, with support for major JavaScript and Python frameworks. This video is a fast tour of what is new across developer experience, production-readiness and governance: AppKit, the purpose-built SDK with a plugin architecture, type-checked SQL, caching and retry logic; Git-based deployment from any branch, tag or commit on GitHub, Bitbucket or GitLab; OpenTelemetry logs, metrics and traces; agent apps on the MLflow agent server; horizontal scaling; and zero-downtime deployments with health checks.
Evan Pandya, Product Marketing Manager for Databricks Apps, explains why apps complement notebooks and dashboards, then covers AppKit, the purpose-built SDK with a plugin architecture, type-checked SQL, caching and retry logic. He demos Git-based deployment from any branch, tag or commit on GitHub, Bitbucket or GitLab, with webhooks for CI/CD. You also see OpenTelemetry support that persists logs, metrics and traces to Unity Catalog or sends them to Datadog, Splunk or Grafana, plus agent app templates built on the MLflow agent server. The video closes with horizontal scaling, routing a user to the same instance, and zero-downtime deployments for business-critical apps.

Chapters

FAQs

What is AppKit in Databricks Apps?

AppKit is a purpose-built SDK with a plugin architecture, type-checked SQL, caching and retry logic that enables developers to build custom applications. It simplifies development by handling common patterns like SQL type safety and automatic retries.

How do I deploy Databricks Apps from Git?

Databricks Apps supports Git-based deployment from any branch, tag, or commit on GitHub, Bitbucket, or GitLab, with webhooks for CI/CD integration. This enables automated deployment workflows directly from your version control system.

What observability tools does Databricks Apps support?

This video highlights OpenTelemetry support that persists logs, metrics and traces to Unity Catalog or sends them to Datadog, Splunk or Grafana. This enables comprehensive monitoring of your application performance and behavior.

How does Databricks Apps handle high-traffic applications?

Databricks Apps provides horizontal scaling to handle high-traffic workloads, with features like routing users to the same instance for session consistency. It also supports zero-downtime deployments with health checks for business-critical applications.

Full transcript

`00:00` Introduction

My name is Evan Pundir. I'm the Product Marketing Manager for Databricks Apps, and today's session is a basic 100 on what we have new, available today: in GA, public release, or beta. So , before we start, I'd like to ask, please raise your hands for those who are new to Databricks Apps. Oh, cool. OK. And for those who are new, have you already created a prototype or is this your first time hearing about the product?
If you've built a prototype, raise your hand. Okay, great. Then that's perfect. These are a few new features. I'll start by talking a little about Databricks Apps as a product. Everyone O. I'm fine. It seems like there were some technical difficulties, but now... Most people seem to be fine, right? OK. I think everything is fine. Thank you. Steeply. Yes, I know many of you are familiar with laptops and dashboards, right?

`00:56` Why build apps instead of notebooks or dashboards?

So, why, why do we need apps? Why did we create Databricks Apps? The thing is , laptops and dashboards have slightly different uses, right? And both have small limitations, right? Laptops are more suitable for technical users, right? You have to run cells, and you can customize the workflow when you create a notebook, but you can't customize the interface, right? So it's a little intimidating for people who aren't technical or aren't used to data platforms or data products.
So you wouldn't hand over a laptop to someone in sales or the operations team, right? Dashboards, on the other hand, have a nice UX, but they're not customizable at all , right? They are passive and the user cannot click on them or perform any actions, right? So, apps, of course, combine the best of both worlds. You can customize them, right? You can use apps for real-world tasks, right? Whether it's writing data to a database like Lakehouse, using an agent or running an agent workflow, executing SQL queries, etc.
And, of course, you can make them very attractive and user -friendly for anyone, regardless of their technical background. Truth? So, apps... Last year, we officially launched Databricks Apps at the summit. So, to quickly explain the value of what we did: we built these apps with full native access to all of your protected and managed data, right? We brought applications to data, unlike in the past where you had to write your own APIs and find ways to feed data into your application, which was very difficult and expensive .

`02:32` Databricks Apps serverless runtime and supported frameworks

So now we have this fully managed, integrated, serverless runtime environment for you. Isn't that right? All of this is within the Databricks security perimeter. So we've made it really easy for you to configure what data these apps can access , as well as who in your organization can access that data or the apps themselves, right? And also, we are open, but we have our own strategy, right? What does this mean? We support all major JavaScript and Python frameworks, right?
We also have custom- designed SDKs that allow you to make more informed choices when building applications based on our best practices and experience gained through numerous iterations with customers. Class. So that's what we'll actually be talking about—new products. We invested in two key product categories: mainly in the developer experience and application hosting, or I would say the infrastructure part, which is the serverless platform.
And this is in addition to what we announced yesterday during the keynote, where we talked about the Genie app builder and the new app space management model . So, these are a few more things that are already available. Those things from the performance will be coming out very soon. So, for the developer experience, I'll talk about AppKit, our purpose- built SDK. We have new integrations with GitHub and Git in general, including Bitbucket and GitLab.
We added surveillance tools. They are based on OpenTelemetry. I will explain this in more detail. And as for hosting, we have our own agents in the applications, which is pretty cool. Additionally, we have horizontal scaling to support more real- time users in your applications. Class. So let's start with AppKit. This is truly a wonderful thing. I've already worked with him a little. It's made my life a lot easier when I'm building small apps, because I, well, work in marketing, you know?

`04:08` AppKit: the purpose-built SDK with a plugin architecture

So sometimes I need to create a small application . This really saved me a lot of time. It is now in general availability (GA ), so it is ready for production use. The main design principle is that we built AppKit so that you can plug it into your programming assistant or any Agnetic IDE, regardless of how you typically develop code, and then point it to AppKit. Then you get this really nice and robust plugin architecture .
This provides modular connections so you can work with Databricks analytics, Genie, our file system, Lakehouse— in fact, all of Databricks’ key resources. This makes it easy to use plain English to communicate with a programming assistant who will be able to properly understand the context of Databricks. In addition, it very intelligently generates type-checked SQL. Is n't that right? Since it's mostly TypeScript, it avoids many of the runtime errors that can occur when you don't use the SDK with programming agents.
They make many such mistakes in SQL, and it can be very annoying. So we wanted to create a system that would help you avoid encountering these annoying errors all the time. And, most importantly, you get a lot of advanced features with AppKit. Isn't that right? These are things like caching, setting up telemetry or observability monitoring, retry logic , error handling. This is something that would otherwise require you to have more knowledge or experience in software engineering .
We solve this with an SDK because a lot of people here are data scientists, right? We don't necessarily have these skills or want to develop them when we have agents who are so good at it . You can check this out by simply finding the Databricks App Kit. We have very good documentation and it is available on GitHub. I'll give you a quick example. You saw us at the keynote yesterday. This is the Genie App Builder interface.
You see, when you just click " connect to Databricks resources"—that's all App Kit's work on the backend. Isn't that right? You connect to your SQL repository, model service endpoint, Lakehouse— all your data and AI. All of this is actually coordinated by App Kit under the hood. I just wanted to show that this is the Genie App Builder interface, but you can of course do this with any other programming agent or Agnetic IDE you want.
Class, the next thing— it's a small but powerful feature, I would say, right? These are applications supported by Git. So now we have integration with GitHub. You can deploy code directly from a Git repository by specifying any branch, tag, or commit. This is very convenient because you don't have to work with the Databricks file system, which most people are less familiar with than Git, which is the industry standard.

`06:36` Deploy from Git with GitHub, Bitbucket or GitLab

This greatly simplifies the process. We are also extending this feature by adding automatic deployment using webhooks: as you work on code and push it to the repository, Databricks can automatically update your application. This is a great tool for CI/CD. I can show you how it works in the app itself. Everything is quite simple here. Let's say I want to commit my application code to a Git repository. In this case, it's a simple Dash-based sales analyzer.
We use the Dash framework to create a sales analyzer . You can see that this is a private repository, yes, it is indeed a private repository. This is important because we support authentication. If you go to Databricks Apps and want to create a new app, you click here. It's in the user interface. Let's name it the same as it is named in our GitHub repository. And this is the easiest step. You just need to paste the URL of your Git repository.
And this is the choice of the Git provider. Currently it's GitHub, but you can choose Bitbucket, GitLab, or whatever service you use. You click here, and you see a little yellow message... What does it say? Authentication. Yes, Git credentials are required. We simplify this process as much as possible. Since this is a private repository, you need to authenticate to continue, so just click here. You can link accounts with one click.
You can log in. This links your account to... So, you see, it's pretty simple, right? Using Git is actually easy. And the main idea is to give you ... Oh yeah, during deployment you can also choose the branch you want. In this case, we are deploying from the main branch, so let's just run the deployment. And in the backend, we initialize the infrastructure, and then you can click on the URL that is automatically generated.
Here, please. You open it, and now you have your little... It's a very, very simple app. I just wanted to show how easy it is to use Git now. The overall idea is that we try to give you the development experience you expect from an application building platform, all on Databricks. This is really important. We get a lot of requests about this from customers. As your applications become more mature and production-ready, you really need to add monitoring tools to maintain stable performance, right?
So, you will be able to monitor the performance of your applications and improve them accordingly . By default, every Databricks application already has a logs tab for debugging at runtime, but these logs are not persistent. So we decided to address this shortcoming, and now we support the OpenTelemetry protocol, which is the largest and most widely used open source surveillance framework. This is very cool. It has huge support in the open source community.

`09:29` Observability with OpenTelemetry logs, metrics and traces

Once you set this up, your apps will automatically record log data, metrics, and span data. Span data is combined to create traces. This data is automatically recorded in the Unity Catalog , where you can index it and even query Genie about it to help you debug your applications in real time. Additionally, we support sending data instrumented through OpenTelemetry to other platforms. If you use DataDog, Splunk, or Grafana, you can send this data to your monitoring provider if you are already paying for it.
We want to make sure the system remains open for you. I'll show you that too. This is again in the product, I want to make it clear. So this is the logs tab, they are there by default, but they are pretty simple. This is something like standard output or errors. So, if you want to configure OpenTelemetry, you should go back to Yes. Does this work? Okay, let's go. Thank you. So, so we'll go over here, and here we have OpenTelemetry.
It's very simple. You just need to add it, right? And then from there you go here. You choose the schema for your Unity Catalog that you want to write this telemetry data to. We can call this table like this. This is a hotel application on Flask. Let's just call it flask. Let's keep this. And in just a few clicks, we configure the ability to keep these logs even after application crashes, right? So, we are recording them again in the Unity Catalog volume.
So when I redeploy this to generate new logs, I just wanted to show what it means to have real log data. Let's go to the Unity Catalog, and you'll see that we have all this log data here. We see the contents of the log. There is additional information there. And to configure spans used to create traces as well as metrics, we support automatic OpenTelemetry instrumentation, which is very simple. It's about seven lines of YAML that you can simply copy and paste.
And there are a few dependencies that need to be installed, for example, for exporters to send telemetry data. And then you can get these spans and metrics in Unity Catalog on an ongoing basis. This is just one extra step. And our documentation is very clear, clear enough even for me to work through it and make everything work. So, yeah, now in the next part we're going to talk about hosting improvements or just our infrastructure improvements.
And one of the main ones is the ability to deploy agents and user agents in applications. Right? This means you can create an AI agent and deploy it through applications, instead of going through a model service where you only get an API endpoint. This gives you a little more control over the agent code, server configuration, and deployment workflow. Let's say you want to continuously develop your agent code using an application development workflow in your IDE of choice.

`12:49` Agent apps and templates with the MLflow agent server

This greatly simplifies the process compared to using a model service. So, the way to get started with this is we have application agent templates, which I'll show you on the next slide, but I also want to explain what's on the screen. We use the MLFlow agent server, which handles all agent requests, and it has built-in tracing and logging, so you get transparency into what the agent is actually doing. And we also have the “Responses Agent”—it’s actually a wrapper that supports your agent regardless of the framework you choose, whether it’s OpenAI SDK or LangGraph.
We automatically wrap it in a “ Responses Agent,” which provides streaming support, multi-agent work, and compatibility with AI platforms and agent evaluation. So, thanks to this wrapper, you get much more options for working with the agent and improving it. This is already a generally available (GA) feature, so it's ready to be used in a production environment, and many customers are already actively using it today.
I want to quickly show you the application interface, where you will see the " Agents" tab. You can click on any of these templates. You can even create your own MCP server very easily. This template is designed to streamline this process. We clicked on the OpenAI SDK. The ML experiment allows us to track these agents, right? We get an idea of what happens during requests: what the requests themselves are and how the agent processes them.
What actions does he take, and everything is in this spirit. Let's call this experiment "demo 1 2 3", hooray! We just needed a place to record traces for an experiment in MLflow. Let's give it a name, and there we go—it's just a simple template deployment. So, everything is very simple. Of course, you'll probably want to edit the code in your IDE, click a button, and make it do something. But I just wanted to show how easy it is to get started with these new templates.
Last but not least, horizontal scaling, which is currently in beta; It supports up to five instances per application under one load balancer and one URL. Right? What is it and why do we need it? This is necessary to prevent the application from slowing down when there are a large number of simultaneous users. Horizontal scaling distributes incoming traffic across copies of your instance to avoid bottlenecks. This means that if one instance fails or “ crashes” for any reason, you can simply redirect traffic to another available instance.

`15:17` Horizontal scaling for high-traffic apps

This is very important as your applications become larger and more business-critical. Let's say this is a sales "co- pilot" for a large outside sales team. If the system goes down , you want to be sure that stopping one instance won't "bring down" the entire application for all users. So we see a lot of customers with more mature applications that really need this. That's why we created it. We also have such a thing as session affinity or “sticky” sessions.
This is really difficult to implement. The engineering work here is truly impressive. But essentially this means that you can direct a request from one user to the same instance. This way, you can store this state and keep the data in memory or in cache. So you don't lose all your progress when the system redirects requests on the backend. Your user experience remains consistent by saving all the cookies you created while using the app.
And another thing that horizontal scaling provides is deployment without downtime. So when you push an update to your application, Databricks can first deploy it to one instance of the build, and then redirect traffic to other instances only after that build passes health checks. So , your existing instances continue to serve users all this time. The main idea is that if you want to redeploy an application, you don't want to interrupt any users who are currently actively using it.

`16:32` Zero-downtime deployments with health checks

So , yes, this is a stable deployment with no downtime. Steeply. So, yes, a call to action. This is, again, a short blitz presentation. I just want to tell you what we've been working on. This is an addition to what we talked about in the main session, and this is what lies ahead for us. All of this is already available. I hope I've inspired many of you to create your first app. It's not scary at all. With App Kit, again, it's very easy to connect your coding agent to the SDK and you can start building these apps with real data.
Explore these new opportunities. They are interesting. It's useful to practice learning how to create them. Some of them are designed for mature applications ready for industrial use, but understanding how monitoring works and deploying an agent from a template is quite an exciting experience. I encourage you to try it. And yes, reach out to your customer support teams if you're interested in creating these real-world demos.
Or now we have our specialists—I mean, it's very easy to build apps now . It's easy enough to work with a solution architect to just say, " Hey, we want to solve this problem with an app." Can you create it? So they can explain to you how the process works and teach you how to do it yourself. Um, yes, that's actually all. I hope you found it interesting. Um, this is the last day of the conference, so have a safe journey home.
That's all I wanted to say today. Um, but I have 2 minutes, so if anyone wants to talk to me or ask a question, I'll be here. So, yeah, thank you for coming, and bye.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.