Skip to main content

Deploying Databricks Asset Bundles (DABs) at Scale

Summary

  • Git-driven declarative deployment: Databricks Asset Bundles enable version-controlled, infrastructure-as-code workflows for deploying notebooks, jobs, DLT pipelines, dashboards, and ML models across environments.
  • CI/CD automation and governance: GitHub Actions and Azure DevOps integration automate deployments while managing Dev vs. Prod differences and ensuring reproducibility across teams.
  • Python interface and workspace UI: New features include Pythonic APIs for bundle definition, workspace-based deployment, and support for AI/BI dashboards and Databricks Apps.

Deploying Databricks Asset Bundles (DABs) at Scale

Watch: Deploying Databricks Asset Bundles (DABs) at Scale
Talk by Pieter Noordhuis, Sr. Staff Software Engineer, Databricks ; Saad Ansari, Product Management, Databricks
Managing data and AI workloads in Databricks can be complex. Databricks Asset Bundles (DABs) simplify this by enabling declarative, Git-driven deployment workflows for notebooks, jobs, DLT pipelines, dashboards, ML models and more. Join the DABs Team for a Deep Dive and learn about: The Basics: Understanding Databricks asset bundles Declare, define and deploy assets, follow best practices, use templates and manage dependencies CI/CD & Governance: Automate deployments with GitHub Actions/Azure DevOps, manage Dev vs. Prod differences, and ensure reproducibility What’s new and what's coming up! AI/BI Dashboard support, Databricks Apps support, a Pythonic interface and workspace-based deployment If you're a data engineer, ML practitioner or platform architect, this talk will provide practical insights to improve reliability, efficiency and compliance in your Databricks workflows.
Here’s more to explore:
Production ready data pipelines for analytics and AI: https://www.databricks.com/solutions/data-engineering
The Big Book of Data Engineering: https://www.databricks.com/resources/ebook/big-book-data-engineering-2nd-edition
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

Chapters

FAQs

What problem does Databricks Asset Bundles solve?

DABs simplify managing complex data and AI workloads by enabling declarative, version-controlled deployment workflows—eliminating manual configuration and reducing deployment errors.

How do DABs integrate with CI/CD pipelines?

DABs can be deployed automatically using GitHub Actions or Azure DevOps, enabling teams to automate testing, approval workflows, and production deployments with consistent governance.

Can DABs manage environment differences between Dev and Prod?

Yes, DABs support configuration overrides and environment-specific settings, allowing teams to use the same bundle definition across development, staging, and production workspaces.

What assets can be managed with Databricks Asset Bundles?

DABs support notebooks, jobs, DLT pipelines, dashboards, ML models, Databricks Apps, and more—enabling end-to-end infrastructure-as-code management for analytics and AI workloads.

Full transcript

0:00 Introduction to DABs

All right. Hello. Uh, can folks hear me? Good to see very some familiar faces in the crowd. Thank you for joining us. Uh, let's, uh, fill up the room here. We've got space on this side as well. Uh, I know it's a full house, so really appreciate uh, everybody making the journey here. I'm a product manager uh, at Data Bricks. work with my colleague here. Um, and uh, we're going to talk about DABS or data bricks asset bundles uh, today.
So, if that's what you came here for, you're in the right room. Um, you'll see this slide in virtually every presentation today. Uh, so just keep this in mind that uh, we are among friends. uh we'll share as much as we can uh with uh folks here, but um things might change a little bit. Uh but just keep uh keep this in mind. Uh so thank you for accommodating that. And then uh just to make sure you're in the right room, um we'll cover the what, why, and how of uh data bricks asset bundles.
And uh we'll go into some level of detail, but don't worry, we'll take everybody along for the ride. We'll go through some of the best practices uh on what we've learned uh from customers, what we've actually built into the platform and obviously the goodies uh all the new stuff that we've been working on and then obviously talking a little bit about u what the future is. So we'll gaze into the crystal ball. But before we begin uh maybe by a show of hands uh how many people here already are familiar with uh data bricks asset bundles?
So, okay. Of course, it's in the title. Yeah, I know. That's that's um how many have actually used them? Okay, keep your hands up. Uh and how many uh are using it in production? Wow. Okay, we have to bring our a game. Yeah. And uh how many have 100% of everything on DABs? Wow. All right. Nice. So, maybe you should uh we might ask some of you to come up here and give the talk. But anyway, enough talking. I'll uh I'll hand it over to uh take us through this.
All right. Yeah, this this is it's great to see a lot of lot of folks familiar with DABs. We're going to cover some of some of the basics for folks who are not yet familiar with DABs. So, um uh I hope there are maybe some new bits for you in there as well. Maybe not. Uh but let's let's see where where it goes. Um so, the what, why, and how. How why why did we build data access bundles in the first place? Well, as software engineers, there's a couple practices that you've gotten used to.

2:18 Why and how of bundles

Maybe in the '9s, you would use FTP servers and move code around, but that's like not happening for a long time anymore, right? Like we all use GitHub, GitLab, uh Azure DevOps or other tools to store our code and do uh version control, right? Um we do that so that we have a history of all the changes that went in so that we can effectively collaborate with our teams and get code review on the changes that we um that we propose.
So we want to apply those same practices to uh project data projects that you that you build on top of data that you uh yeah that when working with data bricks you should be able to apply those same practices. Another one of them is is testing right I'm sure everyone has built the odd notebook where you encode everything in a single notebook and you just hit start and you hope for the best. Um, but that doesn't really scale if you have a larger team that you're working with.
And encoding the assumptions that you make about your code or the environment or the input data sets in tests is actually a very effective way to uh to make sure you don't regress over over time. Automated deployment is another one. Like sure, you can copy code between two workspaces or multiple folders. Uh, but how about those jobs or the pipelines that you associate with them, right? you want to take everything that's associated with your data project and deploy that or be able to deploy that to deploy multiple copies uh potentially to different workspaces.
And then the last one is about environment separation. If you uh are working with a colleague, let's say and I are both working on a data project um and I'm working on some new feature. I don't want SAD doing some development actually clobbering my tables and vice versa or that I'm writing some changes in my code and uh sad changes are interfering with mine. Right? We want individual independent copies to do that uh development and then send those changes off for code review to um yeah to get a second pair of eyes and and proceed with the whole flow.
So in the context of data bricks what does this mean? We want to have a reproducible definition of a of a project and that means the code so the notebooks the Python files the dashboards maybe some alerts or some other other things that are part of the workspace we want to take that um we want to get the resource configuration so perhaps you've scheduled a job for uh your code perhaps you have created a pipeline or 10 and they're all part of this definition of your project we want to take that as well that configuration and then lastly we also want want to know about the permissioning.
Maybe you want to share all of these resources like with a with a view permission to all of your teams like, "Hey, I'm developing something. Can you have a quick look?" Right? It's effective to then share all of this stuff as a as a whole with your with your team. Now, that definition needs to be commitable, right? It needs to materialize to a set of files because then we can use version control uh to keep history, collaborate, and it needs to be a portable definition.
Sure, I can hardcode a bunch of ids and URLs and whatnot in it, but that means it's not portable between workspaces. So, we need to be able to make it make it portable so we can target that definition to multiple workspaces. And then lastly, this this um uh definition, we want to be able to interact with it from um from multiple places, right? If I have a local laptop and I'm writing my Python code uh in VS Code, let's say on my on my laptop, I want to edit those source files and interact with that deployment as well, just like doing it from a CI runner or um uh or elsewhere.
So, a typical flow for this collaborative scenario would would look like this, right? So, I'm I have a project s and I have a project. It's chugging along nicely, but we got a feature request. So I'm going to create a feature branch and to start off my work I'm going to deploy a personal copy of this of this project. um and I get an isolated environment again because if I do some iterative work, I don't want to be touching my production tables, right?
I want to write to some development time tables such that well I can make mistakes like like it's it's fine to uh u play around and make sure to iterate before you're you're happy with your changes. But that needs to be an isolation of other people or production. And that holds for all of the all of my team members. If they get started with a new change, they they get their own copy of the of the project. Then when I'm happy, I'll make a pull request perhaps in GitHub Parlament uh in GitLab that's a little different or Ashure DevOps, but the concept is the same.
I propose my changes and then as part of that proposal of changes I'd like some automation to go out and create another deployment because then on that deployment I can maybe run uh the pipeline again confirm with may perhaps data quality checks perhaps with other assertions that this proposed change actually works end to end. Um again an isolated catalog and then once we get the green check marks everyone's happy with the change uh we can release it to like a main branch or release branch and do another uh authorative deployment to our production u production deployment.
So so far I haven't mentioned dabs. So what is dabs? It's a YAML based configuration format that allows you to capture the workspace settings, workspace uh configuration, permissioning all of the resources uh that you have under manage management like jobs, pipelines, but also Unity catalog resources like schemas or volumes. um the code like this configuration uh format or these files live side by side with the code that you're using right that can be just a couple of notebooks or it can be many different Python wheels.

7:49 Data bricks asset bundles history

It doesn't really matter how you lay out your code and we want to be able to support environment specific overrides. So if I have a cluster in dev maybe I want to use only two workers because I'm not processing that much data but in pro I want to have 32 workers then I don't want to well copy that whole definition I just want to do a very targeted override um why is this useful it brings structure and automation to projects makes collabor collaboration easier like no risky manual deployments no room for human error because all these deployments are based off of the spec right the definition these configuration files.
The entry point for a data asset bundle is always a file called dataix. yaml that contains uh some um some some stanas that we'll look at uh in a little bit as well. You can deploy these data bricks asset bundles through our data cli through CI/CD or uh the workspace UI as as we'll see in a little bit. And um we also have template support. So if you have like a data engineering team or data platform team at your company, maybe they want to be prescriptive on the type of projects that people build and code saying defaults and uh and use use templating uh for that.
But I'll hand it back to for a second to talk about how it's going so far. Awesome. Thank you. So um one of the things that you know this question deep does come up a lot. It's like hey can I take a dependency on this and is this thing GA? And um I just wanted to take a little moment to talk about the history of what happened. So uh we ran a preview uh we uh with uh one customer for about a year. Uh there was a lot of foundational elements that we were building.
Obviously we're building with the CLI authentication all of that comes with it and then uh we ran a fairly long uh public preview and this is very uncommon for data bricks because we like to move fast. But here we were building a pretty foundational piece uh that we wanted to make sure was as future proof as possible. We ged last year uh in April and uh now we're basically in the run phase and what that means if you so all my charts are sort of uh go to the right and up you know it's uh uh how we want to see this but what's been happening is like a very steady clip of updates that have been going on.
This is just the highlight reel, but the key point here is that it is uh it's on like we're really really uh launching features supporting new asset types, more capabilities. Uh we'll cover some of them in here. And an interesting thing, we were looking at some of our numbers and 70% of the users are actually human. Um and to any other crowd, people would uh be like, what do you mean 70% of users are human? uh but this crowd obviously knows there's a lot of automation service principles but that means that people use this in their day-to-day uh with uh their inner dev loop um so yeah so definitely this thing is GA it's ready for production so if you're using it great continue using it and then let's maybe do a little uh walk through here walk through excellent so so far uh high level concepts uh but let's see how the rubber meets the road so I have here open um a database YAML file and I'll just talk a little bit about the different stanzas that we see here and also different different resources that are u part of this bundle.

11:19 Bundle configuration walk-through

Um it's a bit of a demo like a little bit hello world that does some some processing with the NYC taxi trips data sets. Uh but for demonstration purposes that's fine. So a data file always uh the location of it is what we call the bundle root and all the code needs to be nested under that bundle root typically right. So all of the code that is in that folder where data. l is located will be synchronized into the workspace and will be able to reference that from your jobs and pipelines and so forth.
Now the data. yaml is only the entry point and we we we call it the entry point because you can actually include other uh configuration uh files. We do this because well it's better to mod modularize this. Maybe you want to have one file per resource or uh have secure variable definitions in another file or maybe reuse some configuration between multiple bundles, right? And maybe you have a common set of resources you want to reuse across bundles.
Uh using multiple files makes that much easier as opposed to just sticking everything in a single file. So, uh let's take a look at one of these resources. So, uh we're going to take a look at a job. And uh what we see here is that we have this resources block. Then we have a jobs block. And then we have uh a job that we call by key here. The NYC taxi job. The entire payload that you see here is one one to one maps one to one to the rest APIs.
So if you ever wonder what are the supported fields uh will like a certain new field in the API when is it going to be available in dabs? Well, the answer is as soon as we got a new CLI release because this supply chain of new fields being specified in our API specs and uh going into DABs is completely uh transparent. Um this job will run daily and again this is just how the the jobs API looks. So uh you can kind of infer from that we have a set of parameters that we define.
So job parameters in this case we pass a catalog and a schema parameter. The catalog is hardcoded in this case. But the interesting part is the schema parameter not hardcoded. Uh we actually refer to another resource here which is a schema and the name of that schema. Now if s and I both do a deployment that name will be different. Right? So this job will actually refer to the s development schema or the peter development schema when we deploy depending on the identity of at the person doing the deploy.
Um we also have a couple of tasks. Uh we run a pipeline and again this pipeline ID uh if you look at any materialized uh job this pipeline ID will be an ID, right? It will be UU ID. But that doesn't make our configuration portable, right? And that's the property we were talking about a little bit before. This needs to be portable. So we refer to the ID by the uh resource specifier here and we can open it it later and it will be materialized at deploy time as well.
So in order to deploy this job, we'll first deploy the pipeline, get the ID, interpolate it here, and then deploy the job. Um we can commute compute some anomalies. Uh the interesting part here is that we refer to uh a notebook path by relative path. So this YAML file is located under resources. So we can do/source and uh index it there. At deployment time, this will expand into the full path in the in the workspace.
Right? So this this makes it again better portable. um uh for uh yeah for different people or different deployments because the full paths will be different for and myself. And then lastly uh we'd like to refresh a dashboard, right? We do a little bit of data processing, we we figure out some anomalies and then we want to refresh a a dashboard and um yeah, the similar similar arguments hold there. If you look at um you may have already seen that when I hover you get a little bit of description and that's also again onetoone with the API definitions.
It also means that we get this neat autocomp completion where we can see what are the different tasks or properties that we can specify here. Uh but I it's just kind of nice to have but I did want to call it out. Uh I won't go over the other resources in this bundle uh because uh well I don't want to run out of time. Um, so let's move on. Uh, bundle variables. What are they? Well, you can hardcode a bunch of things in your bundle definition, but some things you actually don't know when you're uh writing your bundle and they need to be injected at uh deployment time or they need to be different for your development or production uh uh yeah target deployments.
And uh that's where variables come in. uh they can either be uh set uh here like like here uh and take a default in the configuration. You can use environment variables or uh command line flags to set them. But another flavor are uh lookup lookup variables. So some resources in a workspace will always be kind of shared or common resources, right? We expect them to be created by uh administrators. Maybe these are shared uh SQL warehouses, shared all-purpose clusters.
Maybe there's cluster policy IDs or stuff like that. There will be one instance of each in every workspace that you maintain. But again, that portability is very important. So instead of hard- coding those IDs in your bundle configuration, which would turn that bundle configuration tied to a specific workspace, we support lookups by name. So as long as you use the same name to refer to these resources, we'll do that translation again at deployment time. um permissions.
Uh mentioned it before, I'll I'll skip over this actually quite quite quick because yeah, it kind of speaks for itself. This will give all of the people in this group access can view access to all of the resources that are in the bundle. Very convenient if you want to share um the stuff you're working on with your team. And then we get to targets, right? So what are targets? We call them uh deployment targets. and uh they allow for slight nuances to be uh applied to the bundle configuration.
So in here we call it development. It could also have been called fooar. The name doesn't really matter. Um uh but we set it to mode development to tell the bundle CLI to treat it as a as a development development mode target which means all jobs will be paused and u we make sure that the display names of resources are getting prefixes. So we can easily differentiate between s and my copies and so on and so forth. And here we can also again set these variables right we can set the variable to say oh you know what maybe in dev I don't want to read from the from the full NYC taxi trip data but only like a couple of days because if I do my processing on a couple days worth of data that's good enough for development but in production I want to get the full slice right it allows you to apply patterns like this then we have um uh a production um target as well with one more nuance I think is worth sharing is that you can apply overrides at the at the resource level.
So we saw this NYC taxi job uh definition before, but what happens here is that it will be augmented for this production target to say um when I deploy this job in production, we're also going to add email notifications. Um you can override any other property in here. So if you want to like override the number of workers in your cluster, that's fine. If you want to add like change other settings, that's also fine. But it uh removes the need from copy and pasting that entire definition and keeping multiple copies, right?
You only want to change certain parts of it. Um let's see. I'll skip this for a second. Um what I want to show next is like how do I actually interact with this with this bundle? I'll show the file tree for a second here actually. So we have a couple more resources like a dashboard job pipeline and schema. We have a couple sources like a notebook, a dashboard definition, LV-JSON, um, and some pipeline sources. Uh, so that's the entirety of this bundle.

19:19 Bundle demo and CLI tools

And then if I want to whoop, I'll do this. If I want to interact with it, I can just use the data uh CLI, run data bundle validate, for example. And what that will do is load all the configuration, make sure all the paths are fine, and it output a little bit of information about uh the bundle that we're working with. In this case, there's two warnings. Uh but that's just because they're examples. So it's correct that they don't match, but we don't need to uh do anything about it.
Um the second the second kind of core verb that we would want to do is a deployment. So I run a data rich bundle deploy. What that what that does it copies all those files in the bundle route into the workspace and materializes all the resources that we've defined. So the jobs will be either created updated or if you've removed the definition from the bundle configuration it will be removed. Um we're on uh conference Wi-Fi but it is completing so that's actually uh very nice and surprising.
We'll also show the bundle summary for a second and uh it just li lists all the uh resources that are part of that. So I can now just open this this this dashboard for example and uh I don't know where it went. Oh there there there there. Well, okay. So we have my Dev Peter uh NYC taxi trip uh dashboard and we can we can just take a look at how it's how it's going. Um and I think with that we'll conclude this short demo.
Oh actually no sorry sad there's one more thing uh and this has been around for a while so don't expect like breaking new announcements now with the one more thing but uh the VS code extension that we have also knows about bundles and gives you a little bit more information about your bundle state. So, as we're like open this bundle here in the uh in VS Code, the bundle resource explorer pane will show you the list of uh of resources that are part of it and will allow you to interact with them.
So, if I want to kick off a run, I can just click the play icon here. Want to run a pipeline update, I click these. I can also click here, which is which is kind of nifty. Click uh jump to definition and it'll just open up the file with the definition of that pipeline. So it it makes makes it a bit easier to work with bundles from uh from uh VS Code. Let's go back to best practices. We may have to uh run through these a little bit faster.

22:15 Best practices for deployment

Okay, I'm sorry. Go for it. Um so best practice, why do you need separate development and production environments? Now, strictly speaking, you don't need to have separate development and production environments. In fact, we have some customers who have only one workspace and say this is both dev and staging and prod and we we don't care. But if you if you're in regulated industries or you need a little bit more assurances on either information leakage or you want to lock down permissioning and whatnot, it does make sense to have uh environment isolation and use different workspaces for these different uh environments.
So, as we said kind of before, it allows you to tune resource sizes, uh set um yeah, inest paths, alerting, uh that type of stuff. But it also allows you to have these isolated isolated deployments like again the collaborative scenario for SAD and myself where we both want to have separate copies that don't interfere with each other. Um and uh I I guess this is a repeat of what I what I already showed live so I'll I'll skip over this one.
Um variables can be substituted again um in the direct yaml we have top level variable overrides and um uh you can specify them through the CLI flags or environment variables. Another thing that we have that we support and allows you to better modularize your configuration is complex valued variables. So instead of just primitive types like strings and integers or booleans uh you can actually specify whole objects and that allows you to do more reuse of your of your complex uh variables.
So then iterative development how do I test this? Uh the verbs uh again I'll I'll run through it uh quickly as I kind of showed it already. I gave away the beans but validate is the core primitive you can kind of do to do to get a gut check. Are there any blatant errors in my config that I need quick quick feedback on? Right now, we don't run that all the time in the background, but in the background, but for example, in this VS codeex uh um VS Code extension, I can see us doing that in the future to immediately give you some squigglies or um give you much faster feedback on like what's happening in your configuration.
When you deploy, again, all of the code uh um is synchronized into the workspace and the resources are materialized. Uh, that's gonna that's relatively fast in development mode, but all because all of this stuff is incremental and we're looking to speed that up even even more. The last verb that we that I didn't show yet is bundle run. It allows you to interact with your jobs and pipelines uh from the command line. So, uh if you want to let's say in CI/CD run uh an integration test job, right?
You would compose that and say first I want to do a deploy then I want to run my integration test job and that will just block until it either succeeds or fails and give giving you that uh well positive or negative signal on the on the quality of the changes. What if every every new project came with best practices built in um templates. So if you want to get started with a bundle, we don't want to ask you, hey, just open a file, empty file, data, call it data bricks or jaml and start chopping away at it, right?

25:13 Leveraging project templates

That's kind of a high bar to entry. Um, and in fact, we've built templates for bundles to make that first user experience much much better. So you can run this command data bundle in it. you get an pre-initialized uh uh bundle configuration with some boilerplate that you can use as a starting point. Um the thing is though and we have a couple built-in templates right for Python SQL envelope stacks is another popular template um you'll actually see a new template uh probably when gives his talk here he is uh on lakeflow as well so stay tuned for that and um but templates aren't only useful for us to kind of be very prescriptive about the getting started flow if you have or if you're a part of a data data platform team in your organization You might want to author these templates to give all your customers a good start point, right?
To do all the right stuff with respect to these variable lookups, the resource sizing, code layout, and so on and so forth, right? So, if you want to encode the best practices for your organization, encoding those as templates, uh whether or not you use these templates or a different templating mechanism, that's not I don't think that matters much. Uh we prefer of course these because we can give better integration and all that.
Um but being prescriptive about the structure is very useful for well scaling out DABs adoption in your in your organization. What does a modern deployment workflow look like? Well, um that always involves CI/CD pipelines and data. We don't do CI/CD uh like natively, right? We don't host code either. We allow you to check out code. Uh but that always comes from code hosting uh services and that can be private git repositories, but also of course GitHub, Azure DevOps, GitLab, uh Bitbucket and so forth.

26:56 CI/CD and modern workflows

And frequently those uh vendors also have CI/CD solutions where if there's certain triggers on those repositories, you can kick off kick off workflows. And in in these workflows you can interact with a data CLI to again run those same verbs that you run locally on your laptop, but now you're running them from uh from CLI from sorry from CI. Um, if you're on GitHub, we have a very neat uh GitHub action that uh easily sets up the CLI for you.
And one thing that actually we we failed to include in this slide, but which is very interesting is that we recently launched uh um workload identity federation support on the all side. So if you've ever set this up before and you've been kind of maintaining personal access tokens and you had to refresh those and whatnot, that will be a thing of the past because you can use open ID connect with your service principles in data bricks to exchange uh to get shortlived tokens and not have to deal with secrets in your CI/CD workflow.
So that's that I think it's very technical but that's really a game changer for people who have to deal with uh administrating these things. Um, another thing like as we're talking about adoption, how can I bring existing jobs, pipelines or whatever into a bundle? And the answer is yes, um, pipelines associated state, right? You have a pipeline, you compute a bunch of tables, maybe it cost a bunch of money to compute those.

28:33 Migrating existing resources

So ideally, you don't want to throw that out and recomputee the whole thing just for adopting DABs. And that's uh what we allow with these two verbs. So it is possible to first generate the config for any live object that you already have in the workspace with the generate command. You can do that once. You can also do that multiple times if you're changing the code in the web UI and want to see how it reflects on the on the configuration.
And the second verb is the bind uh command or verb that allows you to associate one of these resource keys, right, the NYC taxi job in in the example I just showed and associate them with an with an ID. Once you've executed this, the DABs will take ownership of um of that instance and um uh yeah, updates will will go into that into that job. So, what's what's new? Uh this is like a lot of a lot of content so far, but it's all stuff that we've already shipped, right?
It's it's done. So, um but it's not done, of course. Like, we have a bunch more coming. And here's like this is I I guess the highlight reel. Uh I think so. I think so. Yeah. I think so. One of I I gave a talk earlier today on workflows. That's my other day job. I don't get paid twice, but uh anyway, they uh so customers are like, "Hey, you've got this YAML thing and it's great, but I have a 100 jobs that I need to update.

29:38 Python support in bundles

I need to provide some sort of mechanism to automate the the buildout of my my workflow." So, Python for dabs, pyabs, uh if you will, uh is something we just launched in public preview a couple of months ago. Uh so that helps you kind of take that next level of a next graduation step. I think we might have an example. I'm gonna breeze through this maybe. Um so uh on this one uh just very quickly we have these decorators that we've defined on this one.
It's a it's a job mutator. So that example that I was giving you for 100 jobs, let's say you want to tag them specifically or you want to have uh notification setup, you can use this in Python in a declarative format that can be done in a few lines of code. U now the thing that I'm really excited about uh is uh the and story. So um this we get this all the time. So most likely a lot of folks in this room are uh DAB native. you're probably part of some data uh team that is responsible for building out this governance on code and uh build these guard rails for all your users and um my spirit animal is uh which is the data analyst and I work in the workspace and I it's hard like these uh I don't use an IDE I don't know how to use a CLI what's this YAML thing so we wanted to bridge that together with uh wait for it dabs in the workspace so uh basically What we do with that is we give you a deploy button.

30:55 DABs in the workspace UI

And I actually I think we might have a little demo here. Maybe I can kind of uh switch over. Let's see if we can uh if we can pull it up. There we go. And I'll just hit refresh. That's the magic uh magic button. So um so what you see here is the same project that uh was talking about earlier. um it uh lives inside a git folder because the idea is that the and story I was telling you uh they need to collaborate on the project together.
So it is uh if you open up a git folder you will be able to see this create new asset bundle um dropdown and in this case uh we just cloned uh the repo that uh had. And there's a number of things that I just want to quickly call out. So this is the same stuff that just deployed. And the nice thing here is because it's the same user, it's the same workspace, it's the same bundle. Um, that means I don't want to deploy something new.
He's Peter is working on his uh instance of uh the project. There's a couple of things that you see in here with this nice uh rocket icon. Um, one is you can navigate to the source uh code which is uh up here. And the other thing is you see the target uh so the deployment targets that people was talking about whether you're development or staging. Now one question that I know is already coming up it's welling up uh is hey I don't want this guy to deploy I don't want sod to deploy in production and so that's actually a valid case uh in this particular instance production and uh uh development are the same workspace but typically production uh deploys happen through CI/CD and uh if it's a different workspace it actually gets grayed out so you don't you will not allow sad or in this case to deploy into production um And then uh the other thing is uh the CLI is basically invoked by this um interface using this deploy button and it'll tell you what is going to happen.
So it actually detected a change because just did a deploy. So uh what I had in my definition here uh because I haven't synced from uh git and didn't commit. So, not the best practice. Uh, but uh we uh it detected what changes are made and when I deploy it, it'll give me a nice little deployment output panel to see uh how uh and what was created and then I can actually go jump to those resources as well. Now, one quick thing uh because we're going to run out of time very soon is um let me just go into the git folder and I can actually create this is what I was telling you.
This is the other entry point for the bundle. Um, and when you do that, you have this nice drawer that kind of slides in. And these are the templates that was talking about earlier. And you'll see a nice uh update to this one um tomorrow. Is it tomorrow? When is the lakeflow uh talk? But we we'll we'll list out all the talks here as well. But um over time, you want to be able to support uh custom templates as well because obviously you want to enforce the standards and uh the structure that you have uh within your uh projects.
And then uh the thing that I really like is this little uh icon. So I'm I'm a sucker for good icons. Um I think that is probably probably it on the demo or on the demo. Okay. Yeah, we can probably let's flip back here. Yes. And um see what else what the future holds. Well, uh the most important thing I think for us is to just keep uh steady pace with all the improvements that are happening on the platform. Uh and by that I mean new features that teams are constantly building out and we're making some pretty fundamental changes into how we kind of discover that and so that whatever you see uh shows up as uh you can dabify this uh as soon as possible. like this uh someone was asking me today about Genie Workspaces.

34:47 Future roadmap and conclusion

I I told them to go add that to the thing. The other thing we want to have is some sort of a uh server side state of what the bundle is. Uh now we can go into detail about how we actually implement this but the key thing is you want to be able to track the history of who deployed what and when. Uh if somebody went outside the chain and made some changes and broke glass, you want to be able to discover that. And um then uh I think with the functionality that we've added now within the workspace uh that just opens up the user base quite dramatically right um one of the first things people ask about is like okay I still have to do this YAML thing right and that is true but we are building those capabilities where you should be able to serialize your changes from the UI from the visual experience for the bobs uh and that you can serialize them into YAML so you can actually continue to do your source control and use the same underlying project uh and configuration that you have uh and then adding the you saw the import or the generate and bind command want to bring that uh there as well and then obviously help improve performance uh for this um and you'll learn a lot more we have uh we regularly meet with customers so obviously if you have uh questions do reach out to us and I think we have some talks yes perfect so the one that I've been talking about uh constantly is uh the first one up here which is lakeflow we integrate quite uh closely with Lakeow.
Um our colleagues and are going to talk about that and then a number of actually folks in this room and our friends and partners are doing uh actually did a talk already this morning and then there's a few more uh coming up. So please do check those out and in the 2 minutes 52 seconds that remain we can answer questions.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.