Build and Deploy Databricks Projects Using Automation Bundles and Genie ZeroOps

Summary

Declarative Automation Bundles (DABs) express the entire Databricks project as source code, providing the structured interface that both human developers and AI agents need to safely build, deploy, and operate workflows at scale.
Genie ZeroOps monitors jobs, pipelines, tables, and data assets for failures and optimization opportunities, proposes fixes with root-cause explanations validated in a sandboxed environment, and surfaces them through an inbox for human approval.
This video introduces Deployment History, a new service that creates an immutable audit trail tracking who deployed what, when, and with what result including code diffs, and demonstrates DABs authoring across Cursor IDE, VS Code, and the workspace UI.

Build and Deploy Databricks Projects Using Automation Bundles and Genie ZeroOps

Modern data engineering teams need more than deployment automation. They need projects that are source-controlled, governed, and reviewable, so both humans and AI agents can safely build, deploy, and operate them at scale. Declarative Automation Bundles (DABs) provide that foundation. By expressing your entire Databricks project as source code, DABs not only enable scalable processes for human developers, but provide the exact structured interface AI agents need to interact with and manage your workflows. In this video, we will show how to use DABs in practice, including best practices and the latest authoring workflows across Genie Code, local IDEs, and the Workspace UI. We will also introduce Deployment History, a new service to track exactly who deployed what, when, and with what result. Building on this foundation, we will share a first look at our Genie Code powered product that monitors jobs, pipelines, tables, and other data assets for failures and optimization opportunities. It explains the root cause, proposes a fix, and validates it in a sandboxed environment. Today, data engineers spend 50%+ of their time maintaining data assets. As agentic tools make code generation faster and more broadly available, removing that operational bottleneck is more urgent than ever.
Talk By: Kristóf Molnár, Sr. Staff Product Manager, Databricks ; Lennart Kats, Principal Engineer, Databricks ;

Chapters

FAQs

What are Declarative Automation Bundles (DABs) in Databricks?

Declarative Automation Bundles express your entire Databricks project — jobs, pipelines, permissions, and configuration — as source code that can be version-controlled and deployed across environments. They provide the structured interface that both human developers working in IDEs and AI agents like Genie Code need to interact with and manage workflows safely.

How does Genie ZeroOps work and what does it monitor?

Genie ZeroOps monitors jobs, pipelines, tables, and other data assets for failures and optimization opportunities, groups related incidents, and proposes fixes validated in a sandboxed environment before requiring human approval. Data engineers currently spend over 50% of their time on maintenance, and ZeroOps is designed to reduce that operational burden.

What is Deployment History in Databricks and what does it track?

Deployment History is a new service that creates an immutable audit trail recording who deployed what, when, and with what result, along with code diffs for each deployment. Genie can also use the deployment record as context to explain the root cause of a failed deployment in natural language.

How does Genie Code work with DABs in a local IDE?

This video demonstrates Genie Code working inside Cursor IDE with the Databricks extension, where it can author and edit DAB configurations alongside pipeline code with full awareness of your catalog and workspace context. The VS Code extension and workspace UI provide additional no-code and browser-based authoring paths for teams with different workflow preferences.

Full transcript

[00:08] Welcome everyone. Uh thanks for coming to our talk. Uh yeah, maybe let's get started right away. Um so some housekeeping. Uh enjoy some legalies. Uh basically what
[00:21] it says is we don't have a crystal ball. Uh but we are going to present you our best knowledge. Uh but uh yeah, please don't take screenshots, print them out, for your signature in it and say, "Hey,
[00:34] you committed to this." Um yeah, so um something that I feel a bit more passionate about is the feedback form. So when you get the feedback form, please please fill it out. Uh it's
[00:46] always helpful to know how we can do better. Uh yeah, then actual content. So we have a problem. Data engineers
[01:00] we spend over half of our time on maintenance. How much time you spend on maintenance? >> I spend so much time on this like failing pipelines, data evolution and yeah the 50% statistic I personally
[01:13] recognize that and then I talked to NL practitioners and they said no it's like 80% for us. >> Yeah. And stuff still breaks uh which is pretty annoying. So what can we do about this? Well, use dabs, use GD zerops, and
[01:30] yeah, you can work more efficiently. You can spend less time babysitting your pipelines, more time building and shipping, so you can, you know, out compete your competitors.
[01:42] Now, with that, let's show something. Are we doing video or live demo? >> Live demo on staging. >> Oh, cool. Okay. Well, I also like to live dangerously. Uh, okay. Let's see. I
[01:54] love the combination of staging and AI makes it less predictable. >> Yes. >> Uh so this workspace actually has a whole bunch of things going on. Lots of jobs. Some of them fail, some of them
[02:06] succeed. Uh some of them are important. And then if you look at this dashboard, you can see there there's a bunch of red, bunch of green. So there's a lot of noise. It's hard to tell like what's the health of my workloads, the ones that I
[02:19] care about. So this is where we introduced the inbox from zero box. Zero ops. You've seen this in the keynote. The zero ops inbox shows an overview of everything that grabs your attention.
[02:33] Everything that the agent looked at and decides there's something wrong here. I know how to fix it. I need you to look at it and approve it. And in this particular workspace, we have 36 incidents. There's about 100
[02:46] jobs and pipelines monitored and almost a thousand tables. And then um if I look at this, there's actually this first one is this fan interaction thing. I think this was actually resolved in the keynote.
[02:59] So let me just make sure to filter by the open incident. And then yeah, this one demands my attention, right? So apparently the wonder revenue is down by 15%. >> Sounds like a lot.
[03:12] >> This seems like a very serious issue. And then um what zero does it provides me a summary right of what happened and tells me actually the pipeline stay green right so this was successfully
[03:25] running apparently which is good but then there was anomaly detection so a machine learning based technique that found hey actually the the orders dropped but then with machine learning
[03:37] there can be false positive right so an agent actually went in and verified uh this is a real error a real defect because of a reason code shapes. So, zero ops actually understands my
[03:49] data. It also understands the code behind it and that's what we'll look at especially in this talk. So, if I scroll down a bit, it assess the impact. So, it assess that this actually affects uh number of downstream
[04:03] assets as well and that's why it prioritized this as critical. uh it looks at a root cause. Um it's like I said this was a code change uh something related to like a row drop
[04:15] statement. I can keep scrolling. Um this is the interesting interesting thing. There's a proposed fix here and I I can just go right away and click and create a pull
[04:27] request. But I might have some more questions first. Uh so let me first ask about this code chains. So, so this was a code change apparently
[04:40] like this is where I can just talk to the agent, learn more. Um, maybe get some more confidence that this was a real issue and there's real fix for it. So, yes, recent change to the silver bookings.py added new column and then
[04:55] yeah, apparently this column had no in it. So, then some of those rows were dropped. Um, and I mean as a software engineer I can't help but wonder like someone made
[05:07] a code change and did they not write test for this? So, uh, were there no tests that caught this
[05:25] because as we we write tests for these things, right? So there are tests um but they actually don't cover this case because there was this new column that was added. So our tests they work on fake data right so
[05:38] they didn't catch this case but zero ops works on real data and in production there was this issue. So then zero ops actually detected this issue and yeah this is where I get a little
[05:51] bit more confidence right this is where I'm like okay this actually seems super plausible let's look at what is the propos fix
[06:04] so there's a silver booking stuff pi it's very close to this this drop statement and oh this is the part I like look at It wrote a new test for me.
[06:16] >> Oh, no, no, wait. Now, it's going to automate a way of writing the test. That's that was my favorite part of the job. >> You're a PM, Kristoff. This was never the fun part. >> Ah, okay. Well, that's how I p my stats.
[06:28] >> I can see that. Um, but yeah, so it actually produced some some fake data for a test. So, this will make sure this thing won't happen again. So, let me collapse these things again.
[06:40] And this is uh yeah I might just take uh go create a progress for me and this is the part where still as a human I get to review a pull request right so it still goes through my normal CI process all
[06:52] the garels that I might have there for review and testing and so on it doesn't just make changes life um but so the the main highlight here is that zero ops understands your data but
[07:04] it's also very much grounded in your code it understands what changed in your code it um can propose changes to your code via pull request. Um so the being grounded code is is a
[07:16] key thing of what makes it work and this is this is actually something we've been doing for some time with dabs. We said we can make everything in
[07:29] a data project into source code because because for dabs declarative automation bundles. Did I get that right? >> Uh yes, good job. Yeah. So uh we recently renamed this in case you didn't
[07:41] get the memo. Uh so now it's declaration declarative automation bundles. The bundle name remains the same. So the CLI command is not breaking but yeah de is remain the same. So yeah nice work
[07:53] correcting your muscle memory. >> Excellent. So same command, same acronym, nothing really changed. I think it probably describes it a little bit better of what it does but yeah. So so DAP makes it possible to
[08:05] describe your entire project as source code. Um, and when I say entire project, I don't just mean like your Python statements and your SQL queries. I mean everything, right? Also, your job definitions, your jobs that you might
[08:18] point and click in the old world, you can write them as source code now. um your environments, your dependencies, everything in your data project is source code and and that is super powerful because then you can take the
[08:30] source code and you can deploy to production or you can deploy to staging environment or a deaf environment and then in production that's where zero ops runs right and then zero ops understands your data your production data your actual data but also understands this
[08:43] relationship to the source code and it can make safe changes to the source code because that might be the other magic trick of dabs. Like if everything is source code,
[08:56] you could just hand the agent a copy of the pro. No, actually no, let's not do that. A copy of the source code and not a prod. Yes. Yeah. So that's very important. Uh yeah, this is where DEPS comes into
[09:08] picture and uh yeah, makes things just a lot more efficient. Um but yes so the main thing of dabs uh and we G8 back in 2024 the main thing of dabs has always been uh we make
[09:21] everything source code so so no more click ops um we do proper data ops and having things as source code just enables all of these best practices it's not just agents right it's if you have
[09:33] things in source code you can put them in git you have modularity you have tests you have devstaging prod you can do branching agents love branching just testing many different things. Um,
[09:45] idees, all of these things are grounded in source code and dabs has been actually very popular since we launched it in 2024. You have almost 80,000 active monthly users now.
[10:00] So, so this is actually the time where we want to say thank you all. Thank you all for for using dabs. Thanks for all the feedback. >> Yeah. Well, this is the opportunity for the audience. Yeah. give yourself a round of applause because yeah, we wouldn't be here without your commitment
[10:12] and all the feedback that you gave us. So yeah, thank you
[10:25] much input on dabs. Um and we shipped so many things this last year based on all that feedback. And and this didn't even really all fit on the slide, everything that we did. >> Yeah. And these are just the things that
[10:37] we are not even demoing today. Uh like this is all the thing that we had to cut because I we don't have you know three hours >> if only. >> Yeah. Yes. Um so what we're going to focus on today are the things that
[10:50] changed in the experience for you as a user or your agents. We have three areas that we'll cover. So one area is local ID altering a with IDs or local coding agents because because
[11:04] we hear you loud and clear. Many of you really care about local development. >> Yeah, that's that's something that came across very loud and clear. Like every customer call I'm in, I hear this, you know, I'm a data engineer. I work in my
[11:16] ID. Don't make me go that to the workspace UI. Make the ID experience good. So yeah, that's what we are doing. >> Makes sense. At the same time, we also invested heavily in in workspace authoring in the workspace and gen code.
[11:29] Everything just works out of the box. You don't need to install any tools. It just works. So we'll show that as well. The last thing we'll show is completely new is deployment history. So we we're now making it so that workspaces
[11:42] actually track everything that happened, everything that was deployed, all the code changes. Um, and that is, as you can imagine, super useful for agents and and zero ops in particular.
[12:00] Um, with that, let's look at authoring. This where we have more demos. Um here I'm actually going to show cursor. Uh for cursor I installed the
[12:14] data extension for visual studio code. Uh extension also works for VS code works for compatible IDEs. I use this extension uh to use one of the templates and create a Python
[12:27] project. So this is just a normal template based Python project. And then as you can imagine there's there's Python source files here. There's source files for your data
[12:39] pipeline. Your your job and your ETL pipeline. We use YAML for that. You can also use Python. You have a Python abstraction. Um and there's tests like like tests are imported. The tests come out of the box
[12:51] with the templates. Let me show you one of the tests. So this test is a little bit interesting because um actually made it uh do a a query
[13:04] just to show that these tests uh even though I'm in my IDE actually connect to data compute run it on the data compute with a given version or with serverless compute um and then just run it right
[13:17] there. So I I don't actually need to leave my ID environment. I can just stay here and run tests uh like a software engineer. This is also staging again. It's taking longer than I would like.
[13:31] Um, so the other thing that you might notice in this extension is there's a data bricks tab. The data tab um shows an overview of my project because it
[13:43] declarative automation models is all declarative. So it just understands everything that's described there. It notice that there's a a job apparently there's a pipeline a pipeline of certain data sets
[13:55] and there's a few new things here because now I can actually look at my workspace file system as well. So another thing I can do without having to leave my ID, I can look at what is in my home folder and see what what notebooks
[14:07] I have there, what files do I have there. But then the big thing that many of you asked for is uh can I see un catalog in the IDE?
[14:19] Yeah, that came up many times like uh you know that's the one thing that a lot of data engineers still had to keep a chrome tab open. You know what does this table look exactly like? What the column names are etc etc. So now you have that integrated into the ID.
[14:32] >> Exactly. So you can inspect what catalogs you have, what schemas do you have. You can copy paste these names with a single click to your source code. Or the other big thing you can actually
[14:44] copy paste these table names into your agent because we also made the cursor agent and other agents work really well with dabs and local development. So I can ask for example so what is in
[14:59] samples.bonder Wonder bricks and then we'll see the cursor agent actually working with data bricks and if
[15:11] you look very closely at little gray text it tells me it used data bricks it used databicks core and then yes look it just told me there's samples wonder amenities all of these things and I can ask lot of
[15:24] follow-up questions so the key new thing here is that cursor now has agent skills. So you have data rigs skills
[15:37] is what I'll ask. Yes, I have skills and I use it to query this water bricks thing. That that's super wonderful. Um and then can you
[15:49] tell the audience where to get these skills?
[16:01] Because right now um we're separately distributing these skills. They work with any agent, not just the cursor agent. They work with cloud code, codecs or whatever. Um you can find them on our website. There's a CLI command to install them. There's a web page. They're in the cursor marketplace and so
[16:13] on. Uh we'll make it so that they're shipped as part of the extension as well. Um that actually brings me to the last part because like I said, it's not just
[16:25] cloth. It's not just cursor. I'm personally actually a huge fan of cloth code. So let me try show cloud code here.
[16:39] So cloth is armed with the same agent uh skills. I used to use the direct cli command to install the skills and it works on all the agents that I have on my desktop.
[16:51] I can ask you questions about the current project, right? Um, if I start looking at the project, it will look at, hey, what files are here? I'll do a few steps. Um,
[17:03] there's a read me. Super helpful. Look, it's looking at the resource files, the ML files. And yes, this is a DAB project. Uh, it
[17:16] has source files, it has tests, there's deployment targets. So I can deploy this to dev or to protends
[17:29] all of it. So I can also ask can you deploy this project to my dev workspace and then open it in my browser
[17:45] because there there's as you can imagine lots of CLI commands for this. You can use the the validate command to validate that everything that you described is correct. You can use the deploy command and then select a target workspace. Uh
[17:57] look, here's the validate command. You can use the summary command to find out what are these URLs or ask your agent. You don't need to remember all these commands.
[18:15] Look summary. And yes, I just opened it in my browser. I would need to sign in. Um, >> okay. Yeah. Well, thank you. So, yeah, the ID got a lot of improvements and we are still working on that. Uh, yeah,
[18:28] hopefully that makes lots of you happy. Now, let's switch gears a bit. Uh, so this was mostly for the data engineer audience. Another thing we heard is that okay, but our team in our team we also
[18:41] have data analyst, data scientist people who don't want this very sophisticated setup. They want just to go to the workspace and something that works out of the box and we also want them to use source control devs. So all the best
[18:54] practices so we can scale our operations. So please
[19:06] let's log back to >> it should be the next browser tab. >> Oh yeah. Um tab. Okay. Yeah. So here we have a bundle already deployed to a dev workspace. Uh so let's see what we can
[19:18] do with it. Um well uh let's create a a job for it. Uh you know how do you create a job? You write a YAML file, right? That's how you do it. >> Yes. No, I mean I I designed the YL
[19:32] syntax. I never liked writing these things in YAML to be honest. So this very like the templates. >> Yeah. Okay. So let's see how the rest of us does that. So let's add a new job definition into this. So let's call
[19:45] this, I don't know, analyze customer reviews, for example. Add this to the bundle. Okay, great. Uh let's deploy this. Uh yes, the job build is going to be
[19:57] deployed. Awesome. Okay, so that created the YAML file as you can see uh as it's expected. Okay, then what? Well, let's start editing it. Um so, uh what can I do here? Uh let's open this. Okay. So
[20:11] there is nothing yet. Uh well let me add the pipeline to it. Um so for example uh where is one uh yeah process reviews
[20:28] reviews. Okay so let me add this um then well what is supposed to happen? Well the oh look at that toast on the top right. The ML file got updated. Let me go back to the previous tab. See if that shows up. It does.
[20:41] So now >> I just told you I like writing this by hand so much. >> Yeah. Well, um not everybody does. Okay. Uh so now you just add this visually and uh
[20:53] the pipeline definition shows up uh in the YL file. Let's just take another quick example. Uh I don't know. Let me add the what should I add the dashboard. Okay. So um update dash
[21:08] Let me add that to it. Um, for example, this one. Okay, let's save the task. Um, okay. Job updated. Get that is the dashboard also.
[21:22] What else can I add? Uh, let me add uh I don't know. Uh, a trigger scheduled every day. Why not? Uh, okay. This didn't work. Uh, okay.
[21:36] Well, we ran out of credits, I guess. Uh, okay. But you get the idea. Um, so yeah, from the UI, the workspace UI, you can uh visual edit uh your jobs and it just shows up in the ML file. So this is
[21:49] also helpful because it also teaches the users how it works. So you can see the before, you can see the after. Uh, and yeah, it's just a very helpful, handy uh feature. Now, okay, I added a job uh to this um
[22:04] uh to this uh bundle um and it's deployed. So, um let me uh remove this um and go through another journey that comes up very frequently. Uh so, let me
[22:16] uh delete this from here. Great. Let me redeploy so it gets removed. You know, I'm doing the ClickOps operations. Uh very efficient. Um, now
[22:31] let's go to the jobs page and create a new job here from outside the bundle. Okay. I could not wait. I'm not connected to the internet. >> You should have I mean you were
[22:43] connected just now, right? >> Okay. Yeah. I don't know what's going on here. Okay. There is some problem with our uh system.
[22:55] >> You could use some existing job I guess. Yeah, let me see if I have that. Um,
[23:07] okay. Um, this is the risk of actual live demos. This is not a dab problem, by the way. This is how you know that it was real so far.
[23:26] Okay. Um, okay. Maybe we are back live. Yes, we are. Who who kicked out the cable from the internet routter? Okay, so now I created uh this uh job. Uh let me add a pipeline to it. Just you
[23:39] know uh blah blah
[23:53] >> we have CLMI commands for this, right? There's the generate command and the window. >> Yeah. Yeah. >> Find command and >> Yeah. Yeah. Um now this is a better way. You tell it tell Genie to do it for you.
[24:06] So yeah, let's move this to a job. Okay. So Genie is thinking, okay, uh user wants me to move this to bundle. Okay, create uh existing bundle or
[24:19] create a new bundle. Let's use the the previous one that we had, I guess. Uh that should be fine. Okay, which one? Uh I guess that's a reasonable follow-up question. Um
[24:31] do I have this here on speed diode? No. Uh then okay, let me which one is it? This one, I guess. Okay, let's move it to this one.
[24:44] And let's see if it uh does it. Yeah, it's working on it. Okay, this is Yeah, still working. I think it is going
[24:56] to get it right. Uh, you never know. Um, I hope so. Fail to read file. Okay. Uh, let's see if it figures out. Uh,
[25:09] okay. Maybe I didn't reference the right thing. Um,
[25:28] okay. Anyways, uh, it's supposed to, you know, create a ML file just like we did before. Uh, so your job is now part of the bundle. Um,
[25:40] okay, now it's added to it. Okay, now it's done. Okay, I don't know what went wrong. Okay, let's see the job definition. And yeah, you are going to see that uh YL file uh just as previously. So
[25:53] any minute now. Yeah, there you go. So created the YAML file for the job in the pipe in the
[26:05] without any CLI commands. uh just telling Genie, hey, please do it for me. Yeah. So, great. Then, um yeah, let's move on to the next topic uh which is uh
[26:19] deployment history. So, this is something that came up very often. Again, you were telling us uh well, it's all great that I can use the deployments uh but uh if something goes wrong,
[26:33] how do I find the problem? Well, investigating issues today is pretty uh hard. I guess >> it's easy, right? >> Oh, yeah. Yeah. Easy for for you. But uh
[26:45] you know what is the usual journey? What happens? Well, something goes wrong. I need to go to my CI/CD logs uh and see uh you know what happened when uh what are the what are the information that I can obtain. Um then I need to go to my
[26:58] git logs to find the version uh the diffs etc. Then I might need to go to look at my job logs. Uh stitch some information together from there. Uh
[27:10] going to the UC uh lineage information to see, you know, how these things relate to each other. And then of course in the end I select Leonard because probably he knows it better. >> It's easy. >> Yeah,
[27:22] >> just a few little steps. >> A few little steps. Uh can we make this better? Uh yes. So uh we are going to give you now an actual comprehensive list of all your deployments uh in a
[27:34] single place where you see who deployed what and when exactly. It shows the state of the resources at that given deployment how it is at that point of time and what
[27:48] changed exactly compared to the previous deployment. And very importantly, this is something that you cannot temper with. Meaning you can present this to auditors
[28:00] and tell them, "Yeah, this is exactly what happened, right?" Okay, you might not be that excited about that last part, but there are some people in your company who are going to be. So this is coming to uh private preview before the end of the summer. Um so yeah, if you're
[28:12] interested, let us know. And uh yeah, let's take a look what it looks like uh in practice. Um so I think I have uh a page here. Uh yes. So here are some uh example deployments uh of bundles that
[28:26] we have in this workspace. Uh this is exactly what I described uh just a minute ago. Right? You have the list of the deployments. Uh what target? What is the status of the last deployment? What is the GH with the link uh to your repo?
[28:39] Uh who deployed and when. I can of course I don't know do some filtering. So uh it works as you would expect such a system to work. Um let's take a look at a concrete example. So let's take a
[28:52] look at this revenue analytics product deployment. Okay, what's in it? Um well in v4 deployment. Well in real life you are going to see much num larger numbers like uh v 4,600 whatever. Um but so in
[29:06] this particular deployment it changed uh let me close this for a second. uh it changed uh it updated two jobs. Okay, what else is in this bundle? Uh well, all these things but uh lots of them
[29:19] didn't change. So I can see for example this alert last got updated in V3. Uh okay, awesome. I can of course time travel to see the state at a given uh deployment version.
[29:31] Then in this side panel I have a bunch of metadata related to the bundle itself. Here is the UU ID of the bundle. uh the information that we mentioned uh
[29:43] what the related repo is that was used for the deployment but is the actual commit hash with a link again. Yeah. So you can uh get all this information uh in one place again uh in a yeah a
[29:55] nonmutable uh state. Okay. So let's um go back and let's take a look at another example. For example, one that failed. So for example, this
[30:07] dev deployment uh failed in version 10. So that's not good. Uh let's get some information about this. Uh SQL warehouse 49 whatever does not exist. Uh okay,
[30:19] that's annoying. Uh how do we solve this? >> Don't ask me, just ask Genie. >> Yeah, let's ask Genie. Yeah. Uh that's what we do these days. Uh okay. Well, diagnose this for me.
[30:31] Okay. Why did this dawn deployment fail? Explain the root cause. suggest a fix. Okay, this is going to work a bit. Uh in the meantime, let me show you something else. Um so, you know, uh going to the job list page is great and then you go
[30:45] and you know figure out uh find the find relevant thing in the list. Very often you just go to the job page and you know this is part of a bundle. So there is this entry point here uh so you can more easily access that list uh that uh that
[30:58] I was just showing you. Uh in the meantime, let's see if uh this is done. Yes. So, but did Chin say uh well uh it did some some investigation, identified
[31:10] the root cause. It failed because uh somebody put in a an ID that does not exist. Uh here is the the diff from from the previous one. Somebody did some bad copy pasta, I guess. Um suggested fix
[31:23] update this to the new ID. And yeah, that that solves it. This is awesome. Yeah. So you know now you can uh go uh to the deployment page uh see the error
[31:36] and get the fix uh by just clicking the you know a gen help me solve this button. >> This fixes it for one deployment. I guess that's great.
[31:48] >> Yeah. But can we do better? Well I guess we can um you know and this is where we come back to zero ops uh where we started this talk. Um
[32:07] so deps are foundational piece that enable genops genzops to work a lot more efficiently because as Leonard said when things are described as source code agents can operate on them a lot more uh efficiently. So what does this solution look like? Well, they are configurable
[32:21] meaning that you can just talk to it and say okay these are the the jobs or the tables that I care about uh monitor these in priority and also it learns from your implicit behavior. So so the insights that you are clicking into it
[32:33] it understands that you care about those versus those that you don't. It gives you a much better signal to noise ratio uh because it can aggregate the issues and find common root case
[32:45] root causes and show them grouped together. So you're not, you know, uh, uh, buried under like a million issues that come from the same root cause. It's proactive, meaning that you don't need to do this clicking that I just showed
[32:57] you. U, it's running continuously in the background and is coming to you with a suggested fix and analysis already. And then finally, it can do validation in a
[33:09] safe manner. Let's talk about that just a little bit more. So what does this mean? Like this is a very important aspect because where we are we've been working on this and we are still working on it because it's very easy to get this wrong right you you tell an agent hey I
[33:22] have a problem uh in prods oh I have an idea let me try that and oh what could possibly go wrong uh right like oops I just deleted some pro data that's not great so we don't want that
[33:34] to happen we also don't want it to exfiltrate some data like oh here's a snippet from all the you know salary information uh that doesn't look like can you take a look that's also not Great. So yeah, we are building in these safeguards uh based on the the feedback
[33:47] that we've been getting recently from all of you. And uh yeah, we have some more time. So very quickly maybe show uh yeah want to give it one more go. Yeah, go ahead.
[33:59] Because I can actually take us full circle and and go back to zero ops and then take you to the setup journey. like we just have this workspace where Chris just pointed something and and go
[34:13] to the Genius Ops inbox and then just enable monitoring. Now we made the interface fully agentcentric. So also the setup experience is a chat based experience. In this case I'm not going to have it
[34:26] monitor any schemas. I don't want anomaly detection that's going to take too long. I'll tell it to just monitor all the jobs. So it will maintain those. Um, it tells me it's going to do an initial scan of the current state of
[34:39] those jobs. There's only nine jobs in this workspace. And then this can take a few minutes, but uh let's have a look at the inbox. So the inbox now tells me um there's
[34:51] been zero incidents, which is good, right? Nine jobs monitored, zero monitor tables, and it's currently analyzing ah it just analyzed my workplace. It basically just looked at the recent runs of those jobs and there was one that was
[35:05] failing and this fraud detection job actually failed. And this one is actually a bit more simple than the anomaly detection case that I showed earlier. This is just
[35:17] simply a failing job. The job job turns red. Um and it tells me that this has a typo since the latest commit. Okay, I could look at the actual job
[35:29] here. And then we close this and see it. It used to be green. Now, now it's going red. And I can click into it and try to find out why why it
[35:41] failed. Uh or I just go back here to the inbox because zero ops did all of that for me already. It root caused it. There was a recent commit by someone
[35:53] that broke this. Okay, this was my commit actually. Uh, I changed this quartz expression and I I renamed country to home country and apparently that's a typo.
[36:06] I'm sure it was some and there's a proposed fix uh which I imagine yes it just uh turns back the name and then the fun thing here is it
[36:18] says it validated this this fix in a sandbox right so so I can I could ask a question about this um so you validated
[36:31] this in a sandbox what does that even
[36:44] Awesome. So yeah, this this this very simple job didn't have any unit tests. Um it tells me it ran a describe SQL query. So it has read only access to the entire catalog all data. It does not have right access. So within this sandbox, it just looked at a SQL. It
[36:58] confirmed in production there's indeed no home country condom. There's only country. Um and then yeah it can create a PR for me with a fix
[37:11] right let's go back >> yeah so we are wrapping it up so we have a couple more minutes for just question very quickly so you might be wondering how do I get the most out of Genie zero ops just three things use dabs those
[37:24] enable these things so you know Genie can operate efficiently and not token max uh you know figuring stuff out from from zero add metad data similar reasons and then follow best practices. We just
[37:37] published a guide. So here is here are two links that you might want to take away. One is our recommendations.
[37:51] yeah, read that provide feedback. You know, is it too prescriptive? Does it work? etc. Uh we are hoping that that's that's going to be helpful. And the other link is if you want to get Genie Zerops, we are launching private preview
[38:04] soon. >> Yeah, very shortly for end of July. >> End of July. Okay. Yeah. So, it's coming very soon. Fill out the interest form so we can enroll you as soon as possible. And uh yeah, well, thank you. Get in touch. Enjoy the rest of the summit.
[38:16] Thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.