Agentic machine learning with Genie Code (includes demo)

Summary

AI-Driven ML Automation: Genie Code now features upgraded intelligence for ML engineering with native integrations across feature engineering, model training, serving, and monitoring.
Hidden ML Debt Problem: Production ML systems have substantial infrastructure complexity beyond model code; AI agents are the only way to automate and manage this technical debt.
ZeroOps ML Infrastructure: Databricks' AI Runtime and Genie ZeroOps for ML eliminate the maintenance burden, enabling teams to focus on innovation rather than system operations.

Agentic machine learning with Genie Code (includes demo)

Genie Code now comes with upgraded intelligence for ML engineering and native integrations across every Databricks ML Platform component: feature engineering, model training, serving, and monitoring.
Learn more: https://www.databricks.com/blog/whats-new-ai-platform-agents-ml-engineering-our-deep-learning-platform-and-new-capabilities
Presenters:
Mike Del Balso, Director, Product Management, Databricks
Amber Roberts, Staff Tech Marketing Engineer, Databricks

Chapters

FAQs

What is the hidden tech debt in ML systems?

Beyond the machine learning code itself, production ML systems require complex infrastructure including data pipelines, validation, monitoring, and deployment systems. This hidden tech debt often exceeds the actual ML code in complexity.

How do AI agents help with ML automation?

AI agents are the only practical way to automate the full ML lifecycle. Genie Code leverages agents to automate feature engineering, model training, serving, and monitoring across Databricks Data and AI platform components.

What is agentic machine learning?

Agentic machine learning represents a paradigm where AI agents autonomously orchestrate the entire machine learning lifecycle, from feature engineering through model monitoring and retraining. Rather than requiring data scientists to manually handle each step, agents make intelligent decisions about feature creation, model selection, hyperparameter tuning, and production deployment based on real-time performance metrics. This approach dramatically accelerates ML development timelines and enables organizations to scale intelligent systems across more business decisions without increasing data science headcount.

Which cloud platform is best for production machine learning?

The Databricks Data and AI platform provides agent-ready infrastructure specifically engineered for production machine learning at enterprise scale. With over 500 million models trained on Databricks in the past year alone, the platform offers comprehensive ML lifecycle support including feature engineering, model training, model serving, and monitoring—all governed by default through Unity Catalog. Agents running on Databricks can access complete context about data lineage, training history, and inference performance, enabling them to make intelligent optimization decisions without manual intervention.

What is an enterprise machine learning platform?

An enterprise machine learning platform must provide scalable compute infrastructure, end-to-end workflow automation, unified data governance, and comprehensive monitoring capabilities to support both human and AI-driven development. Databricks delivers all of these through its Data and AI platform, which combines agent-ready infrastructure like AI Runtime for deep learning with a single data model that instruments the entire ML lifecycle. This unified approach eliminates data silos and enables seamless collaboration between humans and agents on production ML systems.

Full transcript

`0:00` Mike Del Balso, Director, Product Management, Databricks

Welcome to the stage, Databricks Director of Product Management, Mike Del Balso. >> All right. was right. This is a deceptively long walk. Hey everyone. So, my whole career I've actually been working in production machine learning. I started out at Google working on the teams, the ads machine learning teams, that actually kind of kicked off the whole ML ops space. Then I helped start the machine learning team at Uber and helped build and lead the ML platform there called Michelangelo.
And for the past few years, I've been running my own production machine learning startup, which recently joined Databricks, and we've been working on those same problems here at Databricks. And the whole time, my whole career, the whole machine learning industry has had one goal, and a single dream, which is to make machine learning as easy and automated as possible, so we can bring intelligent decisions to all of the important places, all the important decisions across our business.
And many companies, many have tried and many have failed, and I'm really excited to talk to you today because that's finally changing. So, this diagram here, many of you may recognize this. This is from a paper that my team at Google published something like 12 years ago. And the point of this paper is to really explain how hard it is to do production machine learning. You know, this diagram shows, if you look at the red, that's the machine learning code, and everything around it is the point of the diagram.

`1:12` The hidden tech debt of ML systems

All these other things around it are all these other systems you practically need to build and run and maintain to run true ML systems in production. It's a really hard problem, and as a hint, this is all a data problem under the hood. And why are we talking about this right now? Well, we're building more software than ever, and you You have thought that GenAI would reduce the demand in machine learning, but it's actually only accelerated it.
With all of the new software and all the new product experiences we're building, we have more products to recommend, we have more prices to offer, we have more fraud to detect, uh more loans applications to decide on, so we have a lot more decisions to make in the company. >> really hard to do this, and so we only use we ration machine learning and only use it in a handful of places, right? Think in your company, how many models you have in production compared to all of the other decisions that you make in the company that could be automated.
And because we're not using intelligent machine learning there, we're leaving money on the table. Right? We're using we're basically making dumber decisions. We're using heuristics and rules of thumb and statically coded logic. And what we really want to do is automate machine learning so we can make it really easy to scale to all the important decisions we have in the company and keep up with all the scale of the new software, all the new decisions we're going to add as we deploy all this new software.
And we just just think back to the diagram I showed you, it's a it's a really hard problem technically, but also, you know, to automate this, you know, there's people in the loop in every single stage. There's a person making there's expert, there's a data scientist, someone with a PhD making decision about which feature to build, which model looks good, what are we really optimizing for? Which model should we promote to production?
So, if we're going to automate machine learning, we need to do a lot to accelerate these folks and enable the automation to make decisions in the same way they do. And there's actually only one technology that can do this. And software, all of software engineering. But they haven't had that same impact with machine learning yet. We're still kind of building and operating machine learning in the same way. >> Well, there's two reasons.

`3:43` AI agents are the the only way to automate ML

One is only a handful of companies even have like the right infrastructure. It's a really hard problem. You saw the diagram of all the stuff you got to build to run this complete machine learning loop in an automated way. The second problem >> making, they need to have all the same type of information so they can make the same quality of decisions. We can really automate that. And that context is just not available today to our agents.
And I'm really excited because Databricks is the first company that can solve this. And we're doing this in two ways. The first is with agent-ready machine learning infrastructure. So, getting that infrastructure into a place that your agents can actually drive throughout that whole loop. >> So, let's look at both of these, right? Agent-ready machine learning infrastructure. If you're here, you're probably building your ML models on Databricks.

`4:44` Agent-ready infrastructure with Databricks

Databricks has been at this for more than a decade now. More than 500 million models have been trained on Databricks in just the past year alone. And so, Databricks already has a complete leading platform for machine learning. And that's why Databricks is recognized as the leader in data science and ML. Databricks is the dot in the most the highest up, most top, and most to the right. And so, that's why you're probably building your ML models on Databricks today. >> ML.
But now we're going to have agents do it for us. And that's what we've that's what we've spent the past 18 months optimizing for at Databricks. We made the stack agent ready. So, what easy for an agent to spin up and spin down, and scale to whatever uh scale they need. Secondly, this whole data loop, this whole machine learning life cycle is now instrumented with a single uh data model. So, your training data, your features, your serving data, your monitoring data, your inference, they all land in the same governed tables.
That allows your agents to immediately see the impact of any changes they make and have a single source of truth for the entire machine learning life cycle, making it much easier for them to optimize and drive this loop. And thirdly, everything is governed by default. >> So, your models, your experiments, your serving endpoints, all of that's already governed in Unity Catalog today. But we now have governance on the actions that your agents can take, so that you can trust any changes that your agents are making to automate this machine learning loop.
And I want to I'm really excited because we're extending this further today. We are launching a new compute type in Databricks that makes it really easy to build um uh deep learning models and LLMs and that is called AI run time. >> we've actually had this in preview for the past 2 months and we've seen huge adoption from it. There's hundreds of companies that have already moved their deep learning workloads to this platform and are finding huge success.

`7:46` Introducing AI Runtime

Yesterday Ali mentioned Merck's latest drug discovery models and we've been really fortunate to partner with them and help them build on this platform. So I want to announce one more thing which is we're expanding it today to support multi-node training. So now you can build these models at any scale you need on any size of data. We're really excited about this because this brings a whole new domain of machine learning within the Databricks platform and there's no reason that you shouldn't be bringing this to your deep learning teams today. >> Okay, so that was the infrastructure.
Been talking about infrastructure for a long time, right? Let's talk about the agents to actually drive this loop, right? This is getting to a a more interesting thing here. Agents that drive the loop, it's important for them to do it with your context in the same way that your team would actually build machine learning, right? So let's talk about the first part here, it's building. >> also to announce Genie code for machine learning.

`8:55` Driving the agent loop with context

This is an This is an enhanced set of capabilities and intelligence for real-world machine learning tasks built into Genie code and it's available today. >> Nobody has seen more machine learning in the real world than Databricks. So, we've taken all of those lessons, what works, what doesn't work, what good looks like in production, and we've built that into Genie code. Genie code is also natively integrated with the complete machine learning stack.

`9:19` Introducing Genie Code for ML

So, it knows how to use all of those components, all that infrastructure I just showed, better than any other uh coding agent. And what I'm most excited about is we've upgraded the context in Genie code. So, it now knows how your machine learning teams and how your data science teams actually work, so it can make recommendations and do work alongside your teams. And I'll show you exactly what I mean by that. So, you remember this diagram?
This diagram's a little bit deceptive because, as I mentioned, this is from like a decade ago, and that was the problem we were thinking about at that time. That was solving the human problem. But when people are making decisions and driving that whole loop, they're actually drawing on many other sources of information to make these decisions, right? What What experiments do we run already? What uh What does this uh business metric mean?
What What worked, what didn't work? Where do we keep that evaluation script? So, that and all this context is is scattered across our whole platform. And if you don't have access to this data, if your agent doesn't have access to this data, it's going to have the wrong It's going to use the wrong data. It's going to use the wrong evaluation. And it's going to tell you something that kind of looks right, but it won't be right.
And you won't be able to trust it. So, what we've done, yesterday announced Genie ontology, we've built all of this into integrated it with Genie ontology and empowered Genie code with ontology. So, all of that context is built into Genie code, and allow This allows Genie code to know how your machine learning teams work and how they build models, evaluate models. And now let me show you specifically what I mean, right?
This allows Genie code to do things like building features in the same way that your team builds features today, using the exact same patterns. It allows Genie code to reuse the training scripts and understand what are our training workflows. How do we actually build models? What's our existing pipeline? And reuse that. So it actually fits in with the workflows that your team already has. But it's not just about building.
It's also about, you know, bringing these models to production and figuring out what are the better models. And so Genie code also respects the evals you have. It finds the eval scripts that you have, and it reuses those. So when Genie code says some a model is good, you can actually trust it because you know it's holding a model to the same bar that uh your team already trusts. And that allows Genie code to really act like a senior machine learning engineer on your team.
We've found that Genie code performs way better on real-world data science and machine learning tasks than other leading coding agents now. And this allows companies like Danfoss, a multi-national engineering company, to go from no machine learning to a complete production machine learning system in just 90 minutes. And this is a type of project that would take weeks, months, and I know for a lot of companies, a lot of customers in this room, sometimes it takes quarters, right?
It's really hard to do this. Uh this is a gigantic speed up. And so we're really excited about that, and that's just the build phase here. >> 80% of their time on maintaining their machine learning applications. It's just part of the problem. And if we're building models 100 times faster, we're going to be building 100 times more models. And so we can't take on 100 times the maintenance burden. No team can scale to that.

`13:29` The looming AI maintenance burden

And so that's why we need to think about the operating side of this loop, as well, right? After a model's in production, how do we keep it healthy? >> can have latency spikes. There can be any number of reasons for errors. Up- upstream data pipelines can break. There's a long tail of a thousand issues. And to solve this and automate this, something I'm really excited about, and I think it's the biggest launch in machine learning in the past 10 years.
I'm very excited to introduce Genie Zero Ops for machine learning. This is an agent that runs your machine learning systems for you. So there's a really big paradigm shift with this. Instead of typical monitoring systems that, you know, are are looking at your metrics, and when they find an issue, they bring you a problem and say, "Hey, this this thing looks wrong." Zero Ops for ML doesn't stop there. It finds the problems, and then it takes it upon itself to discover the root cause, come up with a fix autonomously, and bring you the solution.

`14:26` Introducing Genie ZeroOps for ML

So instead of go getting problems given to you all the time, you're getting solutions brought to you. And so it it monitors every model continuously. It investigates these issues and resolves them autonomously, but it still leaves your team in the loop to approve before anything deploys. And the flip here is that for the whole history of machine learning we've always had a team member on call for our models. And zero ops for ML inverts that and allows an agent to be on call and your team is just in a position to approve the recommendations from that agent.
And that allows Genie zero ops for ML to solve all of these kinds of problems we just talked about while still leaving your team in the loop just for the final approval. And so rather than me showing you a bunch of more slides about this, I'd like to invite on stage to actually show us how it works live. Thank you. >> Thanks for setting the stage and hello everyone. I'm here to show you how Genie zero ops not only finds issues with your ML systems but fixes them automatically.

`16:08` Amber demos Genie ZeroOps for ML

At Berkshire Sports Global, my team and I have a model in production that forecasts ticket sales for every game of the Data and AI World Cup. And this model is mission critical because that forecast affects tens of millions of dollars in event staffing and operation spend. And we don't want to end up with a lot of empty seats because we over priced tickets. With this model, I need to be alerted as soon as any issues happen.
So I'm going to quickly use Genie code to help create some alerts. Going to create an alert. And I'm going to ask Genie code to create an alert that will trigger if observed ticket sales are not within plus or minus 5% of the forecasted sales for any game. Here we can see how Genie code builds and registers the agent for me automatically. No extra setup needed. All right, I'll allow this um yeah, alert creation here.
And the alert's created. Now, let's fast forward to the end of the week. It's Friday at 5:30 p. m. I'm coming out of back-to-back meetings. I open up my email and crap. That alert just fired 30 minutes ago and the ticket forecast has suddenly become less accurate. This is a critical issue where every minute could cost my company millions of dollars. So, I need to just drop everything and debug what's going on. But wait, there's this new button.
View investigation in Genie Zero Ops? That sounds promising. Let's see what it does. When I tap it, it opens up the Genie Zero Ops inbox where we can see an open thread with the highest severity. So, I'm just going to go and click into that in order to see the investigation that Genie code started automatically about 30 minutes ago. So, instead of waiting for me to investigate this issue manually, Genie code got started immediately and has already explored and tested dozens of hypotheses.
That's amazing for me because these issues are costly and every second counts. Let's see what it did. First, Genie code tries to understand what triggered the alert and this is where Genie code's context and ontology shines. Most agents would provide a surface-level generic suggestion that still requires me to do a majority of the investigation myself. But Genie code goes deeper. It's powered by the ontology of my full ML project, so it analyzes uh notebooks that built the model.
It looks at Unity catalog tables. It runs SQL queries. It's checking for changes in the feature values. It's running ML flow experiments and more. Overall, it really approached the problem like a real ML expert, continuously iterating until it found the root cause. And my favorite thing is all this is done in the background. Work that would have taken me hours to do manually, Genie code root caused it before I even realized there was an alert.
And it looks like it figured out the underlying issue. Our model was missing a feature that conveys the stage of the tournament. This means that the model was unable to account for the increased demand and fan attention that the knockout rounds receive. So, as soon as the tournament entered the knockout stage, our forecasts became less ac- accurate, as you can see here. And then, it didn't stop there. After figuring out this issue, Genie code code went even further.
It started building and testing for the solution. It figured out that we had it existing data for both the games, tournament stage, and match-up. And then, it created features based on that underutilized data. Genie code then decided to retrain a model with these features, did hyperparameter tuning, registered the model in MLflow, all while following best practices for ML on Databricks. That's awesome, and it essentially solved the problem for me automatically.
And since Genie code has full lineage across our ML workflow, it reused the same evaluation criteria from the previous model to confirm the new model's accuracy. And it gets our forecast back to where it needs to be. Now, it's time to roll out this new model to a small amount of traffic, and I'm feeling really good because I can trust what Genie code did cuz I saw the evaluations it used. Now, I can deploy this model in one click.
Let's do it. All right. See version two. And I'm done. >> Now, normally a stale model like this is a major team-wide incident that massively affects our business for hours, if not days, potentially costing tens of millions of dollars. But, Genie code zero ops resolved it quickly and with minimal disruption to my team. And as mentioned earlier, we're entering the 100X era of ML. But with 100 times more models comes 100 times the complexity.
And that's really what we have to protect teams from. And that's really the opportunity here. Give ML tool Give ML teams the tools that they need to stop spending all this time fighting operational toil and start building breakthrough AI. And that's Genie Zero Ops for ML. Thank you. >> Okay. Pretty cool, right? Going to have to jog here. >> your deep learning models and your LLMs with it. Go try it now. Second, Genie code for machine learning is generally available.

`21:47` Mike wraps up

It's built into the Genie code you have right now on your laptop, so you can start using it today. >> this talk with. The machine learning industry has always been about automating machine learning, and we're finally at the point where we have the infrastructure and we have the agents to drive the loop. And that allows us to build a hunt machine learning 100 times faster, 100 times more models, >> truly personalized customer experiences at every part of our product, and we can really optimize every business decision.
So we're really excited uh to see what you build. Um let your agents do the work now, so your team can focus on what's next. So, thank you very much. >>

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.