The Next Chapter for AI Agents

Summary

This video introduces Omnigent, Databricks' meta harness that composes multiple coding agents—including Codex—into coordinated workflows, addressing three problems teams encounter at scale: fragmentation across agents, all-or-nothing control, and lack of collaboration.
Omnigent provides programmable control over agent autonomy, real-time session sharing with read and edit permissions, and cross-review patterns that let multiple agents check each other's work before changes reach production.
The video covers Databricks sandboxes for cloud-hosted agent execution, Unity AI Gateway for dynamic model budget policies, and intelligent model routing that selects the right model per task—including a live demo of a sandbox provisioning in under one second.

The Next Chapter for AI Agents

AI agents are reshaping how engineers and teams work, but the path from experimentation to real production looks different than anyone expected. Drawing on what we've built inside our own teams and alongside customers, we'll share where this technology is heading: how agents are being trusted with real work, what's holding teams back, and where the biggest unsolved problems still sit. You'll leave with a clearer view of how to think about AI agents in your own work, and what to watch for in the months ahead.
Talk By: Elise Gonzales, Staff Product Manager, Databricks ; Kasey Uhlenhuth, Director, Product Management, Databricks ;

Chapters

FAQs

What is Omnigent and what problems does it solve for coding agents?

Omnigent is Databricks' meta harness that wraps and composes multiple coding agents—such as Codex—into coordinated workflows from a single interface. This video explains that it was built to solve three specific problems: fragmentation across disconnected agent sessions, a lack of middle-ground control between full autonomy and constant approval prompts, and no built-in way for teams to collaborate on shared agent sessions.

How does Omnigent give developers control over coding agent autonomy?

Rather than forcing developers to choose between giving an agent full control or approving every step, Omnigent introduces programmable control settings that let teams define what the agent can do independently versus what requires a human checkpoint. This video demonstrates cross-review patterns where multiple agents check each other's outputs before changes are applied.

What are Databricks sandboxes and how do they work with Omnigent?

Databricks sandboxes are cloud-hosted environments in which coding agents can execute code safely, separate from production systems. This video shows a live demo where a sandbox provisions in under one second, and explains that sandboxes integrate with Unity AI Gateway to enforce dynamic model budget policies and intelligent model selection per task.

How does Unity AI Gateway support multi-agent workflows in Omnigent?

Unity AI Gateway acts as the governance layer for agent model access within Omnigent, enforcing dynamic budget policies that control how much compute and how many tokens each agent or task can consume. This video shows how the meta harness uses intelligent routing to select the most appropriate model for each subtask rather than routing everything through a single model.

Full transcript

[00:08] Hi everyone. My name's Elise, joined here by Casey. We're both on the product team. Really excited to talk to you about Omnigent. Raise your hand if you've used a coding agent recently, like in the last week. What about in the last like 24 hours?
[00:21] Yeah, that's what we thought. We use them prolifically internally. A lot of you are using them as well. Um and we've come across a lot of problems that coding agents have introduced. They've
[00:33] obviously solved a lot. They're really great at writing code. But, as we interviewed internal developers and external developers, we found a few key problems that we are looking to solve with Omnigent.
[00:46] I'll start out by explaining a little bit of what Omnigent is. It's a meta harness. If you saw Matei's keynote, he talked about this a little bit. It's essentially a harness of harnesses. So, how can we wrap pod code? How can we
[00:58] wrap Codex? How start to compose them together? Give you more control over the experience and a better collaboration. We'll dive into all of these pieces with you today.
[01:11] I really want to dive in though to why we built Omnigent and go a little bit deeper there. So, why did we go and build this thing? What problems were we actually seeing? The first is composition.
[01:24] This is actually a few screenshots internally of some tools that people at Databricks have built. They've been building these custom UIs so that they can manage all of the sessions that they already have with pod code. They didn't know where they were
[01:37] going. They have all of these tmux sessions. And so, they built these custom UIs for it. But now they're going to have to port that to Codex as we're starting to roll that out. So, how do you start to compose these together and give people an experience that doesn't
[01:50] live in the terminal? The second one here is control. There's really no in between for autonomy. We have a lot of engineers who want to run things more autonomously, but maybe not give it full control. We don't want to
[02:03] end up with a deleted prod database, for instance. But so there has to be something in between. I think we've all experienced on the left where Claude will ask you, "Can I run this?" I have no idea what it's trying to do. I don't
[02:15] know what this means. Uh But on the right, I've we've essentially given it permission to everything, to do just about everything so we can avoid the approval prompt. So we really wanted something in between that allowed you
[02:27] more autonomy where you could trust the agent to do more, but not break outside of the bounds. The third piece here, again, is collaborations. This one's going to keep coming up. Your context can't be shared.
[02:40] We've seen so much copying and pasting between you and the agent, between you and your colleagues, and between Harnesses. This is a real screenshot between Casey and I where we were where we were sending a plan file back and
[02:52] forth so we could write an agent together. There's also no good way to view plans without some sort of viewer. Personally, I've been using Obsidian for a lot of this, but I don't want to have to use multiple programs. I don't want the
[03:04] terminal plus Obsidian plus other things in order to work effectively with my agent. We've also seen a lot of engineers putting up PRs just to see the code that the agent has written because often they're using a remote machine with a
[03:17] terminal. How do you actually see the changes that the agent has made? So these were really all the problems we wanted to solve with Omnigent. And so I'm going to jump in and start by showing you a demo of the open-source
[03:31] experience, and then Casey will show you the managed experience inside of Databricks. Conveniently, we have a native app, which is honestly one of my favorite features here. Can't tell you how many
[03:43] times I've lost terminal windows or lost Chrome tabs. And so having a native app where I can have this concentrated experience has been really awesome. Overall, this is Omnigen. This is essentially our web UI
[03:56] uh UI for it. You can run it in the terminal as well. I definitely prefer the web UI. I find it much easier to manage, but we give you that flexibility because some people do really enjoy working in the terminal.
[04:08] The first thing you can see over here is I have a number of different agents available out of the box. So, I have Claude code just like I normally use Claude code. I also have Codex. And then I have Polly and Debbie. These
[04:21] are two agents we've introduced in Omnigen to help you take advantage of being able to compose different models and different harnesses together. Polly here is our multi-agent coding tool. It has a couple of built-in skills. So,
[04:34] the first one here is cross review. So, you can start by implementing in Claude code and then have Codex review it. We've seen It actually gives us a lot better results than just having Claude code implement and review its own code.
[04:47] Add some more sort of dynamism to that review and we end up getting better results. You can fan out here. So, you can actually send independent tasks to different harnesses and start to parallelize your work more effectively.
[05:00] And finally, we have investigate as well. So, you can do a read-only investigation, debugging, audit, search, really get a good understanding of your code without having it actually write anything to your files.
[05:13] So, I'm going to start here in uh with Polly and I'm going to do a cross review. I'm actually going to switch repos here. Immediately, we do tell you when other agents are working in the repos that way you don't accidentally clobber changes.
[05:25] So, I'm going to switch repos. So, this is just running on my laptop. I could connect a new host. And what Casey will show you is you can't connect to our new Databricks sandbox as well.
[05:37] I can create a work tree. So, I'm going to do Omni demo. Start from demo start. And I can get advanced features here as well. So, I could switch harnesses. So, we offer Claude SDK, OpenAI agents,
[05:50] Codex and Pi. And during the session, I could also switch harnesses as well. So, if I wanted to experiment with a different one, maybe I encounter a bug with the Claude SDK, I could actually change here.
[06:02] So, I've been working on this store ops demo for a session that I have tomorrow. And so, what I'm going to have it do here is I have an agent running in Databricks apps. I want to update the model it's using. So, I want to say, "Can you update the model
[06:16] in store ops agent to high queue?" Oops. You all will see I'm not very good at typing, but that's all right. Agents don't care, which is really nice. So, we give you a really nice chat
[06:30] experience here. You can see it's calling the cross review skill. You can also see it even sent my typos over, which is totally fine. It'll understand what I'm mean here. We give you a native terminal view as well. So, if you do want this,
[06:44] we will skip that part. Still in the middle of working on this. So, we can see it's starting to churn through. I also get my full working directory on the right. So, I can natively view markdown files in here. I
[06:57] can see what the uh read me for this repo that I created for my session tomorrow is. And so, that way I can really work inside directly with my agent. If I wanted to make updates, I could also
[07:10] just quickly add a comment here and directly tell the agent what I think is wrong with the read me. So, rather than copying it from this doc, pasting it back into my terminal window, and telling it to update the text, or just updating it myself, I have this really
[07:23] natural collaboration directly with my agent. Here, I also will be able to see it's like PR already exists. I was running this demo before. I'm just going to do it again.
[07:35] Um and so, we will be able to see when it spins up uh Claude Code and Codex under agents here. I can also directly spin up a shell. So, often we still want to run terminal commands, we still want to do some double-checking of what the agent has
[07:48] done. So, you do get a native shell experience directly in here that you can execute commands on. So, I'm going to exit there. Perfect. So, we can see that Polly has launched
[08:00] uh Claude code here. I can jump directly into the sub-agent. I can see the prompt that was sent. I can see that I need to do a few approvals. I did not run this in auto mode. I can actually switch to my terminal and switch to auto mode and
[08:14] let it run. So, I'm going to let Polly kick this off here. We'll be able to come back and see Codex reviewing. I also have another session going on where I want to work in the agent
[08:28] framework repo that we have, but I want to be able to collaborate with Casey on this. So, I've done I've gone through, I've looked at some issues, I've started to make some code changes, but I don't want to ship a PR to her yet. I'd like for her to come in and see
[08:41] what changes my agent has made, maybe add some additional context, help me actually test this before we ship a PR and have to make changes later. So, I'm going to have Casey jump in, and
[08:53] I've actually already shared the session. So, you do get either read or edit permissions. So, edit will let her come in and uh look at files, make changes to files, but you could also just come in and read the session, and potentially
[09:06] we're working in person or over a VC, and then she would be able to just see my session without making any changes. And you can actually see directly up here, so Casey has joined my session.
[09:19] So, just like you get in Google Docs, you can see who your collaborators are. You can also see the messages that have been sent to them. Oh, perfect. So, Casey has sent summarize what you've
[09:31] done recently. So, she wants to understand what my agent is doing, that way she can get up to speed on this session without having to go read all of the text and back and forth that I've already had with this agent.
[09:44] Awesome. So now let's go back and see what Polly is doing on my cross review here. All right. So it is still working on making this change. And so once it's done that, it will be able to kick that
[09:58] over to Codex and Codex will do a review, make sure all of these changes work really well. And that way I get higher quality code out of the box and I didn't have to copy and paste all of my diff or copy and paste a PR to Codex for
[10:11] it to then add its review. We're really just trying to eliminate a lot of the copy and paste we've been doing and that you all have been doing as you've been working with coding agents as well. Awesome. So I'm going to transition back and I'm
[10:23] going to let Casey show you what the managed experience on top of Omnigen looks like in Databricks. >> Sorry. Oh, but okay. >> Sorry. >> All good. >> We don't need slides, right?
[10:36] >> It'll be fine. >> slides? >> All right. So what does it mean to have managed Omnigen? So Omnigen has You can't see the architecture anymore, but it has an architecture where there's a runner and it has a server. And so the
[10:50] runner we're going to allow and it's usually on your local machine, but you can actually connect it to a cloud-based sandbox environment if you want. So we work with Modal, I forget what the other like I think a couple more came in this weekend that we now support. And we also
[11:02] support Databricks sand sandboxes. Sorry. Here we go. Here we go. >> Sorry, I didn't switch monitors. >> Great. Uh and so the thing that we do with
[11:14] managed Omnigen is you can one, put your runner to run on this Databricks sandbox. As we mentioned in the keynote, Databricks sandbox is actually an amazing technology that we built. It spins up agent sandboxes in milliseconds. And then you can easily
[11:26] create and destroy them as you need and can use them to host your agents securely and allow them to, you know, run amok in that VM uh to do work. But, what's nice about it running from managed Omnigen is once you connect your
[11:39] hosting to a cloud, that means you can then close your laptop and then just go work on your phone. So, you can go on a walk, you can go home, you can commute somewhere, and your agents will still run cuz now they're all running in the
[11:51] cloud um on that sandbox. So, that's kind of the benefit in there. The other thing that we provide as part of managed Omnigen is we allow you to connect to Unity AI Gateway. So, you can connect all your model access through Unity AI
[12:04] Gateway so that you can get access to your different models and model capacity for your coding agents. But then also all your policies can be uh defined in there as well. So, you can have your organization define all your global policies in Unity AI Gateway and
[12:17] have them funnel through to all of your Omnigen agents that you create. So, I'm going to do is I'm going to show you sort of like how to like illustrate what this like meta harness concept means with intelligent routing. So, it's also what
[12:29] we showed in the keynote, but I'm just going to kind of go a little bit deeper into it. So, the benefits of Unity AI Gateway is that we have a utility called UCode. We're actually merging it into Omnigen, so you'll be able to get into
[12:41] Omnigen so the like wh- whatever we call this thing is going to end up changing. Uh but right now it's called UCode. Right now, if I launch uh UCode, and I'm going to do it with watch so you can see what your spend actually is.
[12:54] This is going to spin up Codex in the left, and then I'm going to be able to see my company's budget on the right. And so, these budgets are coming through Unity AI Gateway. So, I can see what my spend is, and then you can start to see the policies that the organization has
[13:06] created over here on the right. So, until I hit 80% of my budget, I can use GPT 5.5. Once I hit 80%, it's not going to switch me to open code with Kimmy models. And so, Kimmy is like a
[13:19] lightning-fast coding model um that's come out recently in open source, and so we're hosting that for you here on Databricks. So, if I ask this a question, I don't know. Waste of tokens, but when is Data and AI
[13:33] Summit? I can send fire this off in Codex. It's going to cost some money. I think it's just over 50 cents. So, then it's going to make me exceed my budget in here. And
[13:45] so, this is where we can have these policies that are contextual and are dynamic based on state. And so, it's actually taking budget as a variable into that policy. So, now I can see here that I've hit it. It's warning me that
[13:58] I've hit this. I can continue working in the session cuz we didn't really like you know, you don't really want your developers not to code. You don't want them to be like, "Oh, I have my budget. Going home for the day." You know, and so you want them to keep being able to
[14:10] work in that session. But now, if I were to start up a new session, so if I were to launch this again, it's now going to warn me that, "Hey, I'm kind of reaching the end of my
[14:22] budget and you might want to actually switch to open coding Kimmy to do your work now." And so, this helps engineers better understand like how much things are costing them and to try to be a little more intentional about their
[14:34] activities and which models that they're using for their various tasks. But this is kind of a pain in the butt, for lack of pardon my French, cuz you're like, "Ah, I don't know what I want to do. I just want to write code. I don't want to
[14:46] think about which model I should be using. I'm always just going to want to use the best model because why wouldn't you?" So, the way to get around that is by using the meta harness to sit above all of your organization's coding agents and
[14:58] having the meta harness intelligently route requests from the agent to various models. And so, that's what we're doing by combining Unity AI Gateway where you have your budget set and your model capacity and combining that with the
[15:12] meta harness layer in Omnigen. So, if I come over here then, good enough. Okay. So, here I am in Omnigen. You
[15:24] don't have this yet, but it's coming soon. We're on some hot fiery bits. Uh we have added this little brain icon, which now will enable intelligent routing on top of our models that we provide. And here are little super
[15:37] harnesses for Omnigen in here. So, what I can do in here is I've already attached to my agent framework repo, so I can say something like, "Launch an agent swarm to find four bugs in this repo."
[15:55] I think my file is too big, maybe. So, what this is going to do is it's first going to call my routing. It's going to say, "Oh, well, we actually needed like a medium-sized model for the routing in order to do this." And then you can see it's going to be launching for all of my agent swarm. It's going to
[16:08] choose based on the task which model it should use. So, when it does this initial surface scan, it says, "Oh, this is like cheap and mechanical lookup task, so I'm actually use Kimmy, which is your cheapest fastest model."
[16:21] Uh and it's like, "Okay, now we're going to do this edge case review." So, for that one, we're actually going to use Gemini Flash. Uh and then so, based on the different things, it's going to uh route it to different to different tasks. And you can see when
[16:34] it does then the deep audit, that's when it says, "Okay, this is actually a complex task. You want the highest quality model for this task." And so, you see I didn't have to do anything before it launched all of these agents. And then you can see your your like, you
[16:47] know, multi-agent swarm working for you over here in Omnigen. So, that's kind of what the power of Omnigen is is you kind of really want to work above all of your coding agents, intercept their calls,
[17:01] and then be able to route across them. And we've really found that on our team that combining multiple agents to do your work across different vendors has a huge impact in your quality. And I I can't go back to just one anymore, which
[17:14] is not great for spends, but hopefully that will all get fixed soon, especially as open source models are getting better and better. All right, so that was intelligent routing where we're using the meta
[17:26] harness to change how we route across different things, leveraging the Unity AI Gateway so that you're able to sort of connect it to your budget to be able to get this. You can't really do it with your own router right now because you
[17:38] don't have the budget set up. Um so right now managed on a jet will be where you get this intelligent routing. So this part of managed on a jet is where you want it to be able to run in the cloud. So you can close your laptop, do other things, move to your phone,
[17:51] etc. So that's where you can connect it to the data brick sandbox. So just how you were able to connect to your local runners from this drop down, you can now or you will soon, I think as of today, yeah, today, you will be able to
[18:06] connect this to data brick sandbox, which is in beta. You'll be able to connect to this and then you can do similar things where I guess I can connect to the on a jet repo and I can do like the same thing like uh mm
[18:18] find a bug in this repo. Oh, this is Debbie. I'm in Debbie. Great call. We don't want that. Debbie is our multi-agent debate skill, which is really fun. Uh but it will not code. So that's why
[18:31] we're switching back to Polly, our coding multi-agent coder. >> It will debate the repo merits. Okay, wait, wait, wait. Oh, crap, I messed up my demo. Uh-oh, it was so fast. Okay, so
[18:43] the thing I need you to look at is how fast our sandbox has spun up. So we're going to try that one more time. Uh find two bugs with different agents in this repo.
[18:55] And then as soon as I click enter, you see provisioning sandbox 1 1,000, done. Already built the sandbox. And now it's hosted. And so now it's just spending most of the time just starting my agent
[19:08] and cloning the repo. And that's the sandbox. So it's able to spin these up incredibly fast uh and that's how you can now start to like close your laptop and things like that in there. I would close my laptop, I don't have a phone hooked up for you to
[19:21] be able to see, so kind of defeats the demo. I guess I could. >> In the session we collaborated in earlier though, you did use your phone. >> That is true. >> So, you guys got to see some mobile here.
[19:35] >> Yes. >> Laptop open. Hope I didn't ruin my display. >> Still running. >> No, it's still running. >> go. >> Um so, that's what this allows you to do by connecting to the Databricks sandbox and so, now you can take your coding
[19:47] agents on the go because we all wanted to code more, you know? Now you'll never stop. The vibes are forever. Um so, that's the kind of stuff that we're trying to build in the Databricks managed version. What's actually happening behind the scenes in here too
[19:59] is it's connecting to a Databricks workspace and that's where it's like ultimately hosted. >> Um >>
[20:11] um connect to this and then hopefully I had a question earlier like, "Hey, can your identity actually pass through this as well with sharing?" That's kind of the stuff that we'll be working on um in the Databricks managed version. All right. Awesome. Um
[20:24] with that, I don't have the slides, so >> Okay. You'll switcheroo. >> We'll switch one more time. >> One more time. >> All right, we just wanted to show you all the cool demos, but they're
[20:48] >> sure. Okay, so this is like a nice diagram of what the managed version It is not on the wrong display.
[21:02] Things just not moving. There you go. Okay, you did it. >> All right, you good. >> Okay, so this is where you can connect your UniAI. This is just now a picture of what I just showed. Where it's UniAI
[21:14] Gateway is where you can get your model access. Your runner is now running on the Databricks sandbox. The server is technically on Databricks workspace. And then that allows you to work across files and everything in there. And from any device.
[21:32] Okay, and Jonathan. Oh yeah, so this when you actually set up Omnigent, this is where you can actually go to set up sort of the manage version in there. There'll be an option to connect to your models to Databricks. It's all in the docs, don't worry.
[21:44] All right, we did that. All right, so we actually built Omnigent with Omnigent. It was one of our core tenants of building this thing where we said, "Well, we only succeeded if our entire engineering team and PM team and design
[21:58] team has fully moved over to using our own product by the end of this." And so so happy to say we did. We'll say the PMs were the first first on the line >>
[22:10] >> to move from just using a standalone harness to using Omnigent in here. And so all of our features that we built here were built often collaboratively with Omnigent. It was kind of Initially,
[22:22] our eng team didn't really understand why we have to do collaboration. We have to do collaboration. It's like the only feature that doesn't exist in all these other sort of like IDE-like environments. And then once they started using it to do pair programming and
[22:35] their code reviews, they like understood why collaboration was so important. We did all of our product management work with this where we we spawned all these like sub agent swarms to do all the research for us, see what else is going
[22:49] on out there. We built our UIs. The logo was built from this. I mean, a lot of stuff was happening and we all built it on Omnigent. And as we were actually building a lot of our sort of like open source agent sandbox connections. We
[23:02] actually found bugs in their code bases by having Omnigen swarms uh go and investigate um in there. Uh and so I think the big thing here to
[23:14] sort of like end on is like why did we open source this thing? Uh and it's mostly because we believe that there needs to be this open layer where everyone can collaborate and work on it together and it needs to be in the open
[23:27] rather than tied to one vendor. Um so Databricks has kind of like always and making things open with Spark and MLflow, Unity Catalog now, our gateway. Um and so Omnigen just makes sense to have this
[23:39] metadata layer in the open. Uh and so we're actually building it in the open today like fully. So we would love to have you all join us. Uh we have already a thriving community there and on we're on Discord and we're literally doing
[23:52] open development uh as we develop this. So we would love to have you join us cuz we think this is going to be the future um of agents.
[24:05] I guess the community is already excited as you can see. We had some pretty uh big ballers who contributed this weekend like literally a day after we launched to our open source repo. So the community is thriving and well.
[24:17] Uh we're pretty excited about what they did. Uh yeah. So the main thing is just you can try Omnigen today. Uh here's a QR
[24:29] code. I mean the GitHub I think this is just omnigen >> .ai >> omnigen.ai/omnigen omnigen.ai is the website where you can get all the docs and all the getting started.
[24:41] >> And managed Omnigen is in beta as of this morning. So it is now available for you to enable in your workspace. It connects connects directly to Unity AI Gateway. You're not required to use the sandbox but you can also use that as
[24:54] well. So all of that is now live on our doc site, too.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.