Matei Zaharia on Enterprise AI: Unity Gateway, Genie Ontology, and Omnigents

Summary

Matei Zaharia, Databricks co-founder and creator of Apache Spark, Delta Lake, and MLflow, discusses Unity AI Gateway, which provides budget controls, security policies to prevent PII leakage and unauthorized tool use, and multimodel routing to help enterprises avoid vendor lock-in as frontier models evolve.
Genie Ontology automatically builds semantic knowledge of an organization's data without manual curation, using the OntoRank ranking algorithm to enable agents to instantly discover and reason about the right tables at runtime.
Omnigents is an open-source meta-harness that lets multiple AI agents collaborate across different models and harnesses while sharing context, designed to enable team workflows at enterprise scale.

Matei Zaharia on Enterprise AI: Unity Gateway, Genie Ontology, and Omnigents

Managing generative AI at scale requires solving cost, security, and model choice simultaneously. Matei Zaharia, Databricks co-founder and creator of Apache Spark, Delta Lake, and MLflow, discusses Unity AI Gateway. Budget controls prevent runaway costs, security policies block PII leakage and unauthorized tool use, and multimodel routing helps organizations avoid vendor lock-in as frontier models evolve monthly.
This conversation also covers Genie Ontology, which automatically builds semantic knowledge of your data without manual curation, enabling agents to discover and reason about the right tables instantly. Learn about OntoRank, the ranking algorithm powering this discovery, and Omnigents, an open-source meta-harness that lets multiple AI agents collaborate across different models and harnesses, sharing context and enabling team workflows.
🤝

Chapters

FAQs

What is Unity AI Gateway and why did Databricks build it?

Unity AI Gateway helps enterprises manage the costs and security risks of running AI agents at scale, addressing challenges that rapidly emerged as companies moved from experimenting with agents to deploying them in production. It provides budget controls to prevent runaway costs, security policies to block PII leakage and unauthorized tool use, and multimodel routing to avoid vendor lock-in.

What is Genie Ontology and how does it work?

Genie Ontology automatically builds semantic knowledge of an organization's data without requiring manual curation, enabling agents to discover and reason about the right tables instantly. It uses the OntoRank ranking algorithm to surface the most relevant data assets at runtime, removing the burden of manual data documentation from data teams.

What is Omnigents?

Omnigents is an open-source meta-harness that enables multiple AI agents to collaborate across different models and agent harnesses while sharing context. It is designed to support team workflows where agents need to coordinate across different underlying systems and model providers.

Why did Matei Zaharia receive the ACM prize?

Matei Zaharia received the ACM prize for his contributions to Apache Spark, Delta Lake, MLflow, and the broader open-source data and AI stack. The prize recognizes a combination of the research behind these systems and the real-world impact of releasing them as open-source tools that other people can use.

Full transcript

[00:20] Hi and welcome back. I'm Holly. I'm here with Ari and we are very honored today to be joined by Matei. Hi Matei, how you doing? great. Yeah. Excellent summit so far? Yes. Yes. Excellent. So, um, very busy man who works on many areas of the platform. What's been keeping you
[00:35] the busiest recently? What are you most excited about? Yeah, I mean there's so many exciting things. I'll just mention the things I've been working on which are mostly Unity AI Gateway, bunch of the AI platform things and Omniture which is, you know, the Wednesday morning keynote.
[00:51] So. Awesome. We're going to get into all three of them, but first I wanted to give a big congratulations for people who didn't hear the ACM prize in computing. We actually have that photograph of you in Times Square. So now you're super duper famous.
[01:07] Yeah, I mean it's it was definitely a surprise. So ACM is the sort of research association in computer science. They they give awards for research and and also for contributions to like, you know, important systems or things like that.
[01:23] And they gave one to me for Apache Spark, Delta Lake, MLflow and sort of this open source data and AI stack that we've been building. So I think it's for a combination of some of the research that went into those and some of actually releasing
[01:38] systems that other people can use. So definitely a surprise. It's one of the prizes they give to sort of like mid-career computer scientists. All right. So in terms of things that you've been working on. So I think the first one out of your mouth was kind of Unity AI Gateway. I'm sorry if I got
[01:54] that name right. Yeah, it's a I have. Excellent. Well done me. Uh so, what's been the the big challenge for that at the moment? Like, I know there's demand for it as a feature. Why now? Yeah, so it basically everyone uh every company, you know, went from playing ground with agents to now really having
[02:10] them deployed and seeing both the costs and the security challenges with them like really skyrocket. Uh so, we we saw this internally, too. We have thousands of engineers here who are using agents heavily. Uh we also have many like
[02:26] hundreds, maybe even thousands now of internal agents from all the teams and and we built things for customers. And so, we had these issues with them. So, there's actually there's a a few uh things that the gateway does. Um the first thing is uh kind of purely a
[02:41] commercial thing, but but very uh useful as a customer. Um if you you can use Unity AI Gateway to access all the frontier models, GPD, Claude, Gemini, uh through Databricks using your Databricks uh uh credits,
[02:57] basically. So, um so, uh since the best model for each task seems to change like every month or so, uh you don't have to like commit to like, "Okay, we'll just get OpenAI or we'll just get Claude or whatever." You can actually like, you know, uh multiplex between them. And as a as a
[03:13] large software provider, like we have enough demand for all of them that we can, you know, get get a good uh deal on that. So, that's like purely financial, but that solves the choice problem of like, "I set everything up for one model, but now like someone else has the best coding model, so I wish I was using
[03:29] that." Um then the other pieces are the uh security and cost control. Uh so, for security in the gateway, um you can set up you can intercept any call to any of these models, you even um your own model capacity if you're not
[03:45] using the Databricks one. And you can centrally set policies on them. So, like make sure people aren't sending PII to the model uh or make sure the model isn't uh you know leaking information to the web when it's doing tool calls. And you can also have a central catalog of your tools or MCPs and your skills in
[04:02] your company. So like all our engineers who do coding in Databricks, they use that. That's how they access all the internal resources like Google Docs and so on and the security team can very easily inspect everything that's happening and and lock it down as it
[04:18] comes up. And then the final bit is cost controls. We allow you to set a budget for users across all the models whatever they're using, you know, you can't spend more than $20 a day or $100 a week or whatever it is that you want and to warn them or to cut off usage of
[04:35] the expensive models or you know, when they hit that limit. So that doesn't really exist from any of the model providers, but it's it's hugely helpful for managing cost. So it's a new class of product basically for us like it's related to Unity
[04:51] catalog cuz it understands all the data, but it's actually very different use cases and it's something we've used very heavily internally and with our lighthouse customers that we developed it with. So really excited to bring it out. Yeah, and that that's a big deal. All of what you said big deal, but as companies
[05:07] are scaling their AI use that cost control or cost monitoring is such a big factor. I've talked to companies that have hundreds or thousands of developers and then that adds up quickly. Yeah, we had like even a very good developers here. They do something and they say, well, I don't know what I did,
[05:23] but it spent $500, you know, on on this thing. Usually it's like reading a lot of logs or code or whatever and it's quite a surprise and this helps you deal with that. And then we also have people who are maybe you know, just being inefficient or like on purpose
[05:38] like you know, kind of doing silly things like turns out there's a API for like ordering your lunch in Databricks. Each day you can at least in in the SF office you can select a restaurant to get lunch from and some people were asking like Opus to review all the lunch
[05:55] options and and recommend one. And you you don't need a volunteer here. Yes. Yeah. That's awesome. And then yeah, the theme you're also talking about being able to future-proof. That's what companies are looking to us to do, not get locked in since a couple months from now new
[06:11] models will be coming out, new you know, in this whole harness concept. But one other topic you were talking about was ontology and when that was announced at the keynote, I don't know if you heard but there was a lot of applause. Yeah, so yeah, my team is one of the teams that I work with is the
[06:27] research team here. So we have these AI researchers who worked on building large language models and scaling them and also on doing information retrieval and other things with them. So we told we asked that team to help make data agents
[06:44] a lot better and the you know, one of the first things that came up that would both make the quality better and make them run faster and lower the cost was can it you know, can it have some kind of memory? Can it know what all the data means in your company? And a lot of
[07:00] stuff is not written down but can be inferred by you know, looking at how people use the data or reading code or reading dashboards and stuff like that. So Genie ontology runs in the background and builds these knowledge snippets based on how people are already using
[07:17] and creating the data you manage with Databricks and then it we have agents that can use that to much more quickly and more accurately answer questions. And there's a whole ranking algorithm we built called OntoRank which is kind
[07:33] of a silly name but user takes a look at these different signals like recency, authority of who created the asset, agreement like do you see many things computing this metric in the same way or is it just one? And ranks things that
[07:48] way. And it's it's it's remarkably effective. I I remember using Genie, you know, before this it was totally random. Just the first thing it found, it would get answered based on that. That's why we limited it in spaces to have you tell it which tables to use.
[08:03] And now just the Genie one, the global one, you don't set up any instructions, it just zooms in on the right thing right away. You know, I've been using Genie one in my pocket. You can do it on the desktop, too. All the time I'm sitting marketing, find it extremely helpful.
[08:19] Yeah, yeah. For for my own benefit. Yeah, I I set up like regular jobs each morning, too, and then I get a an email and it tells me like the things I care about. If it looks weird, it'll just tell me. And I'm really excited about alerts where it would just like message me like when something changes.
[08:35] Yeah, I got to say having it part of like the wider ecosystem is really useful. I don't know about you, but something that I've noticed is even casual meetings that I have with people where I want my agent to know what happened in that meeting. I'll record it and the transcript happens. And then and it's it's driving some really interesting behaviors. But have you had
[08:52] the chance to talk to any customers to who are using this yet? And how do they And if if you can share, how does this differ their experience differ from like other ontology products? Like I know people have been asking for an ontology for a very long time. It means different
[09:07] things to different companies. What are customers saying? the big thing is just how much happens automatically. I think people people were shocked that you don't need to curate a lot of stuff. That a lot of CIOs I talked with have, you know, they have these projects to clean up the data, make it more ready for AI, set up
[09:24] ontology. And they have the plan of like let's have lots of people, you know, we here are the owners in each domain. Here's what they'll do. And I showed them I actually showed them our internal ones. It has a little bit of curation, but not that much of the of the domains we set up in Unity Catalog,
[09:40] and and I show like how easily it can answer questions about about various topics. And when you think about it, it's great because even if you do that massive effort with humans to write down the ontology, things are going to change, you know, businesses actually only going
[09:56] to move faster in the future. So so things will change, and it will be out of date. It will actually be confusing. Whereas if if the agents can can automatically update those and highlight if there are conflicting sources, it's a big deal. Yeah, I think that's one of the unique
[10:12] things about working at Data Bricks, should we say, is that everything moves so quickly. Like I mean, you've been here a very long time, as have I. Anytime you write down a guide of like, this is how you do the thing, it's fine. It's obsolete. And what I liked hearing is you, you know, mentioned faster, you know, better
[10:28] context, and lower cost, and then also simplicity. Usually with a solution, you pick one or two of them, but now you get all of them together, and it's so simple. If you haven't tried any of this, try it. Yeah. Yeah, and the speed is also awesome as a user because, you know, you
[10:44] see the sources, and you even maybe you weren't clear about your initial question, but you can much faster get the answer you want by iterating with the agent. So yeah. So that's not the only thing you've been working on. You've also been working on Omnigents as well. Do you want to tell
[10:59] us a bit about that? Sure, yeah. So yeah, Omnigents came came out of this uh I experience with coding agents and custom agents internally as as people adopted them. And basically, what we saw is all the all
[11:15] the engineers who are like, you know, really effective with coding agents are using multiple agents, multiple types, you know, Claude and Codex and Coursera and so on, and getting them to work as a team to to produce results. So you you and same with the product teams. Like
[11:30] the the Genie product went through so many different models. It's it's now a combination of different models for the different pieces and those are changing regularly to make it better and and faster and cheaper. So so every serious like user of agents is
[11:45] is using a bunch of these and they today in the agent like software world, you've got LLMs and then you've got this layer of software around it and that's the interface to the outside world called the harness. That's something like Claude code or OpenAI SDK
[12:03] and but these harnesses are not really meant to talk to each other. So to build anything on top you have to do it from scratch and it maybe is okay to vibe code a little like multi-agent thing from scratch but as soon as you want to do it at scale in a company,
[12:20] support all the harnesses, also support collaboration between users and and security and so on, you don't want that to be vibe coded. So we created this open source project Omnigen that's what we call a meta harness. It's like harness of harnesses where you can
[12:37] easily combine different harnesses and you get great security and collaboration features for any kind of agent. Like literally now I can use the Omnigen app on my phone and see what my coding agents are doing and ask them to continue and if I'm really busy I can just hand off my session to another
[12:53] engineer and they work with the same agent, same context and and get work done. So there are just these issues above the harness level that weren't being solved that I think we can solve with this new layer. Yeah, now I find this fascinating cuz gosh, couple of months ago, couple
[13:10] quarters ago, a team really doubled down on like we had a workshop on like setting up with Cursor and then I went all in with Cursor which has been fine, don't get me wrong, but then Genie coding agents came out and they're like, oh, they're really good now and you've got all these MCP tools and I realized how much of a lift it was going to be to
[13:26] move and I thought what a silly migration project I have to do now to like move everything across. I also skipped like sharing skills. Like I find sometimes they're like some skills you see are like really trivial and it's like why was this a skill and then some there's like three that are ripping round data bricks right now that are
[13:42] phenomenal and it's like why am I sending this in a slack message that should be sent Yeah, yeah. So all of these can live in the server and I'm the agent can be shared and also we we created the wiring to hook them into any of the agents that you can hunt. So yeah, it's much more
[13:57] portable. All right, thank you so much. This has been an absolutely fantastic time. I'm really grateful that you were able to join us today. Is there one thing you maybe want to leave everyone with before we wrap up today? I I mean my main thing is if try Amethyst like I love using it.
[14:13] Our team you know built like all of it you know in there collaboratively product managers, engineers they're all in there. It's a it's a little hard to describe I think how much it helps until you actually try it but it's it makes it way more pleasant to
[14:30] work with agents and to work as a team and and share stuff and we think this will be very important layer. That's why we're making it open source to make sure it's it can be an ecosystem where everything integrates really well and you can get these cross agent
[14:45] workflows that are just so much so much better. Awesome. Well, you have a lot of fans and a lot of customers all over the world and partners that are super appreciative of what you Matei personally have done and and your team and the whole company. So thank you. Thank you so much.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.