Scaling AI Agents at Mercedes-Benz: Unified Governance and Multi-Cloud

Summary

Mercedes-Benz built the Agent Garden, an enterprise-wide marketplace for discovering and governing AI agents across multiple cloud environments and global business units, built on the Databricks Data and AI platform using Agent Framework, Model Serving, and AI Gateway.
The architecture features self-service agent publishing with a request-approval governance flow and a gateway proxy that routes queries across workspaces while enforcing strict access control, enabling agent reuse across the enterprise.
The system supports thousands of AI practitioners building agents independently across business units while preventing fragmentation and duplicated effort through unified agent discovery and centralized governance.

Scaling AI Agents at Mercedes-Benz: Unified Governance and Multi-Cloud

Scaling AI agents across multi-cloud environments requires more than distributed computing, it requires unified governance and discovery. Mercedes-Benz operates thousands of AI practitioners across global business units, each building agents independently. Without centralized coordination, this creates fragmentation, duplicated effort, and governance risk.
Learn how Mercedes-Benz built the Agent Garden, an enterprise-wide marketplace for discovering and governing AI agents across clouds and business units. Markus and Sai detail the technical architecture combining Databricks Agent Framework, Model Serving, and AI Gateway with multi-workspace orchestration. Discover self-service agent publishing, request-approval governance flows, and a gateway proxy that routes queries across workspaces while maintaining strict access control, enabling agent reuse and enterprise-scale AI operations.
🤝

Chapters

FAQs

What is the Agent Garden at Mercedes-Benz?

The Agent Garden is an enterprise-wide marketplace that Mercedes-Benz built on the Databricks Data and AI platform for discovering, publishing, and governing AI agents across all business units and cloud environments. It enables teams to find and reuse existing agents rather than rebuilding similar capabilities independently.

How does Mercedes-Benz govern AI agents across multiple cloud environments?

Mercedes-Benz uses a gateway proxy architecture built on Databricks AI Gateway that routes agent queries across workspaces while enforcing strict access control. Teams submit access requests through an approval workflow before they can invoke agents from other business units, maintaining governance in a self-service model.

Why did Mercedes-Benz need a centralized agent governance solution?

With thousands of AI practitioners across global business units each building agents independently, Mercedes-Benz faced fragmentation where multiple teams were building similar agents without knowing counterparts existed. The Agent Garden solves this by making all approved agents discoverable and reusable across the enterprise.

What does Mercedes-Benz's enterprise AI strategy look like?

Mercedes-Benz views AI and data as key drivers across their entire value chain and organizes their AI ecosystem around three pillars: takers, builders, and shapers. The AI Gateway, Model Garden, and Agent Garden form the technical foundation on the Databricks Data and AI platform that enables teams to move from experimentation to enterprise-grade solutions.

Full transcript

[00:08] Hi everyone. So, thank you so much for coming. Um We are so excited that so many of you are here today, and I hope you all enjoyed uh the keynote this morning and with all the latest Databricks products. So, before we get started, a little bit uh forward-looking statement. So, this
[00:24] presentation has been prepared for information purpose only, and uh the forward-looking statement should not be treated as guarantee for the future performance or outcomes. And we'd love to hear your feedback, so feel free to submit your
[00:39] uh feedback by choosing my service from the menu on your Databricks event app. Uh we'd love to see uh how you feel about this session and how we can improve in the future. So, I'll hand over to Markus to kickstart the presentation.
[00:55] Thanks, Sai. What a room to talk to. It's uh inspiring to see so many different perspectives from different industries, different companies, different roles, but with a common ambition to turn AI
[01:10] into business impact. And today, we will share you how we are scaling AI agents at Mercedes-Benz to make them usable, reusable, and specifically in a way that works in multi-cloud enterprise
[01:27] environments. We will show how we designed our AI ecosystem around agents, creating a setup that enables teams to move faster without losing control, and importantly, it's not just a theoretical
[01:42] view. It's something that we built together with Databricks in practice. Before we dive in, uh quick introduction. My name is Markus Haubach. I am a product manager for the AI ecosystem at Mercedes-Benz, and in my role, I lead multiple teams in an agile
[01:58] safe setup building the AI foundation that provides the capability to take AI from experimentation to enterprise grade solutions. So my focus is not on one individual application, it's on creating what
[02:15] empowers numerous AI use cases to be developed, to be connected, to be brought into enterprise use leveraging impact beyond the single solution. And I'm joined by Sai from Databricks.
[02:32] Thank you, Marcus. Um hi everyone. My name is Sai. I'm a solutions architect at Databricks uh supporting Mercedes-Benz accounts for over 2 years. Uh normally I'm based in Frankfurt in Germany. Very happy to be here with you and co-present with Marcus
[02:47] today. Cool. Then let's kick things off um by zooming out to the bigger picture AI plays in the role of the broader Mercedes-Benz strategy. At Mercedes-Benz we see AI and data as
[03:06] key drivers of our company strategy. With AI becoming used across the entire value chain, we are taking that forward through several dimensions and some of them are listed here. First, the AI North Star where we got
[03:22] initiated a group-wide AI target vision across business units anchored in business KPIs. Second, it's the AI value at scale where we build an end-to-end framework to turn AI ideas into
[03:39] measurable impact which is reflected in more than 1,500 AI use cases. There are of about 350 already live in production. The The third is the AI adoption and people enablement. The daily AI adoption
[03:57] at Mercedes-Benz has grown more than five times since 2025. And we also built and established a network of AI champions where we have more than 2,000. Fourth, the AI technology foundation. Um
[04:14] we as the AI ecosystem we established a standardized AI platform that allows models and agents um to scale enterprise-wide with a central governance in place. And the fifth is
[04:29] the strategic AI partnerships where we have strong collaborations, strong partnerships with leading AI players to advance state-of-the-art innovation. And these dimensions they enable to move AI
[04:45] into business processes throughout Mercedes-Benz. To make that work, it takes a foundation that connects agents, data, and tools in a way that business can truly use. And
[05:01] that matters because not everyone works with AI the same way. Some they want to have ready-to-use solutions. We call them the takers. This can be integrated AI opportunities within tools.
[05:17] Others, they prefer to build their own automations, their own workflows with low-code opportunities, with low-code tooling in context of composable AI. We call them the builders. And then we
[05:33] see the third pillar of teams who need full flexibility. They build tailored solutions in more advanced use cases. We call them the shapers. And within the AI center of competence, where we are
[05:49] part of, we address all of these pillars. From us as the AI ecosystem and the data ecosystem team to partnering and scouting to responsible and secure AI as well as enablement of people
[06:04] throughout the company. And with that foundation in mind, let me show you what the AI ecosystem behind this actually look like. To walk you through some major elements.
[06:20] In general, we provide the AI tech foundation as a portal. Because we kept hearing the feedback that people don't know where to start with AI. Every day, they face evolving technologies and at the same time a lack
[06:35] of guidance. So, we bring AI capabilities together at one place, covering the journey from experimentation to enterprise deployment. One important part is the model garden. It serves as our central gateway for
[06:51] large language models. They go through an standardized model intake process, where they get assessed. And these trusted model models, we make them available for the entire company. We also provide their guidance when it
[07:07] comes to strengths and weaknesses of the respective models in order to guide the people appropriately. Another key component is the agent garden, where teams can discover, consume, connect agents across hyperscalers in a cloud agnostic way.
[07:24] And we will come to that in a minute. Together, these elements, they form the backbone of how we provide AI capabilities in a reusable and structured way over the enterprise. And that isn't fiction. It's something that we already
[07:42] built and run at global scale. So, today the AI ecosystem is um adopted by especially the builders and shapers where we have more than 5,000
[07:59] monthly active users in almost 50 countries above six continents. We have connected approximately 3,500 apps to more than 40 general-purpose AI models performing about 100,000 tokens
[08:16] in average every second. And once you operate at that kind of scale, another challenge comes into focus. Yeah, how do we make agents easy to find to use and to connect enterprise-wide?
[08:34] And that's what I would like to to unpack next with a closer look at the Agent Garden. So, let's start by taking a look how our Agentic AI user journey looks like in
[08:50] practice. I already mentioned the Model Garden. People and our our builders, shapers, they can consume AI models from the mentioned Model Garden and teams choose the best-fitting model for their use
[09:06] case. They build then their agent leveraging Databricks and connect these agents to data via Unity or to tools via MCP. And once your agent is ready,
[09:22] they publish them onto the Agent Garden. To understand why we built the Agent Garden, it um helps to see the idea behind. That real value comes from agents that work
[09:40] together, not from agents that remain isolated. And for us, that's not just an conceptual thought, it's a direction we have explored, for example, in hackathons. Without the agent garden, useful agents,
[09:55] they may exist. Yeah, but they are rather fragmented bricks with limited transparency and duplicated efforts. With the agent garden, that changes. They become discoverable, they become accessible,
[10:12] easy to use, and especially also interoperable throughout the enterprise. Allowing them to be used more in a more systematic way, joining forces cross teams and cross use cases.
[10:27] And it's not just about listing agents, it's about creating conditions for meaningful collaboration. Building on that idea, the agent garden is in place as our multi-cloud
[10:42] marketplace for agents. Think of it as an corporate-wide catalog of agent cards. Each card gives clarity on what the agent does, on the purpose, who owns,
[10:57] and more information. Cuz we are convinced that if agents are meant to be used beyond the building team, we first need to have transparency. And once an agent is onboarded, it enables them to be applied
[11:14] over teams, over domains, over clouds, instead of being built over and over from scratch, which leads to the fact that we can also drive synergies. And making that work requires a
[11:30] consistent governance layer across platforms. And that's crucial because in reality, agents are distributed. They are not living all at the same place. They are hosted in various clouds and
[11:46] platforms. Managing each environment separately would lead to the fact that the operational complexity would grow tremendously. So, our approach is when people build agents in their decent
[12:03] use cases, we do not want to copy these agents in yet another system. Instead, we establish a technical connectivity during the onboarding. So, the role of this layer is that
[12:18] to connect those environments in a unified way. And here we benefit from the big data bricks capabilities. We connect what already exists and make them usable in an broader enterprise context.
[12:33] One tangible example that I want to highlight here is an agent from our colleagues from Mercedes-Benz Korea. With um Faris Jamal is their CIO, who had already a great presentation here at the Data and AI Summit. They on boarded
[12:49] already their agent in order to make that agent reusable for others. Publishing is meant to be straightforward. And that's why we designed a self-service experience
[13:04] um with minimal efforts required. Because we want to avoid that sharing agents becomes an obstacle in itself. The agent owner, they enter information like the agent name,
[13:20] a short explanation, the region, the deployment platform, and for Databricks, the serving endpoint. By enabling our gateway, um with a one-time activity, they allow us to automate the setup
[13:37] across environments. So, with that standardized onboarding process, we facilitate and accelerate enterprise-wide reuse. But, onboarding alone is not not enough. Um to scale agents in an controlled way,
[13:56] governance also need to work in a consistent and scalable manner. And that is what That's what I would like to next uh explore in a bit more deep detail. Every agent comes with a clear ownership. It's transparent who stands
[14:13] behind and who's responsible. Same applies also to the access. If someone wants to use an agent, it does not mean that access is granted automatically. The consumer goes into the agent garden,
[14:29] checks out what kind of agents are existing, takes a look at the information, then raises a request, which we then provide to the agent owner. And access is only granted if the agent
[14:44] owner approves that request. So, with that clear request and approval flow, we keep usage controlled and traceable, and also have with that some kind of technical handshake enabled. In the Databricks setup, this is um
[15:01] supported by a role-based access control on the serving endpoints. And these guardrails, they will build already into the agent garden from the very much beginning. It's nothing that we added later. And how we
[15:16] operationalize that from an architectural point of view, Saeed will now um show us a little bit more in detail and then we also show a brief demo. Thank you so much, Marcus.
[15:34] So, as Marcus mentioned, um, Agent Garden provides a centralized governance and Databricks uh provides the capability to support that. So, in this architecture you can see that on the left-hand side, without Agent Garden, um, if you want to access a agent endpoints deployed in multiple
[15:50] workspaces, uh, beyond your own workspace, you have to add your own identity into the target workspace in order to access those, uh, agents there. However, we may have the situation where, uh, we want to restrict the user access to the target
[16:05] workspaces. For example, that users should only be able to access the agent that is shared by Agent Garden, uh, but we don't want the users to have additional, uh, permissions, for example, creating notebooks, creating compute, uh, running
[16:20] SQL queries inside the target workspace. Um, in a setup without the Agent Garden, if you want to grant user access to a serving endpoint deployed in the target workspace, we have to add that user identity always into the target workspace and this creates a lot of
[16:36] challenges for us. For example, you may have those unintended privileges as I mentioned, um, the notebooks that are the the user can create and also the access to other workspace resources like dashboard, genie spaces, and so on. So, additionally, there's also no
[16:54] central governance and central allow list for this endpoint access control because it's, uh, separated in different workspaces and, uh, always need to configure those individual entitlement for workspace entitlement for a particular user in the target workspace.
[17:12] And with Agent Garden and integration with, uh, Databricks app solution, uh we we solved this issue. That is with Agent Garden on the right-hand side, you can see that um it leverages a gateway app. So, we we
[17:28] built a Databricks app, and this Databricks app acts as a intermediate layer for routing. And this Databricks app has a service uh service principal, and that service principal is added to every target
[17:43] workspaces. Uh and this process is, right, it's also fully governed from the Agent Garden. Uh if a agent is published in the Agent Garden, and someone requests access to that agent through the Agent Garden, uh that access control is also going through a allow list uh in
[18:01] a database. Right? So, once the user receives the approval to access the specific agent, um and that user just need to call the Databricks app URL, and that Databricks app will route the user request to the target workspace on behalf of the uh the
[18:18] user. So, in that way, you have a centralized endpoint access control. And this has multiple benefits. First of all, it has one single governed entry point for all users without having to managing and configuring uh the users'
[18:34] identity inside the target workspace. And secondly, only the app's service principal has access to the target workspace. So, then you don't have to worry about um the entitlement of those individual users. And finally, because with this single allow allow list, uh you can configure
[18:51] that uh with flexibility, and you can host that allow list in any database you want. For example, if you have a existing um cloud uh infrastructure, uh for example, AWS, and then you have a database there, and then the allow list is stored there, you can reuse that and
[19:06] integrate that with Databricks app, and this app will act on your behalf to uh to access the individual agents you want to use. And so, that can allow you to manage, right, the uh the central governance uh among hundreds, even thousands workspaces on
[19:22] that complexity level. So, we will give you a quick demo. And Marcus, I think you have pre-recorded the video on how it looks like on this uh governance process and access control process in the agent garden. So, we'll
[19:37] run that for you now. Yes. So, we put ourselves now into the perspective of Sai because Sai will outline in a couple of seconds how the connectivity between one workspace to another workspace works. And that's
[19:55] conducted via service principle. Yeah? So, I now start the video to show you the the request flow. So, you can see the available agents on the agent garden. You select the one you're interested in, check the information
[20:11] out, and then request access to. You can do that either for yourself or for a service principle at the idea, at the country, a business reason, and also the potential consumption estimate, and submit it.
[20:27] Now, we see the agent provider perspective. Yeah? We see for the same agent now that an access management is in place. And there you see then Sai's request. But you have also full full transparency about the approved and rejected
[20:44] users. Here, Sai's demand comes in with the service principle. You can approve that, and with that we facilitate then the access onto the agent owner's agent. And now, let's jump back to Sai's
[21:01] perspective of the agent consumer. Sai has now the opportunity to view the endpoint that is here blurred in that context, but he sees now the the endpoint to access the agent on the other workspace.
[21:17] Thank you, Marcus. So, let me switch to my workspace and demo demo the uh workspace from there.
[21:36] So, here I have a workspace where only I have access to uh because I don't have access to um the target workspace where the agent is hosted, but I want to access the agent endpoint uh in um in a target workspace. So, this workspace uh is a self-service workspace where users can come in and
[21:52] then uh do servers self-service analytics. Uh and now I have a service principal, as you can see in in the second cell here. And uh that service principal has a uh client ID and a client key stored in the secrets in Databricks. Now, I want to
[22:08] use the service principal to access the agent deployed in the target workspace, which is a workspace different from this one. Um I have also stored the tenant ID and also the host uh which is the URL from the target workspace in a variable.
[22:25] And as I run through this, here I I can I use this process to um request a token for the target workspace and then exchange that uh the the after token for a Databricks token in order to uh authorize myself to authorize my service principal. And then
[22:42] with that service principal and the agent endpoint URL, I'm able now directly um to call this uh Databricks app gateway app URL to access that agent. And then with that agent endpoint and also um the
[22:58] token I have received from the target workspace, uh I'm able to run uh REST API requests and then ask questions directly from my workspace, even if I don't have any access to the target workspace where the agents are deployed.
[23:13] Now I can run this in the real time just to show you that it works. So simply it takes the the client ID, client secret, and then it's running now this request, sending a request to this gateway app, and that gateway app is
[23:30] routing my request workspace and I'm waiting for the response. Once the response is there, it will send back to me so that you can build this solution into any application you have without having access to any target workspaces
[23:47] that you shouldn't have and where the Databricks sorry the the agents are deployed. Yeah, that's a quick demo here and we want to show that this provides you with the flexibility to create agents for yourself and to share
[24:03] agents via Agent Garden and eventually also have that centralized governance across your entire IT landscape. And for that, I will hand it back to Markus to continue our
[24:19] presentation. Cool. Thanks, Sai. Let me close by a brief brief look at the road ahead. Yeah, what are essential topics that we want to address in the future? Some of them are listed here.
[24:38] One important topic is for us agent orchestration. The Agent Garden already sets the stage of bringing agents together, but we want to provide agent orchestration capabilities in a more
[24:55] systematic way. Why we already see that in the respective business units, they start ramping up their own solutions for orchestrating agents, and we want to drive their synergies providing a central capability
[25:12] in order to orchestrate, coordinate multiple agents and capabilities across the organization. The second focus is um to seamlessly embed agents and MCP. Right now, it still involves an
[25:29] additional setup. You just saw it via the Databricks proxy app. And going forward, the goal is to have an out-of-the-box functionality from Databricks to make it even easier for us to operationalize that kind of
[25:47] connect connectivity independent whether MCP or agents are located in Databricks or outside of Databricks. And the third topic is semantics and ontology. If agents are meant to act and reason
[26:07] together, they need to have a shared business understanding. And that shared business understanding is coming via a common ontology, which we want to strengthen in our organization. And Sai, let me hand back
[26:23] to you with the Databricks perspective on how we can solve these challenges. Thank you, Markus. So, as you have all seen this morning, right, from our keynotes, there are many new capabilities including Genie ontology
[26:38] and Unity AI Gateway and additional capabilities for Unity Catalog to support this unified AI governance. And specifically to the Mercedes-Benz case, in order to help them achieve their future vision. And we also have those
[26:54] features on our road map to support those required capabilities. First, in terms of agent orchestration, um the customer supervisor agent API is available as beta for you. Anyone of you can uh turn that on, and then you can
[27:10] start uh trying to create a supervisor agent using API. Before, you could only do that in the UI And in this API, you are able to choose um the underlying LLM model um by
[27:27] yourself, and Databricks will manage the agent loop for you. And you can also do the async um um architectures uh for your agent Databricks supervisor agent, and then it runs in the background for those long-running agent workflows.
[27:44] In the second part for agent governance and MCP governance, uh we are continuously uh those features into Unity Catalog, and you probably hear more about that tomorrow in our keynote about the Unity AI Gateway and global Unity Catalog. Um
[28:03] so, the agents and MCP servers will also be native objects inside Unity Catalog going forward. And that will also resolve um the challenge we have today, right, with uh cross-workspace agent access. And because now agents are and
[28:20] the endpoints are not workspace level object anymore, they will become the account level object where you can explore, discover, and access through Unity Catalog. And that will mean the gateway proxy app we developed as an interim solution will not be needed
[28:35] anymore because you can surface those uh agent endpoints directly in Unity Catalog and manage the access there uh with uh fine-grained access control. And last but not least, we also have the custom service policy that you can
[28:50] attach to RLM or MCP, where you can have a more granular control, even in natural language, right, as policy as service policy to your individual RLMs or the agents you deployed and manage in Unity
[29:05] Catalog. And finally, for the semantics and ontology part, you have all seen the Genie ontology demo this morning from the keynote. We support that continuously, and we will also have
[29:21] two ways, right, to to allow you use ontology in Databricks. One way is that you see this morning from the keynote there, Genie ontology is a bottom-up approach. You define your glossaries, you define your business concepts, and you like you write a wiki
[29:38] page, and then Databricks will automatically create a knowledge graph for you continuously using the OntoRank algorithm. And then from there, you don't have to worry about the knowledge graph in in the backstage, because Databricks creates that for you.
[29:54] There's also another way to do that. If you have any existing ontologies defined in open-source formats, for example, resource definition framework or OWL, online web language, right, for ontology. If you have those existing
[30:10] ontology defined, you can also use our open-source lab project called Ontos. And Ontos will allow you to integrate those predefined ontology in open-source format into Unity Catalog to push down that ontology to the Unity Catalog business semantics.
[30:27] And that allows you to create ontology on existing Databricks Unity Catalog objects, and then it allows you to reuse them across your generative AI. So, you have those two choices, both top-down and the bottom-up approach, depending on your use case and scenarios.
[30:44] And I think that wraps up our presentation today. Uh we are happy to answer any questions you have.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.