Scaling Enterprise Multi-Agent AI with Federated Governance on Databricks

Summary

BASF Coatings built MAUI, a governed multi-agent interface on Databricks, to give domain teams a secure, reusable access layer that turns governed data into usable answers without requiring central team involvement or risking shadow AI.
MAUI uses a five-layer architecture built on Databricks Apps, Genie Spaces for structured data queries, Lakebase for knowledge bases, and Unity Catalog for access control, with federated governance distributing ownership to data stewards across 16 business domains.
A self-improving feedback loop continuously optimizes agent quality, with a future roadmap including document generation, proactive notifications, and MCP enterprise system connectors to extend MAUI's reach across BASF Coatings' 70 sites in 50 countries.

Scaling Enterprise Multi-Agent AI with Federated Governance on Databricks

Enterprises demand AI agents across supply chain, marketing, and operations. Without a unified platform, teams rebuild infrastructure independently, creating fragmented architectures, data silos, and shadow AI risk. BASF Coatings tackled this by building MAUI: a governed multi-agent interface on Databricks that enables domain teams to create, operate, and scale AI agents while maintaining unified governance.
Learn how federated governance distributes ownership to data stewards and domain experts while centralizing control through a shared orchestration layer. Discover MAUI's five-layer architecture built on Databricks Apps, Genie Spaces for structured data, Lakebase for knowledge bases, and Unity Catalog for access control. See how self-improving feedback loops continuously optimize agent quality and how future capabilities including document generation, proactive notifications, and MCP enterprise system connectors unlock new workflows.
🤝

Chapters

FAQs

What is MAUI and how does BASF Coatings use it?

MAUI is BASF Coatings' governed multi-agent interface built on the Databricks Data and AI platform, designed to give domain teams a secure and reusable layer for creating and operating AI agents. Operating across 70 sites in 50 countries with around 11,000 employees, BASF Coatings uses MAUI to turn governed data into answers for everyday business users without routing requests through central data teams.

What is federated governance in enterprise AI and why does BASF Coatings use it?

Federated governance distributes data ownership and agent creation authority to domain experts and data stewards, while a shared orchestration layer maintains centralized control and compliance. BASF Coatings adopted this model across 16 business domains because central teams were becoming bottlenecks and shadow AI risk was growing as business users sought workarounds for accessing governed data.

How does BASF Coatings prevent shadow AI when scaling enterprise AI agents?

MAUI provides a single, governed access layer that is reusable and secure, giving business users a sanctioned way to create and operate AI agents without building their own infrastructure outside of IT control. By making it easy for domain teams to deploy agents within governed guardrails, BASF Coatings reduces the incentive to use unauthorized AI tools.

What Databricks technologies power the MAUI multi-agent platform?

MAUI is built on a five-layer architecture using Databricks Apps for the interface layer, Genie Spaces for structured data queries, Lakebase for knowledge bases, and Unity Catalog for access control and governance. This combination, developed over six years of working with the Databricks Data and AI platform, allows BASF Coatings to scale governed agents across multiple business domains simultaneously.

Full transcript

[00:08] Thank you all for joining. Um, it's super great to see so many interested people here in our session. Um, with me on stage is Bernard, product owner for AI solutions. Um, my name is Lars. I'm heading the data and AI office, data and AI team um, in central IT for BASF Coatings.
[00:26] Um, and I'm really happy to be here for the second time. Last year was my has been my first summit. Um, now being here again it's uh super exciting to see how many data and AI enthusiasts are coming together in San Francisco to to see the great news Databricks is about to share.
[00:41] Um, we are here to show you how we scale enterprise AI agent in a governed way in a governed way using one shared foundation. Um, this is Genie 1. Uh, no, this is Maui. But, basically whatever we've seen this
[00:58] morning, I think we somehow built a very similar architecture. So, our approach scaling multi-agent systems on Databricks with federated governance. Um, so if AI becomes part of everyday work, we need an enterprise access layer that is reusable, secure, and owned
[01:14] close to the business. Um, but let me start with some context of BASF Coatings. So, we are operating at scale within 70 sites in 50 countries with around 11,000 employees and then revenue of annually um, about 4 billion
[01:30] euros. Um, our three main business areas are automotive OEM coatings, refinish coatings, and surface treatments. So, most likely every second car you see will have one of our coatings layers um, applied. We've been working with Databricks for
[01:46] Databricks for the last 6 years and we managed to create quite a strong data steward community. So, within 16 different domains, we have data stewards operating, owning data products, creating data products um and making them that life on Databricks.
[02:01] So, the problem is not really missing data. Um the problem is that business users still lack a simple everyday everyday access layer that turns governed data into usable answers. So, even with Databricks in place,
[02:17] getting a reliable answer is still slow and manual. Knowledge is still spread across tables, documents, and expert knowledge. So, people can't really find what they need. They are central teams and they manually stitch data and context together. So, this is creating three problems from
[02:34] our perspective. Data stays underutilized, central teams become a bottleneck, and shadow AI risk grows. So, the question becomes, how do we solve this in a scalable and governed way?
[02:50] This is exactly what Maui tries to solve here. Maui is a governed multi-agent AI interface on Databricks that lets users interact with internal data and documents in a secure and self-service way. It is basically built on four key principles. One interface for many agents, steward-led self-service,
[03:06] modular reusable components, and governance by design. But before we built Maui, and I think this is something you guys seen in your organizations as well, we actually tried something else. And that shows you why the platform approach here matters the most.
[03:22] So, across organizations, um also in our organization, we already had many initiatives to build AI agents. Marketing, supply chain, operations, all working on their own use cases. And what we observed as a central AI team or central data and AI team,
[03:38] um even though the business questions were different, technical requirements of course were almost identical. Every team needed a front end, orchestration layer, data integration, monitoring, and of course data governance. Um so, every team started building the
[03:53] same stack independently. The result is duplication, fragmented architectures, and weak governance. So, instead of continuing like this, we changed the perspective. So, instead of building many isolated agents, we made a conscious shift and
[04:09] moved from thinking in use cases to thinking in capabilities. So, Maui is this shared capability layer. It provides everything almost every agent needs. The interface, the orchestration, the data access, the monitoring, and because its foundation exists once,
[04:25] every agent every new agent becomes dramatically simpler. So, the the domains no longer build infrastructure, they only bring the data and their business logic. So, that's a key shift from rebuilding infrastructure to only bring the data um
[04:40] from rebuilding everything again and again to reusing a common platform across the enterprise. So, now, once you have that kind of shared foundation in place, you unlock something critical. Scaling no longer means complexity, it means leverage.
[04:56] And this is what we see already with our three first pilots being in production. We are see we see um we are seeing measurable impact. So, value grows from 0.5 million to 1.5 million annually as adoption scales. So, most value comes from efficiency, but also from faster decisions and
[05:12] better quality. Importantly, the platform allows us to scale this impact much faster. In addition to that, we do have more than 20 agents in the pipeline. And the reason we can scale so fast is on how the platform is built.
[05:32] Every new agent agent reuses the same shared building blocks. So, we have reusable patterns for genie spaces, rack pipelines for unstructured documents. We have skills and tools. We have MCP connectors, feedback, notification, permissions and access concepts, um proactive notifications,
[05:49] and all is already built. So, all is already there. So, when a new domain wants a new agent, they don't rebuild any of these infrastructure pieces. They can concentrate and reuse the foundation and focus on the data and the business context. So, that's why the solution that used to
[06:06] take months now take day takes days only. And every new use case makes the next one faster. And every reusable building block reduces duplication across the enterprise. That's basically the platform effect.
[06:22] The scaling AI only works if we keep control without creating a central bottleneck. So, that's why we use a federated governance approach. We use the organization we already have. Like I mentioned, we have 16 different domains. Each domain has at least two data stewards that take care of
[06:39] what we call our federated domain governance. They're responsible for data quality and availability and compliance. Domain experts now come into the game and bring the business context and validate that results actually make sense in practice. And together they become agent owners.
[06:56] So, every agent is owned where the data and the knowledge actually lives, not centrally. So, our role as a central data and AI team changes Oh. Sorry. Our role as a central data and AI platform team is um
[07:12] changing right now. Um we don't build every agent ourselves, but we enable others to build it. So, with the platform, reusable components, and best practices, and guardrails, they are enabled and equipped to build agents more or less autonomously. Um
[07:28] The best way to show you how we did this is to guide you and walk you through the agent creation process. So, Bernard will now guide you through a user journey on how to create a Maui agent. Thanks, Lars. Um So, yeah. For that, I will show you now how um Simon built his own agent. So,
[07:46] for that let me introduce you um to him. So, Simon works in marketing and asks himself regularly the question which of his top customers are at highest risk and what are the key drivers behind that. And for that he already knows that he needs on the one hand the structured
[08:03] data, so our sales data, and on the other hand also our unstructured data. So, he wants also to understand the drivers by analyzing our internal visit reports out of Salesforce, for example.
[08:18] So, in the next steps he sits down with the data steward, the responsible one from marketing. So, he knows exactly that our data is already lying in Databricks as trusted data products, so our sales data, and on the other hand also our unstructured
[08:35] data, so our internal visit reports are already in SharePoint folders. Only once the data quality, the data completeness, and also the permissions are set uh complete uh completed or validated, Simon and the data steward starts to
[08:52] build the agent. So, for the structured data we are using Genie Spaces, of course. So, here you can see now in our workspace how we can create here a Genie Space. So, we select the different sales
[09:08] data tables, and then add it to the Genie Space. Afterwards we rename the Genie Space and also enter here a description. After the initial creation um both of them, so the data steward together with
[09:25] Simon, receive from us as a central data and AI office some onboarding materials. How to optimize their Genie Spaces, um for example with benchmarking questions, with grounded truth, or with for example,
[09:41] guide how to write a perfect instruction. After we have now connected our structured data, so our sales data, we can in the next part connect our unstructured data.
[09:56] For that, we jump now into our Maui front end, and here you can now see how we can create here a new space. So, we enter here the name, so in this case our internal visit reports, we enter a description, and also set the permissions.
[10:13] Afterwards, we choose that we want to upload files, and upload here our internal visit reports in this case. Then we select our standard rag pipeline, but we can also alternatively choose an advanced pipeline, where we
[10:28] can for example change settings, like our embedding model, or also re-ranking, or hybrid search. So, now we have connected on the one hand our sales data via Genie spaces, and on the other hand also our
[10:45] unstructured data via these kind of vector indices. So, basically in the background, we create out of these files volume in data bricks, and then the rag pipeline creates out of that this vector index. And now you can see how we can create,
[11:03] or Simon and data steward can create here now the agent. So, they can enter here the name, so Market Mind, they can enter a description, and also um in the next step then choose on the one hand the agent prompt itself, and also
[11:20] the model. In the third step, they can then connect on the one hand the structured data and the unstructured data. So, our coding sales Genie and our just created vector index. In the fourth step, then you can see
[11:37] that they can select different tools and also MCP connections. For example, a simple calculator, but also a send mail tool, for example. In the last step, we set again the different permissions, and then the agent is already created.
[11:55] Now, Simon can really chat with MarketMind, so with the agent. And now, remember the initial question, which of his top customers are at highest risk, and what are the key drivers for that? And exactly this question he can ask now
[12:11] here into MarketMind, and you can see how the question is rooted on the one hand to the Cellchini, and on the other hand to the vector index. You can see that Orion Motors is a customer at highest risk,
[12:26] and you can see also in the answer how it combines now both data sources. So, you can see the sales revenue decreased by 9%, and also you can see the primary driver behind that from our internal visit reports.
[12:43] In this case, there are some supply chain issues, so delivery issues, and also competitor trials. Now, Simon wants to go a step beyond that, and really wants to understand the key drivers behind that. And for that,
[12:58] he asked if it could be broken broken down by the different plants of Orion Motors. And also here you can then see that it's been rooted to the Genie space and the vector index again, and then he receives
[13:15] the answer that Orion Motors Barcelona, in this case, stands out most. And it also shows the reason behind it, and that is that there is a competitor that is already doing uh some trials on our premium line.
[13:31] And now this is a big issue for us, of course. And so Simon, from his marketing role, wants to now send a mail to the responsible sales rep in this area. And so he can now write, "Okay, send a mail." And you can now see our human in
[13:48] the loop tool, um where you can see a nice structured mail with all the information. And he can now edit it, reject it, or just approve and send it. So Sharon, in this case, as the sales rep, receives this information in a nice
[14:04] structured way, and they can now take action on that. And Simon really likes this use case, and he thinks, "Okay, that is not really a one-time task or one-time question. It's more a use case that also all of
[14:19] his other colleagues in marketing have." So he wants to create now a skill out of that. A skill that captures exactly this behavior of this conversation and make it also available for all other users.
[14:34] And you have to remember, as Lars said, Simon is not only in this case a user, but also an agent owner. So now, in the next step, you can see how he creates a skill. So um he can click on the top right on
[14:52] create skill. He choose the different parts of the conversation, click on generate it, and then an AI generates like a skill that captures exactly this behavior. You can see a description, and you can
[15:07] see in the body the different step-by-step instructions, and also some guide rails, and when to use it. Then Simon can save the skill and add it also to the Market Mind agent. And that is how he can now make this
[15:24] skill and this behavior reusable for also other users. And through that, he can optimize the agent, so in this case MarketMind, uh step by step for specific use cases.
[15:39] And this is one way how we try to optimize our agents. And another way how we want to optimize our agents are our self-improving AI optimization cycle. So,
[15:56] now imagine Simon, more and more users are onboarded to MarketMind. And so, Simon and also the data steward receives more and more feedback. What we are doing here is then to analyze on the one hand the direct
[16:12] feedback, so for example, thumbs up, thumbs down, but also free field tick feedback. And on the other hand, also the indirect feedback. So, one example here, the agent could interpret SF for example
[16:28] for um say for San Francisco. Also, the user meant uh Salesforce. And when the user now writes something like, "Okay, that is wrong. SF stands in our case for
[16:45] Salesforce." This is a issue that the agent or the AI directly recognize and save also as indirect feedback. And so, we're analyzing all the traces of the conversations of the agent to
[17:02] really also detect this kind of indirect feedback. And what's happening then is that we try to give the agent owner some suggestions regarding improvements. So, for that, we're comparing the
[17:18] feedback against the agent settings. And with agent settings, I'm not only meaning like the agent prompt that you've seen before, but also the underlying modules, like genie spaces, vector indices, skills, and also MCPs
[17:33] and tools. And even more, so we try also to analyze all the metadata behind these genie spaces, for example, like column descriptions, table descriptions, primary key, foreign key, so different
[17:49] table relations. And as I said, we compare that against the feedback. And then try to give the agent owner automatically some potentials for improvement, and they can just accept it, and it will be added to the
[18:04] settings. After this improvement, the user that triggers these kind of feedback, uh receives a mail that it's automatically now implemented, and that he should test it again, and in the best
[18:21] case, of course, give again feedback, in this case in the like a thumbs up. Um and through that, we optimize the quality of this agent, and we also build trust. And when we when a user trusts the
[18:39] system, of course, he will reuse it, and also he will recommend it. And that is kind of our cycle, where we say, "Okay, we have more users, that leads to more feedback. More feedback leads to more improvements. More
[18:55] improvements leads to a higher quality. A higher quality leads to more trust, and more trust leads again to more users." And now we've shown you a lot about these kind of conversations, but now we
[19:11] should go a step beyond that. So, for that we are already working on some some topics and uh three of them you can see here. So, we want to go from ask and answer to generate, trigger, and connect and want
[19:27] to be here like the proactive enterprise uh workflow orchestration layer. For that, we want to integrate here document generation so that you can directly like generate reports out of that or other documents.
[19:45] Then we want to integrate here proactive notifications like alerts or also schedule prompts. And last but not least, a huge topic, we want to connect our enterprise systems. So, SAP, Salesforce, Office to Maui via
[20:02] MCPs. And with that, just with these three features, we unlock a lot of new capabilities. So, let's go back to our example with Marketmind. Now, Simon could in the future send on a Monday morning to the
[20:19] responsible sales rep a nice structured document with the most important information about the customers that are at highest risk and also with some action buttons where they can, for example, add to a Salesforce directly a risk, where they
[20:36] can like call the customer directly, or where they can generate, for example, a mail. And all that based on our structured uh and governed data that we already have in Databricks. And now Lars will show you a little bit
[20:53] more like how the technical architecture behind it looks like. Thank you, Bernard. Um you see it's super easy to create an agent for business users themselves and you may relate now to the um relation I made to to Genie 1
[21:10] earlier. Looking into the architecture, it's basically not too complicated what we built here. So you see how Maui is built as a fully governed end-to-end system and not just a chatbot. It's an orchestration layer
[21:25] on top of Databricks as our data platform. The top layer number one is our front-end layer. It's a single React application that serves different user types and different personas. So business users interact via chat, while data stewards and agent admins
[21:42] use the same interface to configure agents, manage knowledge bases, and monitor single agent usage. Super admins and monitoring screens are available for platform owner team platform owner teams and central data and IT teams like ours. So already here,
[21:58] we unify consumption and configuration in one place. Layer two is the configuration layer built with fast API. So all interactions go through the central backend where we split interactions in two parts basically. Real-time interactions like
[22:14] chat run synchronously and stream responses to the end users. Longer running processes like REC pipelines and REC indexing are handled asynchronously via Databricks jobs. So we have full control over every interaction.
[22:29] When we look at number three, layer three, that's basically the core of the system. So the agent and runtime. Every agent isn't hardcoded, but it's dynamically constructed from configuration basically. At runtime, we build a graph that orchestrates LLM reasoning, tool usage,
[22:46] and data retrieval. This includes structured access via Genie or SQL, unstructured access via knowledge bases and additional tools like visualizations and actions. This is really where the intelligence happens. The agent decides what to call, how to
[23:02] combine it, um how to combine the results, and how to generate response, all within one controlled governance layer. Layer four, layer four is the native Databricks layer. Um so, we're not building around Databricks, but we are basically building on top of it. So, we use model
[23:18] serving, vector search, Unity Catalog, and identity management. Um and critically, all requests are executed executed on behalf of the user. So, permissions are always respected end-to-end. Finally, layer five is our governance
[23:34] and observability, um making use of Lake Base here. So, everything is tracked, conversations, feedback, cost, agent configurations, and changes. We have full full traceability of what happened, um who triggered it, and how much did it cost. So, the key takeaway,
[23:51] this is not just AI on top of data, it is governed orchestration across the full stack service with transparency, control, and audibility built in from the start. So, you can read this architecture basically as one system, one interface, one orchestration layer,
[24:07] um and intelligent agents on top of it. So, this is our project Maui. Um you can ask yourself, why do we why now? Why do we need that? For us, in our company, we have three reasons. First of all, demand is exploding. Teams are contacting us on a
[24:23] daily basis for, "Hey, I want to have a new agent for this, for that, integrate with system A or B." Um if now if Maui isn't the path forward internally, um teams will build their outside governance, or build governance within their own domains.
[24:38] Second, foundation is already um I mean, we are here on the largest data and AI summit ever, um and scaling is no no additional risks, it's basically high opportunities at stake here. Um and third, cost and control, of course. GenAI can be rate limited um and
[24:54] costly. Without control um in the central control point, usage fragments um and becomes unpredictable. So, for us, the decision was no way AI is not an option, and the real choice is rather shadow AI or governed AI. Um
[25:10] with this, thank you, Benny, I would like to close. Um this is our project Maui, um scaling enterprise multi-agent with federated governance on Databricks. Um If you want to learn more, connect with us here on LinkedIn. We are here at the summit until Thursday.
[25:27] Um

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.