Skip to main content

Building Agents in Production: adidas' Five-Pillar Governance and Cost Strategy

Summary

  • adidas manages 300+ model serving endpoints and 6,000+ models on the Databricks Data and AI platform using MLflow Tracing, Unity Catalog system tables, and automated observability to govern agentic AI systems at enterprise scale.
  • A five-pillar framework covering product design, shipping, governance, observability, and developer experience addresses the unique challenge that agent intelligence is rented from third-party model providers whose pricing, lifecycle, and behavior are outside the enterprise's control.
  • A central MCP gateway provides security control over all agent tool access, an RAI board categorizes AI risk across platforms, and a plug-in infrastructure enables standardized deployment that allows complex agent systems to be built and launched in as little as two days.

Building Agents in Production: adidas' Five-Pillar Governance and Cost Strategy

Watch: Building Agents in Production: adidas' Five-Pillar Governance and Cost Strategy
Enterprise AI projects face unprecedented challenges: agents are unpredictable, intelligence is rented from third-party providers, and cost controls are difficult to establish. As model versions deprecate and prompts drift, traditional ML approaches fail. adidas demonstrates how to build production-grade agentic systems with comprehensive governance, observability, and cost tracking using Databricks and MLflow.
Learn how adidas manages 300+ serving endpoints and 6,000+ models using MLflow Tracing, Unity Catalog system tables, and automated observability. Discover the five pillars for agentic systems, an RAI board for risk categorization, a central MCP gateway for security, plug-in infrastructure for standardized deployment, and an observability suite that tracks versions, lineage, and compliance. See how they build complex agent systems in two days while maintaining governance and audit trails.
🤝

Chapters

FAQs

Why are AI agents different from traditional ML models in production?

Unlike traditional software, agent behavior is non-deterministic—the same agent may respond differently over time as model versions change, prompts drift, or external LLM providers update their models. This video explains that half of an agent's logic lives outside the enterprise in the form of a rented intelligence layer whose reasoning, pricing, and lifecycle are controlled by third-party providers.

What is the five-pillar framework adidas uses for production agents?

The five pillars cover the full agent lifecycle: defining the product and use case, shipping the agent, governing it with an RAI board and compliance controls, observing it through MLflow traces and system tables, and enabling developer productivity through plug-in infrastructure and CI/CD integration. This framework was derived from adidas's experience running classical ML at scale and applied to the unique demands of agentic systems.

What is the adidas MCP gateway and why does it matter?

The MCP gateway is a centralized security architecture that controls which tools agents can access and enforces security policies across all agent interactions. It addresses the compliance risk that emerges when agents call external APIs and services without centralized oversight, which creates audit challenges at enterprise scale.

How does adidas maintain observability for 6,000+ models and agents?

adidas uses MLflow Tracing and Unity Catalog system tables to capture versions, lineage, and compliance information across their model and agent fleet. Their observability suite surfaces insights about model performance, dependency graphs, and resource usage in a way that supports both operational monitoring and governance requirements.

Full transcript

[00:08] Thank you so much for being here and also thank you for choosing this room over 20 others you could be in right now. Uh my name is Mahavir. I'm a part of field engineering at datab bricks. I spend most of my days working with enterprises helping them putting their silk demos to production and together I
[00:24] didn't come alone. Uh I have a friend, colleague and customer Rahul from Adidas. Today we are here to share our pretty honest story on how we are doing agents in production. Yeah. And I mean you
[00:40] would find many talks around people talking about these but what make these talk unique is that we believe in showing you rather than telling you. So we will show you uh things in action in production how these agents works. Yeah.
[00:55] excited. Cool. Then uh let's start just to set the context what we will see today. So it's a fivestep agenda. We will set the stage so we know what we are talking about. Then we will dive deep into how
[01:11] Adidas runs the traditional ML and what learning they can take from there to the putting agents in production and then we will focus on governance and observability layer because that thing exactly makes a difference when you are putting on your agents in production.
[01:26] Yeah. And then we will take this learning to a developer productivity like how we can make things integrated with CI/CD and make whole journey faster. Then let's begin. Let me start with something which is keeping most of you if you are working in agentic systems
[01:43] keeping you most of you in and and awake in the night. What it is? One thing is super clear and we should start thinking about that it is not a traditional software like what what was happening in the traditional time that your software is
[02:00] your code. You run it, you build it, you test it and it behaves exactly same tomorrow the way how it is behaving today. But the agent ST is different. Just look at last 90 days like LLM proxy breach or the malicious package
[02:17] injecting in the pi package or your favorite model getting retired like how Ali mentioned about the methos from Enthropic. Yeah. So these things gives you an insight that and this is the operational reality we are living in that agent tech is not only yours half
[02:36] of it lives outside of your wall and that brings me to something super important part which is that the intelligence layer which we are using today we do not own that we rent
[02:52] it. Yeah. So if you think about it the reasoning, the behavior, the pricing, the life cycle of those models are lives with the provider. What do you own? You own your data, your prompts, your tools,
[03:07] right? But the brain as again you are renting it. And that's the new operational reality we are living in. Which means that the landlord can change the lock any Tuesday morning without letting you know. And you need to build a system in a way that you still be
[03:24] responsible because when the governance key in the audit key they wouldn't take that as an argument that hey we don't own the intelligence layer. So that which gives me an question or at least should give you a question as well that hey who are these two guys talking
[03:40] about these what are their credentials and that's a fair question to be honest. Let me answer it from my angle first. Edidas right the customer who is just standing next to me are not watching this transformation from the sidelines they are leading it 180 plus classical
[03:58] MLS in production millions of traces right and last year I'm not sure whether you were there in our last talk with the edidas we were talking about three agentic workflows and this year end we are having 25 plus in production that's
[04:13] the real scale we are talking about and the stacks are high that makes this talk really worth listening for next 30 minutes. So if we go further um let me just paint the picture before we dive deeper into it that you can think it off as like a
[04:29] three layer architecture what we are building at Adidas. First it starts with the foundation layer where our data lives in the lakehouse. Then comes an agentic layer which is a intelligence layer where our agents are rented intelligence and all the things followed
[04:46] to that leaves and the and on top it is a surface layer where the consumption happens means your applications your customers AIB genie so and so forth are the consumptions one and all these thing glue together with our unity catalog ST
[05:02] right now let's just keep it to that we dive deeper as we go around this talk. Now something which is very fascinating to understand here is that Edidas is building the ML from years. They have
[05:18] started it with the classical MLS like building forecasting models, pricing models, personalization so and so forth. But if you if you think carefully these are the models which are predictable, right? You know what is failing, you know their loopholes, so and so forth.
[05:35] But when it comes to an agentic system, the story is not that the story starts the moment you put it into production and you need to really care about how does the behavior changes the moment the intelligence layer changes right and that's the real hard part and that's the
[05:52] part which we would be diving deep today a lot in the further down the talk now if I go down to that path you would understand that just prompting a claw or open AI is just at the starting point
[06:07] that's not an end when it comes to an working with the agentic system means that the real story begins when actually you put your observability and governance in action and that actually is the most hard part nobody talks about
[06:24] when you go in such a conferences which we will not just going to talk about it we just going to show you as well so at Edidas we believe that there are five pillars while thinking about designing any agentic system it starts with the product fit then it comes to a tech fit
[06:40] quality and the governance and observability the first three three of them the product tech and quality you guys can make sure while building a good PC's right that hey is it a good product fit is it have a good tech is it have a
[06:55] good quality so and so forth what are the metrics and so and so forth but the real problem as I again and again taking these two names the governance and observability are the one which you actually face when something fails right like one Tuesday morning you see that okay pi injection
[07:13] happened then you would like to know that how badly my agents are affected and how many of them are affected so that's and and that too you need to know fast and that's most challenge which we face why these two are important I mean
[07:30] imagine that the tomorrow the audit comes of EU AI act or US AI act right you do not have an argument as I mentioned that hey the intelligence layer we do not own the stacks are super high 7% to the global GDP penalties so
[07:46] you are still responsible where you are still renting your intelligence layer so we need to make sure that these two pillars the observability and governance stay intact stay production grade that you can put your name behind that
[08:01] particular application that enterprise name behind when that goes in production and face your customers end of the day. So that is the starting context as I promised for the first part. Now let me get out of the way and bring Rahul who deals with these every day uh in
[08:18] building such a system, building such a agents and putting it into production. Yeah, welcome Rahul. Thank you. Okay. So, first mic check because this is what I uh what I have been instructed
[08:34] from our u experts that I have to speak louder so that you can hear me clearly. So, people sitting in the back can you hear me clearly? Yeah. Okay. Fine. Good. Okay. So, let's start introducing myself. My name is Rahul. I am a senior
[08:51] solution architect at Adidas. uh with my team we are building engineering solutions standards which most of our stakeholders who are working in our lakehouse platform are using it to build AI products and before jumping into agents I always
[09:08] like to start with a story where are we coming from what we build and this is a story about last Tuesday what happened with an ML use case which we use to run some optimizations in the back end nothing exciting I think it's something
[09:24] most of you might have built some kind of an optimization use case. But one key thing I want you to focus on which will really play very well when we talk about agents. So this ML use case that we have deployed
[09:40] every every day there's a monitoring system that runs and basically what it does it tries to look at the threshold that we have set and based on that if it identifies certain drifts is happening then basically what it does it triggers
[09:55] the model retraining once the retraining happens then around let's say certain timeline some illals are happening so we We always focused on evaluation that before you go to production even for
[10:10] traditional AI. This is where we always started saying hey we you need if you really want to build a scalable system you need to think about how you are going to evaluate your system when it breaks in production. So it runs some evaluation. It it logged
[10:28] a new version of the model and it sends an email to a product owner saying, "Hey, there's a new new candidate version is available. Can you have a look? There's a report attached to it and you get you can compare the version with a with a previous version which is
[10:43] running in production and you can check the evals." she she went through it and she approved that uh that process and then a new version is available and inference pipeline picks up at 9 9:00 in the morning and picks up the new approved
[11:00] version for inference. So this is a sketch of an architecture that we what we what we how we built this system. If you look closely everything is a module. Every module is talking to each other. There's a there's
[11:16] a monitoring module, there's a notification module, there's a there's a model pipeline triggering training retraining modules and so on and they work on the rules that we have set and this is what Mahavir mentioned that things were very predictable so you can
[11:33] set up rules when to trigger what. So this is how we were doing traditional AI in Adidas and we have few more of these use cases. In lakehouse we we are hosting pabyte
[11:49] scale of our data's data which empowers our analytics applications and if you look at the value chain of how across Adidas we are using ML we are using AI for product creation inventory
[12:06] management recommendation engine consumer analytics uh personalization suits and so on. There are hundreds of models that are running in production. They react to drift and they ship into production with the same pattern that I showed you. This
[12:21] is the these are the standardization. As a platform team, we are working on that as a consumer, as a product owner, you don't need to rethink about the same setup or come up with your own strategy of deploying things to production. How you can use the same vending machine and
[12:37] actually use this your your your model to ship it there. So what is the scale that we are talking about? We have more than 10,000 experiments logged in MLflow in production which is and we have more than 1.3
[12:54] million ML runs. We have more than 7,700 models model versions in production and basically we have serving endpoints more than 300 plus. Some of these serving endpoints actually experience two
[13:10] billion plus requests during the peak season. That translates to 10,000 queries per minute. And when you are going real time, you really need to think of how you deploy scalable ML in production there.
[13:31] So now let's talk about pillars. How how we ship things to production. And trust me after five minutes you will start seeing the clear picture how we actually went towards the agent because this is where we were coming from. This is the past uh six seven years of experience of AI or how we were shipping things to
[13:48] production and we actually started pulling pieces from there and tried to think how what we need to do with agents. So pillar one when you develop you need to develop with a product mindset. In the end you are not solving
[14:03] only a business use case that you are solving but when you are thinking like a product you will have your customers and how how you build these scalable products that can serve your end consumer. So which means every product has a life
[14:19] cycle in case of AI you will manage your life cycle with MLflow. Don't write a long 1,000line notebook. try to modularize it as as much as possible because you will be running a lot of AI use case in your team and it's
[14:34] always better to build module that you can inherit in different projects. Eval is one of the main criteria before you are even eligible to move even your traditional AI use case to production. We hear a lot about how eval are very
[14:53] important for agents but it is it was also very much important from traditional AI and if you are doing traditional AI in a right way most of the learning you can take it to agent system. Then we have
[15:08] ship. When you are shipping things to production, basically how are you going to ship? How are you going to ship things to production? You are going to ship this things to production with a production mindset. You need to think of building
[15:25] your product where everything goes through a same standardized process or a same vending machine. Every model that you are building should be deployed as a code. No pickling of a model, shipping it, copy pasting into
[15:41] production and so on. It's everything deployed as a code and you have a standardized CI/CD process and no access to production that you can go and tweak things.
[15:57] Then then comes the govern part and this is where we started thinking from the enterprise mindset. We have so many platforms and we always wanted to see how we are shipping products all these
[16:13] ML products that we are building how at an enterprise level if there's a question about hey what kind of Python libraries you are using this version is not allowed or something like this how quickly we can answer those so this is
[16:28] where we started working or leveraging unity catalog system tables and build our own observ provability suit which I will show you how it looks like. So when agent arrived we asked ourself where to start and I want to make sure
[16:45] that my my clicker doesn't show the next slide before I click because this is happening a lot. So when agent arrived we all asked our question uh question to us where should we start how how should we
[17:01] deal with it and there are there were a lot of blog posts talking about MLOps LLM ops all fine but on on an enterprise level how do you think how are you going to introduce agents that will have a total cost of ownership higher
[17:17] and this is where the first thing we said as a rule across our don't start using agent for sake of it start where it matters and the first thing what we tried to look into what are the categories of agent where we we
[17:32] really see a benefit. So we I we we created like a impact versus value kind of a uh pattern that you see here. And here what you see that there are two kinds of categories everyday and tactical AI for personal productivity or
[17:49] some of the use cases chat bots or something what you want to build. And there is a game-changing AI where you really are using agents to enhance your or increase your development processes or you are also introducing some kind of
[18:05] a let's say you want to use it in content creation and so on and it is a very important thing to notice that this is a$100 million dollar question where it lies which Mahavir was briefly mentioning over or
[18:21] under investment in one of the core categories can lead you that you are bleeding bunny but you are not getting what you are asking for so this is where we said okay let's categorize it this is the playing ground where we will start working then as I mentioned we have so
[18:37] many platforms and now we needed to define our guiding principle how do we measure that from a security perspective what are the capabilities of different platform not instead of okay this platform offer us this feature. This is
[18:54] a cool use case. You go to UI low code solution. Amazing. But rather start from a fundamental question. Where do we see security gaps? So we define nine guiding principle 46 key capabilities align to OA's top 10 for agents and try to rate
[19:12] different platform and try to see which are the gaps we have. And the the idea was not to compete between platforms. The idea was you identify a gap and your engineering team like us start fixing those gap building solutions to fill those gap. Why? Because one thing that
[19:30] we need to understand in agentic world you cannot confine yourself in in into a single platform. You agents will start interacting with each other in different areas. They will tap into one platform for one tool access. They will tap another MCP
[19:46] servers and so on. So how do we secure our parameter? That was the question we tried to answer with this. Then let's think about our consumer. We we have so many new things, new stuffs are
[20:02] popping up in different areas. How do they know know for their specific use case where to start? What is the right place to start? What kind of data you are going to use? Where is your data? If you are building some normal
[20:19] application like uh application interface application agents how do you connect to those systems? So a simple kind of a guideline to our stakeholders that okay you start from here this is your use case there's an RAI board we
[20:35] have which basically consists of legal and infosc that tells what kind of use case that you are building what category it falls so since the beginning you are basically very much you can you can get information about are you lying into a
[20:51] high-risk application or a low risk or medium risk or something like that so we have an RAI board and that RAI board also give some kind of a recommendation about what which platform suits well where you can grow your use case faster
[21:06] and as I mentioned and this is the this is what we started building last year. We said that in future we cannot restrict building agentic system which it can just talk to a certain parameter of the platform.
[21:23] Every platform is offering their own MCP server. You can register your custom MCP servers and so on and so forth. But now I want you to think from an infosc perspective. Imagine you are a person who is working in infosc and now you have 10 platforms all of the
[21:41] is MCP server and now you you identify there is some vulnerability that happened and you people have built some agentic system not simple one very complex one that touches multiple time different platform you will spend months
[21:58] to really identify and do a root cause analysis for this such kind of thing. The other problem is every platform offers different kinds of authentication and authorization. This is also a bigger problem when you are thinking of building agentic system that span across
[22:14] platform. So this is why we built our own MCP gateway. We started last year and you might have heard some news recently for sure. So that that's uh that solidifies that the direction that we were taking. Now we will we also have
[22:30] unified AI gateway from datab bricks. Uh very excited to try it on. But the idea was with this central MCP gateway now anyone who is registering or creating tools it is being registered at one place. You have observability at one layer and just as an example platform A
[22:49] has an MCP tool which platform B wants to use. So currently this is how it works. Basically platform B will go to our gateway tap into this tool say okay this is a tool that I want to have access to and that's it because behind
[23:06] the scene we validate everything with entra and no authentication and authorization you need to take care because everything is baked in in our gateway what what value it brings it's basically you do not need to not every product
[23:23] team needs to think about authentication and authorization it is very well managed through our central gateway there. So the question is after identifying where to start I think we we had a very good start of
[23:39] telling like what we what what is our boundary what is our playing field and basically what are the key areas where we need to focus on and this is where we started building our product. So what actually breaks if you shift from traditional AI to agent?
[23:56] So what carries life cycle the way we were building traditional AI the product life cycle that we were managing it's it stays more or less the same now we have few more component in in between especially from the RAI board that they are in the center to really look into
[24:12] what kind of a C use case that we are building. uh CI/CD same vending machine but actually you can do some customization which is also very much achievable distribution in our from our central team what we
[24:27] were doing was that we were always building templates for example for forecasting computer vision and so on and you always have a possibility to change those template but you still follow the same pattern but here what we did is that we built a complete
[24:43] infrastructure for our community members that because this agentic system is quite new design pattern is keep on evolving. So what we said okay you can build your agentic solution transform it into a template we standardize it and now any of the community member across
[25:00] adidas can use it. So this is how we leverage community to really empower new design pattern for agents in our uh standard platform offering. What breaks definitely uh I think it was
[25:15] very easy three years ago when someone would have asked okay Rahul if you are building a recommender engine and going to deploy it to model serving endpoint connected to adidas.com uh how much we are going to pay for it
[25:31] rule of thumb how much you actually uh born by training that model and looking at your traffic profile you can estimate What would be the cost? But for agent, this is going to break because people can bring build really complex multi-
[25:48] aent system that can have access to multiple tools that can try multiple hops that can do infinite tries. And now if you are in the product manager or and being asked in the beginning of the year, hey, can you estimate how much this agent would cost?
[26:05] I think this is a very difficult question to answer. Then uh failure mode before it was very simple. You have a threshold on certain things. Things are changing triggering a retraining pipeline and your model is up and it can be consumed. That is the example that I gave. But now drift can
[26:23] happen because suddenly model version is not anymore available and maybe due to that prompt there is a drift in the prompt. Maybe there you have implemented something which which causes infinite loop and so on and then the same thing
[26:40] applies to audit and observability. So what we started with we as I mentioned that we offer a kind of a vending machine or a C standard CI/CD. What it does basically it helps you to
[26:57] deploy jobs create tables and so on. So when you are moving to production you just use the CI/CD process and we make sure that every product who is building or ingesting data they follow the same pattern and one of the proof of that
[27:13] standardization how it worked for us during the Unity migration we migrated over more than 5,000 tables in Unity because we started 2 years ahead uh when Unity launched uh we migrate we we migrated thousands of shops, thousands
[27:30] of models and basically because everyone was following similar pattern we from a central team we just ran an evaluation what what is going to break and we created a migration guide and help it and which is which helped our
[27:46] stakeholder to move faster to unity. So this is what standardization bring to the table when you are standard standardizing everything. So here what we did is that we introduced a concept of plug-in because what we believed that hey if you are an AI product owner and
[28:03] most probably you are building already a traditional AI use case and most probably now you want to introduce agent or maybe you are a new product owner who wants to really from scratch want to build agentic system more or less we don't need to redefine the entire
[28:18] framework what we need to do is that we just offer an extension now think like a python python uh package. Now what you can do with this core plug-in concept is that you do import agent deploy, import genie deploy, import governance deploy and so
[28:34] on and basically your CI/CD does the uh graph management and does the dependency management and deploy your things. Now someone might ask a question, hey why you do this over complicated thing? I can just create a job, a notebook, a
[28:52] notebook that says what is my dependency and it can just deploy things there. It it will work for a simple agentic system that you want to build maybe a rag system. But here we are talking about agents that are dependent on different data products. So imagine I want to have
[29:10] access to knowledge base what Mahavir has built in his product. I want to have access to a couple of genie spaces in other products that people are using and now I want to deploy this. So what I need to do is that I need to build this complete dependency graph where I tell
[29:26] before the deployment of my agent that this is my dependent resources and when you deploy check that if everything is working fine and then you also need to implement what what happens if underlying resources changes then it should re-trigger my
[29:43] agent version new version of agent deployment but this is what we standardize here because imagine right now we have 3,000 develop ers who are developing all these solutions and and basically uh if if everyone go and build
[30:00] the same setup we we might see some uh some deviations. So that is why we started building all these core component that you can just import and from from while importing it you can just use it to deploy the these things in production. So basically for example
[30:16] agent deployment what it does as soon as you deploy agent it will just uh it will do the dependency management checks if all of the resources are correctly defined and it will guide you if you do not have access and so on. Now observability suit this is the
[30:33] enhancement what we did the graph the picture is very complicated so just think of three pillars one is a scanning tool that runs every day every time someone is deploying something we get notified or we get grab everything in the back end that this is the new
[30:48] resource that has been deployed resource means maybe vector database then it sends a message to our second pillar which is observability or governance pillar where we have defined centrally and this is the guiding principle which I was talking about how
[31:04] this resource was created. Does it follow certain patterns that we identifi define on a platform level and it does the validation saying okay this resource is validated or not validated and then there is a third one is called terminator that terminates if your
[31:20] resources are not valid. This is one of the example. So what you are seeing here is that uh one of my colleague just went to a workspace created a model serving endpoint from a unity catalog and basically uh he forgot to add tags. Uh
[31:37] this is the account name that we use to bind product to to the resources and this is where the serving end uh our observability job scanner was running and it identified that okay there is a new resource that is being created and it it it is sending email to that user.
[31:54] Hey, you have just deployed a resource and basically you missed these things which is a kind of a validation principle from our side and these are the steps that you can do to make it uh more governed
[32:12] and this is how our observability suit looks like. So basically right now we can just go there we can click and we can go to different workspaces we can see uh how in different workspaces what are the resources that is being deployed uh how many of them are valid or invalid how how many of them are following uh
[32:29] best engineering practices or not. Then someone asked hey what happened uh basically if we if we go and let's say tomorrow uh some
[32:44] clot version gets deprecated. So what we do is that we also have this in our observability by version. So technically if uh tomorrow cloud 3.7 goes away uh this from here on on an enterprise level
[33:00] we can filter and see all data products that will get affected due to this. So we are we are looking from the version perspective how data products lineage are linked together and how they will get impacted as soon as certain model art versions are going away.
[33:17] Then I think uh this is not only limited to this. We can even go right now with our observability MLflow version, DBR version, spark version. So technically we can go on a run level, data product level and we can really see which data
[33:34] products are actually using MLflow maybe version two and let's say as an enterprise we say now only MLflow 3.0 above should be allowed. So technically we can see everything from here.
[33:52] And then now whatever you saw it was how we are governing our product and agent part is already now we are we are incorporating it and enhancing it there. Now how we are helping our software engineers very very simple this is a
[34:10] simple uh diagram that will give you an idea of how setup is being done. If you remember we are we were talking about MCP gateway right? Uh one of the core issue is that uh you can right now go and just install any public available MCP in your local in an stddio mode you
[34:28] can work with clot code. When we introduce clot code in addittors, we introduce it with some restriction that your clot code in adds cannot connect to any MCP that is not being registered in our gateway that is not part of of our
[34:46] MCP registry. So that means if you even install AI devkit in your local computer and try to use cloud code or GitHub copilot, it won't detect those MCP tool that AI devkit brings. What we do is that we have our MCP registry and we do
[35:05] like a version control. So every time new version of let's say AI devkc is coming we create a new version image and then you can as a developer you can pull those image in your local container as an stddio mode or in a from a remote mode and you can connect to your VS code
[35:21] and with that your VS code GitHub copilot or cloud code can detect only tools from there and this helps us to govern from the infosc perspective and if something ch some vulnerability happens we can just have one place to cut the tie and everyone loses access of those
[35:38] tools. So what we did is uh Mahavir was mentioning about one of the product that we built which we presented last year about product review summarization. It was an amazing use case where you just give a bunch of reviews to LLM and
[35:54] say hey figure me out what is happening and can you tell me if some key insight that I can grab and it took us a month. it took us a month to to build that pro prototype. Uh but with this kind of setup, we were able to build it in two
[36:11] days. So this is the major uh benefit that we got. But one of the most important thing of course is that what we built followed the governing principle that and the constraint that we have. So technically all the
[36:26] governance and observability that we have built around and how we are controlling our coding agents uh with that kind of a setup we we managed to bring things to production ready state with a CI/CD and everything through cloud code.
[36:42] So after building this I think the first thing was uh we were very happy. Yeah. Uh we we did something we can prove something. But then later I think after after this dawn subtle and then we were just
[36:58] chatting with uh with some of our friends and said hey now we can ship very fast but you know what now governance actually matters a lot. So if someone asks hey you guys are doing a lot of constraint and governance around it um basically it matters more.
[37:16] Why? going from one month to do two days. This win is not free because it changes everything. If you calculate the new math from the platform perspective or the enterprise
[37:31] perspective, more prototype that you are shipping, more of endpoints you are creating, more of vector search you are creating, more of everything you are creating in the platform and the platform is the only thing that can keep fast enough as compared to us as humans
[37:47] because more assets that we are creating verification from the human take will take months. So one thing we need to accept speed is a new default. But what we can do about it is
[38:03] what happens with that speed. Without governance speed compounds against you. So that means uh cost would explode. That means uh drift goes unnoticed. Ungoverned asset will grow. But with
[38:19] speed, with speed um you can still drive safe, right? Um that's what we do in Germany, right? So you need to have a better observability governance and last one is eval. And
[38:36] that's it from our side. Thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.