Too Many Agents, Too Many Headaches: The Chaos of Coding Agent Sprawl

Summary

This video explains how coding agent sprawl—developers using dozens of specialized AI coding tools simultaneously—creates a direct conflict between developer flexibility and the centralized security, governance, and cost control that IT administrators need, and how the Unity AI Gateway resolves this tension.
The Unity AI Gateway acts as a provider-agnostic control plane through which admins configure dynamic guardrails, manage model and MCP server access, enforce rate limits and budget policies, and gain end-to-end observability—all without restricting which agent tools developers use.
MLflow tracing is integrated to capture complete agent execution sequences and token consumption for real-time debugging and security audits, and Databricks internally reports that AI users are merging 60% more PRs—shifting the bottleneck from code creation to code review and CI/CD infrastructure.

Too Many Agents, Too Many Headaches: The Chaos of Coding Agent Sprawl

Enterprise developers are using a wide variety of specialized AI coding agents to boost productivity, leading to an issue known as "agent sprawl." this video explores how Databricks bridges the gap between developer flexibility and IT control using the Unity AI Gateway. The platform enables organizations to centralize security, manage credentials, and gain full observability without restricting developer tool choices.
𝗞𝗲𝘆 𝗧𝗮𝗸𝗲𝗮𝘄𝗮𝘆𝘀:
• 𝗔𝗴𝗲𝗻𝘁 𝗦𝗽𝗿𝗮𝘄𝗹: Developers use distinct agents for authoring, refactoring, and prototyping, creating severe governance and security challenges for IT admins.
• 𝗨𝗻𝗶𝗳𝗶𝗲𝗱 𝗔𝗜 𝗖𝗼𝗻𝘁𝗿𝗼𝗹 𝗣𝗹𝗮𝗻𝗲:The Unity AI Gateway acts as a provider-agnostic bridge to manage access to models, tools, MCP servers, and external agents.
• 𝗚𝗿𝗮𝗻𝘂𝗹𝗮𝗿 𝗚𝗼𝘃𝗲𝗿𝗻𝗮𝗻𝗰𝗲: Admins can define dynamic guardrails, restrict read/write access (e.g., Google Drive), enforce rate limits, and control budgeting policies.
• 𝗭𝗲𝗿𝗼-𝗖𝗼𝗻𝗳𝗶𝗴 𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗲𝗿 𝗙𝗹𝗼𝘄: Developers access managed infrastructure via thin clients like UCode without needing to manually manage API keys or configuration files.
• 𝗘𝗻𝗱-𝘁𝗼-𝗘𝗻𝗱 𝗧𝗿𝗮𝗰𝗶𝗻𝗴: Integrates with MLflow to trace agent execution sequences, latency, and token consumption for real-time debugging and security audits.
• 𝗢𝘂𝘁𝗲𝗿 𝗟𝗼𝗼𝗽 𝗕𝗼𝘁𝘁𝗹𝗲𝗻𝗲𝗰𝗸𝘀: Accelerating the developer "inner loop" shifts organizational bottlenecks to code reviews, CI/CD infrastructure, and on-call operational workflows.
00:00 — Introduction and Conflict Between Developers and IT Admins
00:46 — Understanding Agent Sprawl & Ecosystem ROI
03:37 — Security Risks and "Blast Radius" of Coding Agents
05:47 — Introduction to Unity AI Gateway
08:04 — Admin View: Managing Models, MCPs, and Governance Policies
10:34 — Developer Demo: Multi-Agent Workflows with UCode
12:38 — Observability: Centralized Analytics & Cost Tracking
13:25 — MLflow Tracing and Live Agent Debugging
13:46 — Historical Context: Adoption Trends and Scaling to 5,000+ Customers
15:02 — Future Outlook: Pressure Testing the "Outer Loop" at Machine Speed

Chapters

FAQs

What is coding agent sprawl and why is it a problem for enterprises?

Coding agent sprawl refers to the proliferation of different AI coding tools used by developers across an organization—each requiring separate API keys, configurations, and model connections—with no central visibility or control. This creates security risks when credentials are managed inconsistently, prevents IT teams from understanding what models and tools are being used, and makes centralized cost enforcement impossible. This video describes the conflict between developers wanting tool freedom and IT teams needing control as the core tension that agent sprawl creates.

How does the Unity AI Gateway address coding agent governance?

The Unity AI Gateway acts as a provider-agnostic bridge that sits between developer tools and the underlying models, MCP servers, and external services they need to access. Admins can configure dynamic guardrails, restrict permissions such as limiting certain tools to read-only access, enforce rate limits, and set budget policies—all without requiring developers to change which agent tools they use. Developers access the governed infrastructure through thin clients without needing to manage API keys or configuration files manually.

What observability does the platform provide for coding agent activity?

The platform centralizes analytics on model usage, latency, and token consumption across all coding agents, giving admins a single dashboard for cost tracking and performance comparison across different models. MLflow tracing is integrated to capture complete agent execution sequences, enabling debugging of multi-step agent workflows and auditing of actions taken. This video demonstrates live agent debugging using the MLflow trace view.

What happens to the development workflow as AI coding agents accelerate the inner loop?

This video notes that as AI coding agents dramatically speed up code generation, the bottleneck shifts to the outer loop—code reviews, CI/CD infrastructure, and on-call operational workflows. Databricks internally reports that AI users are merging 60% more PRs, meaning downstream processes must scale to handle the increased throughput. Enterprises need to address these outer-loop constraints to fully realize the productivity gains from widespread coding agent adoption.

Full transcript

[00:04] My name is Bilal. I work on AI at Databricks. So, I spend kind of most of my time thinking about agents from two sides. What developers actually want and what keeps IT admins up at night. It turns out like these two things are in direct
[00:17] conflict. Developers really love the freedom. They want to use the tool, right tool for the specific task. And no matter what that tool is. On the other hand, you know, admins want control. They want visibility, they want
[00:30] security, they want cost controls. And turns out like right now these both things are in conflict. Every cloud vendor, every open source framework, every model provider is shipping like
[00:42] coding agents. And we have like a very great rich ecosystem. There's like Cloud Code, Gemini, Cursor, a lot of like open agentic harness tools, Open Code. Um every week it kind of seems like new
[00:56] agent agentic harness is coming out. And honestly a lot of them are great. They're adding like real value and we keep hearing from customers that they actually have increased productivity. And finally we are actually seeing that coding agents are the thing that's
[01:09] actually making it real. Um and internally, you know, inside Databricks we actually are seeing like AI users are merging 60% more PRs. Everybody's actually you know, writing code at Databricks and
[01:22] shipping. We're shipping, everybody's shipping. And the the industry kind of has grown from almost zero to like, you know, multi-billion dollars in departmental AI coding in just in just a year. So, many times we do see that
[01:34] enterprises would mandate using a certain coding agent. But the way kind of kind of like developers think is that they want to use like one tool for code authoring, another for code refactoring, and then another for one for code
[01:46] review. And I'll I'll give you my example. Like I use like a bunch of coding agents. Like I use a combination of Cloud Code and uh Cursor for coding, Lovable for prototyping, open code for
[01:59] product management work, and then I use by for running like asynchronous scheduled workflows. And I know I'm I'm not the only one. I see a lot of people like using even more agentic harnesses with a lot more variety than me. You
[02:13] know, as a developer, it's also a little bit of overwhelming to use different tools um cuz all of these tools kind of need like some sort of like resources. Like, you know, first you need to be able to connect to models. Then you need
[02:25] to have a way to be able to connect to your external services through MCP. Then you need some like re- some way to like reuse or share these runtime behaviors that you can do
[02:38] with plugins or skills, and then you need memory. And when you onboard to a new agent, you actually have to manage all of like these pieces independently. So, onboarding to a new
[02:50] coding agent is not that simple. You really need to be able to have some sort of infrastructure that you need to be able to like maintain. And that's really the sprawl that developers see. Now, if
[03:02] you are a admin, things are even more miserable on your side. Like, every time a developer picks a coding agent, this means like you need to do like a new one vendor contract.
[03:15] And you need to onboard like a new admin console. There's like a new billing metering. You have to do legal review. Um you need to like, you know, manage the the whole the security of the setup.
[03:27] So, it gets like really, really overwhelming, and that's why we see like a lot of enterprises are reluctant to onboard all of these like, you know, multiple coding agents. And that's like And beyond these like chaos, there's
[03:39] like a real security concern underneath all of it. And they have been We have been thinking that, you know, the keys of the kingdoms were with database admins or with AWS
[03:53] root account. And in 2026, it's kind of seems that that keys have like shifted to coding agents. Cuz when you're running coding agents, these are not running in isolation. They are actually connected to your enterprise live systems through GitHub,
[04:07] CICD, your databases. And when these agents run, they actually in many cases run on behalf of engineer permissions. And they're running at like machine speed. And then like I said, like you
[04:20] know, every time you install a skill or plugin from from from internet, you don't know what's in in in that in that piece of piece of code. It's many times not being reviewed by by someone. And it's kind of
[04:34] like running without any proper vetting and proper like audit checks. So, we're kind of the concern really here is that the the blast radius of these coding um agents is like significant. So, here's
[04:47] kind of where we end up. Developers they do want to flexibility. They want to use different tools for different tasks. Enterprises admins are really panicking. They want to make sure like they have
[04:59] visibility, they have unified billing, and they have like guardrails that they can put on those traffic. And and the security surface keeps on expanding every time you kind of install or run an MCP or skill from internet. And one
[05:12] typical approach really that we have seen is that, you know, enterprises would just clamp down on these agents. They they would kind of like mandate that use this agent and they would shut down go ahead and shut down all the other agents. That's kind of like really
[05:26] really limiting for developers, like especially if you you know, want to move fast and want to use like many of these like specialized agent to harness. And we also have seen like developers will always figure out a
[05:38] way. And what is even worse, cuz now the problem has kind of gone from visible sprawl to invisible sprawl. So, you really cannot um restrict your way out of this problem. And one pattern that we have seen over the last year or so is
[05:53] that the industry has been converging on this AI gateway concept. Which is kind of very similar to, you know, API gateway. So, just like you know, with API gateways, you can centralize and govern all the API
[06:06] traffic. We are now seeing a lot that enterprises are now standardizing on this AI gateway, which basically is like a single control plane that you can use to govern all your GenAI traffic. And we
[06:18] actually Databricks internally onboarded to AI gateway back in 2023 before we even shipped our first like, you know, chatbot. I I'm not sure if you have been following the story of our AI gateway,
[06:32] but with the latest one, we kind of like have made two big changes. First, we have expanded the scope of resources that you can govern with AI gateway. So, you can now use it to govern all GenAI
[06:45] resources from models to, you know, MCPs to skills and to third-party agents. And the other thing that we have done is that we have integrated AI gateway to, you know, Unity Catalog, which basically
[06:58] means that now you have a central place where you can go and define these, you know, policies. For admins, it means that now they have like one control plane where they can go, see all GenAI resources in one place. They can do
[07:11] permissioning, you know, put do rate limits, put guardrails. And because all the traffic is kind of going through like one choke point, they get like, you know, 360 visibility of that traffic.
[07:25] So, they can now understand like how developers are actually using GenAI and how what changes they can do to actually improve their productivity. And the other thing that also it helps is just like making sure like developers have
[07:38] access to these models that they so that they can build agentic apps in a more reliable manner. And for developers, it means a lot of freedom. They can use like different agentic harness, different models, different tools. Uh
[07:51] and because everything also now is cataloged in one location, it also makes it very easy to browse and and search and find tools and skills that plugins that other
[08:04] team members have created. Let me make it a little bit concrete and I want to show you like a demo of it and I'll show you two sides to it. Um I want to show you how developers can use multiple coding agents and then we'll see how the
[08:16] admin side of the story looks like. So, if you kind of log into Databricks workspace, you would see on the left side, left nav, you'll see like AI Gateway. And when you click on it, this is kind of like the view that you get. Unity AI Gateway actually comes with lot
[08:29] of auto models out of the box. So, you have like, you know, models from all three leading research labs available. But then you can also bring this external integrated with external models. That's kind of like a very core
[08:41] concept of AI Gateway. It is meant to be provider and model agnostic. Like you can use it with models that are deployed on Databricks or are deployed elsewhere. And the same thing for MCPs, like you
[08:54] can see all the MCPs that have been um provisioned and set up by your um platform team. Now, on these models,
[09:06] you can go in and actually you can now do permissioning on it. Uh you can like, you know, set uh guardrails um and we support like bunch of guardrails. Like my favorite one is you can actually
[09:18] set up like custom guardrails, which you can just express in the form of like, you know, a like a prompt. Uh you can do like budgets, can do like rate limits. Um and you can do it at a model level
[09:30] or you can do it at a gateway level or you can just express it in terms of like some like, you know, very complex policies. And with MCPs, you can also choose like how you want the odds to work. So, you can either use a shared
[09:42] auth or you can use like on behalf of authentication. And one of my favorite kind of like a feature here is that you can do like very granular governance. So, you can say
[09:54] that users can read from Google Drive, but cannot write. Or only certain people can write to Google Drive. There is like a dashboard that we provide out of the box. You can basically here get like one
[10:07] single pane of glass of how, you know, developers are actually using Gen AI across your organization. So, you can see like, you know, usage trends. You can break it down by models, endpoints,
[10:19] users. And there's also like a performance tab, which goes deeper into latency, time to first tokens. You can break it down by custom tags, by teams,
[10:31] by users, by service principles. Now, let me show you what the developer developers actually see. And And what you are seeing right now, it basically will log me into uh Databricks AI Gateway and then would open like a Cloud
[10:44] Code CLI. As a developer, you get everything the same, same command, same workflow. You don't need to learn anything new. And then you can basically just prompt it. And now all of the traffic is actually now flowing through AI Gateway. And all the resources that
[10:58] your admins have provisioned can be actually now used by Cloud Code. And then you can also do like models. And when I do model, it basically will tell me all the models that have been provisioned by by admins. So, again, as a developer, I don't have to now worry about like capacity, entering like, you
[11:12] know, keys and stuff like that. Everything else has been handled by like AI Gateway. Now, I could go and actually create like a new terminal, and I could do like something like U code um Codex.
[11:26] And here I can you can see like easily I've now into Cloud Code CLI. And one thing that you would notice is that I have to do like zero configuration. Uh everything was pre-configured. I'm using the shared again model capacity that has been provisioned by my admins.
[11:40] Um all the again the tools, the MCPs can also be accessed by codex as well. One of the really use useful thing is that, you know, when you are debugging you might want to like, you know, live
[11:52] access this data to see like what this data looks like. And a great way to be it to do that is to like, you know, use MCPs. And now it's kind of pull pull the list of all MCPs from that AI gateway. And, you
[12:04] know, you can see like there's a bunch of MCPs. There's a GitHub MCP. There's like lean MCP. And connecting to those to those MCPs super easy. I can just add it and that's it. Now I can access this MCP from all actually
[12:17] coding agents. And this is the open agent economics. So one of the really cool thing is that it can actually use a bunch of models. It can use models from Anthropic, um um OpenAI, and Gemini as well as like
[12:30] some of the open models. Again, you can see that, you know, the MCP that I configured, I can access it um from within inside uh open code as well. The other thing that's really useful other point I want to show you is you can do
[12:42] like you code usage. But this can actually show me my token and dollar usage across different coding agents. Here I can like get like a quick view of like how I'm spending my tokens across different coding agents. And one last
[12:54] thing I want to show you is that especially, you know, when you move towards like building these agentic apps, you really as a developer want to understand like what's really happening under the hood. Like when you send a response to an agent,
[13:07] what actions the agent took, what was the exact sequence did it follow. So you actually integrated ML flow tracing with this AI gateway. If I click on this one, you can actually see what
[13:19] happened under the hood. So when I send this prompt, these are the actions that the you know, the agent took. So when you're trying to debug things, this is super helpful. Like, you know, you get like very granular breakdown of what
[13:31] actions the agent take. Like, you know, and you know, what's the latency and token usage of each of these actions. And we know this works cuz we have been using it for quite some time. Um like I said, we all got onboarded to
[13:44] AI Gateway back in 2020 three. And we have actually worked with like more than 5,000 customers also to onboard them to um AI Gateway. We started with engineering team, but right now all GitHub's employees are actually
[13:57] onboarded to AI Gateway, um including non-engineering um teams as well. So, we start everything with like, you know, measuring it. We have use it number one to increase AI adoption within our
[14:09] engineering team. But then we also use it to understand like what can we do to actually make the lives of developers better. The other thing that really helped us is that everyone kind of like leaned in to improve um the developer ecosystem here.
[14:23] You know, even our CEO has been like, you know, making changes to the AI Gateway client. And then a lot of the teams um engineering teams have also been continuously uh making changes. And we also have a very, very rich and
[14:37] growing plugin and skills ecosystem which is connected to AI Gateway. And yes, we did move very fast. And a part of it was that we actually invested in this infrastructure very early. Um and so that really gave us like a very secure foundation which enabled us
[14:51] to like, you know, onboard multiple coding agents, multiple models. So, now whenever we want to onboard to a new model or a new uh code coding agent, the code analysis kind of relatively easier for us to do that. Before we wrap, one
[15:04] um last quick thing I wanted to talk about is I think a lot of the community and providers have been very focused on improving the inner loop. So, we have seen like, you know, great high-quality agent decoders, great
[15:17] editors, fast linters. And that's awesome. Like, we all are of like benefiting from that productivity gain. But, what happens if your inner loop developer inner loop gets like 10x
[15:29] faster, but your outer loop remains the same? What about What happens to like code reviews when you can actually get pushed like 10x more PRs? Like, what
[15:41] happens to your mono repo, or your CI infra, your observability stack? What about your on-call experience when you are shipping like 10x more features? And what we are currently seeing is that these um outer loop components are now
[15:55] becoming bottlenecks for us to ship faster. And they actually kind of if you look at it, they were created with the perspective that things will sh- things will ship with developer or human speed. But, now we kind of like
[16:09] are pressure testing these things at machine speed. So, we have been now thinking a lot like, what can we do to actually improve the outer loop? And it's a little bit of a hard problem because unlike inner loop, better model quality or better agent quality not all
[16:24] not always can get you know, improve outer loop. There's a lot of process changes that you have to do to be able to like, you know, improve the developer loop outer loop velocity. We've seen like, you know, it can give developers the flexibility to
[16:37] be able to use like different coding agent harness while giving admins like one central control plane to govern that traffic. And so, if that's something uh if that's something
[16:50] you are evaluating or facing a similar problem, we'd love to kind of like partner with you. We'd love to like continue having discussion discussion with you and um help us like onboard to AI Gateway.
[17:03] Thank you so much.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.