Building customer apps at scale: AI agents, unified platforms, real constraints

Summary

Block built Builder Bot, a general-purpose AI agent serving 200,000 operations daily across Slack, desktop apps, and automated workflows, enabling engineers to respond to incidents, explore code, and open pull requests through conversation.
The three-layer architecture uses MCP services for tool integration, the open-source Goose harness as the agent runtime, and smart tool proxies that enable consistent authentication and context optimization across all surfaces.
Industry leaders from Trade Desk, Superhuman, and Databricks discussed consolidating fragmented AI infrastructure, unifying customer-facing applications and multiple data warehouses on the Databricks Data and AI platform.

Building customer apps at scale: AI agents, unified platforms, real constraints

Building AI agents for production use cases typically leads to repeated infrastructure, duplicate authentication logic, and fragmented tooling. As companies scale from tens of agents to hundreds, this ad-hoc approach becomes a bottleneck that prevents engineering teams from moving quickly while safely integrating with dozens of external systems and internal tools.
See how Block built a unified, general-purpose AI agent called Builder Bot that serves 200,000 operations daily across Slack, desktop apps, and automated workflows. Learn the three-layer architecture using MCP services, open-source Goose harness, and smart tool proxies that enable consistent authentication and context optimization. Hear from industry leaders at Trade Desk, Superhuman, and Databricks about consolidating fragmented infrastructure, from managing multiple data warehouses to unifying customer-facing applications on Databricks and Lakebase.
🤝

Chapters

FAQs

What is Builder Bot and what does it do at Block?

Builder Bot is Block's general-purpose AI agent used to solve operational and engineering problems across the company, which operates Square, Cash App, and Tidal. It handles incident response, code exploration, and PR creation from inside Slack and other interfaces, processing 200,000 operations daily.

How does Block's three-layer AI agent architecture work?

The architecture separates concerns across three layers: MCP services provide tool integrations, the open-source Goose harness runs the agent logic online so it can respond at low latency without local execution, and smart tool proxies handle authentication and optimize context passed to the LLM.

What problem did fragmented agent development create at Block?

As Block scaled from tens to hundreds of agents, individual teams duplicated infrastructure, authentication logic, and tooling. This ad-hoc approach became a bottleneck that prevented teams from moving quickly while safely integrating with dozens of external systems.

What adoption results did Builder Bot achieve at Block?

Builder Bot reached 100% usage across engineering teams and now processes 200,000 operations daily. The video discusses how making the bot available across multiple interfaces — Slack, desktop, and CLI — was key to achieving that level of adoption.

Full transcript

[00:08] Thanks. Thanks for the intro. I'm Brad Axen. Um, I'm the AI tech lead for Block, which you're probably more familiar with as the company behind Square, Cash App, and Title. Uh, and today I'm going to be talking about Builder Bot, which is how we're running AI agents inside Block and solving general purpose problems and helping
[00:25] operate and accelerate the way we build software. So, the first example I'm going to show you here uh, is working with Builder Bot in Slack. And this is a an abbreviated version of real conversations that people have where they're using Builder Bot to respond to incidents.
[00:41] So, in this in this example, um, an engineer notices that we're getting an error rate spike. They jump into Slack because we whenever we do incident response, we want multiple people having that conversation together live. Brings the bot in to analyze the logs on
[00:56] Data Dog, discovers where the uh, issues are happening. The bot can then jump into our code bases and go find where um, like a a potential like commit that might have caused this problem and suggest a fix, in this case rolling it back. Um, and I can hop in and say, "Yes, go
[01:12] ahead and do that." Um, and uh, and it's able to inform the team about making the change, actually draft the PR to do the revert, all of this happening inside Slack. So, this is not the only thing Builder Bot does. I'm going to dive into the way we use it
[01:27] as a general purpose AI solution. But, it's a great example because I think it shows the layers that I'm going to get into in more detail. So, one uh, the most prevalent thing here is the interface that we're talking to it from. So, in this case it's available in Slack. I'm going to show how we make that available in many other surfaces,
[01:43] too. The other is the tool calling and the kind of capabilities it has access to. So, here you're seeing Data Dog, code search, and even the ability to like edit code and open PRs. Um, uh, and And course, like kind of implicit behind this conversation happening is that it means that we're
[01:59] running an agent harness online so that we can have this bot uh responding in Slack at like pretty low latency without anyone having to open their laptop up and hop in uh and make the uh connect from something running locally. Um but before I get into those details,
[02:16] I want to talk about where we started from and the problems that I think we've done a pretty good job of figuring out how to address. And you can imagine if you're like if you're concerned about incident response times and you want to build an AI agent just to do that,
[02:31] you would probably build an end-to-end solution. You would make it work in Slack. You would integrate those same tools that I was just showing. And that's how we got started. We built dozens of agents that did individual use cases. What we were seeing is that we're repeating the same glue every time we build an agent that way. So, identity
[02:47] and authentication were a huge problem. Like what bot is allowed to do what tasks? What can it access and read? Was a mess. Um obviously, you're repeating a lot of code for the API integrations. Um you probably have a dozen teams building their own hat like one-off agent loops. And um that's a lot
[03:05] of unnecessary work, but even more important, it means that there's this missed opportunity to share performance improvements that make the bots better at all of the ways we work at Block. Because you might only get it in that incident response bot and not separately in your code review bot. So,
[03:21] um we've landed on making Builder Bot general purpose. And this is uh this markdown file is it's actual like system prompt. This is pretty much all it has. Uh and you can see some of the details about how we're just telling it that it
[03:36] has access to a wide set of capabilities and how it's going to go find out more about what those capabilities are and what it can do with them. And that kind of uh you might have heard this referred to as progressive disclosure um for AI. It's basically loading on the fly what it needs to
[03:53] solve the problem, and that lets it handle a very wide set of solutions and capabilities. Um, and I'll show you more detail about how that works. Uh, but I think the key the key here is that that bot that you were seeing do incident response, it's not an incident response bot with a super specific
[04:09] workflow or system prompt. It's just our general purpose AI agent. Okay, so how is that How can we scale that? How do we make that work for new use cases over and over again? And we build a solution in three layers like I was mentioning. So, the base layer we call our capabilities layer.
[04:25] That under the hood is an MCP service mesh where we've built all of these tools into MCP servers that we run online and we handle identity and authorization there. The second layer is the harness layer. So, that's the actual agent loop. We use an open-source agent that we built at
[04:41] Block called Goose that's now part of the Agented AI Foundation. Um, and that handles all the, you know, tool calling, the models, the support, and we figured out how to run it locally and online. And then the last layer, which I'm super interested in personally, are the interfaces where we make that bot
[04:58] available. So, I showed you Slack. I'll do a demo in a minute of a desktop app. Um, but also this can run in like automations. Like just a, you know, an online batch job that's handling Jira tickets or um, automatically reviewing every PR that's open. Those kinds of
[05:14] things. And so, it's the same bot, the same agent harness running in many different situations depending on what we need it to do. So, let me go into those capabilities. Um, this is what our system architecture looks like. We run a very large number
[05:31] of MCP servers in the cloud or in our service mesh. And that's stuff like you'd expect like in the example I showed Datadog. We have like a Databricks uh MCP, but a lot of different vendor integrations, but also internal things like a a connection to
[05:46] our custom CI system so that the uh bot can go check on how a PR is doing. So, it's a mix of both. Um and a thing that we started with was having these all like kind of separately integrated. What we switched to, pretty shortly, is having a tool proxy.
[06:04] So, that's like kind of a unified front door. Every agent integrates only with that front door. And that's where things like identity and authorization happen. So, you don't have to worry about solving that problem depending on who's calling. It might be me on my laptop and then it uses my permissions in those
[06:20] downstream services. So, like what am I allowed to be see in Databricks? Or if it's a service principal like the Slack bot, that tool proxy also handles like, "Oh, this service principal is allowed to talk to these systems." Um so, this is what it looks like when we're doing something like the Slack bot.
[06:36] Uh we're calling directly from an online Slack bot to that uh service mesh. But, another thing that we've added that I think has been really critical is a CLI wrapper around this. So, this is this is the what the CLI looks like. So, each command in the CLI
[06:53] maps to one of the services, so Datadog. And then each sub command matches to one of the tools, like searching logs. And that CLI has been extremely valuable in us bringing these same capabilities that run online to local use cases. So,
[07:09] um of course, we were I was mentioning we use Goose in a lot of cases. I'll show that desktop app. But, a lot of our engineers is also like Cloud Code or CodeX. And all of those tools can access this CLI and they work great with it. And that lets us have something that's portable in a way that um I if you've
[07:26] ever tried to connect MCPs into the various coding harnesses, it's always like a different story, different levels of support, sometimes they fall over when the context gets too big. So, this CLI gives us consistent performance, no matter what agent we're using. Okay. One more piece of this that I
[07:42] think has been really critical is how we make these tools perform well for agents. And I think the initial instinct you have when you're building MCP servers is that you're just going to wrap the existing APIs. So say you want your bot to be able to interact with
[07:57] orders data. Um and you've got an API that just paginates through all of these orders payloads. This is a mess for LLMs. It's like way too much context window usage. If it wants to actually answer a question, it's going to have to do the math itself, which it's never good at.
[08:13] So a much better design is to switch that to like an operational query layer. And so instead of just getting all the data, you run a you have the LLM write a query that gets executed that tells it only what it needs to know and handles all the math. And increasingly all of
[08:28] our tools we're moving into this kind of format. Where it might be SQL or like an elastic search query or graph QL, but you always give it this layer so that the model can specify what it needs and reduce context window usage and improve performance.
[08:44] Okay, next harness. I'll go through this a little bit quickly. Um if you want to find out more, this is all open source. So uh like I mentioned, Goose is an open source harness built by built by Block. We donated it to the Agentic AI Foundation. Um we run this as a desktop app and a
[09:00] CLI, but also we have a version of it that runs in the cloud with durable session history. Um I think the key thing here for this talk though is that we use a harness that can speak to any model. So those same capabilities we can run them through
[09:16] depending on the use case. So like Claude Opus for example is what we might want for incident triage cuz it's great at fast tool calling. Whereas like code review, we might run on GPT 5.5 because it seems to do a really good job of catching bugs. And so that same harness can work with both. And that gives us some leverage on
[09:33] deciding, okay, this is going to optimize for costs by using a model that's a little bit faster and cheaper when we don't need Frontier or when we need it, we can upgrade to the like highest performing model available. So, let me do a quick demo of this.
[09:51] This is our internal version of the app and actually the slides that you we were just looking at, I built in this application and they're running here. That's how That's how we made this that website. But, I'm going to go ahead and switch over to one of our agents and go talk to Builder Bot. So, these avatars are are like the fun little personas that come with our AI.
[10:08] Um but, under the hood, it's really about the capabilities. So, if I go chat to Builder Bot here, I can ask it, "How does the agent loop work in Keg at a high level?" So, that's that online version of Goose that I mentioned. And we'll let that get started. Hopefully, my internet okay connection
[10:23] is okay in here. Um So, uh like I mentioned, this isn't like I'm not talking to a bot working in the code base for Keg. I don't even have it cloned. It's actually looking through our like entire GitHub catalog. It's I we've
[10:40] indexed all of our code bases and you can kind of see that as I pop this out. So, like I mentioned, the first thing it's going to do is just go explore what those tools are like I mentioned with the command line. It's just going to use the command line help output to decide, "Okay, we're going to get I'm going to go look at
[10:56] uh the uh code search tools." And it's going to iterate that way until it has started to found find the source files. And then it's going to look at the source code. This is none of this is on my laptop here. Like it's all connecting through that uh online mesh.
[11:13] And that matters, of course, cuz like at Block, we have a couple hundred million lines of code. On average, I don't have most of that service area checked out on my computer, but I can just go ask any arbitrary question about how Block works and I'll get a pretty good answer. Um this has been a huge change to the
[11:28] way we work because we're having engineers from different teams contribute to other code bases really consistently, but also even across roles. So, something that we've seen a lot of Okay, we're getting back to the answer here. It usually makes a nice little diagram. So, here's like the description of
[11:44] the um how we go from this push messages endpoint to a loop that runs in Temporal that will actually go do the tool calling execution, all that stuff. So, you get these nice answers pretty quickly, and it's I think to me it's wildly exceeded my expectations on its
[12:00] coverage of like all of the code we have at Block. And then back to that last point, that kind of cross-role collaboration, what's been so cool about this is that you can have a designer who wants to make a change in Cash App just like hop in, start asking questions like this, and figure out how they're going to go
[12:16] change that system and do the feature that they want to build. So, we're having more and more non-engineers also contribute code through tools like this. Okay. So, let me uh wrap this up. That last piece that I wanted to talk about was interfaces. And
[12:31] um I've shown you two so far. Um think of those as the two different ways of working. So, the Slack is a great interface for collaboration. That's where the conversation is already happening. You can see uh what someone else is doing. And so, in incident response, for
[12:47] example, you want everyone aware of everything that's going on. Um we see more and more people coding directly from Slack just because they can ask their colleague for an opinion and like refine as they work. But, we also, of course, have a desktop app. And that's where deep work happens. You get these like longer-lived projects
[13:04] that you can iterate on over and over again and and focus. Um what's been critical between these two is that it's the same bot, same capabilities running in both systems. And you don't get a different set of tools when you're in Slack versus
[13:19] locally. It's always going to be the same, so you expect the same performance, and you get familiar with what it's capable of. So, okay. So, this has really taken off at Block. Um we've now hit 100% of our employees using AI tools every week. And Builder Bot is running over 200,000
[13:36] operations per day. Um driven by employees, but the bot's doing the work. Uh of course, it's been super popular in code. Uh uh pretty much everyone is working with AI and getting their code reviewed by AI. But, another cool thing, like I mentioned with that cross-functional collaboration, is that
[13:51] many employees are building prototypes or internal websites or productivity apps and sharing them internally. And we're And we built over a thousand of those in May. Okay. So, quick recap. What did we learn? Um I think it's really tempting
[14:07] to build an end-to-end agent workflow, like a use case. We found it much higher leverage to build the capabilities and then just reapply it over and over again. So, all you need to do to solve a new problem is make a new markdown file. You don't have to go rebuild that.
[14:22] Another big thing has been aligning the model with the workflow. So, like I mentioned, we use uh like this like Databricks infrastructure lets us switch between GPT-55 and Opus when depending on what we're doing. That both drives down lower costs and can improve performance when we pick the right model
[14:38] for the right job. And then last, um people work in all kinds of places. That's like a normal part of being in a tech company. And we found it extremely valuable to make the capabilities consistent no matter where you're starting the work from, so that you can like expect the same
[14:54] performance, get used to what it can do, uh and just like go where you need to do the work. Okay. So, with that, thank you. Um I would love to chat with anyone if you're building similar things or have some of the same problems. And if you want to check out the open-source half of this, you can go to the AgentlyAI uh the
[15:11] AgentlyAI Foundation's GitHub to check it out. Thanks. Thank you, Brad. That was great. And you know, I love how it's so practical and so easy that he just told
[15:26] us how to build it ourselves. Right? And, you know, I love the the CD's capabilities and stuff. So, now I'd love to welcome up um our panel discussion, which we will do intros while we're up here. So, come on up, guys.
[15:43] Burton and Dima are coming on up. So.
[15:59] All right. So, the first thing I always like to do is give us a chance to do each of us have like really interesting roles and and so forth. Tell us you know, tell us a little bit about your journeys and and who you are and stuff like that. So, uh Burton, you want to go first?
[16:15] Sure. I first want to know about this Slack coding thing. We'll we'll we'll talk afterwards. Sounds a little scary, but um Slack is scary. Slack coding. Slack coding, but we'll see. Um so, I work at the trade desk.
[16:32] Does anyone know what a trade desk is? Some of you better, but stock trading. No, we do we do ads, lots of ads. Uh there's a I probably stayed out too late last night, also, but um
[16:48] It's okay. There I'm here I'm here for you, buddy. Um we have a lot of data. You know, there's you know, every day any any second there's about 22 million ads a second. Uh we generate about over a petabyte and a half of data a day.
[17:03] Uh so, there's a lot of data to deal with, so um I've kind of I've been there 8 years, so I've kind of picked up things over time. You know, I'm like, oh, you manage people before and now you can have this team and this team. So, I manage uh relational team. I I manage a data warehouse team. We have um
[17:20] Databricks, Vertica, EMR, Snowflake. So, we're we're we'll get into a little bit more trying to consolidate to a a more sane architecture. Uh and I own a couple engineering teams for feature teams related to our data
[17:36] egress. So. You've definitely seen the entire data ecosystem. Eight years. That's great. Dima, why don't you give us your intro? Yeah, hi. Uh I'm Dima. I'm coming from Germany. So,
[17:51] uh guten Morgen, liebe Leute. Um I'm with the Superhuman. Uh Have anyone heard about the Superhuman? Raise your hands. And uh probably you have heard about us as Grammarly. Who knows Grammarly? Raise your hand. Oh, wow. That's a good
[18:08] audience. I've been with uh Grammarly for 7 years already, and it's still counting. And uh for us uh last year was very active. So, we acquired uh two companies. We rebranded to Superhuman. Now, it's not just Grammarly. Now, it's Superhuman huge Superhuman Mail. It's
[18:25] Coda. It's Grammarly and Go. So, it's it's full productivity suit. So, it's been quite transformational for us. And we serve uh 40 million daily active users. I think uh to to show the scale on the consumer side,
[18:41] uh we make more than 100 billion LLM calls uh per week. Um that's our scale, and it's growing. That's awesome. 7 years. I didn't even know that that that a Superhuman was that that even 7 years
[18:58] old. Awesome. Well, thank you for flying all the way out here, as well. Um yeah, so really quick intro on me. Uh my name is Bruce Wong. Um I'm the head of our data platform organization at Databricks. I'm not going to ask you guys if you know what Databricks is since you're
[19:13] here. Um but I can tell you that we use Databricks a lot inside of Databricks. Um and uh every single function at the company actually uses data in Databricks. Um and so I get to see all of that. Everything
[19:29] from uh the the story I love to tell is like our facilities team even uses Databricks. Uh and so as we came out of COVID uh you know, we were like, "How do we get back to office? How do we make the office experience great?" Uh we actually
[19:45] analyzed badge data and other data um so that we could actually improve the office experience and stuff like that. So, uh and that all happened inside of Databricks. So, it's a you know, small data. Sorry, it's not petabytes of data uh or like, you know, few billion LLM
[20:01] requests and stuff like that. But I think it's it's one of those things that we we literally infuse data into our culture at Databricks and stuff. So, so I leave all of our uh dat- you know, I own all of our internal Databricks use for production and stuff like that. So.
[20:17] All right. So, let's talk about this like you know, over the past, you know, you seven years, eight years, you know, one of the things that I love about being an engineer is like you kind of have to like as your company's grow,
[20:33] like you weren't always a petabyte of data and you weren't always like, what did you say? 40 40 million users or something like that? Yeah. You weren't always there. And so along the way, there's always different problems or different constraints that you run into. Um and so tell us a little bit about
[20:49] like, what are the current constraints? What are the current constraints you guys are are are encountering um at the scale that you guys are at? I can start. Please. My uh constraint is always latency because when you serve something to end
[21:04] users, you want to make it fast. You want to make experience as smooth as it's possible. Has anyone ever thought, I wish this application was little bit slower? No? I I guess there might be some applications that you want to be slower.
[21:20] Maybe you get pleasure for from from them being slow. But usually you want them to be faster. And I have a an example of extreme latency. It would be my favorite German bank. Uh when I registered an app, I downloaded it in 10 seconds. And then I waited for a week to
[21:36] get the password. Because they printed it on paper, folded it, put it in an envelope, and send it over post. So, that's that's the latency that we face. And if I need to recover the password, if I press this forgot your password button button, it will take another week to get the password back. So, I don't want our
[21:52] users to be afraid of this latency. I want the experience to be smooth when you move from surface to surface, when you move from agent to agent, I want you to get an instantaneous response. And that's my limiting factor, what I can build on a platform with with the latency that it gives me.
[22:11] Yeah, I mean I think the and there's so much complexity in all of the operations that we do. Like latency is one of those those interesting things where it's a it shows up somewhere but never the same place at the same time. Right?
[22:27] Uh and so like yeah, I mean like talk about like how have you seen like where latency has shown up uh in your stack and like how is that changing? What are you looking forward to? Where's the next frontier for you on that?
[22:42] Yeah, with the latency like just few years back I was not thinking about building consumer applications on top of my data platform, right? Data platform was for data and I had to build everything that needs to serve customers separately. I have my team in the front
[22:58] row. They they they will confirm that whatever we built, we are accountable for serving, for maintaining, providing you 24/7 support. So, the sound of pager duty in the middle of the night, the more you build, the the more frequently you hear it. Uh and that's also the limiting factor for
[23:14] me. So, how much I can build, how much I can tolerate, how big of a team I need to support whatever we build. And in this case, uh like if I build on top of my standard solutions, I get more of like
[23:30] peace. Uh Yeah, like you know, I think that's actually a a trend that I've been starting to see is like you know, low latency systems are usually transactional systems that we're all used to. I mean, Block is like a great example of like I swipe my credit card, it's a transaction and stuff like
[23:45] that. Um but more and more we're starting to see like the intelligence of a superhuman come to play uh that becomes a you know, low latency, but also high intelligence aspect of things and stuff. So, I mean, there's also scale too here.
[24:03] Like Bert, you talked about what, 1 petabyte a day? Like that's a lot. I mean, I imagine you I imagine you've hit a number of constraints every day with that. We break stuff. We break stuff a lot.
[24:18] I like to break stuff. Well, sometimes it's fun, but sometimes sometimes it's not. I think I mean, kind of related to latency and the question you asked earlier like kind of what are our constraints and like latency is a big thing for us as well. Fortunately, I don't have to deal with the latency as much. It's our our bidding team that
[24:34] deals with it, but when these ads come in at, you know, 20 22 million events a second, we have to make a decision in 50 milliseconds whether you know, who we're going to bid on, how much we're going to bid on, do we have budget? There's a lot of things that happen. Uh and as you can imagine, there's not a lot of 50
[24:50] milliseconds, you can't really do that much. So, a lot of the there's a tremendous amount of back background processing that we bring in all that data after the fact and try to build a bunch of models and opti- opti- optimizations that we can do to make that bidding process uh a lot more efficient. Um
[25:07] and I think what kind of the challenge at The Trade Desk we've always been a very distributed culture. It was like everyone's responsible end to end for their entire stack. Uh which works really well in some cases, but when you're a fresh engineer and you're like, "Oh, here's that data set that's a petabyte of data. Good luck." Right?
[25:25] We've spent you guys like it cuz we write lots of bad queries. Um Select star, it's great, man. Yeah, there's some of that. But like you know, so what we're trying to do as a platform team, you know, we we don't want to be order takers and take do everyone's work for them, but instead of
[25:41] giving them access to, you know, the entire full set of data, it's like, "Well, you don't get access to that. There's these sample data sets or these exact aggregated data sets." So trying to provide them with um things that they can still self-serve and not be blocked, but, you know,
[25:56] giving them some guardrails so they don't shoot their feet off. Yeah, I mean, guardrails is a great point. It's just I mean, I I I actually uh really appreciated Brad's uh illustration where it was like, "Okay, you can call the API and you can token max that way or you can call SQL and
[26:12] it'll actually do it for you, right?" Uh and I know that like that was like that was like one of the interesting constraints for me of the past 3 months is like that context window is actually a constraint. And I remember blowing out my limit like the first time I was trying to do this
[26:29] and I was like, "Oh, yeah, that's a bad idea. Like I probably should use I had to sell myself a little Databricks. I was like, "I probably should use SQL for this instead of using uh an LLM for the for joining a bunch of data." So Um but yeah, the world is changing. So
[26:47] So, you know, one of the things that, you know, we also talked about is uh a lot of us just our architectures are just integrations of many different many different things, right? And, you know, LLMs and and uh
[27:04] MCP makes it even easier to have just like this like gigantic mesh of integrations and stuff. Um but, one of the things we we talked about earlier was like, how do we think about the difference between integrations versus actually like really building a hack
[27:20] of a solid platform that's like not just a bolt-on integration stuff. Dima, you want to you want to go first on this one? Yep. Uh as as everyone raised hands when I mentioned Grammarly, so you probably have seen our red underlines. If you do something that we want to fix, uh you
[27:36] will see this this red underline saying, "Hey, uh I want to help." And uh probably these underlines are fast, right? Uh you you just finished typing, you see, and it's already there uh trying to help. And we want them to be fast, as I mentioned. Latency is killing uh killing the features. So, we
[27:53] always uh when we do these underlines, we always check uh how this impacts the users, how do you feel, how how's the engagement uh compared to what we had before. And usually, like if the latency is growing, the the engagement is falling. And these red underlines these like red
[28:10] underlines that you see, uh they are served today from uh Databricks. So, uh it's it's a massive amount of data. It's about uh uh 200 K 200,000 uh queries per second. They
[28:25] are served from uh the Databricks model serving. And that's an example. We do not integrate uh with Databricks because before that, that's that's been served uh like uh from from the separate solution. Now, data data lives in Databricks. Now, the training process lives in Databricks, and the serving
[28:41] lives in Databricks. And 80% of this traffic of the red underlines comes directly from our data platform that was unimaginable just 2 years back. We who would imagine that it all will be served by the same platform. Yeah, I mean, I think that's been the
[28:57] the interesting trend for Databricks is like we're becoming more and more online when traditionally we've been the the data platform offline and stuff. So, model serving like is a great example of that. We used to have to copy our data into some kind of sidecar serving serving it architecture and stuff. So,
[29:13] um Yeah, I mean, I think the the more you get out of the box, the the easier things are um with this. So, um you know, I I I usually joke about like part of our job is copying data around.
[29:30] And like how do we eliminate so many copies and stuff like that? So, um but yeah, I mean, Brad uh or uh Brett Burt, tell me about I mean, you guys probably copy a lot of data also. Way too much. Now, I remember I think
[29:45] 6 years ago, so we created a service called DataMover. Moves data. And we're we're we're really slick. We have we have uh DataMovers for SQL Server, Vertica, Snowflake, Databricks managed tables. And it was meant as a
[30:02] temporary thing. You know those temporary things that are short-lived. I think there's 370 of them now. And it's people can just wire them up like, "Well, I need this data, but I'm over on this platform." Oh, it's over here. Just you know, check in this YAML file with a SQL query and
[30:17] just shows up. So, we're kind of like the victim of our own success a little bit. So, but it adds a lot of You made it too easy to move data. I know, but I probably shouldn't have shared it with anyone, but you know, I did it for myself, but then I shared.
[30:32] But uh Well, I mean, I mean, everyone knows with copies of data, it's cost, latency. Think you know, there's bugs every once in a while, consistency. It just adds a whole bunch of noise that ideally would just disappear, which we're starting to see with
[30:47] you know, the standardizing on Unity with, you know, we don't do data mover to Snowflake anymore. It's just uh directly connected, which just eliminates a lot of noise and crap. Yeah, I mean, I I think uh I I've been We've been around the data ecosystem for a while, and uh you know,
[31:05] you we've all mentioned all the different like aspects of that stack. And yeah, managing the governance, even just managing the key even just managing the keys for all these different things and like moving like cuz that's the other part. It's like it's not just moving data around and who has access to it, but
[31:21] it's even just managing the keys and security between all of these different systems and stuff like that. Like that actually adds up tremendously and stuff. So. Um So, you know, by I think this is like one of the things
[31:37] that made me realize in the last 6 months is like having everything in one place, like especially at the speed of I call it the speed of agents, right? It's like basically, it's all hindsight because agents are moving so fast. Uh and and
[31:55] you know, Bertie you talked about like guardrails, right? Like let's talk about like how how these guardrails or safety actually helps us move faster and like we always talk about guardrails, not gates and stuff like that. Um So, yeah, talk You know what? You want
[32:11] to expand on that a little bit? Like Yeah, I mean, it's uh I mean, as as more and more business users are getting access to data, it's, you know, it's honestly, in some cases, it's giving them access to less data. It's less more highly cur- curated data.
[32:26] Um You know, it's been fun figuring out all the all the new buzzwords and what things actually mean. It's like, you know, semantic layers now the ontologies the new one, right? But even with semantic layers, it's like what is that, right? It's like you're like I'm I need
[32:41] I'm a practitioner, I need to implement it. It's like, well you read some random blog post. Yeah, you need one. It does these things. I'm like, all right, how do I do it, right? Right. And then you talk to a guy on the that works on Snowflake. Well, you we don't we don't want to go to Databricks cuz we have the our semantic views already. So,
[32:56] they're too hard to migrate. And I'm like, wow, maybe they're harder than I thought they were. Then then I go look at them. I'm like, no, I don't think he knows what he's talking about. Claw can just convert the semantic view to the to to the metric view. Yeah, you don't need to copy paste your
[33:11] agent. You can do it for you. That's right. That's right. So, it's like, you know, I mean, agents are moving quick, but everything in the ecosystem is moving quick. So, it's trying to be on top of uh just what all these things are and and what the right path forward is is
[33:27] challenging. Yeah, I you know, actually, this is one of the things I love about speed to to dig into the business user. We support a lot of business users, and I noticed like after we started unlocking our more semantics in uh in our own internal
[33:43] uh as well as like unleashing Genie on the entire organization, I actually get less questions now because they're like I used to get all these questions about like how to like just basic data analysts uh analysis questions or like, well, what about this? And like
[33:59] when Genie first came out, we started responding like, have you tried Genie internally? And then like it just kind of grew very quickly. And like we stopped having to answer that and uh I actually have to go out of my way now to be a little more uh social now cuz people don't need me anymore, you know?
[34:16] They can just ask Genie. I know. I know. So, I got I got to find other reasons to socialize, right? Uh but I'd much rather have like a beer and ice cream instead of like telling you what your data looks like. So, uh but that's I think that's the amazing thing we're in is like every single
[34:32] person in the organization doesn't need to know SQL. They don't need to know who I am to help hook you up with a data answer. Uh you you just talk to the talk to our data lake, right? Um And so, you know, I think that's sort of the the the interesting thing that we're
[34:49] we're seeing is like it literally is changing how our culture and how we are working together and stuff like that. Um So, Dima, like you also talked about like agents are really forcing you in a interesting direction cuz you're you're
[35:04] more at the center of this than than we are per se, right? Because you're actually in that serving layer and like part of the Gentic ecosystem. Like you want to talk about that and like how what you're seeing at Like what does safety mean for you guys? Yeah, safety for us is essential. It's a
[35:21] trust trust. Like if you do one thing wrong, trust is super hard to restore. Trust is something that we want to keep as hard as it's possible. We have strict security controls. We have lots of governance and privacy teams and
[35:36] responsible AI teams that make sure that whenever agent responds, it responds very diligently, like respecting you. And if you ask security team, would you like to keep this data like in in many systems like here and there and then we can push and copy it to another system,
[35:53] they will be excited about it, right? No, they won't. Keeping all eggs in one basket in this cases is not they want to keeping the eggs in one basket is thing that we want to do in the case of data privacy
[36:08] and security and keeping your data in in your data platform only is something that it's easy to monitor, easy to control. Uh and that's what we want to have. So, that's and what what I see the trend that when we build customer applications like for example, we are using more and more lake base to serve
[36:25] But, I took support growth experiments, what we what promotion should be shown to user. We serve it from the data bricks we show we serve it from the lake base. That means that the agent if you run it, it has access instantly to all the data that we have. In in the past we had to push it
[36:40] to radius outside of the of the data perimeter that needs extra security controls, extra governance and gives zero visibility to the coding agent what's there. So, we have to to build lots of infrastructure to enable this. I think that the simpler we go the the better it is from not just security
[36:57] but also from the productivity standpoint. Awesome. So, I got one last question for you guys and I I had loved this question cuz we're all data folks, right? You know, for the longest time anyone who's been around data, data platforms, or
[37:13] anything, we've all talked about garbage in, garbage out, right? We've all we've always said that. Has AI made that better or worse? Like has AI saved us? Like we're not having like or is it new gar is it we have no more garbage thanks to AI or
[37:28] is it new garbage now? That's a good question. Trick question? Uh faster garbage. Faster garbage. I love it. not careful. Recycled? Yeah, we can go with recycled. There you go.
[37:47] Faster garbage, faster cycles. Overall we can do things faster and we can delegate like you can let your agent say do this for me and I will come and check it in 5 minutes. So, that means I can do more. I can spend this 5 minutes doing something else. Something else I hope like super valuable as well.
[38:02] Uh but also like we can we can do instead of specifications before like a year back I was writing a document. Now instead of writing document I will prefer like just making a prototype. Just showing you the prototype and see that's how it will look like. So, I think it unlocks things that were just
[38:18] not possible before. Yeah, I like that. Definitely prototyping is a lot faster and I would say that's maybe not faster garbage as prototypes are meant to be prototypes but I definitely have seen like how agents finding the wrong data,
[38:36] you know, finding garbage data like doesn't solve that problem at all and stuff. So but yeah, so thank you all so much for coming out participating on the panel. I hope you enjoyed our conversation and our discussion
[38:53] and yeah, like enjoy the rest of the conference and stuff.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.