Scaling AI agents to production: Claude, MCP, and governance

Summary

Anthropic's applied AI team and Edmunds demonstrate why most agent demos fail to reach production—real data, edge cases, and volume cause tasks that took 40 seconds in a demo to take 40 minutes—and what the teams that succeed have in common: robust harnesses, composable skills, and unified data infrastructure.
Edmunds built Data Dave, an autonomous AI assistant combining the Claude Agent SDK with 50-plus composable skills, Databricks Lakehouse for memory and metadata, and Vector Search for context retrieval, reducing weeks of manual work to minutes of intelligent automation.
The video covers the reliability mathematics that separate demos from production systems, including the exponential capability growth of agents that double their skills approximately every 123 days through a compounding cognition loop.

Scaling AI agents to production: Claude, MCP, and governance

Building production-grade AI agents requires more than a frontier model. The system architecture, skills library, memory management, and governance framework multiply the model's capabilities. Anthropic and Edmunds demonstrate how to architect agents that survive production complexity, from reliable multi-step reasoning to safe autonomous execution.
Learn how Edmunds built Data Dave, an autonomous agent that transformed weeks of manual work into minutes of intelligent automation. Discover the architecture combining Claude Agent SDK, 50+ composable skills, Databricks Lakehouse for memory and metadata, and Vector Search for context retrieval. Understand the exponential gains from combining multiple skills and the reliability mathematics that separate demos from production systems.
🤝

Chapters

FAQs

What is the Claude Agent SDK and how is it used for production agents?

The Claude Agent SDK is Anthropic's agent orchestration engine used to build and coordinate multi-step AI agents. At Edmunds, it serves as the backbone of Data Dave, an autonomous assistant that combines 50-plus composable skills with Databricks Lakehouse for memory and metadata storage and Vector Search for context retrieval.

Why do most AI agent demos fail to scale to production?

According to this video, most agents stall when they encounter real data, real edge cases, and real volume—tasks that took 40 seconds in a demo can take 40 minutes in production. The teams that succeed invest in robust harnesses, composable skills libraries, and unified data infrastructure rather than relying solely on a better model.

What is Data Dave and what capabilities does it have?

Data Dave is an autonomous AI assistant built by Edmunds that transformed weeks of manual work into minutes of intelligent automation. It combines the Claude Agent SDK with over 50 composable skills, Databricks Lakehouse for memory and metadata, and Vector Search for context retrieval, operating across four domains of capability.

How does governance and safety work for production AI agents?

This video explains that governance requires defining what agents should not do alongside what they can do, using MCP skill integrations with appropriate guardrails. The architecture described by Anthropic and Edmunds includes memory systems, autonomous scheduling, and a compounding cognition loop that enables safe, scalable agent operations in enterprise environments.

Full transcript

[00:07] Good afternoon everyone. I'm Josh. I'm on the applied AI team here at Anthropic and I'm joined by Greg from admins. So today we're going to be talking about um building agents in production. So building an agent, it's never been easier. Most of them still don't actually make it to scaling in
[00:23] production environments. So we're going to show you what the survivors do differently. I'll give you the view from inside Anthropic from what I've seen across many companies and Greg will go deep on what he's seen inside admins. Just um quick show of hands out of curiosity. Who here has built some type
[00:39] of agent before? Okay, most folks. Who here has built an agent and like tried to get it into production? Okay, fewer folks. How many people have actually succeeded at building an agent,
[00:54] scaling it into production and like serving it to users or like an internal team? Okay, much much smaller set of hands in the room. You are in the right place. So um before we dive in, just want to give a quick note on where my perspective comes from and what applied
[01:10] AI is. So at Anthropic, our team sits at the intersection of product research and our customers and internally we work on agents like cloud code and co-work and we also contribute to the research that goes into cloud itself. Uh externally we
[01:26] partner with enterprise teams shipping agents for instant response, finance, legal, creative tools, really wide variety of domains. So the patterns we'll share today come from both sides of that. what we've learned building anthropics own agents and what we've watched hold up as customers ship
[01:43] agents. So here's the pattern I see from my seat and it's it's pretty consistent. Almost every team builds a demo that looks magical, right? Capability starts to climb in the demo. People get excited.
[01:59] Maybe someone in the room is saying ship it, right? People get pretty hyped. Um but then you kind of hit this cliff, right? the thing that was flying in the demo tends to stall the moment it actually meets real production use cases. So real data, real edge cases, and real volume and all of a sudden a
[02:16] task that used to take 40 seconds is is now taking 40 minutes. So most companies never end up crossing that gap. And the interesting question isn't can you build an agent? Everyone here can. Almost everybody raised their hands, right? It's what what do the teams that make it
[02:31] across this gap have in common? Um, and spoiler alert, it's it's not necessarily just that they had a better model than you. Uh, everyone in this room can call the same Frontier models. Um, it's what they pair the model with. So, let me take the model half first
[02:48] because I I want to be clear. The model matters and it it it tends to matter a lot. Um, and the frontier is still moving pretty fast um, in exactly what this room cares about, right? How much can an agent do on its own before it needs you? So if we look at the trend
[03:04] not long ago an agent could handle a task that took maybe about a minute right and then many minutes. Um but now our most cable models can stay on task for hours right doing long multi-step autonomous work end. So every agent
[03:19] you've already built gets better as this line keeps climbing. So I'm not here to tell you that uh the models have stopped mattering. It's it's actually the opposite. But this this actually makes the question even a little more interesting which is you know if the frontier keeps moving and the agent
[03:35] should be capable of more right out of the box. Um why is it that people are still struggling to bring these agents into production? Um and it's because the model is necessary but alone is insufficient. Right? So here's kind of the the main
[03:50] idea of the talk in in one map. You can imagine two axes, right? one is how good is your base model and the second axis is how good is the system around it or the harness right if we look in say the bottom left corner um if we have a weak
[04:07] model and a poorly engineered harness uh we can basically consider this a toy or a demo um if we have a really strong frontier model but still that weak poorly engineered harness we can end up getting magical demos that don't survive contact with production use cases um and
[04:24] most teams tend to get stuck sitting right there um at magic demos. The teams that ship live up and to the right. Um and here's the thing, these these two axes multiply. They don't add, right? A model jump raises the ceiling and the
[04:40] harness decides how much of that ceiling you actually reach. So what's actually in the harness? These are the tools, skills, memory, and context all wired around the model. And
[04:55] I can make this concrete with a few things that we've shipped. First are agent skills, which is just a small folder of instructions and examples that teaches the agent one job well and only loads when it's needed. The second are MCPS or the model context protocol. This
[05:11] is an open standard for connecting an agent to hundreds of tools and data sources. And the third and the piece that really holds it all together is our claude agent SDK. It's the same harness that we use to build cloud code. And here's the point that I really want to take you to have you take away. These
[05:29] pieces are designed to work together with each other and with our models. Our models are trained and evaluated inside harnesses with these tools. And the SDK is shaped around how the models actually behave. So teams can actually end up
[05:44] seeing real performance gains just from moving onto pieces that were built as one system. And Greg has some firsthand uh experience with this. Now, here's here's the the point that really makes this combination of the model and harness multiply instead of
[06:00] add. The longer an agent runs, the more the reliability of that base model has to hold up, right? Not for one step, but for this whole multi-chain reasoning. And this is arithmetic. This isn't like some marketing slogan or like some
[06:16] groundbreaking research we we've done here at Anthropic, right? So, let's imagine that each step of a multi-step agent loop is only about 95% reliable. On on paper, this doesn't sound too bad. You're only only failing a few percent of the time. But if we chain 50 tool
[06:31] calls into one autonomous run, uh then the whole run is only going to succeed about 8% of the time, right? That's that's a pretty poor outcome. Um but if we say make each step 99% reliable or even higher on a single turn, you may not notice that difference very much.
[06:47] But over say a 50step agent run, uh you're going to get to about 60% reliability of an end to-end task completing. This gap only ends up widening as the runs get longer and as the models get more capable.
[07:05] So how hard is it to get started building with these tools and building agents that can survive production? Honestly, uh surprisingly not hard. Cloud runs natively on data bricks, right? So your off security and governance all work the way the rest of your platform already does. And data bicks's own genie agent uh runs on cloud today, right? This is a paved road. Um,
[07:22] and one of my favorite patterns from the field that I want to share today is that you can build and harden the agent with our most capable models while you're getting the harness, the skills, the MCPS just right and then run production on the model that's specifically best
[07:37] for the job. Right? If you need a lighter, faster model for high volume runs, you can use something like Sonnet or you can switch to Opus for models or agents that really require the highest tier of intelligence. Edmonds did exactly this and it works for for the whole reason the talk exists. Right?
[07:54] Once you've built this composition in the harness, the capability is not just in the model, it's a function of the two together. Edmmonds has been at the forefront of leveraging our models and our agent SDK. And now I'll hand it over to Greg to
[08:11] walk you through how they have brought their own agents out of the sandbox and into production. Thank you, Josh.
[08:26] How many of you have used Edmonds or at least heard of Edmonds? Show of hands please. Okay, maybe 50%. So, we're proud to have the largest collection of used inventory on the market and also the most accurate pricing. Um, so if you are in a market
[08:42] to buy a car or sell a car, please please come to Edmonds. But um enough of the marketing. This is a technology conference. So we all have a theoretical idea of what AI agents are. Before we dive into details, I would like to show
[08:59] you what agents are at admins in practice.
[09:14] Introducing Data Dave. Edmonds AI data assistant built to multiply every engineer's impact. Return on investment. For every conversation, Data Dave turns hours of manual work into seconds. And it doesn't just save time, it eliminates the hidden cost of getting it wrong. Conversation one, a product team needed
[09:30] to audit photo coverage across thousands of vehicle styles, identifying exactly which makes, models, and shot types were missing from the live website. Data Dave completed in minutes what would have taken two full days. Coverage report delivered for all 180 model year 2024
[09:49] styles. Time saved 16 hours. Value delivered $2,000. Cost of inaction. Missing vehicle photos on VDP pages silently suppress buyer engagement. Estimated 25,000 in lost lead conversions per quarter. Now
[10:06] visible and actionable. Conversation two. The ads team suspected a problem with video ad delivery. They needed daily fill versus unfilled impression trends across video ad units. Daytoday surfaced 35.8 million unfilled impressions in April alone. Time saved 8
[10:23] hours. value delivered $1,000. Cost of inaction, that fill rate collapse was completely invisible. At a $5 CPM, those unfilled impressions represent over $170,000 in unreovered ad revenue. Now
[10:41] identified, investigated, and recoverable. Conversation three, a data scientist needed to run a statistical analysis on an A/B test for an ESD offer reminder experience across 29 dealers. Data Dave corrected three bugs that were
[10:57] underounting the test group by five times. Time saved 6 hours. Value delivered $750. Cost of inaction. A product decision made on five times underounted data would have either killed a working
[11:13] feature or scaled a broken one. A $10,000 cost in rework, roll back, or missed optimization. Conversation four, a back-end engineer needed a code review on a merge request, introducing priority score data points to the inventory search service. Data Dave ran three
[11:30] iterative review passes, catching a lead info merge regression in pass two before it hit production. Time saved, 4 hours. Value delivered $500. Cost of inaction. A production bug in inventory search means dealers lose leads.
[11:46] A single missed regression can cost $15,000 in incident response, hot fixes, and lost dealer trust. Conversation five. An engineering lead needed to know what anthropic model usage would cost over the next 12 months. Data Dave
[12:03] delivered a dual model forecast times FM and Holtz damp smoothing with full confidence intervals. Time saved eight hours. Value delivered $1,000. Cost of inaction without a forecast. Unexpected cloud spend growth goes unbudgeted. The
[12:20] model's projected a 22% monthly growth rate. Catching that early prevents $12,000 cost surprises in the back half of the year. Conversation six, the analytics team needed a cost estimate for loading 500 million events into
[12:36] Amplitude before committing to the work. Data Dave modeled the cost from 30 days of actual billing data, then created the Jira story in the same conversation. Time saved 3 hours. Value delivered
[12:51] $375. Cost of inaction. Building infrastructure for a 500 million event pipeline without a cost model risks $8,000 in overprovisioned compute and a missed budget conversation. Six conversations, 45 hours saved, $5,625
[13:10] in direct value delivered and $240,000 in risk avoided. Risk that was previously invisible, silent, and compounding. Now let's look at what data Dave can do across four capability domains. Data and analytics. SQL
[13:28] analysis, dashboard creation, amplitude queries, new relic monitoring, and time series forecasting. Business intelligence delivered instantly. Engineering and DevOps. Code research, code review, ETL pipeline changes, data
[13:44] bricks operations and airflow debugging. End toend development assistance. Business operations, Jira triage, confluence search, issue creation, sprint reporting, and scheduled automation. Your entire workflow connected communication and content,
[13:59] Slack posting, email, Google Docs, Sheets, and presentation decks. Results delivered, not just analyzed. But here's where Data Dave becomes truly exponential. Not one skill, all of them combined. When a pipeline fails at
[14:16] midnight, data Dave doesn't just alert. It debugs the airflow logs, identifies the root cause in datab bricks, writes the fix in GitLab, creates the Jira ticket, and posts the resolution to Slack. One conversation, every system.
[14:34] When you need a business report, Data Dave queries the data, writes the SQL, builds the visualization, creates the Google Slides deck, and emails it to stakeholders. seconds. The compound effect is not additive. It is exponential. Each skill amplifies the
[14:50] others. A single prompt triggers a cascade of intelligent actions across your entire data infrastructure. Data Dave's usage has grown every single week since launch. Conversations up, users up, teams across admins discovering new ways to multiply their impact. Available
[15:07] right now in Slack and email. Just ask. Believe it or not, we did not hire any agencies to create this video. In fact,
[15:22] we use data Dave with a very very simple prompt to make it. So, intelligence is self-reporting. Uh, Karpathi called this the most important chart in the world right now. What it shows is that the length of an
[15:39] autonomous task by an agent doubles every 123 days. By autonomous I mean not requiring any human intervention. What what happens when something doubles with a fixed interval? It means that the
[15:55] growth is exponential. So for all of us to be successful in this exponential AI age, we need to shift our mindset from linear one to exponential. So in 2014 I attended Hadoop World
[16:11] conference in New York and I went to this I was actually talking at the conference too and I went to this session. It was about notebooks and how to process data. Um and it kind of picked my interest. I researched the company. It was some ambitious crazy
[16:28] Berkeley guys that started it and then I went to talk to Vidia in the in the booth and the rest is history. We we signed as what I believe to be the second customer for data bricks and we have been the customer since the beta product uh of data bricks. Um over the
[16:44] time we worked really closely with data bricks on new products. Um I remember emailing Ali you know late at night and saying it would be awesome if you guys implemented this feature it would benefit us within minutes he connects me with the right people and within few months you know the feature is there. Um
[17:02] so over the years we basically coales all of our data infrastructure on data bricks. Um Tim is here he can second me but we used to have Hadoop we used to have Natiza Informatica Redshift and over the last 10 years we coles all of
[17:17] our data onto data bricks. And why am I telling you this story? Well we believe that AI agents is just the tip of the iceberg. Before you can tackle AI agent um successfully, you really need to get your house in order and you need to
[17:32] unify all your data under one roof. If you're successful with that e effort, you will be able to decouple growth from the headcount and you will be able to scale via cognitive leverage. As you
[17:47] have seen in video, uh tasks that usually take uh a human, you know, one day now take minutes. And because the cost basically becomes LLM cost, you realize a huge value multiplier.
[18:06] So let's look under the hood of data into some of the capabilities and some of the implementation details. First, we decided that the interface for data Dave is going to be Slack. And the reason for that is that we want very low onboarding cost specifically for nontechnical users. We don't want them
[18:22] to install an IDE uh or a harness, configure it and so on and so forth. And there's two different ways you can interact with Slack. One is a private mode and the second one is uh a kind of team mode where you talk to Slack as a team member. The advantage of the second
[18:39] mode is that people can learn from how you interact with data Dave. The way we learn is through mimicking others and that works really well with data Dave and that's specifically important because the capabilities of data Dave are so broad that a lot of people don't
[18:55] even know what's possible and when you see others interface with it you learn. Second we leverage Amazon SQS as an interaction between the interface and the implementation on the worker side. So um that not only allows us to kind of
[19:11] roundroin and scale but also provides the fundamental safety net. Um so the message is never dqed until the worker successfully processes the request if for whatever reason the worker fails. Uh after a timeout another worker picks the
[19:28] message from the Amazon SQS. So you have a confidence that all the requests and all the session completes successfully and all that happens without any code. And finally we leverage uh Fargate workers for serverless processing of all
[19:43] the sessions. Um currently we have I believe four tasks configured per worker. So if there is a lot of demand coming we just automatically scale the system and we can handle uh almost I would say you know as many requests as we as we can.
[20:00] So um what is the brain of the system? Um the brain is really the agent core SDK and in the version one we used to use lang chain. In version two we converted to cloud agent SDK and big
[20:15] props to Josh and the entropic team that upgrade was really really significant for data. Um the uh the fact of the matter is that the cloud agent SDK works really really well. The reason that it works well is it has this agentic loop
[20:31] that allows you to add throw at it whatever you need which is skills and then we'll also discuss memory and exemplars but it just magically works. uh prior to cloud agent SDK we as I mentioned we use uh lang chain and the problem with that approach was that it
[20:47] required a lot of manipulation of the topology of the agents whereas now we don't have to worry about it we can just focus on you know adding skills and auxilary services so what about the system infrastructure I already spoke a little bit about AWS
[21:04] that is kind of uh playing the role of orchestration and containment container management we use Fargate for execution and the container service. We also uh leverage EFS for session persistence. So if for whatever reason the the worker goes down, it will signal
[21:21] the cloud SDK to persist the session in the middle. The the the work the cloud SDK was going to uh persist the session in EFS and then another worker that comes up is going to pick up the session in the middle. Uh we also leverage Dynamob for uh session application and
[21:40] um and cancellations. So it's kind of like an auxiliary service and finally cloudatch just for logging. Now data bricks plays the crucial role in the system. Um it's basically the data platform that powers the whole system and it does it in two different
[21:56] ways. First we aggregate all of our instrumentation metrics in data bricks for the whole system so we can improve the system over time. track it and see how it performs. Uh see which skills are the most important ones, how the users are interacting with the system. So it
[22:11] allows us to continuously improve the system. And second, it it is the data platform that powers our our most important skills specifically um our SQL analyst that answers any questions and it has an overview of what is happening within admins.
[22:28] So you know as most of you we use lakehouse uh for all of our data. uh we leverage lakebased for memory that we will discuss subsequently. Unity catalog uh provides us with permissions and access privileges and vector search I
[22:43] think is one of those underestimated services. It's it's super useful for a lot of our skills because it allows us to find relevant data and metadata for a variety of skills. Also um as a side note I say that you know we use Lovable and Vector search is
[23:00] is a powerful tool that allows us to to build apps because it has uh all the information about what is the what is important at at admins. And finally uh zerobus is super uh super cool to just ingest all the instrumentation metrics into the system with very little uh
[23:17] code. So um here I don't want you to to get overwhelmed and you don't have to read everything uh but the point of this uh this slide is to show you the extent of the skills that we actually currently have powering data and we approach this
[23:34] from the perspective of meta programming. So um the charter have for every team is to not solve the problems but create a skill that solves the problem. That way next time you have the problem, you don't have to solve it
[23:50] again. You already have the skill. So we basically ask every team um you know try to implement one skill and then after they implement one skill everyone kind of has this light bulb moment and going forward they always think about
[24:05] implementing new skills rather than just solving the problem. So we think this is a a super powerful pattern that is kind of like the future of of how development is is going to look like.
[24:22] Um so the skills are not only used to add capabilities they're also used to integrate other systems. So within admins we have uh systems like GitLab, Jira, Confluence, Google Drive, Amplitude, New Relic and many others and the skills uh allow us to integrate all
[24:37] those systems. for for the past decade we had this dream of creating this new knowledge system that will replace all the other ones and I can just go into one one place and get any knowledge about admins that I want and data Dave
[24:54] actually solved that problem uh and it didn't replace other systems it just provided an interface that accesses all those other systems so now if you have a question about anything you go to one place and you get the answer and you don't really care where it's coming from
[25:10] is it coming from Jira confluence I don't care I get an answer so how do we implement the skills uh within entropic cloud uh SDK there's two different ways you can implement skills uh for simple things you can just inline
[25:26] the skills and that's usually for like one step immediate execution uh in that case there's like isolation overhead and it's really fast and the skill has access to all your other tools So that's kind of like the plain way to do it. Now if your skill is multi uh
[25:44] uses multi-phase reasoning like if you have to plan, write and review, then you're going to use sub agents. And an example for that would be SQL analyst. When you access SQL analyst skills, you need to plan um the query,
[26:00] right? So the you know the SQL analyst has to figure out what data to use, compose the query, execute it and then um forward the results back to the reporting agent. Um so another another thing that sub agents do is parallel
[26:15] processing. Example for that would be code review. So the way we implemented code review skill is that we have two parallel sub aents. One of them does the traditional code review with unit test coverage make making sure that your system is uh is
[26:32] written properly but the second sub agent is doing architectural review. So it's basically trying to figure out how does this change fit into the whole ecosystem uh of modules within admins. And the nice thing is that they those things can be done in parallel which
[26:48] kind of speeds things up. But more importantly those sub agents are completely independent. So they are not bi they're not introducing any bias across and uh once both of them complete they report to the reporting agent and then the agent reports to the user all
[27:03] the findings whether there are issues on the architectural side or on the just traditional code review side. Another uh property of sub aents is that you can isolate what those sub aents have access to. And you know for a code review you most likely just want to give
[27:19] access to maybe GitLab possibly Jira but you don't want to potentially influence the agent to kind of look at data somewhere else. Uh that's really not the point of of code review.
[27:35] So you might ask um is there anything that data Dave is uh you know any anything that data Dave cannot do or shouldn't do and that is production system. We do not let data dave touch production system. Uh it can fix issues. It can triage issues. It can help you uh
[27:51] organize your jura stories, do project planning, create videos. It can do a lot of things. But it cannot access the master branch. All of our master branches are protected. All the data they can do is create another branch and create a merge
[28:07] request. But the human has to approve uh any of that. So that gives you the peace of mind of uh you know data Dave not going um you know crazy and doing something inappropriate. So you might ask uh like what's the big deal uh why can I use u you know cloud
[28:25] code or uh uh codeex or anti-gravity? Well there's there's fundamental distinction. First of all as I mentioned it's a zero setup access. So um the system is instantly accessible within Slack and that is specifically very
[28:42] important for nontechnical users. They don't want to mess with installing IDE and configuring them. Second, it's a multi-tenant one brain. So we have more than 100 concurrent users at a time and we can share unified memory for the
[28:58] whole organizations that we'll discuss subsequently. Finally, it's a central governance system. So it's a single control plane and any enhancements, any updates, any new skills are instantly available to everyone. We don't have to worry about,
[29:13] you know, some users having access to some skills and then someone else having other skills and they have kind of getting different results. Um the system is is kind of very unified. So how do we handle memory? Uh we think about memory in two different ways. One
[29:31] is individual personalization. So how do you want to interact you know with data Dave? I I will give you an example of what I do. So I I asked data Dave to always give me topic sentences first. Uh those of you who took writing intensive classes in college know that topic
[29:47] sentence is basically kind of like a summary sentence for every paragraph. And I kind of like it because I don't want to, you know, treat the response as a puzzle that I have to figure out. I want to know exactly what is it about and then more details. So that's just an example of how you can personalize
[30:02] day-to-day but you can obviously do whatever you want and the second one is is what worked and it's kind of memory for the whole organization. So if I learn something I want this learning to transfer to others. So what is an example? Let's say you're a developer
[30:19] and you're doing a unity catalog deployment and you figure out that you have to use specific service principle for that. At that point you would say uh you know data Dave save it as an examplar and next time anyone else does unity catalog deployment they don't need
[30:35] to figure it out automatically you know data Dave will u get better with time um and maybe more businessfriendly example would be lead conversion so you know for our business user lead conversion is very important um um you could theoretically just define it and say
[30:52] lead conversion is you know uh uh sales you know leads submitted over all the sales. So that's a very contrived example, but once you save that, then the system understands what lead conversions are and people don't need to define those terms. For data Dave,
[31:11] one of the uh most used features of data Dave is autonomous scheduling. So what it allows you to do is just use natural language to schedule jobs, list them, and maybe remove them. And the reason it's so popular I think is that
[31:26] historically nontechnical users never really had a good system for that. You know they don't really understand cron jobs uh and some other kind of scheduling tools. So now with just natural language they can schedule jobs and and run them. How do we use it? Uh we use it for
[31:42] meeting synthesis. So what we do is before every data Dave meeting uh we uh basically generate videos and the video basically captures what was done uh automatically over the last week and data Dave has awareness of the whole enterprise so it without without any
[31:59] complicated prompts it knows what happened what is what are the current topics and then as the meeting starts we just watch the video so it you know a lot of developers don't like to prepare for meetings and I think that's like a super useful feature at least you know for me and and and the team. Um another
[32:16] thing that we use for uh you know uh uh for scheduling is automated doc syncing. So every day we sync the docs for data Dave. On the right side you see an example and that example shows you that data Dave is not going to make assumptions. It will ask you to clarify
[32:32] like when you say update docs what does that really mean? Like do you want to refresh confluence pages? Do you want to query some tables and uh refresh stats? Do you want to sync something from data bricks? Um, and u obviously we use sonnet as as just mentioned and uh I
[32:50] think that model is really good at at not asking you too much but it's not going to make any assumptions. It's going to ask you for enough and then uh given that information it will do the best. Finally you we use scheduling for trend research and publishing. So we have a
[33:06] very popular Edmonds pulse report and what it does it researches the web for automotive information um all the trends that are happening in industry and then it combines it with the internal knowledge and our metrics and then based on this combined effort it provides the
[33:23] recommendations that are very business specific again because it has knowledge of the whole enterprise. Um so what is an example? Um you know oil oil prices went up recently and as a result a lot of people start looking into hybrids and EV vehicles and uh it's a very contrived
[33:40] example but data they've provided you know you guys should uh think about those products and enhancements on the website because it also understands our website and our products. So it has very targeted uh recommendations as to what what should be done.
[33:58] One of the more interesting things about datadave is those emerging capabilities. It's once you have so many skills, you basically don't even know what it's capable of. So, um, recently I've been asked, um, can data Dave, and shout out to Eugene, by the way, I've been asked,
[34:14] uh, can we throw slide deck at data Dave and then create a video that narrates the slide deck using narration from the content? And I was like, well, I worked on on that skill. It's not going to work because it basically you need to provide
[34:31] some content and then it creates animations and then narration. So I'm like, ah, it's not going to work. And then I'm like, okay, let me try it. So I wrote a simple prompt through the deck edit and voila, it worked. And u the reason it worked is because we had another skill uh we call it mark it down. And it's a multimodel skill that
[34:49] basically no matter what you throw at it, whether it's images, uh, PDFs, JSON files, HTML, uh, Microsoft files, uh, Google files, it will be able to extract the information, take screenshots, and then feed it back into memory. So, um, the reason that is
[35:07] actually happening is that as you're adding new skills, the cross skill capabilities grow quadratically. So if you have you know five integrated skills the number of connections it's 10. If you have 10 10 skills the number
[35:23] of crosskill connections is 45. It's basically metaf's law right the just the network the way the networks work. So currently we have 50 skills. If my math is correct that is more than a thousand cross skill capabilities. So there is
[35:40] something fundamental that happens when you move from like 10 skills to 50 skills and it's it's pretty amazing like how this how much more powerful the system gets. Um if you if you get to that point you will kind of you see it you know across your your companies.
[35:58] So let's let's summarize. Um we obviously the system leverages LLMs um and we as Josh mentioned we use Opus to write the whole system and uh we use Sonnet uh to actually run the system and
[36:14] set behaves amazingly well um and it's amazingly accurate and also it's more economical than than running OPUS and we don't really uh have a lot of uh need to to use more powerful moment model than on it. Uh for agent coordination, we
[36:32] leverage as I mentioned claude SDK and again that framework grows really well. So we um we're very happy with that and we basically kind of outsource that to entropic and those two things work well and we don't really have competitive
[36:47] advantage in in uh you know in improving any of those uh components. What we really focus on is the skills and the self-improvement. This is the domain specific area that we think we can contribute the most and we
[37:02] get the most value. So it's it's a kind of nice divide between the two where you you just focus on your domain and that's where you get majority of the improvement. And I'm not going to claim that this is the new EMC squared for agents but you
[37:18] know maybe maybe it will become uh but you basically have those four areas and all of those areas are improving dramatically. you know, almost every month you have a new more powerful LLM and you just get it for free and your system gets more powerful. The agent coordination is is getting better and
[37:34] better. You know, Entropic uh has a private preview for dreaming. Uh and we are really excited to at some point try it and it's going to take our memory to the next level. And then skills, when you add new skills, the power of it grows
[37:49] quadratically uh as I showed you. And finally, this self-improving quality is a is is a is a property where the more you use the system, the better the system gets with memory. And if you put all those components together, they kind
[38:05] of have this multiplicative effect. And that's how you really achieve exponential power gain. So, I'm sure you're um you know at at your companies you have a lot of very smart people that leverage a lot of AI
[38:22] tools, but I would guess that maybe majority of them are going to be siloed and secluded. What we're trying to do is create a shared nervous system. Um so when I figure out how to fix a pipeline, everyone else acquires that capability.
[38:40] When one of us gets faster at Edmunds, the whole company gets faster. And that's how we're trying to realize the true collective intelligence.
[38:55] So over the decades companies thought that their mode is to build processes uh implement software uh acquire the best people build a brand acquire data build customer relationships and even build intellectual property.
[39:15] What I think is in this new agentic world, you need to build a compounding cognition loop, a living system that reasons, executes, and becomes exponentially smarter with every single task.
[39:31] So, uh, with that, um, me and Josh have a couple hypothetical or theoretical rhetorical questions to you. Um you know throughout this conference uh a lot of people said well the spending on AI is getting higher and higher and and we
[39:47] don't really see ROI. Um so our questions are are you setting your goals ambitious enough and are you ready to deploy your digital workforce? Thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.