AI Will Go Wrong and the Blueprint to Get It Right
Summary
- Most AI failures stem from people, process, and architecture gaps rather than technology limitations—the model accounts for only 10% of what it takes to build an effective AI platform.
- Building trustworthy AI requires a three-layer architecture combining domain expertise, governance guardrails, and the underlying data platform, with trust coming from business relevance rather than statistical accuracy alone.
- This video walks through a live AI failure post-mortem and a hands-on exercise where participants act as domain experts to design an AI Judge capable of scaling oversight across enterprise deployments.
AI Will Go Wrong and the Blueprint to Get It Right
Most AI failures aren't technology problems — they're people, process, and architecture problems. In this hands-on session, you'll witness a live AI trainwreck: an agent operating without guardrails, governance, or the right organizational design. Then, in a real-time post-mortem, we will collectively dissect what broke and why. You'll map the missing middle layer: the intersection of roles, accountability structures, and platform architecture. And you'll work alongside peers to design practical fixes. The session closes with an exercise where attendees act as domain experts to train an AI Judge capable of scaling oversight across enterprise deployments. Expect minimal slides, maximum doing, and a concrete blueprint you can apply within 30 days. this video is built for executives who are tired of theoretical AI frameworks and want proven patterns that make AI projects safer, faster, and genuinely business-ready.
Talk By: Lexy Kassan, Lead Data & AI Strategist, Databricks ; Maria Zervou, Chief AI Officer - EMEA, Databricks ;
Chapters
00:00Introduction00:33Success defined: trust, adoption, and business outcome01:09Inverting AI: model is 10%, people and process are 90%02:36Three-layer architecture: platform, guardrails, domain expertise04:03Domain experts, metric views, and Genie Ontology05:05Overcoming transformational inertia in AI adoption18:17Designing KPI trust evaluation: ownership, assumptions, evidence19:09Live interactive exercise: building an AI trust framework36:06Executive sponsorship and sustaining AI adoption
FAQs
Why do most AI projects fail to deliver business value?
According to this video, AI projects fail primarily because of gaps in people, processes, and architecture rather than model limitations. Without defined accountability structures, governance guardrails, and domain expertise baked into the platform, even technically accurate AI will not be trusted or adopted.
What does the '10% model' principle mean for AI transformation?
The 10% model principle means that the algorithm or AI model itself represents only about 10% of what it takes to build an effective AI platform. The remaining 90% comes from the people, processes, and architectural decisions that govern how AI is evaluated, deployed, and used.
What is the three-layer AI architecture described in this video?
The three-layer architecture places the AI model at the foundation, the platform infrastructure and guardrails in the middle, and people and processes—including domain experts, accountability structures, and evaluation frameworks—at the top. The people and process layer defines how AI should behave, which is then enforced through the infrastructure layer.
How can enterprises build an AI Judge to scale oversight?
The video demonstrates a hands-on exercise where domain experts collaboratively define criteria for evaluating AI outputs, which then trains an AI Judge. This judge can scale oversight across large enterprise deployments by applying consistent, expert-informed criteria without requiring human review of every AI output.
Full transcript
[00:09] Hi everybody. Welcome to the session. Uh we really appreciate you being here on Thursday morning with us. Uh as Katherine mentioned, this is AI will go wrong uh and the blueprint to get it right. So, we're going to talk first about what it means to get it right and then how we're actually going
[00:21] to get there. Uh so, thank you so much. At the start of this, I want to lay out what is clearly on the screen. Success is a business outcome. Right? This isn't about getting the AI
[00:33] to production. It's making sure that AI is adopted, trusted, and scalable once it's there so that you actually get use and the business gets value. That's the point. Without trust in AI, you're not going to
[00:45] be able to achieve those outcomes. And typically, what we see is that trust comes not necessarily from having AI that has a high accuracy rate for some of the statistic, but that people actually believe that it's representing
[00:57] things that should be representing for their their organization. So, with that in mind, we're going to talk a little bit about what it means to actually put that together and then you're going to be a part of doing that today in the room. Uh it doesn't require
[01:09] a laptop, so uh it'll just be a phone, by the way. Uh just put that out there for the hands-on part. All right. So, taking that to the next level, driving AI transformation and building
[01:21] these platforms, building AI systems takes not just the algorithm, the piece that we all hear about, all the models that we talk about, um that's like 10% of what it takes to actually create an effective AI platform and AI tooling
[01:35] that's actually going to drive the business. Instead, most of what we need is coming from the people and processes engaged in creating evaluations to make sure that the judgment of AI is appropriate to
[01:47] use. So, those people and process are you, your teams, the way that you operate, the way that you measure your organization, how you want AI to act. That's then instantiated in your
[02:00] infrastructure and architecture. That's where it's enforced, so whatever the people and process say needs to be done is that enforced in the architecture. And only after that do we have that
[02:12] model. Right? That that model, the algorithm that everyone's so excited about, whether it's Opus or it's Chat GPT or whatever we're going to do, those are like the smallest
[02:24] consideration. So we need to invert how we think about AI to focus more about people and process. And we think about who those people are. I'll put this in three buckets.
[02:36] Uh we have a kind of a layered system here. At the platform level, that kind of the foundations at the bottom, most people think primarily about the platform itself and the data
[02:48] foundations, starting in that build and run grouping. But instead, we need to think about this cross-functionally and collaboratively. So yes, the platform and the data foundation are important. They're going
[03:01] to give you the solid foundations and the governance to start your journey with AI. That includes processing, lineage, visibility, security, and auditability
[03:13] as core components of how your AI will continue to be trusted. Above that then, we talked about having those instantiated guardrails. That happens in the AI architecture. So as we start to
[03:26] build these capabilities for AI, that's in your AI architecture layer, you get observability, you get guardrails, and this is the piece that we're going to be building today, but not from a technical perspective. So typically, when we think
[03:38] about the AI architecture, it's somebody in IT somewhere is standing up a system and putting this in. That's not actually what we're doing today. Like I said, cross-functional, people and process. You're going to be domain experts in the
[03:51] room, because that's really where it starts coming together. Again, you've got your model and algorithm there. But, where we're going to be focusing today is actually on the side is that left?
[04:03] >>
[04:15] layer that we're creating in Databricks, whether that's through metric views or through the ontologies you may have heard about in our keynotes, that's where ground truth and the understanding of the business actually gets captured. And it's your domain experts who are the
[04:28] ones owning and curating that information so that it's reflective of your organization and its ground truth. You also have your data analysts as a part of that, business analysts creating those definitions, especially when you
[04:41] think about something like metric views, anything that requires the translation of understanding of business context to code. And then finally, for really high-stakes models, you might have model stewards
[04:53] providing independent oversight. All of these types of knowledge can be encoded and created to support scalability across the organization so that it's not just one model that gets governed, it's
[05:05] all of them. And on the other side, I want to make sure that we acknowledge there's a significant amount of transformational inertia that we need to overcome when putting in AI. At the top of it, well, technically at
[05:17] the bottom of this layer, >>
[05:30] This is the new way of working, and we do encourage and require that that the organization transform to accomplish it. Now, that includes setting the ambition in that way, but also assuming that
[05:42] there is risk. There's not ever going to be zero risk in AI. So, we need to understand what the risk appetite is from the top, and we also need to have resources supporting that to both build and govern as we go along.
[05:56] Risk management is also key in creating the assessment and mitigation frameworks to ensure that all of that governance gets put in place, it's reviewed, and it's appropriate to that risk appetite.
[06:08] Finally, we have change management. Change management helps the team to refactor ways of working and and get that AI into the hands of users so that they're seeing it, trusting it, working
[06:20] with it every day. Again, business outcome. I promise we're almost done with the slide portion, and we're going to go into something hands-on, I swear. One more. This is it. And this is where it's going to start looking more like what you're about to
[06:32] do. So, we talked about all of those different roles, how we get it pulled together. First of all, we're looking for business knowledge here. Again, not statistical knowledge. We're going to talk about that some other point in time. This is
[06:45] business knowledge. That's coming from domain experts, you all. We're designating you domain experts. Could also be business
[07:00] say is right or wrong in the AI. How well do we think that the AI is reflecting what we need it to? Then, we're going to put that into measured outcomes, meaning taking all of those evaluations, applying them to the
[07:13] business problem at hand, and seeing how AI uses those to address what we're trying to accomplish with system. And again, that becomes part of system behavior. Once we say, "Yes, that looks
[07:26] good." it gets put into the architecture, it's then available and usable throughout. Underpinning this, of course, is the full foundational platform. What you're What you're actually looking at right now, and what you will see as we continue on, is a
[07:39] Databricks app built on top of Databricks, of course, with a tremendous amount of AI behind the scenes. So, without further ado, we're going to get right in to the exercise.
[07:51] >> I'll sit here. >> Or whichever. >> I think I'm here. >> You good? >> Yes. >> Cool. So, >> this is the product that we're going to
[08:03] actually challenge today, and it's called the Executive AI. And basically, what it represents is a SaaS system, an AI system that your chief of staff saw in a conference, they loved it, and they said, "You know, we
[08:15] want it. We want it in our company. So, we bought it." But what does it do? Basically, we can plug in all the data from our company. Like, think of it like board discussions, financial reports, strategic documents, all our market
[08:29] research, forecasting and planning. Basically, all the data that we have in the in in our systems, we plug it in into this app. And the idea of this app is that it goes through uh any different or potential
[08:41] acquisition opportunities for our business. And judge it how good or bad it is. And then it presents the best acquisition with specific KPIs that we judge against. So, this is the application.
[08:54] But let's zoom in and see what is happening with this project Aura Aurora. So, this is an acquisition candidate that we need to understand if this is something that we need to buy. Think of it as a software company, let's say, that we need to acquire or not.
[09:08] But how would we how would would would we do that in a kind of data-driven way? So, this is how if we zoom in into the AI system, this is what it gives us. It gives us that this is a really
[09:20] high-confidence acquisition, and this is the purchase price, like 340 million we would need to pay for this software to get it in our business. But why? Why?
[09:32] These are different KPIs that the AI system validated against, right? So, we have the financial strength. It says that we have 67 million ARR and 42% year-over-year, and all the data
[09:45] behind this is audited. But, you What you can see here it also surfaces some signals that we need to be really careful about. So, it says that the financial strength is good, but 20%
[09:57] of the ARR is coming from the top customers, and also 84% of all of these is coming from big enterprise contracts, which is good. So, this looks strong. There are some signals that we need to
[10:09] validate, but looks fine. Then, we have the customer health. What's happening here? We're saying that our customer health is actually very good. 3.4% increase year-over-year, and the data are all audited.
[10:22] The average contract for our customers around 4 years, and the gross retention is quite high. So, what this means is that we have good customer health. The customers are retained, so no one is leaving. So, it looks like healthy KPI.
[10:36] Then, we have the market opportunity. This is the potential market we're trying to capture, right, with this acquisition, and we have 5.2 billion TAM, 23% year-over-year. But, what we see here is that
[10:49] this data come from one source, a Gartner source, that was actually an external research happened 2023, right? So, it's kind of saying that we have this opportunity to address,
[11:01] but it's based on old data, right? And the last one is the investment on return, and this looks great. Like, we have 412% 5-year ROI, and we have validated cost synergies, market growth,
[11:13] and all of these different KPIs. What does this tell us? This is the system This is the the acquisition we want to do, and this is all the data we have behind to make us confident in this acquisition.
[11:26] But, all of this data is based on assumptions. So, what I I you to start thinking now is are the assumptions strong enough and verified that you would actually put your name against this acquisition?
[11:39] And this is the exercise we're going to go through today. What's happening behind the scenes as we said is that we bought this SAS tool, we plugged in all our data inside and we were hoping for it to give us
[11:52] business specific recommendations. But what is happening is that this system, while it sees our data, it doesn't understand anything about our business. It doesn't understand how do we make decisions. It doesn't understand
[12:05] what are the KPIs we need to audit before making acquisition and doesn't understand anything really about our strategy, about all our business terms and capabilities. So, how can we validate and challenge
[12:19] these assumptions? I'm going to show you an example because this is the example I'm going to do with you, but then this is the logic you will need to follow for the exercise that we're going to do.
[12:31] So, as we said here, we have an investment over return KPI with 400% 5-year ROI. If we zoom in into this KPI, we're trying to see where is this coming from basically, right? So, what are the data
[12:44] that feed into these assumptions and how all of these were validated? And we say that we have validated the customer demand, there is huge strategic fit for our company, we have executive
[12:56] sponsor and we have compared different acquisitions that we have done before. But there's one KPI that is under review and this is the revenue attribution consistency. And the question now you need to start
[13:08] thinking because it's going to follow along in the whole exercise is, okay, it says this system says that it validates something and gets something. But who actually was the person who validated that? Was any of the business
[13:21] experts in the loop for us to approve and validate this acquisition or it was just a technical team coming up with a random KPIs, calculating things as they think are right, and then not involving
[13:34] any of the business stakeholders into the loop. Because for us to trust it, this means that the customer demand was validated by the VP of sales, the strategic fit was validated by the strategic team, the executive executive
[13:48] sponsorship is coming from the CEO, and the acquisition evidence coming from the corporate development team. And the final thing is the one that we have to come up with the methodology on how we do revenue attribution.
[14:01] So, this is what we did. We actually we worked with all of these people, and we went back and challenged the way the KPIs were calculated for the investment return. And what we found is that
[14:14] actually there were mistakes on the calculations behind the scenes. So, while before we're saying that we had the projected ROI of of 412% and that we would actually be willing to pay a price 340 million for this
[14:27] company, now we're saying that actually we went through the data, we validated with all the business users, and actually we found that the revenue synergy was double counted, and that is why the revenue now should be
[14:40] the fair price should be 280 million, and it is a completely different scenario for this business case, right? So, now we're saying that it's not such of a strong case because the people who had to validate it were in the loop,
[14:52] challenged the KPIs of the AI, and came up with a more informative, business-oriented decision. But the thing is, how we would go there without having to present something to
[15:04] the board? So, how can we do that before we present the full board decision? >> All right. So, this is where all the things you're about to do are going to come into play.
[15:16] What we're going to do instead of having this ungoverned, uncurated data going into our AI system. We're going to start by having corporate data sources that are curated, that are governed, that we can
[15:29] trace, they have lineage. We have metrics that we calculate based on our methodologies that are that are applicable to our due diligence. Those will then go into pre-process judges. This is the majority of what you
[15:42] guys are going to be doing here. So, these are things where we could check for do we have enough information? Is the methodology correct? Do we have source verification? Do we know that the
[15:54] data is up to date? Who's going to be telling us where this should come from? These are the pre-process judges that will help us to create our evaluation for the AI. Only then can we push this
[16:08] to the AI agent for its evaluation. And while it says the brief would be sent to the board, actually, there are a few more steps. We want to then have post-process judges. This is also going to be a part
[16:20] of what will be created. You'll see this happening live while you fill in the hands-on survey. So, in the post-process judge, it's going to make sure that we have the appropriate scope validation, that the evaluations that we've created, the
[16:33] criteria that we've laid out for what is going to be acceptable for this business case are followed, and if it's not, it will flag it in the final results. There's then a human review gate. Again,
[16:45] 70% of this is people and process. So, at the end of all of this that happens, it's not going to just go off to the board. There's additional due diligence. We want to make sure that everyone who should have been approving
[16:58] and and ensuring that validations are there has the opportunity to do so. It gets all of the appropriate sign-offs. Only then do we have a certified brief that can actually get to the board that's audited, reviewed, transparent,
[17:11] has all of the capabilities in there that we need it to, and all of the evaluations have passed. So, it will have that full audit trail, every decision logged, full provenance chain, all the sources available.
[17:25] All right. With that, it's going to be your turn. >> Yes. So, let's describe a little bit the methodology on what we we are expecting you to do so that we understand.
[17:37] I don't want you to I will present again the KPIs that we have to judge this acquisition, but I don't want you to think if this looks nice or not. I want you to see and challenge if you would trust it, and what do you need for you
[17:50] to trust it, right? So, the framework follows these four questions. The first thing is for a specific KPI, we think about the the the health of our customers, who would be the person who owns this,
[18:03] right? Who is the person who knows the most about it, and that if they approve it, you would trust it? Then So, this is the owner part. Then we have the part where we are thinking about the threshold. So, what would you
[18:17] need for you to change your decision? So, what is the threshold under which or over which it would flip your decision against being confident to this KPI? For example, if the ROI falls below a
[18:29] percentage, then the board shouldn't go ahead with this acquisition. Then the next thing is not only about the threshold, but what is the business assumptions we make behind the threshold, right? So, what must must be
[18:41] true for this KPI to be trusted? So, what are the business assumptions that we need to clarify and be there for you to trust it? And then what evidence would you require? So, what are the data points that you need to put in place so
[18:55] that if you need to verify something, you can go back to these data points. And in that way, we will be judged because as you answer these questions, basically we are defining what the business needs. We are co-defining that,
[19:09] and then we are applying to the AI. So, we're going to build now this business layer that will pass the AI through. And so, this is where you play in. So, please
[19:22] take out your phones. Scan this QR code. And you will need to choose one of these three KPIs. And you have to answer the questions, basically. In case you are in iPhone and
[19:36] default browser is Safari, you may see a problem with authori- authorization. Just copy the link and open it to another browser. Um and you have time to fill all the KPIs or one. Just go through this and
[19:49] try to see what would you make what would make need what would you need to trust this, right? So, what are the data? Who is the owner? What is the business assumptions that we need to have behind to trust
[20:01] this financial strength KPI, the customer health KPI, or the market opportunity KPI. So, yeah. >> And feel free to make things up. >> Yeah. >> There is
[20:15] this organization and its due diligence framework on the fly. So, make up people, make up groups, make up data that you would want to have, whatever you need. And what we'll see as
[20:27] we go through is that these uh that behind the scenes this is going to be grouping the information that you put in and creating candidates for judges.
[20:47] And uh what we have done on the Who Would Know is we're trying basically to see who are all the different people that need to be involved in this decision-making, right? And as you can see, we have finance people, strategy people, sales people. So, in
[21:01] order to have an AI system that takes you from data to a decision-making system, we need to have all of these business expertise into the loop.
[21:17] >> All right. And we can give it more than 3 minutes if need be. We I think we have time for it. So, if you want to challenge more than one, go right ahead. Put in as much or as little info as you want. And let's see what comes up.
[21:31] I appreciate the one that says CEO {{}parentheses} me. >>
[21:46] It's often the case that we know that one person knows the thing, and they're the one that we need to check with. >>
[22:02] >> Great. As you can see, there's so many different people that need to be involved, right? That is the point of the discussion here is how do we bring all of these people in the loop, and how do we get their uh their expertise into the system, basically? >> And if we feel like when when you open
[22:15] this up and you look at one of the metrics, you go, "I wouldn't even know where to start." It's a great indication that you need to bring more people into the process who know these things. This is how you get that 70% people and
[22:27] process becoming part of the system. >> So, yeah, amazing. Thank Thank Thanks for sharing. And um yeah, what we do behind the scenes is we collect your
[22:39] responses, and basically we try to group them uh we try to group them into these specific kind of most popular, let's say, judges. And then what we're going to do in the next next screen is basically we're
[22:52] parse parcel of this data, and then try to create some high-level judge definitions, so that we can see visually what we mean, and what are the rules we're applying. >> Yeah, so you can see for example that
[23:04] right now we have 24 people in the market opportunity saying something about market definition and alignment. That that's an important factor. So behind the scenes that classification is actually taking those natural language responses and grouping them to say these
[23:17] have really similar themes. So we're going to create something around that theme. >> Yeah, we can keep >> Yeah, we'll keep keep it up while it's it's refreshing a few more.
[23:29] There any final ones coming in and then we'll go into the judges. So we've got 148 responses. Love it.
[23:46] Getting there. All right. I think it's slowed down. We ready? >> Yes. >>
[23:59] each of the KPIs, basically we're going to group the most the top responses, let's say for the judge and come up with a judge definition. Okay. So
[24:13] this is the first judge and this is about the financial financial strength KPI and basically what we pass is what is the idea of this judge? What does it do? Basically the goes to the financial statements and metrics and says that it
[24:25] has to be reviewed and audited and with external third parties as well. So this is coming from what you guys created. So gross revenue acquisition and retention and customer churn analysis need to be documented and traceable
[24:37] against a specific source. And top customer concentration is measured and documented. Now, this is the idea of the judge, but how do we actually apply it on the system? We need some criteria and this is the pass and the fail, right? So the pass when when
[24:50] does this pass against something the AI product will produce? It passes when all the financial data are verified by both internal and external sources and top customer concentration is less than 10%
[25:03] and GRR trend is documented. So, all of these need to be true for you to trust this financial KPI that our application have created, right? So, basically all your ideas, all your ideas
[25:16] you have from your business expertise for this financial strength perspective, it's coming into one judge that we codified and now we will apply on the AI. >> All right. Next up, we have our customer
[25:29] health. And in customer health, you said that we need to check that it's accurate and verified and also reconciled with any financial data sources. That the monthly active users and retention definitions are documented and
[25:43] consistent with our month-over-month and year-over-year calculation methodologies from defined sources. And once again, that we have a gross revenue retention and net revenue retention meeting specific thresholds that are validated.
[25:57] So, we have to make sure for this to to pass, it has to have all of that data accuracy is confirmed, metrics definitions are documented and everything is at that threshold. If it does not, if it's unconfirmed, this will
[26:10] actually block our moving forward. >> Yep. And the last one is about the the market opportunity. And here we're saying that the market data is sourced from at least two independent and credible sources, right? Because if you
[26:22] remember what we had in the initial brief was just one old data source. All the market data has been refreshed within the last 12 months, so it's not from 2023. I cannot trust this data
[26:35] source anymore. And then the TAM is explicitly defined, calculated and validated against multiple sources. And that is the definition of the judge, but what does the judge do exactly and what is the codified version of the judge?
[26:47] The market data source needs to be more than two and independent and the data refresh needs to be more than in the last 12 months and the time verified by multiple sources. So, yeah, these are the judges that the
[26:59] judges we all created today. Excellent. >> So then, let's see what happens when we actually apply them. >> Cool. So then, this is exactly the brief we had in the beginning, but what we did is we challenged it or passed passed it
[27:13] through this business judging play we created just before, right? So if you see the three top judges we just presented, I'll apply I'll apply them on the brief and what are the pass or fail criteria for each of these judges. So in
[27:26] the financial strength, you can see that actually the financial auditor judge passed because all the data are audited, top customer is less than 30 40% and the enterprise mixes around 84%. So the
[27:39] judge passed, right? So all the KPI criteria are passing in this case. In the customer health, we see that the data integrity judge we just created, it's actually not blocking but raising
[27:52] some concerns because the gross retention is actually fine. So it's failed because it's less than 95% and the concentration is kind of raising a flag because it's around 28%. So we have high concentration in specific
[28:05] customers. And and then you want to do the market opportunity? >> Oh, sure. Yeah. So our market opportunity has has been flagged because we had only one source. And remember we said it had to be a more recent source
[28:17] and it had to be more than one. So because we put that in, it says oh, it's stale, you don't have enough research here. We need to look back at this again. It's it's not going to pass. And then finally we have our return on
[28:29] investment which we already knew about. So with all of that said, it means that instead of proceeding with high confidence a 340 million acquisition, it's under review and we don't know what
[28:41] we're paying for this and its confidence has tanked, which Which appropriate. >> Yeah. So the idea is where we started is that this application was taking some of that data,
[28:53] surfacing specific APIs that no one has validated. Now, we are validating because we built these APIs, these judges, that we codified all the business expertise and
[29:06] knowledge into the judge, and now the judge is applied as a layer on the app without actually changing the data or how the app itself is working, just adding some judges or scorers
[29:20] to see how confident we have on what the agent is presenting. And that is the idea of the session, right? It's like how can we codify all the business knowledge and expertise and apply it on
[29:32] our AI systems. >> All right. So you've seen it in action, the lovely loop. We took our business knowledge. We said, "Who's got that knowledge? How do we get them involved in this process? How do
[29:45] they create the evaluation criteria? How do we then measure the outcomes from that and instantiate that into the system behavior so that our decisions can be trusted and we can actually start using AI and getting our business value
[29:59] and business outcomes." So, with that, I think we have some time for questions, a few minutes for questions. If you have questions, there are two microphones there. You can come on uh >> Hello.
[30:11] >> Thank you so much for all of your responses. That was fantastic. >> Okay, I have a question that we can Oh, yes, let's give them a round of applause.
[30:26] >> So I'll start us off with a question to give people a moment to decide what they would like to ask. You opened the session by letting an agent fail on purpose. What's the failure you see most often in real enterprise deployments and why does it keep happening?
[30:39] >> Stop. >> You want to start? >> Oh, sure. I'll take one. So the the thing that I tend to see is that uh the criteria are not defined for success up front. Um, so whether that's
[30:51] something where you could say this is an adoptable process, this is a something that we could actually put in the hands of users and they know that there would be impact for using it, or if they've not put in any evaluation criteria to
[31:03] say what good enough looks like. Right? Assuming that you can get perfection out of AI is a a falsehood, right? It's a fool's errand. So you want to make sure that you understand from a risk and benefit perspective what is an
[31:16] appropriate threshold that you can reach before you start to actually scale that AI out so that you get the opportunity to put that into production at some point. >> Just to add also from my perspective is usually we see we see people defining
[31:29] very technical KPIs, not aligned to business value as well. And ideally what we're trying to do here is yes, technical KPIs like latency and etc. are important, but we always need to map it
[31:41] to the end business value, which could be customer churn or something, or click-through rate, right? So how can we map it to the real business value? Because that will bring value to our AI systems as well.
[31:53] >> All right. There's another question over here. >> Oh, there is? Great. >> So I think a lot of teams probably have already started jumping on to the agent and AI bandwagon. So what
[32:05] recommendations do you have to retroactively kind of put it through this loop and start getting some of these agents into control with, you know, the the governance
[32:18] around it? >> I have an example that actually failed. And basically we're working with a company who is doing a lot of women health things, let's say, or AIs.
[32:30] And we're working with a technical team on building the model, the architecture of the agent and etc. because it was a chatbot architecture Uh on an on the app. And everything was
[32:43] perfect. We're very happy. We're actually innovating a lot because we're doing a lot of innovative architecture setups. And then when we want to try to put into production, we had to pass through approvals, right? And when we went to
[32:55] pass through these approvals, well, these people had no clue like the medical team had no clue we're doing that and that we hadn't involved them. So then what happened is the project didn't move. So we had to re-implement
[33:08] it and use the medical team as here as we did today as our business SMEs. We're actually them giving giving us context on what is good and what is bad from their perspective then it be codified
[33:20] into our application and then our application was fine-tuned on their feedback. So what I want to say is that yes, many people do this, start from technical perspective and then try to launch it and they will get blocked
[33:33] because you always need to have the person who will approve you into the loop. And I think that was one of the biggest learning. How do we bring them? The person who willing to approve for your product to go live, you need to look them in as early as possible and you need to get their feedback.
[33:46] >> One other thing I'll say is it's never too late to govern. >> Yeah. >>
[33:58] stewards, part of the involvement of the people and into the process of AI is looking at the responses of AI in production and actually continuing to refine what the evaluations should be.
[34:11] So adding those judges, adding evaluations, and being able to then take that and improve the quality over time across multiple judges. Um it is very feasible um and it's something that you know, within the Databricks platform you
[34:23] have the ability to add judges later. It doesn't have to have all happened up front. Ideally it happens at the start, but like if you're if the horse is already bolted, you you know, you can still bring it back in. Yeah.
[34:36] >> Could you provide some strategies that we can utilize to get some of the SMEs to get involved in that process cuz when you're dealing with AI and you're dealing with moving into using agents to kind of make these decisions or create
[34:49] the reporting, there is could be some resistance. So, can you provide a little bit about or some strategies that we can use to kind of get them involved in the process and make sure and I guess kind of communicate how important their knowledge is in to creating these
[35:02] agents. >> Yeah, you're you're absolutely right in that there is resistance and and frankly, it's crucial that they understand that it's not just we want your knowledge so that we can capture all of it and then let you leave the building, right? We we want you to be a
[35:15] part of this process on an ongoing basis. So, to the, you know, previous question around how do we make sure that even if we have something in production, it's continuing to function, we have appropriate guardrails. They need to be part of that process on an ongoing
[35:28] basis, not just up front. And helping them to understand what that looks like as we progress, making them part of the team that's actually going to continue to support the AI that will then support
[35:40] them and all of their colleagues is a great way to get them into that. It you know, if you think about kind of change management, getting people like the desire going for being involved in that, it's you know, there is a little bit of a different style of education than just
[35:53] saying, "This is going to change your way of working and isn't it great? You're going to be so much better at it." Because there is true fear, right? And we do need to make sure that we acknowledge that but also provide them with specifics of this is yes, it's
[36:06] going to change what we expect of you but for the better and in perpetuity, you're going to be a part of this. >> Thank you. >> Also, just to add for what I've seen working well is always have a champion
[36:20] in the team that you are building something for and and get the requirements from the team and the champion. And the idea of the champion is it's so involved in the process that whatever you're building matches really
[36:32] well with the requirements, and then the champion will go to the team and champion the product, right? Will help with increasing the consumption and making sure they understand the reason we are doing what we are doing. Um and so, having a champion that will their
[36:46] role is to go and talk about the product and bring everyone in the loop or involved in this progress, it's key at this point. >> Thank you. >> Thanks. I think we have time for one more quick question. I was just getting the 1-minute warning.
[36:59] >> Fantastic. Thank you for the great session. The live interaction was uh a lot of fun. So, kind of on a similar vein, there are there there's a lot of excitement around AI and a lot of like want for adoption.
[37:11] However, this is competing with kind of normal business operations and prioritization. And so, a lot of times we have teams who are like, "Ah, we really want to do something like this. We just don't have time." Or like, "We'll kind of start
[37:24] into it." And then someone will be like, "Ah, I got pulled off cuz this project." So, what is like a framework or a strategy to then advocate and push and prioritize this framework? >> Yeah, so this is where at the start when
[37:37] I said that leadership is an important piece of this and getting resources an important component to having the ability to move forward, this is where it comes in, right? Because somebody has to say, "No, you're carving out time. Or
[37:49] we're getting additional support so that we can reallocate people to do these types of projects, to get things moving with AI." Otherwise, you're stuck in resource constraint like you're talking about. So, this is where that executive
[38:02] sponsorship really needs to understand not only "Oh, we need to have money for tokens. We need to have people in IT working on things." It's there are people in the business functions in
[38:14] those teams that will be required. Their time and expertise is required as part of this initiative to move things forward successfully, right? So, that's where like that's a piece of it. Individually, if there's the opportunity
[38:26] for other leaders throughout the organization to say, "I'm going to have my team allocate 10% of their time or we're going to kind of second them for a couple hours a week." That's fine, but it's not going to be necessarily as
[38:38] visible to leadership unless you've got that sponsorship at the top. >> For me it's also that you have to the product projects that you're working on, they need to be attached to a business value. At the end of the day. Otherwise,
[38:50] no one will care, right? And different priorities will start drifting you away. But what are you trying to solve for? And always attach it there. And I would plus one what you said on the exact sponsors. If you need someone high that
[39:02] cares about these, and this is the business value we're trying to deliver for them. >> Fantastic. Thank you. >> Thanks so much everyone. >> Thank you.
Learn more about the Databricks Data and AI platform.
The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.