Agentic AI in regulated industries: Merck's GenAI-ready data platform

Summary

Merck discovered that AI hallucinations on enterprise data stem not from the models themselves but from insufficient context, leading the team to transform their analytics foundation into a GenAI-ready lakehouse with six structured context layers.
The six-layer framework adds domain semantics, KPI definitions, use-case rules, enterprise metadata, and runtime feedback loops on top of existing analytics-ready data products, enabling Databricks Genie Spaces and Agent Bricks to deliver 95% accuracy on clinical and commercial questions.
After onboarding 200+ users, the platform achieved a 50% reduction in time-to-insight, providing a replicable blueprint for deploying agentic AI in regulated industries with governance and data lineage built in by default.

Agentic AI in regulated industries: Merck's GenAI-ready data platform

GenAI models built on fragmented, uncontextualized data produce confident wrong answers. Merck transformed its analytics platform into an AI-ready lakehouse by embedding business context in six structured layers, enabling agentic AI to deliver 95% accuracy on clinical and commercial questions.
Discover how Merck evolved from analytics-ready data products to GenAI-ready ones by adding domain semantics, KPI definitions, use case rules, enterprise metadata, and runtime feedback loops. Learn their technical architecture using Databricks Genie Spaces and Agent Bricks for multi-agent orchestration, their rapid enablement of 200+ users, and the 50% reduction in time-to-insight. this video provides a blueprint for regulated industries to deploy agentic AI systems with governance and data lineage as defaults.
🤝

Chapters

FAQs

Why did Merck's AI agents produce wrong answers?

Merck found the problem was not the GenAI models themselves but the data layer—existing analytics-ready data products lacked the business context, KPI definitions, and domain semantics that agents need to generate correct answers. When an AI wrote a correct-looking SQL query against an improperly contextualized dataset, the result was confident but factually wrong.

What are the six layers in Merck's GenAI-ready data framework?

Merck's framework adds domain semantics, KPI definitions, use-case rules, enterprise metadata, and runtime feedback loops on top of their existing analytics-ready data products. Together these layers give AI agents the context needed to correctly interpret and answer clinical and commercial business questions.

What results did Merck achieve with agentic AI on Databricks?

By building a GenAI-ready data foundation and deploying Databricks Genie Spaces and Agent Bricks for multi-agent orchestration, Merck enabled 200+ users and achieved 95% accuracy on clinical and commercial questions, alongside a 50% reduction in time-to-insight.

How does Merck's approach apply to other regulated industries?

Merck's maturity model provides a blueprint for any regulated industry looking to move from analytics-ready to AI-ready data products. The key insight is that governance, data lineage, and business context must be embedded in the data layer before deploying agentic AI—not added as an afterthought.

Full transcript

[00:07] Good morning everyone. Thank you so much for coming over here. What an exciting week this has been with all the announcements and everything. So, before we get started, let me just quickly ask a question. In the last 12 to 18 months, how many of you have created some kind of an AI agent conversational analytics or something
[00:23] where you're trying to ask a question to get an answer? Okay, awesome. Many of you. And how many times have you seen the AI provide you a very nice answer which was completely wrong? Okay, awesome. So, this talk is for you.
[00:40] So, we're going to share our journey around when we were faced with a similar problem when a brand analytics team came to us and asked a very simple question, what are total new patient count for this particular brand? And how the AI really created a really nice sequel, created that answer, and
[00:58] only to realize that the answer was completely wrong is when we started thinking about what exactly is happening. And we realized that GenAI is not the problem, but the overall context that is needed for it to create the right answer is actually the problem. So, let's get into it.
[01:14] Hi everyone. My name is Anuya Iyer. Uh I am the lead for US commercial data services and solutions at Merck, the human health uh division. And my name is Nishesh, and I'm a partner at ZS, and we work together to solve a lot of the data analytics
[01:30] problem at Merck. One of the main objective that my team is trying to drive uh at Merck is having a digital uh first mindset and culture. That basically means simplifying our core data products and processes to be able to adapt uh to the uh
[01:49] GenAI processes and being able to deliver high-value impact. And that is what we are here to talk about today as a part of our presentation. That is where most of the organizations are heading head-on right now. That what it actually takes to make
[02:05] agentic AI work on enterprise scale. At Merck we've been on this journey for a substantial period of time now. We have a very very solid and robust data foundation that we're extremely proud of. And that foundation works extremely
[02:20] well for all of our analytics use cases. But when we got to a point where we wanted our agents to work with this data foundation, we realized that we're hitting a wall. And like Nishant said, the problem was not really that the agents were not right. The problem was that our data
[02:36] layer, the way the data ecosystem was being set up, was not ready to talk to the agents yet. And that is what we're going to talk about here. How did we begin our journey and how did we evolve and got to a place where we were able to
[02:52] have the agents talk to our data platforms the right way. But before that, I want to talk about why this is important to us. Merck's purpose is to be able to power the use of leading edge science to be able to save and improve patient lives.
[03:09] This just doesn't This is not just a tagline for us. This is kind of the foundation or something that we use when every data and technology decision is being made. Getting the AI right in commercial life
[03:26] sciences is so much more than just making a decision. It is about getting the right information, the right products to the right patients at the right time. And the stake is not abstract. The stake
[03:43] is clinical. And that's the reason why it is important for us to be able to have these agents work right and to be able to provide the right level of information through this agentic setup. Our mission is about improving life and AI agility is now a
[03:58] part of that mission. And for those who don't know about ZS, we are a global services firm and we work very closely with all the life sciences and health care companies to solve a lot of the complex data and AI problems ultimately with the aim of
[04:13] improving patient outcomes. Awesome. Now, let's talk about the tension that every leader in this room is feeling. The business is moving faster than ever. The the market dynamics is changing at a
[04:30] very very fast pace and the data strategy is flexing along with it equally fast. The brand and the marketing organizations demands are scaling up super fast. They want to be able to have the insights and answers to the questions available at
[04:45] fingertips in real time. But unfortunately, that is not how our core data systems are being structured today. Most of the times when marketing or brand lead comes up with a with a business scenario, the business insights and analytics lead would take some time
[05:01] to understand what the requirement is, go back, design, develop, go through a couple of iterations of testing before the product is delivered back to the marketing and brand teams. This is typically a cycle that takes a couple of weeks and by then the business context is completely changed and evolved.
[05:17] The AI era is demanding agility and for us to be able to stay up to date with that agility, we need to be able to move away from the operating models that we have set today that are built enough to be able to tolerate hand-offs and queues. We want
[05:34] to be able to define systems that are agile enough to be able to provide the answers in days if not minutes. And that's the problem that we were set to solve in Atmark. Now, let me introduce you to something that we are extremely passionate about.
[05:52] Um the the end-to-end connected data ecosystem journey that we started at Merck a couple of years back, we call it Data Genie. I I know it it has the word genie in it, but this was much before uh Databricks decided to call their tool sets genie, right? But, uh Data Genie is
[06:10] basically Merck's commercial data product ecosystem. The idea was very very straightforward. We had different commercial teams trying to build their own data pipelines, trying to devise their own business rules, trying trying to create their own transformations, and in the process they were we were
[06:25] creating a lot of redundancies in the systems. When we started on this journey of Data Genie, we wanted to build a platform which was scalable, which had a standard set of business rules and repositories, uh which was governed in the right way, which had security enabled, which had
[06:42] the right level of quality standards set in it. The way Data Genie was being set up was we organized all of the similar source agnostic data and put it in in respective domain setups. We've got 10 domain sets that were established right
[06:57] now. We are marching towards 12, but we've got patient patient genie, we've got sales genie, we've got market access, we've got customer, we've got activity. We've been able to create specialized domains by subject area as a part of our um Data Genie setup.
[07:14] We have also created curated last mile products which are very very specific to the end-user uh use case requirements where they're trying to solve a specific problem related to a particular brand or to a particular focused user group. We've got solutions which give answers
[07:31] from a precision marketing standpoint, promotional optimization. Uh we have solutions that cater to uh brand launch brand launch analytics, performance analytics, so on and so forth, right? So, DataGenie at this point is a place where we have extended
[07:47] scalability to a large extent where we provide a single version of truth for the commercial organization to access. There's high quality of data that the system is able to guarantee. We have built in the right lineage, the right governance, and the system just works
[08:03] fine as far as BI analytics, self-service analytics, data science model is are concerned. Um However, when we tried to point our agents to this platform we realized a reality that we were not
[08:19] really agent ready. Now, I want to be clear, right? In our journey to make DataGenie agent ready, we did not really have to start from scratch. The way the DataGenie environment was being built was based off of a medallion construct.
[08:35] We have the architecture that runs across four layers. We've got bronze, we've got silver, and then gold is split into gold L3 and L4. Um the bronze layer is where we talk to the external systems like IQVIA, Commodo, um
[08:50] the GP organizations to land and stage our information and to ingest the information in the in the right way. Silver is where we start standardizing, harmonizing the information, trying to bring it into a standard format uh where we start to uh create a common
[09:06] data model. As the information flows from silver to gold is where we start curating these data sets and and trying to make them very very focused to a particular subject area. Like I spoke in the previous slide the the DataGenie uh
[09:22] gold L3 is where curated domain data products sit, and DataGenie gold L4 is where curated solution data product uh pro- product product sits. Now, each of these uh domains are governed by certain set of business rules.
[09:37] Um there are uh there are a high level of quality expectations that is being uh mandated out of these data products. And these data products just work fine for any self-service analytics or for any BI dashboards or AI/ML model. This
[09:52] is a very very mature foundation. There's years of investment. There's There's years of institutional knowledge which and uh this knowledge is being built into how the business rules and how the data models are being set up. We have close to uh 500 plus uh data
[10:09] products and we have more than 1,000 users who access uh uh these systems today for their day-to-day uh analytics needs. Yep. So, as you can see, right? Like there's a strong data foundation which was already in place. But, what happens uh when we first
[10:25] pointed an agent to this data foundation? Very quickly we realized that there is not one but there were four different things that actually broke, right? And all of these four were connected but like separate things and none of those were related to the LLMs
[10:41] or the GenAI model that we were using, right? So, the first one was the low semantic depth. So, while we were creating all this uh data foundation, we had all the documentation. None of that documentation was available for the agents to be leveraged, right? So, it was not able to figure it out what
[10:57] exactly does a column like a TRX mean, right? It's in your head, it's not coded. Uh it doesn't know how to calculate rolling 3 months because there's formulas that KPI is not uh defined anywhere. So, that was the first one. The second one was the disjointed metadata, right? So, we had created all
[11:12] of these domains, but these domains were completely separate. They were not talking to each other. So, when you wanted to combine the two uh or multiple domains to create an answer, it was not able to do that. Then the third one was the outdated architecture. Not in a sense that it was still built on a date
[11:27] uh Databricks. It was using the latest and the greatest technology, but it was outdated in the sense that it was not for agentic use cases. So, we really had to thought about like how do we bring all these different layers together to really make it work for agentic AI and we'll see I will show how we went about that. And then the fourth
[11:43] one was the governance gaps, right? So, all the ARDs, all the data products that we had were well tested, well governed, but then as we started putting context on top of it, how do we make sure that the governance of the context is being taken, it is not becoming stale, and how do we go about it? So, these were the
[11:59] four key things that we saw uh that broke the moment we pointed an agent on top of Data Genie and that started our entire cycle around how do we solve this problem in a way that we can really scale every week, every month, every year so that we can continue to get the compounding effect.
[12:16] Right? So, before we go into the details, right? I just want to focus a few minutes and this is something that was also covered in the keynote. So, majority of the ecosystems, if you think about it, they were as of today created only for like humans to consume and probably BI
[12:32] dashboards and all of these analytic system to consume, right? And that is where the interesting thing happens, right? All of these are very forgiving system. You create a KPI, you showcase in the dashboard, but there are so many humans in the loop that even if there is some
[12:47] kind of a confusion, either an analyst will come in to fill in that confusion or the user has a lot of experience and they can infer the information which is not part of the dashboard mentioned explicitly. But that is exactly what is changing when we are including agents into the loop, right? So, in addition
[13:03] for humans to be the consumer, even agents needs to consume all of this information and agents are not as forgiving, right? They need to understand each and everything as is or they will make up stuff, right? So, as we start thinking about the future layer, then in addition to having
[13:19] your data foundation, we also saw that there are two two new layers that are needed. One is what we're calling as is context layer or the knowledge layer, which is which will make your data AI ready, machine readable, and all of those things. And then the second one is what we're calling as the intelligence layer. So, these are the set of agents
[13:35] and the capabilities that you need to build so that an external agent, while it's coming to your data ecosystem, they can understand, interpret, create the SQL or the code or whatever is needed to really get the answers. Right? So, that is basically the shift that we are seeing.
[13:50] So, let's take a look around how did we go about, right? Right? So, on the left-hand side is what all the journey was all about from a from the last 18 to 24 months, where we set up all the domains, we set up the data products, and we are calling that as ARDs, right? Analytical ready data sets.
[14:06] What we need is actually GenAI ready data sets, which we are calling as GRD, and that entire journey is basically what we're talking about right now, right? So, and overall, we can think about this as four key steps around what we need to do. So, the number one is adding the meaning
[14:23] and the standardization, right? So, you have all of your tables, you have all of your columns. So, the first one is adding that semantic information on top of it so that as an agent comes to your data, it knows what exactly this table is used for because you will have multiple tables, which will look all the
[14:38] same, but you will want to use it for specific purposes. And within that table, you will have a lot of columns, which you need to tell because you will you be using your standard naming convention, right? You will not have like descriptive names in your like tables by default, right? The
[14:53] second one is the context and the lineage. And in over here, what we mean by context is the business context, right? And this is the most important thing because this is often where, if you don't have this, GenAI will try to make up an answer for you, right? It will never say like, "Oh, I don't know the answer." It will be like, "Okay, I'm
[15:10] making an assumption, and I'll give you an answer." And then you will you need to figure out whether it's right or wrong. And in addition to that, the lineage is really important because you need to figure out how the tables are connected, when to go to which one, what are the relationships, and all of those things. And then the on the right hand side what
[15:25] we're talking about is doing all of this without governance is what will make this really, really bad when you go from your POCs to your pilot to actually scaling because your context needs to be up-to-date, otherwise you will get a stale answer. And then you also need to make sure that
[15:41] there is consistency across your domains even for the same kind of things so that it is always always picking up the right kind of thing. And on top of it you need to have the security applied so that agent knows what to access, when to access, what to answer, and what not to answer.
[15:56] Especially, right? And then the last one is the retrieval structure because if you just dump all the information we know, agents will continue to go around, it will take to consume a lot of tokens, and it will not have the most efficient part to retrieve that all the information and give you the answer that is needed.
[16:14] Right? So, when we started thinking about the context, the first time we just started defining it, but very soon we realized that there are six different types of things that we are really talking about, and we started on the journey on really putting all together,
[16:29] uh providing a structure so that when we are defining the context, we are sure about what we are what we have already defined, what is missing, as well as when the agent is coming in and trying to retrieve the information, it exactly knows how to traverse this information, right? And these are the six layers that
[16:46] we're talking about, and then you will see at the bottom is the strong foundation. So, you still need your ARDs, you still need to invest in building that strong data foundation, but then the first layer that you do is data context, right? So, this is all of that table and column information.
[17:08] Yeah. So, this is all of that information that you need to provide to to make sure that the agent can understand your entire environment. And the good thing is a lot of this can be AI generated as of today, right? So, you can really use some of the capabilities, even Data bricks points out of the box capabilities to provide like create
[17:24] curate the first version and you can work on fine-tuning that for your use case. The second one is where things starts get interesting, which is the domain and the system context. So, these are your KPI libraries, catalogs and all of those things where you need to basically say how exactly do you want to calculate XYZ
[17:41] things, how exactly it should be done for one thing versus the other thing. Right? And as we will see, the higher we go, the more specific starts becoming. So, Arnie will walk through the remaining three layers now. Thank thanks, Nishish. Um so, the three layers that Nishish basically spoke
[17:57] about is something that the tools can deliver to today, right? This is where most of the technical metadata enrichment around how your tables are being set up, what kind of column definitions exist, uh so on and so forth can be defined. But, as you go into the top three layers, uh the lay- layer
[18:12] four, five, and six, that is where we really need the uh SMEs to come in to be able to set the right context. Uh use case context is where we are very, very focused on use case based business rules, brand specific, uh
[18:28] security aspects, uh something which has to be defined very, very specifically to answer a use case uh focused question. So, the use case context is where uh our domain SMEs or solution SMEs come into play and then they work very, very closely with the marketing organizations
[18:44] to define the right level of business rules that have to be uh applied to answer a brand specific uh question. As we go into the enterprise context, this is where we really want the entire end-to-end data ecosystem to be able to talk to each other.
[19:00] Uh like we explained in the previous slides, we've got different domains, we've got different personas, and each of these domains and personas are being set up to be able to uh cater to different needs. Uh the enterprise context is when we kind of bring all of these domains and personas together. It is also something
[19:16] that uh, would make our uh, chief security officer or compliance officer happy because that this is where we kind of define what kind of information is available to what part of the organization and what is uh, what can be exposed to the external world.
[19:31] As we move into the memory and runtime context, this is the self-serving uh, loop where the system is continuously evolving. Not all of the questions that you would ask agent are going to come back with accurate answers. There is going to be a certain confidence that the end user would have
[19:48] to provide to the agent based on the answers that are being given. This is a learning loop where the feedback is being captured and based on the feedback, the SME comes in, creates a con- uh, creates additional context. It gets into a process of
[20:04] getting revalidated and and that is how uh, the entire six layers of context work with each other. Now, establishing this level of uh, knowledge system or knowledge layers is important for our agents to work because this is where most of the routing of
[20:19] where uh, the agent goes to get what information is being uh, decided. I also want you to focus on the right side of the screen where we have three key aspects that we want you to focus on. The first, you as an organization where you are in
[20:35] this uh, six-layer journey that we are talking about. Most of the organizations today, uh, like us, what we were a couple of uh, months back, are where our foundations are analytics-ready but not necessarily JNAI-ready. The real differentiator actually uh,
[20:52] sits between layers four to six where we have started defining business rules specific uh, context, uh, KPIs and metrics uh, definition we where we exactly tell the agents where to go to get what information. Also, the way this context is being
[21:07] defined, the context is defined at an enterprise scale, so the agent should be able to provide the same set of answers to different questions coming from different parts of the organization, right? So, you built it once, but you should be able to use it across various personas, uh different co-pilot uh
[21:22] co-pilots that you've set up uh in your organization, and to be able to answer different use cases the the way they are being set up. Do you want to walk through the example? Sure. So, I'm going to try and uh talk about a real-life scenario where we had like a before and after.
[21:39] Um so, like I said, right? Data Genie was a Medallion construct. We had a very very solid robust foundation where our analytics was just doing a perfect job. So, we decided to take it one step further, and we put a agent on top of it. We tried the agent, or we made the
[21:54] agent ask a question. The question was very very simple. Any brand and marketing organization lead might want uh to ask that question, right? How many total new patients did Winona as a brand get in the last rolling 3 months?
[22:10] What the agent basically did was it realized that oh, we are asking for total patients. So, the the the basic tendency was to go and find the sales table, look at the look at the TRX column, look at the 90 the last 90-day period, uh aggregate the information, throw out a result.
[22:27] Now, the scare the scary part in all of this was because it was agent-based result, there was a very high level of confidence score that the organization or the team had on it. We said, "Okay, agent have has returned this result. It must be right." But in reality, there were at least three things that the
[22:42] agent was not doing uh doing correctly. Uh Winona is a specialty brand, so the information of uh sales is not necessarily what comes from standard uh secondary data providers, but it comes from specialty pharmacies.
[22:57] Uh the the total new patient is not a metric that you you just randomly count your total patient. It is basically somebody who has a enrollment form created for the first time. Enrolling three month also has a very very specific definition depending on how
[23:12] that calculation works. Now, all of this was being set up as a part of the six layer architecture that we spoke about when we were defining the context layers. Uh fast forward, when we created all of these context layers and we ran this query one more time, this time the agent exactly knew where to go uh to fetch
[23:30] fetch what level of answers. And most of the answers that the agent was looking for was in layer number four, where the data context was getting defined. It exactly knew that this is a specialty product. It has to go to the specialty domain to get the answers to the information. The business rules were
[23:45] offer also defined in the data context layer. It exactly knew what rolling uh three months meant and also the calculation for new patients, right? Um this was all because our domain SMEs were able to bring in the knowledge that they had uh gathered as a part of the
[24:01] data journey and provide it in the in the respective context layer to be able to do the job right. Awesome. So, now we also wanted to share some of the strategic bets that we made as part of this journey, right? And because we
[24:17] feel that was the difference that was again a big differentiator why we were able to do this very very quickly, right? So, the biggest thing that we aligned on was what do we want to focus our effort on versus what do we want to like leverage, right? And in the age of GenAI, what we
[24:33] are saying is building software is very easy and it's fun. So, everybody is trying to build custom solutions, right? And the good thing about that is when you build that, you have a lot of progress that you can showcase because every two weeks you can have features that are being released and you can
[24:49] showcase like we are this is how we will deliver, right? But very soon what you'll realize is the business outcomes are getting dragged on by months and months and months because what you're building is actually a platform and not the business outcomes. So, at that point of time, we made a
[25:04] decision that we will start leveraging Databricks as an out-of-the-box capability and rely on whatever features they are providing knowing that it's an early part of the journey, it will continue to evolve. And but we said like we will want to focus our effort on the middle portion, which was the building
[25:20] the context. Because that is not something that anybody can provide. That is where the subject matter expertise come into picture. We really need to pull it all together. And that will that is as we saw in the example, that is what will create the final business outcome. Right? So, that is what we started
[25:36] doing. So, leveraging some of the features which were available at that time, which is Unity Catalog, Genie Spaces, as well as Agent Bricks, we started on this journey of curating that particular context for all the different use cases and enabling that on top of the Databricks platform.
[25:52] So, again, this is a little bit more detail slide we just wanted to share because this is on the left-hand side is a double-click on how we are defining the different layers of the context and how exactly that translate into the Databricks as a platform. And you will see we are using different things like
[26:09] Unity Catalog to define the data context layer, as well as we are using things like Genie Spaces to define all the different business context, the business rule, and all of those things. Right? So, this is like a high-level picture around how did we go about using the different kind of things.
[26:25] And given that this is a tech conference, nothing will be complete without a technical architecture diagram. So, this is how everything comes together, right? So, you have your basic data foundation on the bottom, right? Which is where you're building all the ARDs, all of those things. You have your data pipelines bringing the
[26:42] data, bronze, silver, gold layer. And then gold layer is where we started focusing on building this overall context layer. We thought about would how do we go about using Genie spaces and the best way we thought about was maybe we can treat that as subject matter experts. So we started building
[26:59] Genie spaces not to be super broad but broad enough that it can solve a lot of business questions but at the same time it can answer those questions very very efficiently and we are like not overwhelming it with like thousands of lines of context, right? So we started building multiple Genie spaces each for
[27:16] either a use case or a brand or a domain, right? Depending upon the complexity and the broad set of questions and then we started using an an orchestrator agent to really make sure that as the business question is coming in, the business user doesn't want doesn't need to worry about where
[27:31] to go to get the answer. The orchestrator agent can figure out which Genie spaces to go to and basically pull up the answer. Obviously as you can already see with all the announcements that we have seen, how this architecture will evolve. So Genie one will come in, we'll not have to create our own custom agents.
[27:48] They're also like a trading ontologies so we'll need we are super excited to see how the ontology offering can fit into this overall architecture and make it even more efficient and that is what we're saying that we don't need to continue to build custom solutions, right? The companies will create that.
[28:03] This will get better and it will become make our lives very very easy. The only thing that is not changing is nobody can create that context for you. So that is where you need to focus on and have a really good strategy around how to curate that context, how to maintain that context and how to really put it
[28:19] all together to make sure that you can start generating very very high confidence answer. So maybe let's see about like the impact that we were able to deliver through the solution. All right. Um So we we are we are still in the in the journey that Nichesh basically spoke
[28:35] about. This is just one of the many agent take AI use cases that we solve for but we wanted to be able to measure the output or the value story that this agentification process was able to create for us. At this point from an accuracy
[28:51] standpoint, we are at a place where we can confidently say for the for the use case that we solved for, we are at a 95% accuracy. Now, I want to make sure when we talk accuracy, what does it mean? We are not saying that 95 out of 95
[29:07] questions was what the agent was being able to answer correctly. We are saying that there was Out of all of the queries that were being asked to the agent, 95 of those queries came back with accurate answers. There was still that 5% where there was
[29:22] a little bit of ambiguity where we are not able to get the right level of insights, but that was what our process was able to capture as a feedback and we were able to bring in the human in the loop to be able to create that additional context and get this process to be to a better level of accuracy.
[29:38] The speed. Typically, the the way a domain setup would need, we would need to be able to create all of these ARDs and GRDs in a time frame that spans between months. We are at a case we are at a place now where we prefer configuration
[29:56] over code. We have we have been able to activate our domains within within weeks that previously took months for us to generate. Also, if you go back to where we were talking about the AI era demands agility and we were
[30:12] talking about the marketing organization and the brand organization wanting answers to the questions in in days, we were able to solve it from there for them as well. What something took 2 weeks to be able to create the right data product and the right analytics
[30:27] answer for them, now takes days. We have been able to bring bring down our time to insights by by a substantial period of time and we've also been able to build on the aspect of scalability where each of the use cases now inherits the
[30:43] basic foundation that has been set and we are able to configure new TAs based on all of the brand launches that we have planned in in front of us and we're able to use a multi-agent orchestration system that is able to drive the scalability to a
[30:58] large extent. Awesome. And again, like cannot underscore the importance of the platform itself. So, as you will see, right? Even with the 95% accuracy, the key was like all the out-of-the-box features that we were getting like like guardrails, benchmarking, and all of
[31:14] those things. That really made it super easy for us to try out new questions, making sure that over a period of time as the data gets refreshed, we are still getting the right answers. And as well as like we are not we don't have to like build all of these together as different
[31:29] features, right? So, that is where it really helped us in accelerating the time. Even the 50% that we are showcasing, this is the average, right? So, for a lot of the complex things that is where we saw time decreasing from days and weeks to like few hours on how
[31:45] the agent AI was able to bring out the question. I think the the biggest win right according to me is where the system went from providing confident wrong results to near zero hallucinations, right? The way the system was being built, I think we were able to really drive the
[32:00] confidence of the organization to our data ecosystem when they were trying to use conversational AI substantially. So, that according to me is a is a big win that we should be able to register. Awesome. So, before we go, we wanted us to showcase a maturity model that came to
[32:16] our mind as we were building this out. So, the way we were thinking about the six layer, we thought like maybe that's a good way of showcasing how as you continue to build each layer one at a time, how your accuracy will continue to evolve, right? And if
[32:33] you would have read the Anthropic's recent article as well, they also mentioned the same thing that if you will directly use the LLMs that they are providing, the accuracy will be close to 25 to 30% and it is really the overall enterprise context which will help you go from 25 30 to 90 plus and still the
[32:48] best in class is still close to 95 96% and I feel like the question is not how do you will get 100%? The question is for the remaining 5% what exactly does your agent provide you as the answer and how easy is it for you to figure out whether that's a right
[33:05] answer or the wrong answer, right? So, that this is what we wanted to share that maybe you can do a quick assessment just to see where you are and this this was our journey like as we added each layer, we started seeing the accuracy go up and like you can see like what exactly will it take for you to go from
[33:20] one level to another level and get the results that we were also seeing for ourselves. Yeah. So, thank you so much for being here and and listening to our session. I think at the end of the presentation, this is the question that I would want to leave you all with. Where would your organization
[33:36] land on this ladder today? And what would it take to take your organization to the next level? What we learned on our journey was that the highest leverage investment that we made on our connected data ecosystem was when we invested in setting up the right L3
[33:53] domain and system context. That was where the accuracy curves bends and we were able to see substantial improvement in the way our agents were reacting to the questions that were being asked of them. So, I I think the layer three, the data
[34:09] context layer is where the maximum the maximum work when it comes to agentic AI setup to be able to drive that zero hallucination and and and fairly robust accuracy resides.
[34:25] Awesome. Thank you so much for coming. We'll be around here if anybody has any questions, more than happy to have a conversation.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.