How to Solve the Context Gap: Engineering Reliable AI Agents

Summary

The context window—what an agent sees, when it sees it, and in what format—is the primary determinant of whether an LLM reasons correctly or hallucinates, making context engineering more critical than model selection alone.
Practical techniques include tool naming and namespacing to reduce attention competition, optimizing tool call results to cut hallucination vectors, and using multi-agent architectures to provide fresh context windows for complex workflows.
Lakebase provides managed PostgreSQL on the Databricks Data and AI platform for persistent agent state and long-term memory, while Agent Bricks enables production feedback loops that let agents improve from real interactions over time.

How to Solve the Context Gap: Engineering Reliable AI Agents

The biggest hurdle to production AI isn't the model — it's the context window. What an agent sees, when, and in what format determines whether it reasons correctly or hallucinates. Most teams stitch together RAG, tools, and state management ad hoc, then struggle to evaluate whether their context decisions actually work. this video is a deep dive into context engineering on Databricks: the systematic practice of designing what agents see at every step. You'll learn how to configure RAG systems that go beyond basic retrieval, use Model Context Protocol (MCP) for secure tool and API integration, and manage long-term memory across complex agent workflows. You'll leave with a playbook for building measurement pipelines that empirically evaluate your context decisions — and the patterns to ship agents that stay reliable when users, data, and tasks evolve.
Talk By: Badr Slimani, Sr. AI Engineer, AI FDE, Databricks ; Pradeep Ravi, Manager - AI FDE , Databricks ;

Chapters

FAQs

Why is the context window the hardest part of building production AI agents?

LLMs can only work with a fixed amount of information at once, and simply filling a longer context window does not guarantee better performance—as demonstrated by needle-in-a-haystack benchmarks. What the model sees, when it sees it, and the format of that information determines whether it reasons correctly or hallucinates, making context engineering a critical discipline.

How should tools be designed to minimize agent hallucination?

This video recommends careful tool naming, namespacing, and description writing to reduce attention competition between tools, as well as optimizing tool call results to remove noise that can introduce hallucination vectors. These patterns apply across all model families and agent frameworks.

When should a multi-agent architecture be used instead of a single agent?

Multi-agent architectures are recommended when a task is too complex for a single context window or when subtasks benefit from isolated, fresh context that is not polluted by earlier conversation history. This video presents multi-agent patterns as a key technique for managing context at scale in complex enterprise workflows.

How does Lakebase support long-term agent memory on Databricks?

Lakebase provides a managed PostgreSQL layer within the Databricks Data and AI platform that agents can use to persist state, store semantic and episodic memory, and maintain context across long-running workflows. This video also covers Agent Bricks feedback loops that enable agents to learn from production interactions over time.

Full transcript

[00:08] Right now, we're going to turn the stage over to two forward deployment engineers. But before I do that, I just I want to make a mention and and this is, you know, kind of leading into our conversation here. What really makes bringing AI into production challenging
[00:20] is the context window. What the model sees, when it sees it, and what format sees it sees it in. That's really the toughest part. And so these two gentlemen are going to help us close that gap. We have Praep Ravi and he's
[00:34] actually a manager of the AI FDE team and with him is B Slammani and they've worked across multiple industries. They're going to bring all of that incredible experience to the stage.
[00:46] They've worked in media, entertainment, global energy and workflow solutions. So please join me in welcoming them to the stage as they walk us through how to solve the context gap. Thank you.
[00:58] >>
[01:11] important topic of uh context management and why uh context is so important in uh designing and implementing reliable uh AI solutions.
[01:23] Quick introduction first. What do we mean by context? So uh large language models can only work with a set amount of text or tokens at uh a set time. Uh that's what we call the context window.
[01:36] It increased with time. So we started uh in the early uh 2020s with the context windows of a few thousand tokens to now regularly working with the models that support up to 1 million and even 10
[01:50] million uh uh long context windows. The question though is um does fill in longer and longer context windows mean um better performance? We will see that
[02:03] it's not necessarily the case and it will be a recurring uh talking point towards uh throughout this uh presentation. So 40 minutes is quite short to cover such a big topic. Uh what we decided to
[02:15] do is to condense uh some uh tips some important tips and best practices that u deal with the context windows. We structured it and grouped it into a couple of the main categories that of uh
[02:28] type of context that fill the agents that we are working with today from instructions uh tools tools definitions and returns and and these apply across all models all model families and all
[02:40] frameworks where you are implementing your agents. We will then move to uh more specifics around data and AI workloads and building uh agent solutions on uh data bricks.
[02:52] So we mentioned earlier that filling in um bigger context windows might not necessarily translate uh to uh better performance. One of the earliest tests of this is called needle in a haststack.
[03:04] And as you see on the right an illustration of this um it is a simple retrieval task where we are checking whether the model can
[03:16] buried in the middle of longer and longer uh context windows. However, with time models became increasingly good at this and were able to solve uh this uh issue with good performance.
[03:30] But when you start uh adding more and more complexity, so um in the form of ambiguous question pairs or distractors, we noticed as you see on the chart that performance uh starts decreasing um
[03:44] quite drastically and this applies across all model families. Um and also for state-of-the-art models, the rate of degradation uh might be uh different and state-of-the-art models might be more
[03:57] forgiven. But the fund fundamental uh principle behind the uh cell applies performance is not uh uniform across the full uh context length. And so we will
[04:09] be covering uh in this uh presentation how uh we can optimize what we provide to the context window. We will do this in u different categories uh of what uh
[04:21] slowly fills in the context window. And so let's deep dive into that to start with. Initially we started with prompt engineering and in this case we our main lever for optimizing performance that we
[04:34] iterated over is the system prompt. So we had a combination of a system prompt and user input and we u studied how uh what we provide and how we provide it across uh these two uh helped steer the
[04:48] model towards the direction that we were interested in. Now however in uh multi-turn conversation and tool equipped agents we have the system instruction stills still but we also have a combination of uh tool and MCP
[05:02] definitions uh tool call results uh memory if the agent has access to external memory and the everinccreasing conversation history with uh all of uh
[05:14] these uh different categories of text that fill our context window. And so we will use these different categories throughout the next slides to illustrate a couple of uh tips and best practices on uh how to optimize what we feed to
[05:28] our context window. Uh to start with the instructions. So if we keep that same scale from the uh slide before uh to the left we could be too broad. So uh we are being vague and
[05:42] uh provide minimal context to the model. Assume that the model has enough context. uh we underspecify behavior and to the other extreme being too specific uh would mean that uh we hardcode some
[05:54] uh if else uh logic uh we um think of all the edge cases uh that come to our mind and we uh provide them to the model but this uh means that the model is not
[06:06] able to to generalize um being just right is somewhere sits somewhere in the middle where we are specific enough uh to guide the decision isions and get um
[06:18] the model to the direction that we are interested in but allow it to uh reason at the same time. We instead of providing all the edge cases we can think of instead provide a few curated u
[06:32] canonical examples and um as usual and this will be a recurrent theme throughout the the discussion as well. We start minimal and then measure and try to add uh incrementally. One thing that can help here is uh uh our tools
[06:46] like MLFlow prompt optimization uh with the JPA uh algorithm. The second category here that we mentioned earlier is um tool uh and MCP
[06:58] uh definition. So first of all why do we use uh tools um is to offload any deterministic and uh exact uh work uh to code instead of relying on the model
[07:11] itself. So if for example you need to perform a mathematical operation, it's better to use a calculation tool instead of assuming that the model is able to perform to do such operations. It also
[07:23] allows to extend access to life context and external context for example web search tool or reading from an external uh database. And an MCP is just a way to standardize these tools and reuse them
[07:35] across different agents or uh different use cases. And so how to do this um in a way that improves the performance and reliability of our systems. Um a good thing to do and to think about are um
[07:50] the names and the descriptions of our tools as well as their schemas in their definitions because this is what will help the model decide what to use and uh when to use it. um and prefer we will
[08:02] see this in the next slide. A few well-designed and completely uh namespaced and separate uh tools over closely defined tools that could compete for the attention uh of the model and
[08:15] return which will be our next category is around what should we return from uh the tool result back to our context window. So as I mentioned here, this is a very
[08:27] quick example of um being ambiguous with tool definitions. So instead of having as you see on the top three different tools, one called query, one get data, one run that do something that is very
[08:40] closely related, having a single uh condensated tool with a good name, a good description and a clearly defined input schema would allow the model to very quickly uh and better decide to use
[08:53] this tool over um relying on the uh in the previous schema uh in the previous example um which tool uh call the the next category is around tool
[09:06] call results. So um tool results are a big uh hallucination and error vector for uh uh agentic solution and this is even more the case in multi- aent
[09:19] architecture. So I referenced a research paper there called from spark to fire. Um imagine you're in turn 10 of a conversation and you have called uh tools earlier that provided some results. It is very difficult for the
[09:32] system to differentiate what are actual if you have inaccuracies in some tool results whether these are inaccuracies or are now grounded in u in truth. So the research paper introduces an
[09:45] interesting uh graph based approach to help solve this problem. But without going um uh to these depths, simply making the tool results if you uh are
[09:57] defining them informative and actionable for the model, typing them and validating them before bringing them back to the context window of the model can already make a big impact on the reliability of your agent. So quick
[10:11] example of this here on the left we see um the row output from a tool call uh that we provide all almost 200,000 tokens of it back into the context window or if it errored out with a
[10:24] cryptic error message. Is it really needed? Um, back into the context window versus uh reformatting uh everything summarizing the results. Uh if it's an error uh for example an imprecise uh
[10:38] column name, maybe it's better to simply provide a hint to the model uh that it should retry with the corrected uh tool name.
[10:53] Um an interesting um concept here uh is progressive disclosure. uh this helps with uh the conversation history that as we mentioned earlier in multi-turn conversation have has a tendency to grow very quickly. So um one way to uh
[11:05] decrease the amount of context that we load uh into the context window is progressive disclosure. One of the con concepts that uses this is skills uh that you're likely familiar with under the skills.md format. They are not only
[11:19] markdown files with instructions. They can also be uh folders and scripts and uh the token uh amount that they contain can grow uh quite fast. However, they they are loaded dynamically and this is
[11:32] the progressive disclosure um concept. We are able to discover dozens of skills at the cost of fully loading a single activated one. we only read the front
[11:44] matter of the skill and if the model decides it's needed for the task it's working on then we fully activate the skill somewhat similarly uh when we're working with tools for example uh we
[11:56] could use a tool search tool um that allows the model to uh check uh across all the tools that we provide to it which ones are most relevant to what the model is working on instead of loading
[12:10] uh everything at the same time and we will see an example of this uh in a demo shortly. And uh again very similarly just to show that it applies to multiple uh objects. So if you're working with Genie and we need to decide which Genie
[12:22] space to load then uh behind the scenes uh Genie space search uh allows us to first uh decide uh which uh space makes most sense for loading instead of loading everything uh at the same time.
[12:36] This is an illustration of this. Um uh first level we only load the front matter can be as low as 70 tokens. Um and so we can load multiple skills at the cost uh of a single fully activated
[12:50] one as you see uh down uh there which can represent savings of uh thousands of tokens.
[13:03] Another interesting uh concept here uh around saving what goes to the conversation history and also optimizing uh tool call results is what we call programmable scratchpad and you might have uh heard of it called programmatic tool calling by anthropic for example.
[13:16] Um the concept here is if the model has access to multiple tools, we need to do the tool call, serialize the text, send the result back, um call the next tool,
[13:29] etc. If the model has access to a Python kernel where it can combine these tools, uh and format the the results and only output at the end after validation,
[13:41] etc., what we need to the model. Then we save back and forth and serialization to to text. And this is what the research paper from our uh internal research team
[13:53] um um that I referenced here uh calls mex which is part of the code as action uh family and that I can illustrate in the next slide where as I mentioned so you see on the left uh four uh tools
[14:07] that the model needs to call. Um in the classic scenario you would call the first tool. If they are sequential it's uh also challenging. You need to wait uh for the results of the first tool and then call the second one and with all
[14:20] the serialization uh costs versus to the right you provide the Python kernel. Um and we decide how to um run uh these tools with uh an await or running them
[14:33] in sequ sequentially and then combine the results uh validate uh type etc and then after bring back uh the output to the uh context window. This was shown by
[14:45] the research team to increase uh quite a bit the results uh for our models as you see here uh whether they are frontier models or um open- source models. So
[14:58] with uh um increases uh twofold sometimes in uh accuracy and a decrease in uh the the cost. And um lastly around the conversation
[15:10] history itself as it builds up we're faced with the question of what should stay. Um if as we discussed earlier we have uh 10 turns 15 turns um it does it
[15:22] really make sense to keep uh row tool dumps from much earlier uh turns and uh some redundant retries? Probably not. At the same time, some decisions and implementation details
[15:36] even if they happened uh earlier are um very likely needed in later conversation terms. One concept that helps here in this scenario as well is compaction. You
[15:48] might have seen it in some of the coding tools that you have used which consists in when we start nearing the uh context limit to uh summarize it and uh re
[16:00] reinitialize it. However, this is not a magic solution. It relies on an algorithm. So sometimes it's worth it if you know uh that some uh decisions and implementation details are important to manually reinclude them in uh a fresh uh
[16:14] context window. Um, another approach for uh fresh and isolated and distilled um context windows is simply multi- aent architectures because in this case you
[16:27] could have completely separate namespaced agents with access to their own tools or sub agents that you start fresh with an empty uh context window that you then uh provide what you need
[16:39] uh to it. It also adds parallelisms parallelism and speed. Um but as usual there is a trade-off and in this case it is uh by u added complexities and uh
[16:51] boundaries so only needed if uh without it uh we faced some limitations on the uh single agent architecture. So as I mentioned um I will show a very
[17:05] quick demo here. Um there are extensive benchmarks and studies some of which we um referenced in the presentation. What we thought we'd do here is just a quick way to uh visually demonstrate some of
[17:18] these concepts in a in an interface that we're familiar with. And so that's what I will be showing here.
[17:35] So the scenario is uh we're working with an agent uh that we equip with tools. Uh let's say it's around customer success. So we are uh looking we have access to tools around getting uh customer
[17:47] information subscription details updating cancelling etc. And initially it's a few dozens of tools maybe a couple of MCPS and as our tasks uh start growing and the complexity starts
[18:00] increasing we are faced with a decision of how to provide all of this context uh to the model. So a first approach simply providing the entire catalog of all the tools that we have access to and all the MCPS and to demonstrate one of the
[18:14] concepts we discussed earlier. Uh second approach we embed uh the description name description uh of our tools and we only equip our agent with a single tool
[18:27] that initially that allows it to uh check which tools are most relevant for the task that it's working on.
[18:41] We start with the basics of uh every agent that you are working with which is the agent loop. Uh what this uh consists in is so you have a message uh array that contains this all the system and
[18:53] user messages that uh accumulate in your conversation and then the agent loop that also might have seen uh with simply a loop that goes uh through um that loops and
[19:08] chats with the model providing the tools that it has access to and getting the results whether it is the final text and uh the conversation uh is done or a
[19:21] decision to call uh one of the tools that the model has access to. So this is the agent loop uh that is uh the starting point of all agentic uh implementations and what we do is as I
[19:34] mentioned earlier is we start with um these dozens of tools that we have access to
[19:48] and we compare with a tool search approach. So we notice here that accuracy defined by whether the model is able to call the correct tool or not is good for both scenarios. Average tool calls uh is uh we we notice that we have
[20:02] one additional tool call for tool search which makes sense because we need this additional tool call to check uh which tools we need to call afterwards. However, we already see a big impact on
[20:14] um uh token use. So, uh since we're not loading the full catalog of tools that we need um so the tool usage will be much lower for the same performance using uh tool search. And now what
[20:27] happens if we push it uh further? So we keep increasing complexity of what we need to do and providing more and more tools and we get closer to the uh
[20:39] context uh the limits of the context window let's say uh hundreds of of tools that we equip our model with. We notice that
[20:51] so for our tool search approach the accuracy remains uh uh similar to before the average tokens uh are contained. Uh and when we compare it with
[21:05] the regular approach um so first of all the latency uh is uh has doubled the token count uh is uh much higher than before but more importantly we start
[21:17] seeing this decrease in uh performance of the approach um which uh demonstrates uh this um the fact that the context um that the performance is not um uh
[21:30] constant when we keep increasing what we feed into the context uh window.
[21:43] And so to summarize, we have seen in this simple uh quick uh demonstration two approaches um when working with the same exact underlying model where the one where we are careful about what we feed into the context window uh able to
[21:57] uh decrease cost, talk and use latency while uh getting um better performance uh especially for uh complex uh tasks and very long uh contexts. Um and in
[22:11] general what this um uh demonstrates uh is yeah we need to um think about what we feed uh the context uh window and nowadays with more and more uh powerful
[22:24] uh models coming on a regular basis uh we notice that for enterprise tasks and you can uh reference uh office QA which is a benchmark that is focusing on these types of tasks the model is no longer
[22:37] the bottleneck. the actual bottleneck is the harness around the model and how we manage uh the context uh when we're working with these models and that is why also all the announcements that you
[22:50] will be hearing around meta harness and the genie ontology are uh so exciting. So thank you very much. I will over to you brother. Thank you brother.
[23:15] context gap. I would like to start with how we can scale it. So let's start with a thought experiment. Let's imagine human without memory. What would happen?
[23:27] It's going to affect us in every decision making. It's going to affect us with respect to the reasoning or every decision whatever we are going to make is going to be from the scratch. And would that be a solid call? I don't
[23:40] think so because we don't have anything in our memory or we don't have any rules saved in a memory to think whether we are making the right decisions. So that's the beauty of memory, right? If
[23:52] it is gone, we can't get a solid call. That is applicable to agents as well. There are two things which we have to be very careful when we are designing the agentic solution. First is planning.
[24:04] Dumping a very big prompt and asking agent to come up with a solution is one approach. But with that scale, no allin needed output the context moistening and everything is can happen with respect to that approach. How do we do that? How do
[24:18] we tackle that? We have to plan break the big agentic system as a atomic level a code level function level so that we we are trying to decompose the complex task into a
[24:31] finer grain number of steps and in every step we have a validation gate to ensure we are building a deterministic system instead of throwing a big prompt. Just to give you an example let's talk about
[24:44] uh research agent. So we can easily go just provide a big prompt ask uh to come up with a very novel topic and do a research and give a very heavy prompt.
[24:56] The outcome might not be novel it will be an allucinated output. But how we can actually achieve the same thing by effective planning which is break down everything into a smaller steps pl put
[25:08] put that steps very much logged on the plan MD and have a validation gates. validation gate system like we are trying to evaluate every step and ensuring only one step is solved. We are moving on to the next step. In a way we
[25:20] are saving context here at the same time we are breaking down the bigger problem into the smaller problem. Whenever we are addressing the contract gap how we can scale it right we are not designing the solution anymore for the one user.
[25:32] We are designing the solution for the n number of users based on the use case what we are determining. It could be internal or external customerf facing but it has to scale. It has to understand the pattern of different user understanding. How do we do that? That's
[25:44] where we need to externalize of the context window. So there are different types of memory. I'll again go back to the same human example. When we think everyday events, whatever reactions, whatever we are doing, it is being
[25:57] recorded in our episodic memory and it is timestampsed and we know when we are trying to take a decision those episodic memory come as an example to us and we are taking with the help of the episodic memory like every instance every day
[26:10] whatever we are doing right and with respect to the semantic is nothing but outside of the 10 episodic memory what I had what is something which I learned what is something the rule which I
[26:22] created That is what semantic memory is about. Agent need both. I have to take if we have to take a decision. I need what rule I need to apply and what is the example. So example comes from the episodic and the rule comes from the
[26:35] semantic. So that is where the two types of memory we have episodic and semantic. And when it comes to uh personal this is organizational which is very important I really don't not I
[26:47] don't want to disclose my salary to the entire organization but if you're building a wrong agent that can happen so that's where the unity catalog comes into the picture in terms of managing how we can have a very fine grain level
[26:59] of access control user data remains within user organizational data is very generalized and it is applicable for the entire organization.
[27:12] So this is a kind of an experiment. So there is a lightweight red framework meline has been run to uh build this uh charts. On the left what we can see is we took 10 genie spaces which is different into domains. We scaled the
[27:25] memory incrementally and uh the the baseline what we see here is 5 percentage which is human curated baseline just by providing the instructions. This baseline is something
[27:37] we have approached we we have achieved. So if you could see here iterately when we scale the memory when we add more memory we can see the performance is reaching till 70%age. So more the memory, more the reasoning
[27:50] capacity, more the intelligence. On on the right, what we see is the number of reasoning steps it take it took for a model to perform from 20 to five which is we are not heavily
[28:05] depending on the model anymore. When the reasoning steps are coming down which means it gives me a leverage. I can play with any model at some point of time. I can go to the cheaper model or I can actually swap different models and play around.
[28:22] So how we can scale? We talk about the memory but how we can scale that's where linkbase comes into the picture. Uh when we are building a stateful agent it has to be pretty fast because whatever application we are developing is either userfacing or enterprise level it has to
[28:35] scale for the n number of users. So it has to be very fast. So how this is achieved by lakebased lacebase is a managed postgress offering by datab bricks. And how it works is it is un the the conventional postgress is very
[28:48] different from the lake base. The storage is actually decoupled from the uh compute so that you'll be able to perform uh faster operations in terms of faster instant recovery or scale faster.
[29:02] Everything is possible because this architecture is very unique. So that we were able to achieve 5x lower right latency times and not necessarily all the agents are going to work for 24
[29:14] hours. Agents also need some rest and at times if it has to take rest. So when it is on the off when we are not using it has to scale to zero. So if you're scaling to zero we are also handling the cost problem here. So at the same time
[29:28] someone immediately comes in ask a question agent has to scale up. So that that is being taken care by the scale to zero feature and it context is not just a problem only for the agentic solution. It is
[29:42] also a problem for a classical solution. How do we address again the same like ways the feature serving endpoints can provide uh the context to the classical level solutions like if you have a use case like recommendation system or fraud
[29:54] deduction or personalization your feature serving endpoint can actually serve the context in the real time evaluation we talked uh very briefly about evaluation in different slides but
[30:06] I wanted to just spend a moment here in terms of breaking down what is evaluation on a high level right So how we can evaluation should not be on a component
[30:19] or or in a system level. It has to be on a component level. It has to be on every atomic action level. So it has to be on every function level what we are writing. So how do we do that? We should
[30:32] instrument the code have a solid ML traces in terms of every function what it is doing. so that I can pinpoint when I have a problem with my agent in a system. I'm not looking the problem only
[30:44] on the system level. I'm also looking the problem on function level on a very finer grain level. I know exactly where the problem is because I have the I have instrumented the code with MLflow tracers. So I have a solid observability
[30:57] to figure out where exactly the problem is. So I know whether it is a problem because of the tool calling or is it a memory or is it because of the multiple components what explained? So that is something we definitely have to do in
[31:09] terms of plan. We have to plan the validation gate on every function level and instrumentation is required. So instrumentation can be done through the MLflow traces. LMS as a judge is a go-to solution for
[31:22] any of our validation approaches. But how we have to really teach how to measure with a solid rubrics. So we have to help LM to quantify we should align the LLM for a specific use case or a
[31:35] specific validation and we also have should have a solid CI/CD approach the DevOps in terms of whenever we change even if it is a small change on a prompt level we have to run a
[31:47] complete regression suit to figure out is there any kind of a regression impact integration impact because of the small change. So we need a golden truth data for that. So but it helps us to understand the complete integrity of the
[31:59] whole en whole system and the finally length accuracy parto view is about at times if there is a lengthy response is generated by your agentic system it can trick the LMS
[32:12] judge to say this is the valid response how do we actually figure that out length of accuracy parit one such approach which can help you to figure that out
[32:26] so we talked a brief about different optimization technique etc. So but do we really need to take care of that or we have to just build the agent from the scratch every time? Datab bricks managed
[32:39] services like agent bricks knowledge assistant is one such uh solution which is basically go to solution for building your rag agent where context engineering or the continuous learning is already happening in build where we don't need
[32:52] to worry about it. All we need to do is as a human users we have to provide a proper instructions and just configure it. So TAOO is one such approach which where we we could see that the automatic
[33:04] optimization is happening. Basically there is a reinforcement learning method is happening for every response generation. There is the reward mechanism and different kind of a judges judging the response and there is a
[33:17] feedback loop which helps you continuously learn and this is one of the internal of agent bricks and the second thing is how I can learn from agent right now the contacts which is available in the form of document and
[33:29] everything we can replace it but what do I do with respect to the context which is available in everyone individual mind so that has to be actually put into the system that's where we have to interact with the agent Let let me give you an
[33:41] example. You have a angel brick solution chat application. So we are asking the chat application to generate a query and when we ask to generate a query it it responded but the theme feels as anme we know this is not the SQL I want I the
[33:54] SQL what I wanted is something has to be compatible with data brick SQL. So we are giving a feedback can you generate a compatible SQL with data brick SQL. Then it generates the correct version. So the agent is learning it is building its own
[34:07] episodic memory and semantic memory for us. We can see from the graph there are different agents which we attempted and I just wanted to focus on the last two bar where because of agent bricks continuous planning we can see agent
[34:20] bricks defeats its own performance like 77%age and eventually it goes to the 80 percentage. So we talked about agent bricks which is more of a rag agent. The other kind of
[34:33] agent which we always wanted to do was uh text to SQL. So genie space is not anymore only for text to SQL. We we heard about a lot about ontology. So I just wanted to start with ontology which
[34:45] is not in these points. But how do I actually learn about anything and every asset and every object which is there in my workspace and build a context on top of it. That is what ontology is about.
[34:59] I'm going uh it is happening automatically for us. So going to every workspace set learning what is this asset is about with the help of business semantics which is available in the UD
[35:12] catalog layer and I'm understanding the business context and I'm figuring out building a knowledge graph on top of it indexing it providing that knowledge graph as an input to genace that is what ontology is about so it eliminates the
[35:26] schema blindness we are not uh so the agent is well aware about what it is dealing with well aware about what is the schema? We don't need to teach it and genie we and whatever we are doing
[35:38] is a curation process as upstream when we're building that ontology layer. So that when we are providing it we are no more building a system which is guessing we are building a system which is
[35:50] deterministic. When a system is guessing, it can consume more context. But when when we building a system which is deterministic where it is going to help us to actually execute the formula, it is going to
[36:03] provide the exact outcome. The accuracy is going to increase the cost is going to reduce and to build a system deterministic system there are multiple features what uh Genie spaces is having. For example, business metrics we are defining our own
[36:16] formulas where it's just a formula. We are not guessing any creating any formula on the fly. It it has a whole dimension where the dimension is applied during the runtime and we are doing the data modeling inside the genie to help
[36:30] genies to understand what is the relation between this table and other table. So those are those are the some of the approaches which we are helping genie to teach the deterministic system and another one is a trusted assert. So
[36:44] if there is a pattern of always some 50 questions we know these are kind of the questions or the SQL questions which user has to address why not create them as a trusted asset so that instead of
[36:56] genie recreating that it would use it as an example and provide the uh create the response for us instead of recreating the complete context. Again it goes back to the same topic of how episodic and semantic memory help into us like
[37:09] basically we are creating a semantic memory with the help of trusted assets. So we did everything but are we good to go to production? So if you're going to go to production we need to be have a
[37:22] very solid application where the mistakes are not going to happen. How do we do that? Guardrail comes into the picture. some content maybe during the develop we we couldn't figure it out but we wanted to take a control over it so
[37:35] we try to redact it uh we are trying to redact the content and ensuring we are not allowing the PI content and if you wanted to actually build a custom policy it is just a matter of coming up with a
[37:47] prompt and determining the custom policies always there will be one developer who might create an agent which can be in the category of runaway agent which can consume more of a cost how do I control it I I'm I'm more
[37:59] concerned about my team using how much of uh productivity. At the same time, I also want to be careful about building a budget aware system. So that's why setting the QPM, TPM and uh having a budget. For example, I want to set only
[38:12] $500 is allowed for the old team to be actually come up with this productivity approach. So we can set that budget uh with Unity AI gateway. And finally coding agent is something we all use but
[38:24] do we really track how much of token we are consuming and is that are we really being productive with respect to that so we have to always compare productivity with versus tokens what we consume every
[38:38] token is a deal every token is going to be costly right so that's where integration with unity gateway is going to help us to figure that out finally we talked about multiple things
[38:50] the core optimization technique what brother was explaining and the product level understandings where context engineering is already done for you where you don't need to build everything from the scratch but how do I sequence
[39:03] my approach what is something I should do instead of building solution from the scratch the custom agent from the scratch please start with the built-in options please start with agent bricks genie AI functions and there are a lot
[39:16] of solution accelerators you you could start with that that would be the best play to start because most of these solutions will be able to address it within that uh built-in options. But if in case you would like to build a custom agent then start start prototyping using
[39:30] AI playground. So you are you can actually have your multiple models you can provide your prompts add the tools add MCPS internal external managed and if you are convinced this is what you
[39:42] want you can export that as a code and then deploy it in the apps. That's how it we can develop. do not start coding it. Start playing with the AI playground and then eventually iterate it and but already you might be having an agent in
[39:55] your own platform at this point of time. Maybe if that is a case you would like to deploy and test how it works in data bricks. Uh that is possible using deploying the existing agent on the app. All we have to build is just interface
[40:07] it. And finally uh all we had talked about is addressing the context gap. On a similar line, databris has launched a very great course on context engineering. Uh, please enroll for that.
[40:20] Thank you so much.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.