Multi-Agent Architecture at Scale: AstraZeneca's Agent Bricks Strategy

Summary

AstraZeneca scaled from a 5-agent proof-of-concept to 20+ production agents using Agent Bricks on the Databricks Data and AI platform, solving three core problems: trust through on-behalf-of authentication, adoption through Vega-Lite visualizations, and scale through a continuous evaluation loop called MaxGenie.
The federated architecture routes queries through a centralized multi-agent supervisor to business-owned Genie spaces for structured SQL queries and a Knowledge Assistant for unstructured documents, with Unity Catalog and Entra ID integration enforcing permission boundaries automatically at every layer.
MaxGenie, an automated statistical evaluation loop, increased Genie space accuracy five-fold in 12 minutes by continuously optimizing the SQL generation configurations used by business-owned domain agents.

Multi-Agent Architecture at Scale: AstraZeneca's Agent Bricks Strategy

AstraZeneca scaled agents from a 5-agent proof-of-concept to 20+ agents in production using Agent Bricks on Databricks. The challenge was solving the tension between business-owned agents and enterprise governance. AstraZeneca addressed three core problems: trust (ensuring users see only their authorized data through on-behalf-of authentication), adoption (delivering rich visualizations using Vega-Lite specs that travel through APIs), and scale (continuous optimization of Genie spaces through MaxGenie, a statistical evaluation loop that increased accuracy five-fold in 12 minutes).
this video covers the federated architecture uniting business-owned Genie spaces for structured SQL queries with Knowledge Assistant for unstructured documents, all routed through a centralized multi-agent supervisor. You'll learn authentication patterns that preserve permission boundaries across every conversation layer, how to embed visualizations so business users understand agent answers in seconds, and scaling evaluation beyond one-time optimization. The architecture demonstrates how Unity Catalog, Entra ID integration, and MLflow traces enable operations at scale while keeping governance simple and enforcement automatic, essential for regulated industries like pharmaceuticals where trust is non-negotiable.
🤝

Chapters

FAQs

What is AstraZeneca's federated agent architecture on Databricks?

AstraZeneca's federated agent platform routes business user queries through a centralized multi-agent supervisor that delegates to business-owned Genie spaces for structured SQL queries or a Knowledge Assistant for unstructured document retrieval. Each domain team owns and maintains their Genie space while central governance enforces security and compliance.

How does on-behalf-of authentication work in AstraZeneca's agent system?

On-behalf-of authentication passes the end user's credentials through every layer of the agent stack—supervisor, sub-agents, and data sources—so that Unity Catalog permission boundaries are respected even when multiple agents are involved in a single query. This contrasts with service principal authentication, which grants agents an elevated identity that would bypass user-level permissions.

What is MaxGenie and how does it improve Genie space performance?

MaxGenie is an automated statistical evaluation loop that continuously tests different Genie space configurations—instructions, sample questions, and SQL generation settings—to find the combination that maximizes query accuracy. In the demo shown in this video, MaxGenie increased Genie space accuracy five-fold in 12 minutes without manual tuning.

How does AstraZeneca use Vega-Lite to improve agent adoption?

Instead of returning raw SQL results or text tables, AstraZeneca's agents generate Vega-Lite visualization specifications that travel through API responses and are rendered in the frontend as rich charts. This approach makes agent answers immediately understandable to business users without requiring them to interpret data tables.

Full transcript

[00:08] Hello everybody. Thanks for coming. My name is Brian Burke. I'm a senior director of platform engineering at Astroenica in the data analytics and AI space. I'm joined by Hamu Moradi who is an AI forward deployed engineer at datab bricks. And today we're going to talk to you about
[00:24] our journey um growing agents and scaling agents using agent bricks at Astroenica. So first a couple of words about Astroenica. We're a global pharmaceutical company. Um we're relatively large. A lot of metrics on the screen there. U basically just carry
[00:40] that message about being large. We are nearly a $60 billion company. Last year we had close to 200 products in our pipeline. These are medicines and therapies. we're trying to bring to market to get in the hands of patients. Um, but I can say the number that I I'm
[00:56] personally most proud of and I think most people at Astroenica are most proud of is the 320 million. That's 320 million patients whose lives we've been able to improve um through our medicines and therapies. So, a lot of us in the room are are engineers and we have the luxury of working across different
[01:13] industries. Um, I consider myself fortunate enough to be in the healthcare space to work for a company like Astroenica. So if I can contribute even a tiny slice um to that benefit uh it it really makes going to work rewarding. So how are we able to bring this benefit?
[01:29] Um we are a science and innovationled company. Um we work primarily across three therapy areas. Oncology is one. Another one is our biioharmaceutical business. This is a lot of uh cardiovascular, renal, metabolic
[01:44] medicines, respiratory medicines. Uh and then finally we work in the rare disease space. Uh and as we're trying to improve and and make progress in these areas, we always do so with a keen focus on people, society, and and trying to be good stewards of the planet. We take
[02:01] that very seriously. So let's talk about agents. All right, agents have never been easier. Um at Astroenica, probably like a lot of your companies, we're cranking them out. We've got Microsoft C-Pilot. We've got
[02:16] notebook LM from Google. We've got Genie Spaces. The business has really started to dig in here. Um the problem is we're starting to wind up with that pile of agents. Uh I get people building these things coming to me and my team saying, "Hey, this is great. I built this agent,
[02:32] but how do I take it to production?" Or, "I need a different UI or hey, I I have this agent. How can I share it?" Right? At the core, these questions are really mostly about scale. Right? So you quickly get to the point and the question is how do we actually federate
[02:48] AI in a way that can scale. We started to get an idea at Astroenica about how we wanted to try and pull this off but there were some technical challenges we're going to have to solve along the way. So about those technical challenges I'm going to hand off to my colleague Hamune who's going to talk to you a
[03:03] little bit about those. Yeah thank you very much Brian. So before we jump into these three problems that we solved at Astroenica building a multi- aent systems let's think about why do we want to bother with this question why should we
[03:19] federate AI in the first place and throughout our experience we noticed that enterprise apps usually fails because of moving towards one of the two extremes. One extreme is that
[03:36] business teams try to make their own small app. They love it. They are very happy with it. They know it very well. But the problem is that is not one app. It's going to be 40 apps. But you have one question and you want one answer,
[03:53] not 40 answer. So there is no single source of truth. At some point the enterprise comes to this point that okay, stop. We're not going to do it this way. We are going to the other extreme. There's going to be one centralized app for the whole
[04:09] enterprise and everybody's going to use this one. But what's the problem with that one? It is fairly scalable. It is fairly standardized. The problem is that business teams have not built it. They don't like it and hence they're not
[04:26] going to use it. So the problem with centralized platform is that they are detached and they are rarely loved. That's why we are introducing a healthy medium which is federated agentic
[04:42] platform that business teams make their own apps and agents but there is a centralized place for them to all fit. So one example of it would be agent bricks which we're going to cover in
[04:57] this presentation. So here's a rough architecture of how this would look like in terms of a architecture design. All these genie spaces here it means that there are different teams for
[05:13] example far team is working only on one specific drug. They know that data very well. They make their own genie spaces. Another team makes their own genie spaces. Now there are five or 10 different genie spaces. They might have
[05:29] other agents. For example, when they are working with unstructured data, when might they might be working with different specific business logics that they know them the best and they build it themselves or they want to connect to external MCP servers. They have all of
[05:45] these but they are not lost. They all can connected into a one centralized system multi- aent supervisor. So when one person at the company ask a question what is the sales of Faro in the last
[06:01] six month there is a routter that knows where to go where to get which agent to get and what is the basically the answer to that question. So this is the simple architecture of a federated agentic platform. But what were the problems? So
[06:16] what's wrong with this? What was the issues of bringing this into the production? So we faced a few issues. We categorize them into three main categories of problems which are going to be trust,
[06:34] adoption, scale. And to be honest, each of these topics, they require their own presentation exclusive 40 minutes each to go through all examples. But what I'm going to do in this talk because I also when I'm
[06:52] listening to a talk, I don't want general things. I want to understand one specific example. And for this reason, for each of these issue, I'm going to tell you one specific problem and how we solved it. And by the end, everything
[07:08] that I'm going to show to you, all the code, all the blog post, everything will be shared. So you're going to walk out of this room with all the asset you need to basically replicate the same thing. So let's start with the trust. Going
[07:25] back to this architecture, what does this three problem means if we want to pin it into this architecture? So trust means that how can I trust that the user is going to get the right data the data that the user is allowed to see.
[07:42] Adoption means that how can I make sure that I visualize the answers in a way that users can understand. And the third problem was that how can I make sure the quality of genie spaces remain optimal and they don't degrade.
[08:00] So these are overall some three issues but I'm going to dive into one to each one tell you what exactly were those problems and how we solved them. So let's start with the first one trust.
[08:16] As Brian mentioned, I am a AI engineer. My job mostly is working with customers. And I learned this very early in my career that if the customers does not trust me, the project would not be successful.
[08:32] And I understood the same way that if you do not trust the app or any other application that application would not be successful. Now what is the uncertaintity inside an app? Nothing but the agent because the agent is the part that has the agency.
[08:50] So if the user or the platform does not trust that agent, there is no way that app would be successful. There are several dimensions that we need to make sure that trust is there but one very specific part that was
[09:08] extremely important for Astroenica was data access. How important is that? So at the beginning of our engagement whenever I asked that uh can we have some subject matter expert so we can work on the evaluation and increase the
[09:24] benchmarks I was seeing that there is some hesitation to bring a subject matter expert and for me was a question mark that why does it take so long what is the complication about just bringing another subject matter expert then I understood later why that's such a big
[09:40] deal the simple reason is that this team of fara is only allowed to see their own data. Another team working on another drug only is allowed to see their own data. They're not allowed to see each other data. So it's not so easy to bring
[09:58] a subject matter expert from another team to basically look at this data and Astrazenica team they knew that in theory databas platform allows this level of authentication
[10:14] that you have full trust but they have not seen it yet. They were thinking that this going to come when we get to the production. So what I did is that how about instead of waiting for the UI team to arrive and make a sophisticated
[10:30] beautiful UI, I just make a very easy streamlined one day app. But I implement this authentication so they can see them for themselves that how this authentication works. Then the trust was there and hence was so much easier to
[10:47] get any subject matter expert that I basically needed for basically working on this and now this was basically the idea. How did we do it? We basically instead of relying on the agent to decide whether the data should
[11:05] be shared or not, we completely erase the problem. Is not up to the agent to decide. it is enforced by the platform. And how did we enforce this? Basically, our goal was that trust has to survive
[11:22] the conversation. Every prompt, every tool call, every row, the user identity should travel with with it. So to explain to you that what is this new way of authentication compared to the older
[11:38] version is that is the difference between service principle authentication and on behalf of the user authentication. So what does this means is that in the traditional way if Brian
[11:54] and me Brian you're taller than me I didn't notice. Okay, there is it's not 100% accurate sometimes this these photos. Uh so if Brian and me they would we would like to get a elevator in the
[12:09] service principle case elevator remains blind to our identity does not know who we are right so if I would like to go to the bottom floors and if Brian would like to go to the top floors the
[12:24] elevator or the app has to make a compromise has to decide which floor is possible for example makes this compromise that you all can go to the floor 3 4 5. The problem with that is that I would be a bit unhappy because I would like to go to the bottom floors.
[12:42] And Brian also might be a bit unhappy that he used to go to the top floor to the penthouse and enjoy his own privacy. Now he has to tolerate me as well. So how do we solve this problem? And by the
[12:57] way, this is how most app works that they basically compromise. they become blind to the user and they allow basically only one type of authentication. It's much simpler way is that basically the elevator would not be blind to the
[13:13] identity. It knows that it's Brian. So Brian can go to the penthouse topest floors of Astroenica and enjoys his time while keeping me in the basement of Astroenica working on agents.
[13:33] So that is basically the part about the trust. Now we solve the trust. The next part is adoption. So then in Astrazenica many of developers were working on genie spaces. That's where how they started. They were
[13:48] business subject matter experts. They built their own genie and they like their genie space because they built it. They are familiarized with it. And if you know that genie space also has this builtin visualization.
[14:03] So whenever the answer comes you don't need to read all the table. There is this nice visualization that shows you this is the graph. What most people might not know is that if you want this visual, if you want to make your app and
[14:18] connected to a UI, connected to Microsoft Teams or in your mobile, this visualization is not going to travel with the API. The visualization remains in Genispace. Only the table and the
[14:34] answer of Genie spaces would travel through the API. So when I was working Astroenica, they came back and said we need the visualization. And me being a bit maybe arrogant, I
[14:50] thought but how important is that? Do we really need visualization everywhere is we can just look at the table. Then I immediately understood why this is such a big deal to have visualization. Because tables do not drive adoption.
[15:08] So imagine a sea level looking at this left or right screens less than 10 seconds. If it looks at the le left screen might not get so much insights but if it looks at the right
[15:24] screen less than 10 seconds can see that okay EU had a quite good quarter and that drives actions and basically creates adoption as simple as that. So how do we do this? So when we got to
[15:43] this problem, we tried to recreate basically think from the scratch how should we think about visualization and one part that I did not at the beginning really like to work on visualization was that the traditional way of working with
[15:58] visualization is was not built for agents. There was no agents and I was thinking that how can we create this in a way that is built for agents. So to explain to you the technology basically this web light technology that
[16:15] we use I would like to talk to you about my favorite meal and is I think after lunch you I hope by showing some photos I'm not going to trigger hunger in you. So let's look at my favorite meal is
[16:32] Persian noodle soup uh called ash. And I have a question from you. If you could take a photo either from left side of the screen or right side of screen, which side would you take photo of?
[16:49] Someone might say, I would take a photo of the left side. It looks yummy. It It looks more entertaining. But the problem is that if you take photo of the left side, when you go home, you cannot eat that. You cannot even
[17:06] smell it. Even you don't know what's below this even you cannot look at the left side nothing you are stuck with that very limited angle you cannot do much on the other hand if you scan the QR code not only you have all other
[17:23] resources that we're going to talk about this slide the other side you have the recipe of this dish and with that recipe you can cook it you can eat it you can smell it you can invite your family, you can cook it
[17:38] forever. Forever you can have this food. So it's not really in basically compared to each other and but more or less that's how the typical visualization is done. They throw this JPEG to the UI and this poor UI has to
[17:56] work really hard and at the end the visualization looks very crappy because it cannot render, it cannot play with this. So this new approach is that we minimize this visualization into a simple recipe and we throw it at the UI
[18:12] and UI grabs this and can do a lot of things with it. So how do we basically do that? A user ask a question, Genie answers and says this is the table and we use this UC function veite that gets
[18:29] that table and ask the agent what do you like how do you like it to be visualized agent says I like a pie chart or I like a histogram or whatever else that's it we just throw this simple like JSON to
[18:45] the UI and that UI can be anywhere render renders it and the beauty of it is that since it's so light and the UI aspect whether is in phone or whether is in Microsoft teams can do whatever it
[19:00] wants with this makes the beautiful visualization that is even interactive. So basically that's the main difference and this was our basically by the way we published a joint blog post Brian and I and a product team and which is
[19:17] available in the QR code and we're going to show it QR code at the end again. So this was about our adoption. The last part of the three part is scale. Now
[19:33] what's what is the there are many parts to a scale. Everybody would like to scale. But when I was thinking about which part is most important to reflect on when it comes to scaling and I don't know maybe you don't agree with me but I
[19:49] think the most important part is evaluation that you need to make sure that evaluation is scalable and unfortunately I think still in the industry we think of evaluation or optimization of evaluation as a onetime job that we
[20:08] optimize this genie, we are done. But the problem is that this genie is not something static. It's going to change. Data is going to change. How you use it is going to change. Sometimes even the business request is
[20:23] going to change. The model that you define your KPI is going to change. How do you make sure that this is going to stay optimized for a long time? Maybe we say that okay we ask Brian from his penthouse to give us someone who is
[20:40] responsible for this genie space to keep it always optimized but the problem is that is not one genie space right now it might be 20 genie spaces and next year we might have 200 genie spaces this does
[20:55] not work we need a new approach to make sure that our optimization for evaluation is continuous So that's how basically I created this simple app that is as light as a skill
[21:13] that basically you're going to upload to datab bricks to optimize the genie space and it's called max genie and the idea is that it's basically optimization loop uh for genie spaces
[21:29] and this is fully open source and I'm going to explain to you how does it work behind the curtain and perhaps this is the most technical slide that I'm going to talk about is that this is how max genie works in data bricks it takes a
[21:47] genie space downloads the whole genie space as if it's a modifiable code so when it downloads it it can change everything it can change its instructions s it's SQL examples data
[22:06] joins name of the columns everything it treated as a code and then it takes the evaluation benchmark and split it into train validation hold out test like a classical traditional ML
[22:25] and why to prevent this becoming overfitting because I see many genie optimiz ganization that they optimize a genie claiming that this is optimized by making it overfitted. It's just add all
[22:41] the failures to examples. Of course, now the optimization would go higher, but if you ask a new question, it's going to fail again. That that's not so valuable. We want it to be scalable and future proof. That's why basically we uh set up this specific statistical analysis to
[22:58] prevent this overfitting. So now we have this train set. We have all the assets of the genie space. We ask a llm to read everything and we have inside max genie we also shared the best practices for
[23:15] optimizing genie space. Now that genie uh that LLM decide which part needs to be changed, which part of instructions, which part of examples and after all of that that becomes one candidate to change and then genace looks to see that
[23:33] if this is worked and it passed or did it failed and continues on this until it reaches 100% full optimization or reaches a plateau. That's basically as simple as that. Now I can show you one
[23:50] uh one minute demo of this. So you can see this is New York taxi geographic genie space which is by the way available on all data bricks workspaces. So you can repeat this for yourself. And as you can see the
[24:06] accuracy is 10%. I also go to the benchmark here and show that the accuracy of the benchmark is 10%. And what I do is that I go to the genie code and I basically call max genie
[24:22] and after calling max genie I just press press enter. That's all that's all I do. What happens is that genie code first detects which genie space this is downloads the old genie code and does
[24:38] not touch the original genie space but rather makes a new copy and tries to re iterate on that new copy creates basically a new job that optimizes this. Now you see the job is
[24:54] here is finished and it took roughly 12 minutes to basically run the full optimization. When this optimization is done then we can have basically the new genius space that is optimized which I'm going to uh
[25:11] basically clink right now and after clicking I see that uh this this new gen space I can review look at all instructions but most importantly I see that 10% improved to 50%. So five-fold
[25:27] increase and I would say it's not that bad for 12 minutes optimization job. So that was basically about Max Genie. What we discussed was trust,
[25:42] adoption and scale. And with these three I noticed that basically trust made it safe, adoption made it useful and scale made it repeatable.
[25:58] Back to you Brian. Thank you Hun. So we have these technical challenges solved right now. We can go about actually building our federated Agentic platform. Right? So first um I just want to take a quick step back and say why do
[26:14] we actually need one? So like Hamune alluded to the it's been happening for decades now it continues today simple questions can be hard to answer right so for example an example that somebody might ask at Astroenica how has farsea market share changed in
[26:30] A&Z to answer that question you need to know what is Farsa it's a brand in our cardiovascular portfolio what is A&Z in the context of Astroenica it's our Australia New Zealand market what is the market share who are the competitors right who Who do we compete against in
[26:45] that space? Where do you actually find the data, the tables, the columns, right? So, we've all been trying to solve these challenges for a long time now. I'm going to guess a lot of people in this room are engineers. Okay? So, we probably have some experience working on the right hand side in the central
[27:00] platform model. You've got your scrum lead. You've got your BAS trying to act as the link between the business and the engineering teams. You have your backlog. I think we can all probably list a long a long list of how that can work and how it can fail, right? Has
[27:15] anyone tried to uh solve the single version of the truth problem? If you've figured that one out, let me know afterwards. I'd love to talk to you about it. Okay, that one continues to be elusive. What has gotten turbocharged over the last year or two is the business owned version, right? Suddenly
[27:31] these tools, these agents are in the hands of the business, right? So they're low code, no code tools tools. the agents have really lowered the barrier to entry. The the problem is you wind up with that pile of disconnected ungoverned agents. So, how do we find that sweet spot in the middle? How do we
[27:47] continue to let the business teams expose their data and their rules while still leveraging the benefits of that centrally managed platform? And so for us, what we've had success with is
[28:04] business owned domains with shared central control. Now to do this there's three basic core principles that we try to stick to. One is we need to empower the business, right? So we need to give them the tools in order to create what they need to build these agents. They already know the data, they know the rules. So with a
[28:21] combination of Genie and knowledge assistant within agent bricks, they're able to solve a lot of use cases and I'll go into a little bit more detail in the next slide. The second thing is we need to govern. Okay. Now there's two big parts to this. One is the supervisor layer and one is the application layer.
[28:37] Okay. So at the supervisor layer, we do this centrally because this is where we implement the charts, the Vega light spec that Hamune was talking about. There's also kind of table stakes capabilities like web search. It makes more sense to do that centrally and then everybody can take advantage of it.
[28:52] There's also MCP tool integration. We use a lot of third parties like a lot of people do. For example, we have um a third party that we contract with. They expose an MCP server for our branded content. This is in the pharma space. We can't just say fargo is the best
[29:08] medicine ever. You must prescribe it, right? There are certain rules we have to follow. So, we use that service. Central agent calls out, comes back with approved content and there you go. Business is happy. Okay? There's a couple other things. The authentication strategy needs to sit in that central platform that on behalf of authorization
[29:24] starts there. And then finally, um it's an engineering project. There are always going to be some technical challenges. Having the engineering team centrally some of those integrations, it helps to have them in the middle. And then finally, we have our operations part. We don't just want to get to production. We
[29:40] want to have this live long term. We're taking a philosophy where the more we can leverage out of the box platform capabilities, the more simple we can keep the approach, the better we're going to be able to scale. Everybody goes into projects thinking, I'm special. I need custom. You very well
[29:56] might be, but as soon as you do, you are really raising the bar and making much more difficult to maintain long term. Plus, you're going to miss out on a lot of the things if you don't account for it that you're going to get on this platform you're investing in in like unique catalog out of the box, ML flow traces, things like that. So, that's
[30:12] been our philosophy. So, with that level set, let's get the business creating our agents, right? We have two major patterns that have really emerged. They kind of fall into the structured pattern and the unstructured pattern. So in the structured side of
[30:28] the house, we're fortunate at uh Astroenica that we already have a lot of data bricks users. We have analysts in their ad hoc SQL queries. They are creating notebooks, scheduling on jobs, um building, training, deploying their own ML models, right? So they know the
[30:44] data. They're already familiar with data bricks. The step to genie was very linear and simple. In fact, so much so in many cases they found it on their own and that's how we wind up with the pile of agents, right? So we're trying to fix that problem. Okay. Um the second is the unstructured pattern. So I give an
[31:00] Astroenica business a lot of credit. They have managed to compile an quite a number of uh unstructured documents, PDFs, word docs, things like competitive intelligence reports, market research. Some we pay for, some are just freely available. They've got it in SharePoint.
[31:17] A lot of cases it's cataloged nicely. A lot of cases is it cases it isn't. But what we've been able to do is when it's sitting there, it's hard to unlock value. But what we've done here is we've been able to integrate it into data bricks volumes. We happen to use Fiverr for those that care about such things.
[31:33] And once it's in volumes, you can layer a knowledge assistant on top of it. Now, because agent bricks and knowledge assistant just went GA a few months ago, for those that aren't aware, knowledge assistant is basically a manage rag capability in data bricks. Okay? It
[31:48] takes care of the chunking, embedding, vectorization for you. It also does the document retrieval. So our users have really found a lot of value from that. So the other thing because it's a managed service, it is supervisor ready. To plug it into an agent brick supervisor is trivial. It also gets ML
[32:05] flow capability under the covers. All right. So with these two tools in their toolbox, um they've been able to bring a tremendous amount of value. So with all that taken care of, we get to the governance slide. Now I will I
[32:20] will be honest with you. When I'm sitting in that chair to get to the governance slide, that's when I go and check my email, right? I'm mute. Thanks. Anyway, so uh this isn't governance around SOPs and change control boards and all that kind
[32:36] of stuff, right? This is actual practical governance that ties all this together and it's all centered around Unity catalog. So imagine we have our application SSL authentication against Entra. We have groups you get in the application that OBO authentication goes down to the supervisor. Okay, that's the
[32:54] technical challenge we had to solve with Hummingune. Once you're in that front door, once you have the supervisor, it's very straightforward. Unique catalog governs the supervisor, all of the all of the agents that the supervisor can access. It even goes down to the data
[33:10] layer, right? So which tables can I access? Rowle security, column masking. So from top to bottom, it's all governed through UC. All you have to do is make sure that you get your groups right, split people out. With that in place,
[33:25] and of course, we all know you can put one user in multiple groups. But with that in place, it allows you to scale agents horizontally, both at the sub aent level and at the supervisor level. We are now at the point where we don't just have one supervisor with a bunch of sub aents. we have multiple supervisors,
[33:42] right? So maybe with our business, you have an oncology supervisor for that area and you have a bioarm uh supervisor for that area. Some people only get one. Some people get both if you're high enough, right? But we can manage that through a combination of the application
[33:58] layer and the groups and it all just works. So with that in place, we can't forget about operations. It's important for us to keep the lights on. uh getting it to production is only the first step. So what we have found important is one is
[34:13] establishing a publish process CI/CD. If the business is out there creating their agents, how do I get that stuff into the overall capability? I will admit this has been a little bit of a challenge. Our business likes to live in an underscore v1 underscore v2
[34:28] world, right? So getting them to kind of buy in um is a work in progress. Okay. In some cases, the APIs you need to actually implement like GitHub actionpowered full-fledged CI/CD either isn't there yet or it is there and it's
[34:44] kind of a heavy engineering lift and maybe we do or don't invest in that. At the very minimum, you need to have some sort of SOP, some sort of process. Partner with the business to get their work through that pipeline. Okay. Second thing, eval benchmarks absolutely critical. It's not just getting it to
[34:59] prod, it's keeping it alive. Um, we use uh MLflow traces quite a bit. If you are starting to use agent bricks and you haven't looked at MLflow traces yet, um, data bricks basically made an agentic version of MLflow, a new version of what's been out there for a long time for traditional ML. You can go in there,
[35:17] you can see who because we have on behalf of authoriz authentication, who made the call, um, input and output parameters, how long did it take, there's a Gant chart in there that describes everything that went on. Okay, we've also got our feedback mechanism. The thumbs up, thumbs down on the on the
[35:33] the UI and the application, calls an API, writes rows into a delta table. We decided to make that one to one to the ML flow traces, you get negative feedback, read the comment, look at the MLflow trace, maybe it was slow, maybe the parameters wrong, right? So that helps our agent ops team. And then
[35:50] finally, in the operations um realm, take advantage of all your enterprise capabilities. For us, a lot of this keys off of the fact that we're integrated with, for us, we use Microsoft Entra. So, let's get it and engineers out of the business of deciding who should have
[36:07] access to what. The Farsa team that built their agents, make them the approvers of that AD group. They're the ones that knows and can decide who has access to that agent. That all gets integrated through the on behalf of authorization into this stack. And then, of course, you have endpoint management
[36:22] and stuff like that that makes it part of something that you're deploying out to the business. Okay. So I will say this was a big project. It continues to be. It has not all been smooth sailing. We've encountered a few challenges. So one of them was the CI/CD challenge I
[36:37] talked about both organizational and technical. Um we like to run fast and try and be at the forefront and keep up with data bricks who runs even faster. We live in a public preview and private preview world. So with that comes risk.
[36:55] Sometimes the products that we're looking to implement don't come with what I would consider uh table stakes capabilities example that continues to stick in my craw is applying a policy to tag it so I can track how much it costs. So that's my upvote for that feature in the
[37:10] backlog at least if you're in the room. Um and then finally uh agent bricks is a managed service within data bricks. It comes with some limits right so you have to operate within those limits. So, let's do wait, we'll do a quick trivia, right? Here's a question for you. If anybody's been playing with agent bricks, how many agents can you connect
[37:27] to one of those supervisors? Does anybody know who thinks it's who thinks it's 10? Show of hands. Anybody think it's 10? 20. Anybody think it's 20? How about 40? 40. Correct answer is 20. 20 as of today. Uh, I've been told
[37:43] that's going to go up. Data Bricks is always pushing the pushing the limits. I think it's going to be 40. Keep an eye on the uh release notes. Next question. What's the max file size uh for a single file in knowledge assistant? The unstructured rag. How big a file can I
[37:59] put in there? 10 megabytes. Anybody? 25 megabytes. What's the max size? How about 50 megabytes? Anybody? 50 is the winner. But again, a little birdie told me it's going to go up. So that's good. That's another reason we invest in the data bricks platform. They continue to
[38:15] make improvements, right? And then finally the last question is how many files can you put into a knowledge assistant? Right? So this unstructured drag has access to files in a volume. How many is it? Is a thousand? Is that the max? 10,000.
[38:32] Unlimited. Is it actually unlimited? You can print as many files as you want. Believe it or not, it is unlimited. Well, it's limited by how many files you can put in S3 bucket, I suppose, but I'm going to call that unlimited. So I give a lot of credit to the the product team to bring that capability to us. Right. So
[38:49] this has been our story. I appreciate you guys paying attention. Um this is how we built a federated agentic platform at Astroenica. Keep in mind creating agents is easy but you need a strategy in order to scale. For us at Astroenica um we've had some
[39:05] success combining business owned domains with a shared central control. you're going to have technical challenges. For us, we had to solve the problem around trust, adoption, and scale. Um, if you want more details on exactly how we solve those problems, that black QR code
[39:22] there has a bunch of links. Um, it is the how you do the OBO setup. We have a link to the GitHub repo for Max Genie. Uh, the blog that, uh, was published around how we did the Vegaite integration to do the graphics. And there's also the recipe for my new
[39:38] favorite Persian soup, Ashte. Asht. Ash. Okay, close enough. All right, I hope this information helps you on your journey to scale AI. Thank you very much.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.