Automating Taxonomy Generation With Compound AI on Databricks

Summary

• Compound AI Architecture: Combining multimodal and text-based LLMs, internal data, and external APIs creates sophisticated taxonomy generation that improves accuracy and coverage.
• Automated Taxonomy Solutions: Modern approaches solve taxonomy challenges in retail, manufacturing, and e-commerce by automating classification that previously required manual effort.
• Scalable Classification at Enterprise: Implementing compound AI on Databricks enables organizations to build, deploy, and maintain taxonomies that evolve with business needs at scale.

Automating Taxonomy Generation With Compound AI on Databricks

Talk by Allistair Cota, Lead Consultant, Data & AI, Lovelytics ; Sudhir Gajre, Managing Director, GenAI, Lovelytics
Taxonomy generation is a challenge across industries such as retail, manufacturing and e-commerce. Incomplete or inconsistent taxonomies can lead to fragmented data insights, missed monetization opportunities and stalled revenue growth. In this video, we will explore a modern approach to solving this problem by leveraging Databricks platform to build a scalable compound AI architecture for automated taxonomy generation. The first half of the session will walk you through the business significance and implications of taxonomy, followed by a technical deep dive in building an architecture for taxonomy implementation on the Databricks platform using a compound AI architecture. We will walk attendees through the anatomy of taxonomy generation, showcasing an innovative solution that combines multimodal and text-based LLMs, internal data sources and external API calls. This ensemble approach ensures more accurate, comprehensive and adaptable taxonomies that align with business needs.
Databricks Named a Leader in the 2025 Gartner® Magic Quadrant™ for Data Science and Machine Learning Platforms: https://www.databricks.com/blog/databricks-named-leader-2025-gartner-magic-quadrant-data-science-and-machine-learning
Build and deploy quality AI agent systems: https://www.databricks.com/product/artificial-intelligence
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

FAQs

What is taxonomy and why is it important for businesses?

Taxonomy is a systematic classification system that organizes products, content, or concepts into hierarchical categories. For businesses, complete and consistent taxonomies enable better search and discovery, support data monetization, and ensure consistent customer experience across channels.

What challenges does incomplete taxonomy create?

Incomplete or inconsistent taxonomies fragment data insights, miss monetization opportunities, and slow revenue growth. Manual taxonomy creation is slow, inconsistent, and difficult to maintain as businesses evolve, making automation essential for scaling.

How does compound AI improve taxonomy generation?

Compound AI combines multiple approaches—using both multimodal and text LLMs, incorporating internal business data, and calling external APIs—to build more accurate and comprehensive taxonomies. This ensemble approach captures nuances that single-model approaches miss.

What business outcomes does automated taxonomy enable?

Automated taxonomy enables faster data monetization, improved customer experience through better discovery, reduced manual classification effort, and taxonomies that scale with business growth. Organizations can build dynamic taxonomies that adapt as product catalogs and business needs evolve.

Full transcript

All right, good afternoon. Can you guys hear me? Okay. Well, thank you so much for joining. I know it's Thursday. It's the last session. Apparently, told me personally that they saved the best for the last. So, thank you all for coming. I think we have about 35 or 40 in the room. Uh the topic today is automating taxonomy generation with compound AI/ agents on data bricks. I head up uh the genai practice at loveletics.
With me I have uh my colleague Alistair. He's a lead consultant in our team. I am good at making slides. He's good at writing code. So we make a good team. Um so hopefully this this will be a fun session. I'm sure you guys have seen this. You are fully aware of this. would truly appreciate. Oh, there was a feedback slides. Would truly appreciate if you guys can give us some uh some feedback. Uh so the agenda is kind of divided into three sections.
I'll talk a little bit about taxonomy, origin, importance, why all of that. I'll talk a little bit about the compound AI agents. What is all that about? And then Alistister will bring everything together for us from a compound AI for taxonomy perspective. you'll get to see a pretty cool uh demo as well and then we'll kind of conclude. Um hopefully this time at the end we'll take the questions if not we'll be hanging around so please feel free to ask us questions we have done several implementations of compound and uh and agentic system so we'll be happy to share any feedback any information knowledge uh we do have all right so having said that um when I first got into taxonomy uh with a client several uh actually months back and I started to research I kind of realized that The roots of taxonomy literally go back like thousands of years during the times of in the ancient civilization.
In the recent past which is about 1700s KL lenius father of modern taxonomy created the binomial nom nomclature and the hierarchal hierarchical structure still used today. So obviously taxonomy is as relevant today as it was back then. I mean we all know a proper taxonomy serves as a shared language uh creates consistency in naming and classification helps reveal relationships which can then further drive search and discovery.
Again in today's world with AI classical machine learning genai we can all appreciate the meaning of a well ststructured detailed and curated data. Hence, taxonomy is very important, very relevant today. Obviously, I have to kind of mention this as the famous philosopher Confucious said. Beginning of wisdom is to call things by their proper name. I think that's that's pretty uh uh pretty important quote over there, Claude Shannon, father of information technology.
Best way to understand something into is to organize it. Now I personally come from a a background decades ago in uh rel relational systems where everything was structured. There was a technique called normalization where you would take data, organize it, structure it. So that importance even though we live in a world of highly unstructured data um still still exist. All right. So the uh the business impact of taxonomy, right?
So when we worked with real clients in retail and manufacturing kind of becomes apparent to us that in today's world it serves as the foundation for structure, organization and meaning. Uh first of all it enables consistent categorization across systems and I have yet to meet a client who has one system and one system only. I know many clients who have like several ERPs, several relational databases, several data warehouses.
So as you can imagine uh this takes a very very kind of important significance. Um it is hard to piece a complete picture u when you have like missing data or incomplete data right how can you do analytics or predictions or any of that type of stuff when you don't have all the information kind of you know pieced together and as I said when we work with customers in manufacturing retail you know ecom they mentioned that taxonomy is a very very critical problem for them right u it actually impacts business function It impacts economics.
We have seen a very recent client use case where they completely express their inability to upsell and cross-ell to their clients because they just don't have a complete picture of who the retailer is or who the client is. So obviously uh it's got huge impact. The other thing we realized is working with business experts is taxonomy is not a one-time task. It's not like you can sit down in a room for like a week or 10 days and get the whole thing done.
Why? Businesses change. New products come in. You you launch new product lines. New customers come in. So obviously it has to be kept updated on a on a regular basis. Hence it is very important to have some sort of automated system some sort of AIdriven system ideally in our in our world which can keep things automated. So having said that what are some of the typical challenges right I mean it is important we know that having a great taxonomy a great classification would be good but why is it so hard so traditionally taxonomy was sort of like a byproduct right or rather an afterthought I mean and it was always done manually so there was no systems per se I mean a lot of the ERP systems will have metadata tagging and stuff like that but it was almost almost like an afterthought.
Not only that, different people of across different business groups are responsible for taxonomy in a typical and traditional system. So that leads to inconsistent classification, inconsistent tagging. We all are data people in the room. We know exactly what happens when the same thing is called by two different labels, so on so forth. Scalability becomes an issue, right? Most of the clients we deal with in retail or manufacturing or whatnot have tens of thousands of products.
And the other interesting thing is if you use traditional software techniques which are typically rulebased techniques is very hard to come up with rules to generate taxonomy. But guess what? Modern AI especially Gen AI using multimodal approaches which you will shortly see. As I said he is the real smart guy. uh you will see a cool demo where we are doing multimodal things and you know all that type of stuff semantic search semantic similarity vectorization all those kind of techniques now help you piece together a almost near perfect or a near accurate taxonomy picture.
So when we did this project I think um some of the analysts told us that we were like in the upper 90s from an accuracy perspective uh in taxonomy. So the results are actually very very good. All right. So now that I have talked about taxonomy, finally let me tell you what taxonomy looks like. Right? So here's two examples we made up of taxonomy. One is from manufacturing and the other one is from ecom. So in manufacturing as you can see there could be like say multiple levels right all the way up to like what is it level five.
So let's see it's hard for me to read from over here. Maybe you know level one could be a pump which is an equipment type. Level two could be a centrifugal pump. So like a subcategory or subclass. Level three could be an impeller. Level four could be maybe stainless steel 8 in. And then level five could be a particular classification or a code. Now imagine a table having this level of rich and detailed data with all your 50,000 products in it.
Right? With this type of enriched data, you can do some amazing analytics on it. Right? The other example is uh there's e-commerce there, but I'll give you an example of a retailer. So, not too long ago, we worked with a retailer who had sub retailers in South America, and they had no idea what the sub retailers were. Some of them could be like really really small stores like literally like a vegetable vendor with a cart, but once they knew, and Alistister will actually do a demo, which kind of is somewhat similar, but once they figured out who the retailer was, they could very very easily then do cross-ell and upsell.
So if it's a if it's a chemist or a drugstore, they can do certain things. If it's more of a general store, then there are certain other items that might be relevant. So it has got, you know, as you can see, real business impact. Okay. But then why are we so excited about it today, right? I mean, why now? And what is the compound AI um or the agent opportunity? So about last year, maybe right around this time, there was a very famous paper published by mate and others from data bricks and UC titled the shift from models to compound AI systems.
Uh quick raise of hands. Have you guys heard about it, read about it, whatnot? No, some people. Okay. Uh if you have not, it's a it's a pretty good read. uh a basis of lot of our work after I saw this paper was actually derived from this. So of course it talks about a component architecture which then kind of feeds into agentic systems as well. So a lot of the work we have done in the recent sim about a year um we have used a lot of concepts and sort of kind of almost inspiration um from this from this paper.
Um, so what's kind of the big deal about this, right? So, basically, compound AI systems are AI systems that solve tasks through multiple inc multiple uh interacting components including different types of LLMs, information retrievalss, APIs and all that type of stuff. So, simply put, it is a very modern architecture with LLMs as just a component. And I I write frequent blogs and in fact I wrote a blog about this several uh months ago about how most modern architectures these days are not LLMcentric.
Right? So when Genai came into existence number November 202 when chat GPD was launched soon after the API was available all the applications that were created or most of the applications were kind of LLMcentric. You had an LLM you had some sort of chat interface. It bounced against the LLM you got a response and then the world realized oh wait a minute there's a big problem with that. LLMs elassinate they don't have awareness of your data they only have general information through the pre-training phase whatnot and hence the evolution of compound AI and we have had just tremendous success implementing these systems so obviously LLM are not enough lms are powerful but they are not all knowing right the pre-training phase goes through uh the general data which is on the internet uh but not your data so that's that's kind of the important point uh the other thing is initially there was a notion that throw the biggest hammer at it, right?
So, find the biggest LLM you can in terms of number of parameters and hopefully that solves your problem. But that's not necessarily true, right? Uh a larger model size does not always mean better performance or uh you know better accuracy. Again, LLMs don't have a memory. You have to provide them memory. They can't access well they can access tools but you have to write the tools uh internal external all that type of stuff.
So now the age where we are shifting or probably have already shifted from monolithic models uh to now these modular compound AI system. So I think that's what we find as amazing. Um so essentially LLMs external tools retrievers APIs and logic and Alistister will actually show us a demo when we build bring all these things together into something like which I frankly think is pretty amazing. I mean, people don't realize that it was just impossible to do some of this stuff like maybe about a year ago or something.
The other thing I like about compound AI systems is the best way to design complex systems is to break it down into modular architectures. And this is like a classic example of a of a modular architecture. The other thing I also like which was I was immensely inspired by a quote by or one of the presentations I saw of recently is you can actually take a a GBD 3. 5 model which is an older model a smaller parameter model and make it produce the same efficiency and accuracy as the biggest hammer available today.
Why? because you are basically augmenting its capabilities with tool calling and APIs and memory and all that type of stuff. And that really resonated with me. So no longer do you have to chase down the biggest hammer out there, you can use from an inferencing, you know, quality, cost, whatnot perspective, a smaller model potentially. uh and then using all these different tools, contextual reasoning, uh contextual awareness, uh whatnot, which you will see shortly, you can actually make it into a pretty compelling uh and a pretty powerful system.
Now, having said that, um this actually had tremendous impact on our design philosophy on how we build systems and how we work with clients and how we are uh actually implementing love uh implementing uh geni systems. Um so this is how we kind of position the whole geni application journey when we work with clients right now this is a pyramid structure and again um it's not a strict pyramid because a lot of the different components and the different things we are mentioning over here can be intermixed and can be exchanged and all of that right but one of the biggest frustrations I personally had is when we talk genai and again maybe this is about a year year and a half ago So people automatically related to that to chat bots because that was the world of chat bots and rack patterns and I call that as the level one application and again they're great and they're very powerful like if you put one of these in a in a customer support type center um it can have a huge impact.
So I'm not at all against it but all I'm saying is you know a typical rack pattern just a rack pattern with a with an LLM think of it as level one. If you go one level up the pyramid then we go to what is called as the intellig or what we call as the intelligent information platform. So these are slightly complicated system right an example of this could be a document extraction system where you upload documents either batch real time extract key value pairs uh interesting information whatnot.
Then we get into a reasoning and an and an acting um type of paradigm where now you can have complex context use an LLM to reason against it and make some business decisions. Uh maybe about a year and a half or two years ago I was very fortunate to actually work on a business application where we used reasoning. So we would pass health information data use an uh use an LLM with a reasoning type of prompt and make it reason to figure out uh a risk rating from a life insurance perspective.
So that was a fascinating experience where we did something beyond a a chatbot. Uh then is the compound AI systems and again that's what the talk is and you'll see some great examples today. Uh but again ability to use multimodal LLMs API calls tool calling all of that and of course at the top of the pyramid is the agents. This has been the week of agents with everything being announced. We have already been working in agents.
We have implemented uh a couple of iterations of that. Uh and again these are not this is not a strict pyramid right like for example you can use a rack pattern as part of an agent or as part of a compound AI architecture but this is just to kind of explain what that genai journey should um should look like. Um and of course um you know all the work we do is all based on the real bicks data intelligence platform data plus governance plus apps.
We build a lot of apps as well in u in data bricks apps. All right having said that now turn for the smart guy. Uh he'll walk us through the specifics of compound AI for taxonomy and then we'll see a pretty interesting uh demo. So all yours. All right. So let's go over the core components of what makes up a compound AI system. Specifically in this context, taxonomy generation. Of course, you need to have an LLM and specifically what we're going to highlight is the power of multimodal LLMs.
So the ability to handle not just text data as an input, but text, image data, structured and unstructured data that can lead to some really powerful results generating product descriptions, um attribute classification, and other predictions. Second, you may want to supplement the system with a classical ML model as well. So, classical ML models are still around and it's still going to be there. So, they can be used for more narrow tasks like specific attribute or rule based uh rulebased mapping predictions.
Now, external APIs, of course, we're going to show you how you're going to use your own data in UC to run inference and generate predictions, but what about enriching your data with from external sources? So the retailer example that Cir mentioned for example the data that the client had was really not that useful. It was just addresses and we don't know the rest about the retailer. So but what if you connected it to an external API like the Google Street View image API where you provide an address and you get an image result back.
The LLM can then process that and then give you some more metadata information about what that retailer might do in terms of their own business. So connecting those APIs is definitely something we're going to show today as well. Of course, your internal structure data is very key. So, texonomy has different rules and definitions in each organization. We can all any day go to an LLM and let's ask it to predict texonomy.
It'll give us something. But we wanted to be grounded in the values that we recognize in the organization at hand or in the business domain at hand. So, tying this system with your internal structure data with taxonomy definitions and taxonomy rules is key. Now, as your data gets bigger, you may want to use retrievers and vector databases um to do semantic search to supplement and provide those rules um on inference time.
And of course, all these are multiple pieces. They have to be stitched together by a controller or a planner block. So, this is the sort of the orchestration of all these different components taking your input and getting those those intermediary predictions until you get your final uh combined prediction. So why now and why data bricks for compound AI? So data bricks provides some great features that make this system very easy and useful to stand up.
So Mosaic AI agent framework of course uh allows you a great platform to build AI agents and it supports multi-turn conversation. It also supports evaluation and of course the deployment of those models is just very easy and something which will support us for the for the batch inference as well. Then foundation model APIs and AI gateway. So like I said we're going to show multimodal LLMs over here. But how do you make that accessible and easy to use within data bricks.
Um that's where AI gateway comes in. You also have the foundation models APIs which are ready for you to use and um so you have access to those. You have access to external models like GBT40 if you use AI gateway and even your own custom LLM and or custom models sorry and then AI query. Um what I'll show first is going to be real-time inference but where the power of texonomy generation comes in is batch inference. So AI query is a great feature to allow you to do batch inference at scale and you can basically build workflows that predict texonomy for you know a lot of rows of products or retailers or whatever the records are that are missing taxonomy today in your tables.
Unity catalog and ML flow helps you to track uh version and govern the life cycle of your models or agents and of course allows you fine grain permission controls and access controls on your data and of course delta lake and pispark allows you for for inference and and orchestration at scale. Okay, the example I'm going to show in the demo is around a clothing retailer. So we're talking in the product domain over here and this is a reference architecture uh and what you'll see in the demo.
So think about in Unity catalog say you have some product data an image data. Now imagine this product data has very little information at most a few a few words as a product description and you have an URL to an image or just the file itself to the image. What can you do with that to actually generate taxonomy and generate this at scale? So that's where you can build and what we've built over here is an MLflow Pyunk model um that you feeds in that you feed in the in text and image data as input.
The first prediction you get out using your LLM that's you know hosted through AI gateway will be your initial texonomy predictions. So I showed that slide level one to level four for example that's where you would get your first interior intermediary predictions. But there's a there's a catch over here. How does it know what level one level two is appropriate to assign? It could just make up something but we don't want it to do that.
So that's where you see the data bricks API uh block over there I want to inference. We are also giving a call sending a call to the data bricks API to read our tonomy rules definition table. So this could be like a list of allowable texonomy values or could be like literally definitions of what level one, level two, level three is because a certain value of level four needs to match with a certain level of level one.
It has to be a hierarchy is what? So this helps you stay grounded in that fruit and avoids hallucinations. All right, you get your first texonomy predictions. Then there's a call to a second LLM and over there again you have to read up what are the allowed attributes for those given predictions. Again it still has the image input. It has the text input but now you can predict your attributes. So in the case of clothing for example attributes could be things like what kind of seasonality is it?
Is this a winter item? Is this a summer item? or is so seasonality is one thing it could be also the material the color some things like that and then finally the final LLM predict um LLM prediction that we can be done is predicting some additional metadata for this example and where the external API comes in we want to know and try to infer what the brand is of this product or this clothing item because again this is a retailer they don't make their own clothes they just you know it's wholesale they're just um selling different clothing items And then metadata, it could be, you know, where what are the competitors, who are our competitors who are also selling this product.
So for this example, we're using the Google Lens image search API. So Google Lens, I'm sure you've if anyone's used Android or even Google Photos on your iPhone, you've probably used that Google Lens feature directly. They have a great API now as well that you can use. Uh you provide an image as an input and you'll get search results. So not only will you provide the LLM your input your text your input um text and image data but also the search results that you get back from Google lens and now with all that information try and predict the metadata.
So that's what the final step is is is doing and all this is then stitched together the final output to delta tables. Before I go to the demo why one thing I want to say is why are we doing three LM calls or even more than one LLM call. It's to minimize the cognitive overload. When you ask an LM to just predict 20 different things in one go, you don't often get the best results. And that's what we learned with that, you know, the project we were talking about.
We realized that once you actually get to break it down into these sequential steps, that's where it you're really feeding in the LM with some proper logic because you really need to first predict the taxonomies before you then get to try to guess what is the seasonality for that product because it may not apply if you know if they if the LM doesn't know anything about the taxonomy in the first place. So that's the reason for breaking it down into three steps.
With that, we'll go over to the demo. Yeah, thank you. Okay, cool. So, imagine you are a business analyst and you've been tasked with assigning texonomy values, attribute values, and metadata values for this new product that the business needs to onboard very, very quickly. So, for this example, I'm just going to provide an image input. Again, we're doing clothing. Got a looks like a shirt over here. Let's provide the image URL and this body Wi-Fi.
But now we've got a preview. So before we call the LM just want to we this is what we expect to send over and just going to query our LM. Now while that's running you may have noticed we're using data bricks apps at Love Litics. We love using data bricks apps. It's great for um for basically spinning up something really quick to showcase to your end users, your your executives about an idea you have for a model. We're using a streamlit template over here.
It's connected in the back end to our model serving endpoint that we stood up and very soon we'll get the prediction so we can see the results. But it's very easy to stand up even if you don't know you know detailed HTML and uh CSS. Okay. So we got some outputs first and like I said we we're it takes some time because we're doing a three-level approach here. So the first you know let's look at level one, level two, level three and four.
We've got men. So the level one is just is it men's, is it kids wear, is it women's wear or is it some other accessories. So you can see it's in this case it's predicting men. Level two is apparel. Level three topear and level four shirts. The back end for this way we're using GPT440 as the you know the multimodal LLM. You can use another one as well. But I also want to point out this reasoning. So we've actually and we really like this approach of asking the LM when it makes its prediction to provide textual reasoning as to why it did that.
So you can see here it says that the image that it sees is a men's formal shirt. Um so based on the rules that we're we're feeding it from our lookup to the texonomy rules, it there's a category called topware for level three. So it's looking at that. That's why it's assigned level three. And um by based on its cut and style is what it says. You can see here it aligns with the men category and of course it's a shirt semiformmal attire.
So you can kind of see this reasoning. I think it really helps with auditability in case you especially once you get into batch predictions and you get a a bunch of predictions and then you realize something is wrong and you want to investigate that why not have the reasoning there to understand you know what did we feed the LLM in the first place? Did something go wrong or did the LM totally just misunderstand uh what we were providing in the first place?
So I think reasoning is something that's for this for this exercise of texonomy generation is is good to generate. Then the second call was the predicted attributes. So it's predicted the color purple. I think you can infer that from the image. Nothing about it really s uh suggests that it's for a specific season. So it says all seasons and formal usage. Okay. And the last one is the metadata. So this is where the API uh to the Google image search API comes in.
So, we're providing that image as an input to Google's lens API. Um, it's predicted the brand as Read and. And we can actually just double check this. Let me zoom in a bit. And you can see here, you can kind of see on the logo, um, I believe it's a brand from India. That's a Read and uh shirt. So, it's been able to identify that. And then it's started listing some of the possible vendors who might like for example competitors who might be selling this product.
So, Amazon, e, eBay. And all of this is again coming from the retrieved search results. Okay, so this is the app and this looks great. But what is happening in the back end? So let's switch over to data bricks. Okay, so first off, we'll start here. I mentioned Pyunk. Uh I'm a great fan of Python models and they have great support now for agents obviously because you can customize whatever model you want and log that to ML flow.
Um you're not tied down to the specifics of any library or flavor like scikitlearn or or um or lang chain specifically. You you you have the freedom to choose what you want and the customization you want. So I'm not going to go in the code in detail, but I'm just going to show some some key snippets. We want structured output, right? output even though we're working with LLMs, it's actually tabular data that we want as an output.
So that's where structured output um specification comes in play. So all these JSON definitions, these are just specifications that hey, I want these kinds of structured outputs in my response as a JSON object because once I get JSON with Pispark, it's very easy to flatten that to tabular data, tabular fields. Okay, I'm going to scroll down a little bit. We have some lookup functions like for example we said we had to look up what the taxonomies are.
So again we're using the data bricks um in this case the statement execution module from the workspace client or data bricks SDK rather if you've used that which under the hood is just using the data bricks rest API to query your data and then of course we have uh we want to load the attributes filtering for specific levels. So once you get your taxonomy predictions, you want to filter for those and then look up what are the allowed attributes for those um for those tonomy combinations.
And then finally just here's your your your um API call. And finally going right to the bottom where we put this all together the predict function. Every MLflow by the way every MLflow model uh Pyunk model needs to have a predict function. So we have our step one where we get our predicted taxonomy and then providing that texonomy as an input we get our predicted attributes finally the metadata attributes and stitch it all together pandanda's data frame output and you're on your way served this as a model serving endpoint.
So obviously remember to you need to provide your your credentials for authentication purposes otherwise the LLM is going to try to or the model is going to try to query the UC without the proper permissions and nothing's going to come back. So try make sure you have the right credentials in place. Now let's go over to inference. I talked about uh batch inference for example. So let's look at our inference data before we get any predictions.
So this is what some sample data that we have here. You can see we have very basic product descriptions. Some of them are a little bit more detailed than others, but um not great. And that's all we have. We have images on the right, all the different links. And in some cases, you may even have missing data here. Um we worked the client we were talking about earlier. Um they also had product information where they wanted taxonomy generated.
And out of the eight or 12 columns that we thought we had available to us, most of them were unusable because of just missing data. and there's just no everyone's too busy in the organization to go and clean that up but taxonomy is still required. So that's where multimodality at least having one of these inputs and specifically the image helps. Now let's just look at the use of AI query like I mentioned. So AI query as as as you as I've shown actually in a few sessions this week at summit is really great.
So we stood up a model serving endpoint of our own texonomy demo. Now using SQL query and you know in a batch inference u paradigm you can now query your endpoint but providing multiple rows in the input uh in the input request and you get you know your batch output. So if I scroll to the right, you'll start to see you get basically a nice JSON object, a JSON field, uh which is you can expand that out as I've done on the right hand side here to tabular fields which are then you know ready to go and you can populate them to your output tables.
So this is I think a really great way of doing uh batch inference using compound AI to generate taxonomy. Okay. So in conclusion, where we are today is we've clearly witnessed a shift from standalone LLMs to compound AI and even agentic based architectures for comp uh for generating taxonomy. And why does this matter? So and especially with the architecture we we showed over here we want to reduce hallucination by grounding the system with the right external tools with the right access to the specific tables in your data and providing the right business logic as to uh how it should generate those predictions and it also increases your precision and adaptability to real world scenarios and why is data bricks the right platform for doing this like I said mosaic AI agent framework allows us to create a custom model that was the Python functionality.
It allows us to serve that model using model serving um and as well as the agents deployment uh SDK and you have AI gateway and foundation models that make it easy to access these LLMs within data bricks with the right guard rails that you set up and also to set up inference and monitoring tables if you need. Unity catalog and ML flow allows you to manage the life cycle of those models in the appropriate way while also controlling fine grained access controls.
And of course the power of Delta Lake and Pispark handles the scale structure and speed of your inference. So that's where not only are you able to support real-time inference thanks to model solving but with AI query and spark under the hood batch inference is also now very much possible with taxonomy generation and compound AI. All right, I think we have some time for some questions.

Learn more about the Databricks Data and AI platform.