10x Faster AI Batch Inference with AI Functions | Databricks Week of Agents
Summary
- Serverless AI Functions: Databricks now offers fully serverless AI Functions with zero infrastructure setup required for batch inference at scale.
- Performance Gains: New batch inference features deliver 10x faster AI query processing on your Databricks Data and AI platform.
- Structured Outputs: AI Functions enable you to define JSON schemas upfront, ensuring extracted data matches your exact requirements automatically.
10x Faster AI Batch Inference with AI Functions | Databricks Week of Agents

Scaling AI-powered batch workflows just got a whole lot easier! As part of Week of Agents, we're introducing major updates to AI Functions—making them fully serverless, 10x faster, and deeply integrated with the Databricks Data and AI platform.
With zero configuration, AI Functions enable enterprise teams to seamlessly run batch inference at scale, powering agent-driven workflows across massive datasets. Watch this demo to see how you can supercharge your AI applications with Databricks AI Functions.
Fully serverless – No infrastructure setup required
10x faster performance – Speed up AI batch inference
Deeply integrated – Leverage the full power of Databricks
Ready to scale your AI workloads? Watch now and explore the future of AI-powered automation!
Chapters
0:00Introduction to AI inference0:33Setting up the AI query2:14Analyzing results and action4:37Conclusion
FAQs
How fast is batch inference with AI Functions?
According to this video, the new AI Functions deliver 10x faster performance compared to previous approaches. The demo shows processing social media reviews in just a couple of seconds using Llama 3.3 through Databricks' foundation model API.
Do I need to set up infrastructure for AI batch inference?
No. The updated AI Functions are fully serverless, requiring zero configuration or infrastructure setup. Databricks handles all the backend efficiency and scalability automatically.
When should batch inference be used instead of real-time inference?
Batch inference is ideal for scenarios where you need to apply AI across large datasets without requiring immediate results, such as scoring historical data, processing overnight analytics, or applying models to periodic business reviews. Batch processing dramatically reduces costs compared to real-time inference since resources process many records efficiently without the overhead of request handling. Use batch inference when latency tolerances allow hours or overnight processing, preserving real-time inference capacity for customer-facing applications requiring sub-second responses.
Can I define structured output formats for extracted data?
Yes. This video demonstrates how you can define a JSON schema with specific field types and structure before running the query, ensuring all extracted results follow your exact format.
Full transcript
0:00 Introduction to AI inference
Hey everyone today I want to show you how easy it is to apply AI across your data using batch inference on data Bricks now ai is all the rage these days but it isn't always easy or obvious as to how you can implement it at scale today I'm going to show you how our latest release of batch inference works and how it's easier than ever to efficiently use AI to draw insights out of your data now to set the stage we're going to take a rather basic but important challenge imagine that we have a ton of social media reviews that we collect that all say different things about our company's Global stores we simply want to use AI to extract some key pieces of information that will help us take action such as the location urgency and ratings of service and product quality now instead of having to design an entire AI system from scratch we can simply create a query so what we're going to do here is we're using AI query we're giving it an endpoint name in this case we're using metal llama 33 which actually comes ready to go out of the box through our foundation model API and we're providing it a prompt which is letting it know what we want out of it so in this case we're telling it would like you to extract the following information from each social media review we want to know the location the service score the product score uh any product names that might be mentioned an atmosphere score and an urgency high or low so that we can kind of take action and know which ones are are severe then we can give it some general guidelines here and that's enough for us to get started however there is one very awesome new feature in batch inference which is the ability to Define an output format so in this case we're telling that not only do we want these responses but we actually want them to be uh in this Json format with these types and in exactly this schema so we can just tuck this away and now we can actually just go ahead and run this query so what this will do in the back end data bricks will make this efficient fast and scalable and now you can run this across your entire table without having to build or stand up any AI infrastructure on the side so we'll just give this a second to run here awesome so in just a couple of seconds you see that we have now taken these key pieces of information and thanks to AI query and llama 33 we now have these nice Json objects uh with our new features now we can also use some SQL functions in order to look at these a little bit more cleanly and unpack them so now you can see that for each review we've broken out the location a service score a product score the product names that were mentioned and then we have our structured review which is the Json object in its entirety and then of course the original review just so we can verify awesome so we've extracted some key pieces of information now how do we actually take action well we've set this urgency field let's use it so we can query this data again and then this time we're going to look for any reviews that have urgency of high so if we run this you'll see oh we do get a bunch of stuff back now it looks like these reviews got really bad service scores for a bunch of our different products but you'll notice that it's all from one location let's go ahead and take a look at what this review says oh boy avoid like the plague right it looks like someone was very unhappy with their experience at one of our stores so this urgency of high looks like it was pretty accurate now with this information in hand we can actually go and start remediating this make sure that this store is properly following company policies making sure that it's clean and that their employees are well trained and now thanks to AI query and batch inference you could see how this could become a pipeline where on some Cadence maybe daily you can check just the high urgency see reviews and have a much better pulse on how well all of your stores are actually performing so to wrap it up batch inference through AI query lets users apply AI to reason over their data where it lives in an efficient governed and scalable way which helps make data bricks a true data intelligence platform
Learn more about the Databricks Data and AI platform.
The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.