AI Agents in Action: Structuring Unstructured Data on Demand With Databricks and Unstructured

Summary

• AI agents for data workflows: This video shows how LLM agents autonomously ingest, process, and structure unstructured data using Unstructured and the Model Context Protocol.
• Dynamic document workflows: Agents discover documents across sources, build ephemeral workflows, and export structured insights directly into Databricks Delta tables.
• Operational efficiency: The system responds to natural language requests, transforms unstructured data into usable formats, and surfaces insights rapidly without manual intervention.

AI Agents in Action: Structuring Unstructured Data on Demand With Databricks and Unstructured

Talk by Christopher Maddock, Head of Product and Engineering, Unstructured
LLM agents aren’t just answering questions — they’re running entire workflows. In this talk, we’ll show how agents can autonomously ingest, process and structure unstructured data using Unstructured, with outputs flowing directly into Databricks. Powered by the Model Context Protocol (MCP), agents can interface with Unstructured’s full suite of capabilities — discovering documents across sources, building ephemeral workflows and exporting structured insights into Delta tables. We’ll walk through a demo where an agent responds to a natural language request, dynamically pulls relevant documents, transforms them into usable data and surfaces insights — fast. Join us for a sneak peek into the future of AI-native data workflows, where LLMs don’t just assist — they operate.
Here’s more to explore:
Production ready data pipelines for analytics and AI: https://www.databricks.com/solutions/data-engineering
The Big Book of Data Engineering: https://www.databricks.com/resources/ebook/big-book-data-engineering-2nd-edition
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

Chapters

• 0:00 Introduction to Unstructured
• 1:11 Data pipeline challenges
• 3:15 Unstructured platform tech
• 5:56 Model Context Protocol
• 8:29 Agentic workflows
• 11:27 Multi-agent demo
• 14:18 Conclusion and call to action

FAQs

What does the Model Context Protocol enable for AI agents?

MCP allows agents to interface with Unstructured's full capabilities, discovering documents, building workflows, and exporting data. This enables agents to operate autonomously on unstructured data at scale.

How do agents handle multiple document types?

Unstructured transforms PDFs, Word documents, images, and other formats into canonical JSON schemas. Agents can then process these consistent representations to extract structured insights.

What is an ephemeral workflow in this context?

Ephemeral workflows are temporary, on-demand data processing pipelines that agents create in response to natural language requests, process documents, and then terminate—eliminating static infrastructure overhead.

How does this approach improve data workflow productivity?

Rather than building custom connectors and pipelines for each data source, agents can dynamically discover, process, and structure data in response to business questions, reducing development time and enabling self-service analytics.

Full transcript

0:00 Introduction to Unstructured

Everybody got me? Okay, great. Right, let's dive in. So, let's go over what we're going to talk about today. who our little company is. Uh a look at model context protocol and how we use it and apply it unstructured aentic workflows a deep dive into that and then I'll walk you through our implementation of it and what you can expect from it as well. So unstructured is just coming up for its third birthday. I lovingly refer to what we do as the most boring part of generative AI.
Nobody in Gen AI does anything more boring than we do. We take unstructured data where it lies. We connect to it. We transform it into canonical JSON schemas. And we prepare it for consumption by Genaii models. GM 3 on July the 4th, USA. It's picked up 10,000 paying customers. There's now 35 million product downloads. We have 87 of the 100 in ongoing usage of this software. Mostly because nobody else wants to do it because it's terrible, terrible work.

1:11 Data pipeline challenges

This is the work we do, right? We have a fundamental belief that if you want great results out of AI, you need to put great data into it. 80% of the data, I'm now hearing 90% of the data from Gartner is unstructured data. We bring that to bur. So think about traditional modern data stack, Excel sheets, CSV files, rows and columns into tabular data stores. This is about everything that didn't fit in those data stores. images, audio, video, PDFs, word documents.
And this is effectively what it looks like for us. Call it the rat's nest. And I see this in company after company after company. You've got some data that lives in your world somewhere in your system of record. Maybe that be S3, Google Drive, or something along those lines. Then we're going to connect to that data. We're going to transform it by identifying if it's a PDF, it's a PowerPoint, it's a Word document, it's a text file, it's a log file into our opinionated canonical JSON schema.
Once we've done that, we're going to clean it with regular expressions and a bunch of other custom processing scripts. We're going to chunk it, we're going to create embeddings, and we're going to write it into downstream vector storage. So, we have there's nothing here that you can't do yourselves, right? It's just utterly utterly miserable to do it yourselves. So what I see all the times when we talk to customers is either they've built this themselves and they're like oh I wrote a quick connector to S3 I grabbed some data I transformed it all looked great I'm happy and then they get told ah I've got to get data from SharePoint now I need PDF files now we need PowerPoint files so we see this growth of this code I've talked to companies that have literally had 50 engineers maintaining this stack unstructured exists to just make that easy for you so I say we sit to the left of graph graph and vector storage.
We connect to data where it lies. We transform it into a canonical usable format and we write it into graph and vector storage. We do that on loop, expose it to your LLMs for retrieval, for model training, for whatever your use cases may be. This is what it looks like. So there are today around 50 connectors that are available right now. Our connector framework is open source. You can contribute them yourselves. We work on 67 different file types.

3:15 Unstructured platform tech

Once we've picked up the data, so we've grabbed all these file types. There's a pre-processor here. Its job is to check out the file and understand what it is. If the pre-processor says, "Hey, I've got a word document." We know how to get the data out of a word document because it has a common schema. So, we'll take that data with just straight Python heristics, regular expressions. You really want our fast strategy to work.
It's the quickest way to get the data out. is highly accurate. It costs nothing. Hello mate. Uh high- res strategy is based on object detection and optical character recognition. So that works on the PDFs that don't have extractable text and we need to bring machine learning models to bur. If we can't get the data with those then we'll look at a VLM. We'll say how can we get this data out with a large language model.
We do this in cases like maybe last week at the snowflake summit some shouldn't say that name should I here people are drawn a hand written table with Arabic in it like the only option for us is LLM so LLM is a solution for when things get very complicated but we don't want to use that all the time because the most expensive and the most latent all we're always driving to the cheapest cost of performance what you get out is the canonical JSON schema if it's PDF PowerPoint whatever it's always the exact same schema and then we offer enrichment Python nodes can run on the data.
LLM based enrichments. You can plug in your own API calls. We spit out at the end chunks embedded JSON that can write directly into Delta tables. Along the way, we connect to any third party models. We got integration with together, Voyage, Vert. Ex, Bedrock, OpenAI. We don't care. We're not opinionated about models. If we use a third party model, we charge you 0 on the pass through. So this is what it looks like when we take a document.
We'll pick up a complicated PDF title text favorite view of your table. A pre-processor will grab it. It'll say, "Hey, I've got some text." That'll go to OCR processing. It'll say, "Hey, I found an image." It'll base 64 encode that image and write it into the JSON document. And there's an option for LLM based enrichment of that too. What was in the image? See a summary of it. Write that out. any tables we detect are transformed into rows, column cells.
That data gets reassembled back together and written out into the same canonical JSON schema. Okay, so that's who unstructured is that's what we've been doing for the last 3 years with our little company last October. The NTP spec came out. I was uh interested in it. So I had a read through it. I thought this could be compelling for us. It's a way for the LLMs to directly talk to the tooling and the connectors that we provide.

5:56 Model Context Protocol

I didn't think about it for about three months until my favorite venture capitalist and investor called me one weekend and said, "Hey, you need a solution for MCP fast. Do something now." So, like a all good heads of engineering, I spent all weekend vibe coding our first MCP connector. It's simple in what it did at first, right? It just said, "Hey, here's our API. We're going to talk to a natural language." I integrated it with cloud desktop and it worked.
I could talk to unstructured in language through cla unstructured pick up all of my files from the S3 bucket, run the workflow, chunk it and embed it and write it into my vector storage and have it comply. Then I did the next best thing that all good engineering leaders do. I say, "Hey engineering team, I built this in two days, vibe coded the hell out of it. Why can't you deliver me a full working version by Wednesday?"
So we had some fights said it's not magic you can't vibe code everything in the world MCP is basically just an open standard for how we talk to tooling right it brings LLMs closer to the tools capabilities that the my team have began to expose through it are access to resources ability to prompt it and tool control so they started standing up the actual version of the unstructured MCP server that now lives in our GitHub if you look at docs. structure on structure. io.
It's all listed out there. The whole thing's accessible to you right now and downloadable and works. And you can make requests like this. Transform these PDFs into JSON. Convert any data from this S3 bucket to JSON. Chunk it. Embed it. Write it to vector storage. The full tool list of what it does. You can see here. Create a workflow to move data. Create a connection to data. Delete workflows. You have full control. Everything that's available through our Python SDK.
Previously where you'd write code to interact with unstructured is now available through natural language interface. There's the tooling is integrated directly into cursor. We use it ourselves inside our little company. It just makes it phenomenally easy to transform your data, get it into a format that's understandable by our limbs. Okay. So now we'll talk about where we are going next with this. So the next iteration of what we're doing with it and what's happening behind the scenes at Instructured now and what all my engineers are busy at home doing right now is think of this more like the LLM talking directly to unstructured.

8:29 Agentic workflows

So the idea is that at some point an LLM can come along and we won't have pre-ingested all your data. Right now what I see from most of our customers use cases is hey I've got all this SharePoint data. I need a chat your data solution with an LLM. So I'll pick up all that data. I'll transform it to JSON. I'll chunk it. I'll embed it. I'll enrich it. I'll write it to vector storage. They'll point an LLM it. They'll do retrieval augmented generation on it.
Everybody's happy. Some of those systems work out well. Some of them work out at five times the cost of the business case value that they're providing because the sheer cost of number of steps and models inside them. So what we're trying to do is address that with MTP. So now the idea is that the LLM via our agent has control of the platform and it understands the data it's in there. So we have all our suite of 50 connectors available to you.
You make a request to it. Hey Claude, I need all of the HR data on the last performance review and instead of us having chunked and embedded everything and constantly refreshing it, the agents know how to connect to our connectors in the places where the tooling can make a search. it makes a search, identifies the documents it needs. In the places it can't, we're building cataloges on top of those connectors. So, we have document indexes of what is inside those data stores and then in quick retrieval straight back to the LLM.
So, we'll return back to the LM, hey, I think these are the documents that you need and these are how we'll answer the use case. You can reply to the agent, no, you've missed, you brought me 2023's data, 2020 2024, and it will go back, issue the queries across the top of connectors and drag that data back. So why this is better for our customers is that we're no longer taking on that massive weight of cost of ingesting all of the data. when it's a use case that makes a ton of sense and there's a ton of business value.
Yeah. Well, then maybe it's okay to spend $200,000 a month on compute to do this. But in a lot of those use cases, those lower value ones, the agentic approach opens it up. In some use cases for massive scale and search, we still do it the traditional way. It's the fastest way for us to do it. And those those quick hit things, we can save a lot of time and resource by doing it through agents. The idea is this like I talk about unstructured being ETL plus for generative AI whereas traditional ETL move rows and columns to rows and columns we move all of the unstructured data in the world to large language models.
What agents make it is that you don't need to come and log into my platform anymore, learn my SDK to connect your system to it to write integration code. You just straight away can start working with structured out the box day one. Okay, demo. I don't actually have an internet connection, I found out, which actually made me very relieved. For those of you who work with agents, it's great. 98% of the time, at least 2% of the time, they have publicly humiliated me.

11:27 Multi-agent demo

So I have a recording of the interaction talking to the unstructured agents behind. So you can see this is just a streamlit application sat on top of unstructured MCP server and its agents. So let me start this. So it's saying hey I'm the multi- aent chat system. What can you do for me? And then I've pasted in from the PDF documents in the S3 bucket. Can you give me an analysis of the strategic positioning of uranium production companies?
So the models replied and they said, "I think you're looking about uranium production companies and based on PDF context. I think I can get them from S3. Can you confirm this is right?" And we're saying, "Yes." And then the model is saying, "All right. Now, are you interested in specific geographic regions? Would you like any focus on time period analysis?" I'm like, "Not really. Five years." And get me anything from a PDF.
So now the model says, "Okay, I'm preparing to capture the most relevant information. Are you interested in any specific aspects?" So I'm going to talk to the model and tell it this is the data I need. This is the things that I'm most interested in. Now it goes away and another agent is called and it'll do task definition. So you see right now it's gone okay I know I'm outputting everything to delta tables. So it's defined a delta table on the fly.
It's grabbed in the data that it needs. It's transforming it into our schema and then it's going to route it into delta tables. Now it asks I'm going to operate unstructured. Do you want a new workflow or would you like to use an existing workflow? I say I think I've already got a workflow for S3 to delta table. So find one. So it lists all of the workflows that I've got available to me. And it will execute that workflow and start moving that data into delta tables.
And it says, hey, I moved all the data and here's the things that I've learned about this process. So we didn't have to first go and pick up 10,000 PDF documents and ingest them all and have this massive vector database which is a replica of my S3 bucket which is a replica of SharePoint. We're issuing these queries like on demand through this application then you can go on to have your typical LLM based interactions with it.
So agents for us are about how we can define another method of go to market. So we have the big option like I say move all the data into a vector database replicate everything. But if you've got lighter weight tasks like this and you don't want to ingest pabytes of data that are going to cost you a fortune, build indexes, work with connectors that offer real-time search, group that all together, manage the task with agents, and get it all back.
Okay, we have 5 minutes left. I'm bang on time. First of all, I am at D 106. It's like right down the other end. We have the best trucker hats and swag you could ever imagine. So, please come and help me get rid of the 2,000 hats. I really do not want to be shipping any of those home. You're welcome to five, six, seven of those things. And secondly, our little company is going bananas right now. If any of you are I don't even know if I'm allowed to say this, but any of you are like staff senior level Python engineers and you want jobs, please come.

14:18 Conclusion and call to action

I need you. Like booth D106 up there. Ask for. I run engineering and I will chat to anyone who wants anyone who wants a job at a high growth super exciting fun startup that thinks turning PDFs into JSON is a fun job. It takes a unique personality to join our company. If you're one of those people, please come and be bored with me and make this work for all of Genai. Thank you very much.

Learn more about the Databricks Data and AI platform.