Intelligent Document Processing with Lakeflow: Transforming Unstructured Data for Agents & Analytics

Summary

Even frontier AI agents score below 50% on the OfficeQA enterprise document benchmark because they struggle to accurately read and extract information from real-world documents like invoices, contracts, and forms at scale.
Databricks Document Intelligence, built on Lakeflow, provides standard and high-accuracy parsing modes, structured extraction via natural language instructions, and multi-source connectors for SharePoint, OneDrive, Google Drive, and email.
This video walks through building end-to-end intelligent document processing pipelines that take raw PDFs to rich answers in Genie, enabling more accurate agents and analytics without maintaining a fragile custom tech stack.

Intelligent Document Processing with Lakeflow: Transforming Unstructured Data for Agents & Analytics

A huge portion of the data that runs your business doesn't live in clean tables — it lives in millions of documents: invoices, contracts, forms, and filings. This is some of the richest context your enterprise has, and today's agents finally have a shot at unlocking it. But there's a catch: agents can't actually read your documents. On our OfficeQA benchmark, even frontier agents score below 50% on real enterprise document tasks because they struggle to read them. Close that gap, and every agent, dashboard, and decision downstream gets sharper. In this video, we'll show how Document Intelligence on Databricks turns unstructured documents into accurate, governed insights that power more accurate agents and analytics. We'll walk through how to build intelligent document processing pipelines with Lakeflow and AI Functions purpose-built for document understanding — taking your documents from raw PDF to rich answers in Genie. Walk away knowing how to turn your document corpus into the data layer that powers smarter, faster, and more trustworthy agents and analytics.
Talk By: Archika Dogra, Product Manager, Databricks ; Jason Ping, Product Manager, Databricks ;

Chapters

FAQs

Why do AI agents struggle with enterprise document processing?

Even frontier AI models score below 50% on the OfficeQA enterprise document benchmark, which tests real-world tasks like reading invoices and contracts. At enterprise scale, challenges include documents arriving from multiple source systems, managing high file volumes, and maintaining accuracy when stitching together OCR services, visual language models, and parsing libraries.

What is Databricks Document Intelligence?

Databricks Document Intelligence is a built-in capability on the Databricks Data and AI platform that provides standard and high-accuracy parsing modes for unstructured documents such as PDFs. It supports structured extraction from natural language instructions and integrates with Lakeflow to build scalable document processing pipelines from multi-source connectors.

How does Lakeflow support document ingestion from multiple sources?

Lakeflow Connect provides managed connectors that incrementally ingest documents from sources including SharePoint, OneDrive, Google Drive, and email. This eliminates the need to build and maintain custom ingestion pipelines for every source, which often breaks down at volumes of tens of thousands of files per day.

How can processed documents be queried with natural language on Databricks?

Once documents are parsed and structured data is extracted into Unity Catalog tables, Genie provides a natural language Q&A interface so analysts can ask questions directly against the document corpus without writing SQL. This turns a raw document library into a governed, interactive data layer for both agents and analytics.

Full transcript

[00:07] Hey everyone, my name is Jason >> and my name is Archica. >> Thank you so much for taking the time to attend our session. Intelligent document processing with LakeFlow transforming your unstructured data to for ready for agents and analytic use cases.
[00:20] It's just a standard forward-looking statement. And also please remember complete your surveys after each session really helps us out. Awesome. As a bit of an agenda overview,
[00:32] over the past year, we've been talking to hundreds of customers. So, we'll start off by sharing a typical journey that we often see when customers try to implement document processing applications on their own. Then, we'll talk about how simple it is to ingest
[00:44] unstructured data onto data bricks. After that, we'll walk you through how to make your documents agent ready through parsing and processing. We'll show how deploying document intelligence at scale can be done
[00:56] incredibly easily. And finally, we'll finish up by sharing what's next for document processing on data bricks. I want you all to meet Maya. Maya is a data engineer at a software company
[01:08] called Spark Bricks. Her engineers are all token maxing. AI usage is through the roof, but their budget's thin, and so her seuite executives want to understand where the money is going and where they can save.
[01:21] So Maya's job is to process the company's invoice data, ingest it, extract out the spend, and build an agent and a dashboard so that the seauite can ask questions and get reports. Should be simple enough, right?
[01:33] Maya starts off vibe coding. Things are going well. She's a system that was able to read a few locally uploaded PDFs and she got a rudimentary dashboard working. Things were looking good at first, but when she takes a deeper
[01:45] investigation at some of her findings, she realizes that some of the numbers are missing or some of the insights were entirely wrong. To fix the quality problem, she tried stitching together Python libraries, OCR services, visual language models, and a bunch of other
[01:57] tools. And suddenly she realizes that her tech stack is getting increasingly complex. And when it came to graduating beyond just using a local sample set, she realizes that invoices were coming from
[02:10] everywhere from Sharepoint, from One Drive, from Google Drive, and even from emails. She spends hours creating incremental ingestion pipelines to each source. And her pipelines start to break down and slow down once she's handling volumes of tens of thousands of files a
[02:22] day. Now she's on call 24/7 when things break, which they always do, to figure out what changed and where. schema evolution changes, change data feeds that don't load properly, corrupt files,
[02:35] rate limiting, random changes of thirdparty APIs, you name it. And of course, hearing about her project, now every other team wants Maya to build them an internal chatbot, an external chatbot, AIdriven search, AI
[02:48] powered dashboards, and 17 other AI agents. Maya was exhausted from just setting this up once. She never expected to deal with any of this and simply can't move fast this way. So, we just watched Maya struggle a lot.
[03:02] And the unfortunate reality is that Maya is not alone. We've heard the same concerns from hundreds of customers over the past years. First, their documents live across scattered sources and they struggle to ingest them into one place.
[03:15] Then, once those documents are consolidated, they struggle to get their agents to actually accurately reason over them. And lastly, this all breaks down at scale. The issues that they hit at a 100 documents look very different
[03:28] than the ones that they hit at a million documents. And all of these problems are particularly important to fix today because your documents hold some of the most important enterprise contexts that
[03:40] you have. 80% of enterprise data is in unstructured formats and this number is rapidly growing. There are 2.5 trillion new documents created every single year.
[03:56] Every contract, claim, invoice, filing holds really important context that your business operates on. And this holds true across every single industry. We've worked with healthcare customers generating insights from patient records to financial service companies and
[04:09] customers extracting information over complex financial filings and even manufacturing customers processing logistical documents, invoices, and more. And everyone wants in. In today's world,
[04:23] businesses expect their teams to use the data that lives in these raw, messy documents to do things like making existing business workflows more efficient and automated, fueling search
[04:35] systems and analytics, and serving as the backbone and context for the decision-making agents that they want to deploy across their enterprise.
[04:48] So let's see how we can help some of these customers generate more value in particular Maya uh faster on data bricks from their enterprise documents. Let's revisit Maya's first problem scattered sources. Jason, would we happen to have
[05:00] anything here that would make Mia's life a little bit easier? >> Absolutely. Today, for those of you who aren't already aware, we'd like to introduce you to LakeFlow Connect. For those who aren't familiar, LakeFlow Connect is our native ingestion platform
[05:12] that makes it incredibly easy to bring your data into the lakehouse. Layflow Connect connectors are simple. They take just a few steps to set up and they're fully managed. This means in so
[05:24] far as we have the ability to solve a problem, data bricks takes on the responsibility of holding the pager when something goes wrong. This means automated schema evolution, data lineage, automatic retries, error handling, connector upgrades when
[05:36] thirdparty APIs change, change data feeds, and many more features designed to lower your maintenance costs. Leaflow connect is also unified with the rest of your lakehouse. So you can do ingestion in the same place where you're doing orchestration, analytics, and ML
[05:50] without needing to learn another platform and it's all governed with handling by UD catalog. And finally, our connectors are efficient end to end. Incremental reads, incremental writes, incremental transforms to ensure that your pipelines
[06:03] are as efficient as possible starting on day one. So now let's just take a look at just how easy it is to actually get started using LakeFlow Connect.
[06:16] So let's imagine we're Maya. We're given a SharePoint URL to a subfolder full of PDFs. We can take a look at what some of these PDFs look like. And so here's an example of one of the invoices. We'll go to another and we can go to a third. What we're
[06:28] seeing here is that these PDFs are pretty messy. Many of them are just images taken by someone's phone and because they're coming from different vendors, they have different structures, different tables, lopsided tables, barcodes in different locations, as well
[06:40] as different naming for key entities. And this is a pretty classic example of the document use cases that we're hearing our customers are dealing with. So, let's see just how easy it is to get this data in data bricks.
[06:53] First, I'm simply going to go to our add data page. I'm going to scroll down until I see the SharePoint tile, and clicking on it opens up this nice wizard. On this first step here, I can choose
[07:05] from either existing UC connections I already have access to, or I can make a new connection. UC connections are UC governed objects that contain the credentials that allows data bricks to access your external data sources. Creating a new connection is simple.
[07:18] Simply hit create connection, give it a name, copy and paste in some information related to your SharePoint site, and then you can sign in via OOTH. So, I've actually gone ahead for the sake of time and already pre-filled out
[07:30] this wizard. So, in step one, I chose a UC connection I already had. In step two, I simply give the pipeline a name as well as choose a location for my event log. And we see here I have two
[07:42] options after I can either ingest structured data. So these include things like Excels, JSONs, and CSVs, and I can land them directly in as their own delta tables or I can ingest unstructured data, PDFs, images, videos. Really good
[07:57] for these AI and document intelligence use cases. In my case, because our invoices are PDFs, I'm going to select unstructured data and then move on to step three. Now, in step three here, all I need to
[08:09] do is copy and paste in that subfolder URL that I was provided earlier. It's really simple. And once that loads, what we can see is we can actually browse and explore our SharePoint site within our data bricks wizard. And so I can check
[08:22] and yeah, this looks about right. These were the files that I wanted. So I can select the entire folder, give the destination table a name as well.
[08:34] Next, I just choose where I want my destination catalog and schema to be for my destination tables. And then finally, I can also set up my own schedules as well as notifications. If I wanted this to run, let's say once a day, once an hour, once every 30
[08:47] minutes based off of the different cadences of my use cases, then all I need to do is click save and run pipeline. And this is an example of what output pipeline you'll receive. So we see here
[08:59] is we we have uh a managed pipeline that is creating a destination streaming table where all our PDFs from SharePoint are landing into this destination table. Let's take a look at what that actually
[09:11] means. So here what we get is a file directory table where every PDF lands as its own row. Each row contains information such
[09:23] as the original path and URL to the file. Rich metadata such as any custom SharePoint tags, labels your business users might have added as well. Uh who
[09:35] created the file, when was it last updated, what was the original mime type of the file. But I want to bring your attention to this one column, this content column. Here, if we expand this out, we actually
[09:48] see that for each row in this column, this itself actually is the respective PDF's entire content simply stored as a binary format. We'll show in a little bit just how easily you can access this binary file. But otherwise, that's it.
[10:02] In just a few short steps, we're able to ingest our PDFs from SharePoint and land them into a file directory table within Data Bricks.
[10:17] Awesome. That was super cool and super easy. So now we've gotten the documents into data bricks, but the next step is to generate powerful, accurate insights from those same documents. And today we know that Frontier agents are remarkable. We've watched them write
[10:30] thousands of lines of sophisticated code and they've even passed the bar exam. However, for some reason, we keep hearing customers come to us and say that their agents really struggle to perform basic reasoning tasks on their
[10:43] raw enterprise documents. And we set out to figure out why. A few months ago, our research team gave three frontier agent harnesses a set of
[10:55] diverse realworld enterprise documents that often look like this and asked them to perform analytical reasoning tasks on those documents like this one to varying
[11:07] degrees of difficulty. What we actually found was that these state-of-the-art agents failed on these tasks over half of the time. Now the benchmarks were definitely difficult but our team had a
[11:21] different hypothesis something different from agent reasoning alone. That hypothesis was actually rooted in reading these enterprise documents that were often getting across invoices, claims and filings are messy, variable
[11:35] and riddled with complicated things like handwriting, mixed layouts, images, charts and mangled tables making them difficult for even the human eye. forget about the agent eye to actually
[11:47] comprehend, analyze and provide accurate insights on top of. So once we did this and realized this, we decided why don't we take this a step
[11:59] further and see what happens if we gave our agents glasses. In that same research paper, we tested that hypothesis. Instead of feeding the agent raw PDFs, this time we intelligently
[12:11] parsed those raw PDFs into structured text via AI parse document, which we'll get to in a second. Then we fed that same structured data to those agents instead of the raw PDFs.
[12:24] And we ultimately asked them to reason over the exact same tasks without changing anything in the agent harness. The results were pretty amazing. agents not only got smarter, but they got
[12:37] insanely fast. They had a better grasp of the data from the get-go that lived in that document, made less tool calls to accomplish these complex reasoning tasks, and they ultimately stopped going in circles. So,
[12:50] what we saw across the board across these three frontier models at the time of that benchmark was accuracy gains and speed gains and efficiency gains. And that's what really motivated us to
[13:03] build the best-in-class document intelligence solution on data bricks. Over the past year, we've been building out composable document intelligence building blocks to parse, classify, and extract key information from your
[13:16] enterprise documents. We set out to build these building blocks with three core goals. First, they need to offer state-of-the-art accuracy at lowest cost. Second, they need to be capable of
[13:28] operating at high volume enterprise scale. And third, we really wanted to be intentional about providing an end-to-end composable story to customers so that they have a repeatable doc uh
[13:40] playbook for every new document use case versus having to build a Frankenstein pipeline from scratch over and over and over again. Our R&D team spent a lot of effort training
[13:54] systems and harnesses and constantly updating every AI function with the latest and greatest techniques to really push the frontier on quality and efficiency. This enables us to get better quality at state-of-the-art
[14:07] levels and also push cost down at the exact same time via specialization. We've made a lot of progress in this domain over the past few months and I'm really excited to share some new
[14:19] benchmarks hot off the press for some of our latest um developments across these functions. Let's take a look at the latest benchmark for AI parse document that you see up here across the market. What we see today is that we perform
[14:32] with the highest state-of-the-art quality yet are the most cost effective solution at scale. So you can actually unleash these insights across not just a 100 to a thousand documents at the cost that work for you, but scale this up to
[14:45] hundreds of thousands to millions of documents. And we're doing this across the board. We see a very similar story for AI extract, which performs document extraction tasks as well. On this cost
[14:57] versus quality graph, you see two modes. Our standard default mode and then a new precision mode that's coming soon that's tuned for some of your enterprises most difficult and complicated document extraction tasks. No matter which mode
[15:11] our customer selects based off of the complexity of their documents, both points sit strictly at the front of the Pareto frontier for document extraction.
[15:25] And so the benchmarks look good and the story sounds great, but let's actually see this in action. So we left off with Jason ingesting those documents into data bricks into a file table. I'm going to go ahead to the agents page
[15:37] on data bricks and we'll get a look at what's in these documents. I'm going to click create agents, go to functions, and I see I have a few different use cases here, but I'm going to start off with document parsing because this is
[15:49] the number one task that a lot of our customers are trying to do. Turn these raw PDFs into something that's ready for downstream agents and analytics. I'm going to actually go ahead and select the table that was created from Jason's
[16:02] LakeFlow Connect ingestion pipeline. We're going to use this table and we're going to start parsing. This is going to take a couple of seconds as it's taking in those documents and it's rendering
[16:14] what exactly we parsed out of them. Cool. Awesome. So on the left hand side here we see one of the invoices again slanted scanned a little bit hard to to read a bit blurry. Um and then on the
[16:28] right hand side we actually see a really easy way to see what exactly we parsed out from every element we detected on that invoice. So every element I hover over, it'll map to where that lives on that invoice on the lefth hand side
[16:41] here. Also, if you want to go and see the actual structure data that we generate through AI parse document, you can click on the JSON tab here, which kind of shows you what we're doing. For every document and every page, we have our elements that we're detecting. We're
[16:54] giving you coordinate bounding boxes of where that content lives on that page, which page number it's on, what the content we extracted from that element actually was, and we're providing you some really additional valuable signal
[17:06] on the confidence score of the parsing for that given element alongside the type of the element. For example, if I scroll down here, I can see that this table is a bit mangled
[17:18] and hard to really comprehend, but we do a pretty good job of turning this into a structured table across all the different merged cells that we see in the table. And then if I scroll even lower, we can see here that we have an
[17:30] image here, which is a c a barcode. And we've actually flagged that as an figure and given you an intelligent description of what that image actually looks like in that document. So here we actually figure out that this is a black and
[17:42] white barcode that can be used for any downstream agentic workflow that you want to build on top of the parsed text. So this is super powerful, but this is just step one. Now that you have the structured data that your agents and
[17:54] analytical systems are going to do a lot better on, there's a lot of things you can do downstream. Build search systems, retrieval systems, agents, or extract key information from these invoices that you care about. And we know that that's
[18:06] what Maya wants to do. So, I'm going to go back to the agents page. And as you might have seen the first time around, we have the information extraction task right over here, which is the next thing that I want to do. I'm going to click
[18:20] that and I'm going to add my table or my data from a table here as well in my invoices schema. And then I'm going to choose the column that contains the content of those
[18:33] documents. I'm going to go ahead and click create agent. And what's going to show up once this loads is it's going to process my documents on the right hand side. We can kind of zoom in here and it's giving you
[18:45] an opportunity to define what you care about extracting from those documents that you're giving this extraction agent. So I have a natural language description already kind of ready here that pretty much explains if I'm Maya,
[18:58] I'm telling it that I want to extract the invoice ID, the date, the line items. there's a bunch of really valuable signal that I want to come out in a structured schema when I run this task. So then I'm going to click generate schema and we're going to
[19:10] convert that natural language instruction into ultimately a structured JSON schema and take a first pass at some sensible descriptions for every field to help you hill climb quality faster based off of what we know from
[19:23] your description and the documents that we already have on hand. Before I save and run extraction, we have some really awesome capable features here like citations and confidence scores to give you additional signal and metadata on
[19:36] how well we're doing at extracting every single field that we're going to run this task on. And so now I've clicked save and run extraction and enabled citations and confidence scores. So this is going to
[19:49] return to me the document and then also what fields we successfully extracted from it alongside that additional signal. And we already see some results streaming in here. Awesome. Okay, let's maybe take a look
[20:03] at one of them. For example, this one side by side, I see again the parsed text output of the document that I gave this agent. And then I also see on this side the extracted terms and data points
[20:17] um in the structure JSON schema that I wanted to specify. And if I hover over here, so for example, I have vendor name mobile bricks. I can see a confidence score associated with that exact field
[20:29] and I can also see a citation of where we found that in the document. And this is particularly useful when your documents are really long and complicated and there might be multiple places that the same content could live in. And so this is really helpful for
[20:42] workflows like that. Okay, I think this looks pretty good. Seems reasonable to me, but I can't go to production yet. If I went to my boss and I was Maya and I was like, you know, I checked five documents, everything looked good. let's
[20:54] send this to production. There would be a lot of questions. So,
[21:08] give you guys the first sneak peek into a new feature that's coming to information extraction within the next few weeks. What we're going to be able to do now is I can evaluate extraction quality with maybe a set of ground truths or golden data that I might
[21:20] already have. um whether that's five to to 500 documents and ground truths. And so I'm going to go ahead and add an evaluation data set which I'll find here. So these are my
[21:32] ground truths. Then I'm going to provide the input column and I'm going to point the agent to the expected extraction and I'm going to go ahead and kick off an evaluation run. So what's happening in
[21:44] the back end is that this is creating an MLflow evaluation run based off of the inputs that you provided the ground truths that we are now aware of and then ultimately what got extracted by the agent in the version one of the agent
[21:57] that you built. And it's going to show you this um show you the results across these three different data points. And so we're going to wait a little bit for the first document to load. But we're going to see here that this is starting to stream in. And the results are
[22:10] looking pretty valuable to me. I can see which fields are matching, the mismatch fields, etc. But what this is going to switch me over to ultimately is this overview page that does a really helpful job of telling me how good is my current
[22:22] system at extracting data from these documents. So we see an overall field accuracy of the percentage of fields across every single document that were extracted correctly. And then we see a secondary signal of how many documents
[22:36] were 100% correct, which means that every field within that document was extracted completely accurately. And so I'm kind of like, you know, this looks good. Like, but in the old world, I'd have to go and figure out what was going
[22:49] wrong, where it was going wrong, and what I need to do to enhance my schema and descriptions to actually tune it better to my task. But something that's really exciting and brand new to this product is a Genie Code integration.
[23:01] Genie code went ahead and actually analyzed the same evaluation run and is servicing to me straight off the bat some recommendations for how we can improve our system because it knows my intent and knows the outcome that I'm
[23:14] try that I'm trying to achieve and it can that way analyze and recommend the right improvements for my document extraction task. So I don't need to spend days to weeks doing this myself. So we see a few pretty helpful
[23:26] suggestions here. I think this looks pretty good. So I'm going to go ahead and apply all five fixes and it's going to start kicking off this iteration loop completely driven by Genie code. So I
[23:38] can just sit back here and relax. Can maybe look around or I can actually see what Genie Code is doing real time if I want to get a better understanding of how it's approaching this problem. And here it's pretty transparent. It's saying here's the fixes that I'm going
[23:51] to be applying by the way um just for me to have a good understanding of. and it's rendering the right UI at the right moment in time for you to see that side by side that it's actually going and showing five edited fields that it
[24:03] decided were going to be the highest levers for quality improvements and what those edited fields are actually going to look like. And now it's going to ask you for some permission like, "Hey, now that I've given you the edited fields and I've showed you what I'm trying to do here, um, are you down to update the
[24:16] configuration and actually rerun the evaluation?" And I'm going to click yeah, of course, because it knows that, you know what you're inevitably going to do after you iterate on quality is you're going to want to re-evaluate and
[24:28] benchmark that progress versus the last version that you have. And speaking of versioning, we have that built into the product as well. You can see up here that we're on version two of our information extraction function and we
[24:40] can pretty much track all of our versions here. Um, compare them side by side and restore them in case we run into any regressions along the way during this iteration loop. So now Genie Code is rerunning this
[24:54] evaluation and let's see how well it did. Oh, that's pretty good. It got us up 10 points or I guess 11 points in accuracy to now 98%. And it's telling us that a lot of our documents are now
[25:06] fully correct, which is really awesome. And again, I really didn't have to do much. I just had to give Genie Code my intent and what good looks like and it can figure that out and reason on this on its own. So, I'm feeling pretty good
[25:19] about this. I think now I can maybe go to my boss and say, "Hey, I think this is ready to go to production." Um, so I'm going to go and click use agent here. And the next thing I'm going to want to do is to create a Spark declarative pipeline.
[25:32] >> Awesome. So now let's talk about Maya's final problem, the difficulty with scaling. That's where the combined value of Lake Flow and document intelligence really starts to shine. As Archer described,
[25:44] our AI functions are what power the parsing extraction agents that we just saw. AI functions are research optimized building blocks that can easily be applied and chained together so you can use AI transformations at scale. These
[25:56] AI functions are automatically integrated with the intelligence of Spark declarative pipelines, which means that we can ensure as new documents are coming in, only the new or changed documents are being incrementally parsed and extracted, ensuring that your
[26:08] pipelines stay efficient. So where Archa left off, she was about to create a pipeline. Let's take a look at what that would have looked like had she clicked it.
[26:21] Here is an example pipeline made from our AI parse and extract agents. What we see is that our document parsing and extraction UI actually creates Spark declarative pipeline code for me. So if we zoom in, we see that first we're
[26:33] simply parsing our documents by defining a silver table and calling AI parse oh AI parse document and then passing in that file content column from our Lakeflow connect table I
[26:45] mentioned earlier. And then we define it as a new column called parsed. And that's it. Parsing our documents really is just as simple as a single line of DBSQL code. But once your documents are parsed, your world's your
[26:57] oyster. You can use your parsed data as you would any other typical structured data that might live in a table. We took it one step further and we chained together another AI function, AI extract. To use it, all we needed to do
[27:09] was define another table, call AI extract, pass in that parse column that we just defined earlier, and provide the natural language schema definition that Archa was able to make using the help of Genie code. And again, all of the Spark
[27:23] declarative pipeline code was already pre-created and pre-populated for us just from using the UI. You're given production level code that you can inspect, trust, and edit to your liking. All of which can be initially set up through simple and intuitive no code
[27:36] solution. We can also view this pipeline via the pipelines monitoring page here. I'm sure many of you guys have seen this before. We can set notifications, track run successes, monitor performance, and
[27:48] whatnot. And now that we have these two structured silver and gold tables, something we can do is make an AIBI dashboard. Now Maya can simply present this to her seuite executives so they can learn more
[28:00] about where their money is going beyond tokens. And there we go. We now have a dashboard purely powered by PDFs and SharePoint. We can do another step and orchestrate all this together easily using LakeFlow
[28:12] jobs. First, we just create a new job. Then we select the injection pipeline as task one. We create another task. Select the
[28:24] parsing and extraction pipeline which we just saw was created. And let's say I want to add another one. All I need to do is say add dashboard task. refresh dashboard and select the dashboard that we just
[28:37] had and save it. Great. I can add schedules as well as triggers. Let's say I want this once once an hour feels about right. And then I can go ahead and click save as well. And there
[28:50] you have it. Based on that schedule, once an hour, what's going to happen is it's first going to kick off that ingestion pipeline. They connect is going to go and see, hey, are there any new files or any files that have been changed? If so, we'll bring in just those changed documents. We'll then
[29:02] automatically kick off the parsing and extraction of these documents. And finally, we'll refresh the dashboard as well. All orchestrated so that you can set it up once and truly set it and forget it.
[29:17] >> Okay, so now we have like a pretty helpful dashboard, but if I'm the CFO, I don't want to just look at a dashboard. I can see, you know, a few graphs. Okay, our spend is definitely going up. There's a few vendors and categories that we tend to be spending a lot of
[29:29] money in. But this doesn't really tell me like a full nuanced analytical picture of my business and how we're spending money across the board and what constitutes waste versus what doesn't. So you guys all came from the keynote
[29:41] and you probably heard a lot about Genie1. So let's see how Genie 1 does at this now that we have this structured data that lives in data bricks. I'm going to go up here to Genie 1 and I'm going to go ask it a quick question. I'm
[29:53] gonna say uh I'm the CFO, if I can spell properly, of Spark Bricks. Where is my enterprise wasting money and why? So, I'm going to
[30:07] send it off to the races and a Genie one is now going to parse through all of the data that it has and is context aware of and then figure out the right data sets to pull the right information of and then give us this consolidated report on
[30:21] what it knows about our business, our invoice spend and anything else that might be relevant to tell us how we can optimize our spend across the board. So, this is a pretty sizable effort. Genie might take a minute or two to get this
[30:33] done. So, I've kind of already asked Genie Genie one this uh earlier this morning. So, we're going to go over to this report that I generated with pretty much a very similar question above. And it's giving me this spend analysis
[30:46] report of where my money is going over the past 18 months. The first thing that it detects is that there's a lot of AI and ML tool usage and this is exponentially growing. But a
[30:58] really key insight that it's seeing is that we have 12 vendors now active. So there seems to be a significant amount of spraw here. It's telling me actually a recommendation that maybe I should start consolidating my AI tools to a few
[31:10] strategic vendors and then I can go ahead and go in and negotiate enterprise discounts because I have high volume usage more concentrated across one to or three to four strategic vendors.
[31:22] Something else that it detected that I might have not realized was that for some reason across my company, we're uh spending money on nine different issue and project tracking tools. We have track bricks, board bricks, bug bricks,
[31:35] task bricks. There's too many companies that are doing the same thing for my one enterprise. So another recommendation here is to also consolidate this spend to really drive that down. Another super interesting insight and
[31:48] this really reflects how it's uh reaches across the enterprise context that it's aware of is that we're actually spending more money on office facilities um but we have less on inoff attendance over the last few months. So maybe we need to
[32:01] figure out why there's a discrepancy in increased spend on office related expenses with less people going into the office.
[32:13] And lastly, it seems like we're not doing a very good job of paying our invoices on time. So, seems like an easy enough fix, but ultimately what we're here to show you is that Genie1 did a really good job of summarizing these insights and recommendations from the
[32:25] data that you provided it with those documents living in data bricks, structured insights from those documents living alongside them and other structured data that might be valuable to contextualize spend within the enterprise that Maya works for.
[32:45] Awesome. So today we went on quite a journey. We started with Maya's first problem. Her enterprise invoices live across different external sources. And thanks to Likeflow Connect, we made our life a lot easier in just a few clicks. Then once she reeled in the documents
[32:58] into data bricks, we learned that agents and LLMs aren't great at reading enterprise documents accurately. And we use document intelligence to come in and help Maya unlock high quality insights from unstructured documents.
[33:10] Lastly, this all looks great on a few hundred documents, but Maya really needed to orchestrate this end to end and robustly at scale, which is where Spark declarative pipelines come in and give us everything that we need here.
[33:23] The TLDDR is that your agents are only as good as your document intelligence layer. And we've worked super hard to make this endto-end story best-in-class and unified on data bricks.
[33:36] >> So, what coming next for Lake Connect? We're building out hundreds of more sources. Immediately on the road map, which you'll see in the few weeks to come, will be one drive, Box, and Dropbox, just to name a few. We're also working on built-in permission handling.
[33:48] That means inheriting your source permissions from your files directly in a unique catalog when you ingest them for for governance use cases. And finally, we're working on agentic user experiences so that you can create production level injection pipelines via
[34:00] Genie. And then for document intelligence, you guys got a glance at some of the things that are coming super soon, like our built-in managed evaluation experience coupled with Genie Code for improving your document extraction results. And then this new
[34:12] document extraction precision mode for the most complicated, long, and extensive documents that we see our customers bring to us today. That's going to be going out into a research preview really soon. And alongside document intelligence, we're working on
[34:24] a lot of other capabilities for helping you turn your unstructured data into structured insights, whether it's audio transcription, search, or enrichment within your batch pipelines that are going to be landing shortly. We also have real-time document processing APIs
[34:37] that are come going to come out soon. And all of your AI functions are soon going to be governed by Unity catalog. >> Great. So, we know that we just threw a lot of information at y'all, but don't
[34:49] worry. We created an online tutorial where you can follow along with a demo walkthrough, demo video, and demo code. Just scan the QR code, or you can visit the bit.ly link. Uh, it's bit.ly/databicks-docintelligence.
[35:03] And if you want to get even deeper into some of the things that we showed you today, we have another session coming up later this week on building agent ready data at scale and really going into some of the best practices for intelligent document processing from the moment that you start building your pipeline to the
[35:16] moment that you want to run it in production. And so we have a session on Wednesday uh Wednesday evening, but it seems like that one got overs subscribed. So we have another session on Thursday. And this is a really good opportunity to learn how we've worked
[35:28] with customers and actually help them operationalize and productionize these workflows on millions of documents every single week. >> Great. Thank you all so much for taking the time out of your day.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.