Sponsored by: Lovelytics | Automating High-Stakes Finance Workflows with AI Agents

Summary

Lovelytics deployed two production-grade AI automation use cases for Lippert — lockbox remittance reconciliation and proof of delivery validation — saving over 250 hours monthly with more than 90% model accuracy.
The remittance reconciliation solution uses a deterministic extraction pipeline combining document segmentation, OCR noise cleaning, header normalization, and customer enrichment to produce finance-ready tables and exception-focused audit trails from multi-page vendor PDFs.
The proof of delivery automation combines Azure Document Intelligence, OpenCV smart cropping, and LLMs with a safety-first validation layer to handle challenging inputs like crossed-out quantities and signatures that standard OCR cannot reliably process.

Chapters

FAQs

What is remittance reconciliation and how was it automated with AI?

Remittance reconciliation is the process of matching vendor payment records — in this case from multi-page lockbox PDFs — to corresponding invoices in the finance system. The automated solution uses a deterministic extraction pipeline that segments documents, normalizes headers, cleans OCR noise, validates line items, and applies customer enrichment to produce finance-ready tables and exception-focused audit trails without human intervention.

How does the proof of delivery automation handle crossed-out quantities and signatures?

Standard OCR struggles with handwritten marks like crossed-out quantities and signatures on physical delivery documents. The solution combines computer vision via OpenCV smart cropping with Azure Document Intelligence and LLMs, plus a safety-first validation layer that verifies outputs before results are accepted into the finance system.

What results did Lippert achieve with these AI automation use cases?

Lovelytics' two production-grade automation deployments for Lippert save over 250 hours monthly at more than 90% model accuracy, as described in this video. These outcomes show how applying AI agents to routine back-office finance workflows can deliver measurable time savings at scale.

How do you evaluate whether a finance workflow is ready for AI agents?

This video presents a first-principles framework that starts with whiteboarding the full process before writing any code, mapping each step to understand what is repetitive, rule-based, and verifiable. Finance workflows are particularly well suited for AI agents because they are designed to be boring by purpose — consistent inputs, defined rules, and known correct outputs that make it possible to validate agent accuracy.

Full transcript

[00:08] First of all, thank you so much for uh coming for this session, especially after a big Matei keynote and uh right around at lunchtime. I was not expecting so many people, but uh glad I spent some time on my deck
[00:20] working on it then. So, thank you so much. Uh so, the topic today, and we only have 20 minutes, so I'm going to be pretty quick on this. Uh you can catch me maybe at my booth or uh sometime in
[00:32] the next couple of days. So, the topic today is kind of taking automating uh high-stakes finance workflows with uh AI agents. So, we'll get a little bit into why they are a
[00:44] very interesting workflow. And why they are very common across all industries, right? Everybody has finance going on. Uh so, my name is Sudhir. I'm the head of GenAI for a Databricks partner called
[00:58] Lovelatics. I am also a Microsoft MVP. I'm joined by my MVP peer Tejas over here, so he can also take some questions uh since I have to leave right away. Uh
[01:11] we are a Databricks consulting partner. All we do is Databricks, about 500 some people. Uh we have a booth back there, so visit us um and all of that type of stuff. Okay. Uh
[01:23] brief agenda. I know we only have 20 minutes, so I'm going to be a little bit ambitious trying to cover a whole lot of points over here. Um what is a agent-ready finance
[01:36] workflow? To that matter, any workflow? How do you qualify a workflow uh for AI agents, right? This AI agents technology is pretty cool. But just because you have a silver
[01:48] hammer, doesn't mean every problem should be solved using an AI agent. Uh what makes a finance workflow agent-ready and some first principle thinking uh uh on looking at workflows and uh AI
[02:02] agents and that type of stuff. Then we'll get into two specific examples. And I picked the most boring examples in finance out there. Uh but they're very effective. Sometimes
[02:15] the most boring workloads can be actually pretty compelling and uh powerful. And these are the use cases that uh uh me and my team have actually implemented. So, we'll kind of look at that. Uh of course, this is the Databricks Summit, and there's a whole
[02:28] bunch of technical people over here. So, we will take a look at architectures and uh how we evolved architecture and all the interesting things and some uh operational patterns and some uh how do you build How do you start with a use
[02:40] case and then build a portfolio of use cases. Okay. So, having said that, um well, first of all, why? Uh you always have to start with the
[02:52] why, right? Uh since you have to justify to somebody up in the hierarchy on why you want to do it. So, as I said, finance work workflows, uh if you look at it, are extremely manual.
[03:06] They're very redundant. They are very document heavy. They need a lot of people. And they're actually pretty boring. Nobody wants to do it. But they have to do it. Their day job is really something else, but they have no choice.
[03:19] And I have a bunch of examples there on the screen, like a lockbox reconciliation, proof of delivery validation, accounts receivable, and uh customer-facing finance inquiries, and a
[03:32] whole bunch of that type of stuff. So, those are some examples. And when we studied those with our clients on what's the advantage, what's the ROI, we realized that there is a huge
[03:45] productivity benefit, right? I have some numbers over there that by automating these simple but extremely boring things, you can recover 70-80% productivity gains,
[03:57] faster resolution, just overall the happiness index of people go up because now you're taking the boring stuff away from them. So, the point here is that it's absolutely something worth doing.
[04:09] And it's actually pretty simple to do on the Databricks platform. Uh it's not that complicated of an architecture. We have actually done it. Uh okay. So, as I said earlier, uh finance is pretty unique. And what I
[04:23] like about finance workflows, workloads, is irrespective of your industry, you can be uh CPG, you can be manufacturing, you can be uh insurance, whatever, everybody has
[04:35] finance. And more or less, the workloads are very, very similar. Uh so, what makes a finance workflow workflow agent-ready? By the way, you can remove the word finance, and you can essentially use
[04:47] this slide to gauge any workflow out there, irrespective of finance, that you want to automate. Uh repetitive, high-volume work, such as claims and invoices,
[05:00] document-heavy input, uh document comes in, and analyst is looking at it, trying to extract values, and then feed a downstream system. Happens a lot. Uh judgment-based exceptions,
[05:14] uh meaning there are certain situations in finance where human judgment is better, so you have to divert it to that. Uh multiple systems of context, you know, finance people have like many
[05:27] screens open all at the same time, and they're trying to figure things out. Uh so, you need inputs from many systems. Uh in finance, uh you might be surprised to hear this, finance people deal with numbers.
[05:39] So, there it is very easy to validate. You can post-process an extracted output and do very clear validations. You can basically add up a bunch of numbers and see if it tallies with something or not.
[05:51] So, the post-process validation is something we developed for our client which turned out to be pretty amazing. Uh of course, injecting the human in the loop is always important. And then measurable operational impact
[06:04] as we saw in the earlier slide, it's worth the effort. Okay, so I kind of always like to start thinking from a first principles perspective. And
[06:16] this doesn't necessarily have to apply to finance, it applies to anything. Um so, people ask me a question often is I have a particular workflow process.
[06:28] It has been in production for uh decades. How do I start automating it? So, applying first principles uh thinking to it, the first thing you would do is you take a human process
[06:42] and essentially whiteboard it step by step. What's the input? Is it a document? Okay. What happens next? Is a human analyst extracting the values? When they extract
[06:55] the volumes, are they doing a calculation? What is that? Next, next, next, next, next. Um so, I really ask them to sort of kind of whiteboard that process out. One trick I have discovered working with
[07:08] my clients is especially business users, SMEs, is I tell them that think of me as a junior intern you have just hired. Let's go to the whiteboard and you teach
[07:21] me. You basically teach me how this workflow works and that's what we do. We can uh just uh whiteboard it out. Um and then if they come at a step where they where
[07:33] they say, "Okay, now I have to look at these numbers and I have to add these numbers and then I have to do something else uh or I'm doing reasoning over the number." And I'm like, okay, what is reasoning? Like, explain to me. I
[07:45] capture those inputs, and actually, they are very good inputs for either your system prompt or your user prompt when you shape the thinking of the LLM. Or if you use frameworks like DSPy, like how we use, uh, you can
[07:57] sort of use that for your signatures and your modules. So, that's a very cool trick uh, that I have discovered, which has worked very effectively. So, from a first principles thinking perspective, break down the work and delete as much as you can. If there
[08:10] are unnecessary steps, don't do it. Uh, I ask a common question, why are we doing this step? And the answer I get is, I don't know, we've been doing it for the last 5 years, so I keep doing it. Uh, that happens a lot. So, if you get that answer, just delete it. Uh,
[08:24] isolate judgment. We want to figure out what the contextual reasoning is. The contextual reasoning sort of gets embedded as part of your prompt. Uh, measure the friction points. Uh,
[08:36] meaning, uh, you know, the time spent. Uh, what are they really looking at? How hard is it? Do they have to flip through pages? All that type of stuff. Uh, find the value ceiling. Uh, meaning,
[08:48] if I make this person's life easy, uh, how happy he or she is going to be? Am I going to save them hours, dollar amounts, you know, that type of stuff? And then start automating the friction, right? Try to automate as much as you
[09:00] can. Also, by the way, one of the key lessons we have learned is there is no glory in automating everything on day one. It is completely okay if you take a workflow and start with small automation
[09:14] steps, but still keep the human in the loop involved. In fact, we strongly recommend that approach. Uh, because that kind of helps you measure the quality, do a slow roll out, gain the confidence. Gaining the confidence from
[09:26] SMEs for, uh, AI agents is super critical. If you try to automate something and it doesn't work, uh, it's never going to go to production. So, those type of sort of kind of little best practices we have gathered are are super
[09:39] useful. Okay, so having said that, let me present two extremely boring and extremely redundant, but highly effective use cases. So, this one is
[09:51] what we call as a lockbox remittance reconciliation. Um and so, really this is the problem statement, right? The problem statement is um my client
[10:04] has a bunch of vendors and the vendor sends them these remittances, which are PDF files. Um one file per vendor. But, the file has multiple pages. It can
[10:17] be 25 30 pages. And then they send a check. Now, there is a financial analyst sitting there who is actually going through all the pages trying to figure out what are the actual
[10:30] line items and the dollar amounts that they received. He's adding up all the dollar amounts and then trying to measure it against the check.
[10:42] So, that's called as remittance reconciliation. So, that's the reconciliation process. Anybody would like to trade their day job today and go do that? I have couple of roles open for that if anybody is interested.
[10:54] They're hiring. But anyway, well actually they're not hiring anymore because we automated it. But anyway, so that was super boring. So, when we looked at the problem, we are like, yeah, this is pretty easy to solve with all the current technologies.
[11:06] I'm sure you guys have heard about Databricks AI parse document, AI extract, AI classify, da da da, all that type of stuff and then other from Azure and and whatnot. So, really that's what we did at a very
[11:19] high level. If you go underneath the covers it's actually a pretty complicated problem. Um document extraction, intelligent document extraction is actually still a very complicated
[11:32] problem. In fact, tomorrow morning I have the privilege, me and my colleague over here, to interview Archika, who is the product owner for IDP at Databricks. So, we'll have a little conversation with her. Uh which we will broadcast
[11:44] live later on. Uh but it's still a very hard problem. Why? Because PDFs are very messy. PDFs are fundamentally generated for the humans to read and not for machines to read, or at least AI agents to read,
[11:56] right? So, they have tables and they have nested tables. And if an invoice line item has a long description, what happens is the dollar amount then goes on the next line. It's not exactly where it should
[12:10] be. So, those are just some few examples that we encounter. Um so, we really had to sort of build a a a pipeline architecture, as I say, in the middle over there, which consists
[12:22] of, you know, many sort of components. Now, I kind of hear the common feedback, "Why is that so hard? Just take a PDF and throw it to a VLLM and it'll extract everything." And I have done that and uh
[12:35] that has worked fine for me. My response to that is doing it once versus doing it for 10,000 documents on a daily basis are two different things. So, don't tell me you took a document, sent it to a VLLM
[12:47] and it worked. No, that's not what we're trying to solve here. Trying to solve a real production problem under scale. So, for that, we had to build a complex architecture. We had to take the PDF, we had to segment it out. If it's a table,
[13:00] we had to use like special extraction techniques just for the table. If it's a line item at the bottom, we had to use something else. So, it was a kind of segmented segment out the remittance sections, detect and
[13:13] normalize the headers, clean out the OCR noise before you can extract, uh validate rows and the segment totals, and then enrich the customer data conservatively.
[13:25] Um so, even though you have all these functions like AI bar stock and AI extract and Azure doc intelligence and all that type of stuff, just throwing stuff at it usually doesn't work. If you're serious about production, if
[13:38] you're serious about scale, there is always a pre-processing thing which will help you uh, clear out the noise or reduce the noise. What are those exact pre-processing steps? For that, you guys have to hire us as
[13:50] consultants. Okay, I was kidding about that. Uh, it's hard to generalize that. It kind of depends on the use case and it depends on, you know, what type of documents and all that type of stuff. But, always remember that how can you clean up these documents
[14:03] um, before you send it to whatever intelligent extraction engine you are using. Uh, and then what we have to do is then we have to create these finance ready outputs. Right, we have to essentially
[14:15] think of them as gold tables or platinum tables. We have to take all this messy data, we have to structure it in a nice way, throw it in delta tables, we have to create the exceptions for analyst to look at,
[14:27] uh, so on and so forth. Oh my god, okay. I keep talking a lot. Okay. So, the next one was what we call as a proof of delivery validation. This is yet another extremely boring problem. Um, so, what happens is when
[14:41] our client used to deliver to their customers, they would actually sign on a delivery slip. Uh, and sometimes they would make changes on the delivery slip such as, "Hey, you told me you were going to send
[14:53] 10 units, but I only got eight." So, they would uh, with an eight. Uh, and we would get those things back. There is a business analyst sitting back there who actually is looking at the
[15:05] signature, looking at if any amount is scratched, and trying to figure out what changed. Um, so, really when we looked at it, we are like, "Yeah, this can be automated." Uh, the problem over
[15:17] here is first step is signature detection. You get a PDF. Then you have to use certain techniques. We used a open source library called OpenCV. So, basically we used a
[15:30] segmentation algorithm. You get a PDF, the bottom third is where the signature is always present. And in the bottom third it's really on the right hand side. So, we used those types of segmentation
[15:42] techniques, then we used OpenCV, and then we used AI parsers and at times even Azure Doc Intelligence to then figure out does a signature exist? And is there any quantity that is
[15:55] crossed off? If a quantity is crossed off, we have to route it to an exception table so a human can take a look at it. If everything matches, we just clear it. And now the person doing 10,000 of these in a day or probably in a week has to
[16:08] only look at like 25 exceptions. Now, of course, I'm skipping a very important step in the interest of time. That can be another 2-hour discussion actually. Uh and that step is how do you know
[16:21] your extraction is right? And the answer to that is evaluations. So, we focused tremendously on offline evaluations as we are building. We generated golden data sets, and we
[16:33] bounced it off against our as we were developing. We agreed with the client on what is a reasonable accuracy because guess what, even humans are not 100% accurate all the time. Is it 85%? Is it 90%? We agreed on that benchmark, and we
[16:46] did very, very serious evaluations. Unless you did evaluations, you can't put these things in in production. So, this was another boring, but highly impactful um
[16:58] problem we solved. Okay. So, what does the architecture look like, right? So, these are all sort of kind of cute little boxes I have drawn over here. And what I'm trying to basically tell
[17:10] is you have all kinds of inputs there, uh log box PDFs, uh ERP data sources coming in, proof of delivery documents, emails, sometimes even business rules.
[17:22] And then, in the middle is what we call as a document intelligence component. So, these are the different components we built piggybacking on AI parstock and you know, sometimes even on Azure doc intelligence and all of
[17:36] that. Something to do field extraction. Uh, something to do table detection, smart cropping, document segmentation, OCR, all that type of stuff. And then we created a gold layer
[17:48] where we put all of those extracted things in a nice structured format for finance people to to look at. Uh, what you see at the bottom is kind of the agent layer where how do you do enrichment?
[18:01] Uh, how do you do uh, exception management, extraction, recommendation, validation? And of course, we use quite a bit of agent bricks to do that. Uh, the box on the on your right hand side, the controls and
[18:13] evals is super important. Unless you have all the right types of controls and evals, and especially eval is very important. Uh, we figured out a post-processing technique to do like row level validation, segment level validation,
[18:26] confidence thresholds, uh, ML flow traces super important, uh, human escalation rules. Now, with all the announcements last 2 days, uh, with the Unity AI gateway, a lot of
[18:38] these things can be basically you can piggyback on the Unity AI gateway, but before Unity AI gateway, we would sort of build these frameworks uh, for our customers. Well, one key framework we built, which I was super happy to see in Matei's presentation
[18:51] this morning, is the model selection, the smart router they call it. We totally built that framework for our clients. Depending on the workload characteristics, you can pick a smaller model, go to a bigger model, you know, all that type of stuff.
[19:03] So, those are sort of the controls in the in the evaluation box. Uh, all right. So, uh, from a operating principles perspective to kind of summarize this
[19:16] whole talk, the number one lesson is automate the grind. Let the most boring work be taken by AI agents, deterministic checks. That's why finance workloads are perfect. You can
[19:28] extract quantities and you can do post processing and make sure it all works. So, you know right there whether it worked or not. If not, you can route it to an exception. Uh design for exception. Uh traceability design that is super
[19:41] important and ML flow is your best companion when it comes to traceability. We leverage online offline emails and online emails through ML flow. ML flow judges a lot. Uh build once and reuse often.
[19:54] Very important. As you do these use cases, you sort of kind of build those architectural patterns, the design patterns of how you're solving the problems. If you start doing that, then you come to the next
[20:08] and the last slide over here, that is from single use cases, now you can literally impact every single workflow, take a portfolio approach, not just in finance, but in whole other areas as well because
[20:21] a lot of these fundamental building blocks are very, very repeatable. Once you solve these problems, they can be applied to multiple different areas. All right, I am right at time. Thank you so much. If you have follow-on
[20:33] questions, I have to run for a client meeting, but my colleague Tejas is over here. He's a lot smarter than I am, better person to talk to. Uh and then feel free to connect with me on LinkedIn if you do have any follow-up questions or would like to know anything more. Thank you so much.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.