Transforming Title Insurance With Databricks Batch Inference

Summary

• Batch inference for data extraction: This video showcases how First American used batch inference on the Databricks Data and AI platform to extract data from millions of historical title policy images efficiently.
• Optimization and cost reduction: Learn strategies that achieved a 75% reduction in project timelines and 70% cost savings through optimized data processing capabilities on the Databricks Data and AI platform.
• Business impact through AI: Discover how intelligent data extraction improved operational efficiency, enhanced the title insurance process and ultimately improved the home-buying experience for buyers, sellers and lenders.

Transforming Title Insurance With Databricks Batch Inference

Talk by Madhu Kolli, Sr Director AI, First American Financial ; Prabhaker Narsina, VP, Data and AI, First American Financial
Join us as we explore how First American Data & Analytics, a leading property-centric information provider, revolutionized its data extraction processes using batch inference on the Databricks Platform. Discover how it overcame the challenges of extracting data from millions of historical title policy images and reduced project timelines by 75%. Learn how First American optimized its data processing capabilities, reduced costs by 70% and enhanced the efficiency of its title insurance processes, ultimately improving the home-buying experience for buyers, sellers and lenders. this video will delve into the strategic integration of AI technologies, highlighting the power of collaboration and innovation in transforming complex data challenges into scalable solutions.
Databricks Named a Leader in the 2025 Gartner® Magic Quadrant™ for Data Science and Machine Learning Platforms: https://www.databricks.com/blog/databricks-named-leader-2025-gartner-magic-quadrant-data-science-and-machine-learning
Build and deploy quality AI agent systems: https://www.databricks.com/product/artificial-intelligence
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

FAQs

How can batch inference improve data extraction from images?

This video demonstrates how First American used batch inference on the Databricks Data and AI platform to process millions of title policy images at scale, extracting key information with minimal manual intervention and higher accuracy.

What timeline and cost improvements can I expect?

First American achieved a 75% reduction in project timelines and 70% cost savings by optimizing batch inference workflows on the Databricks Data and AI platform, along with a 15% boost in operational efficiency.

How does batch inference integrate with the Databricks platform?

This video shows how to build end-to-end batch inference pipelines on the Databricks Data and AI platform, combining data engineering, AI capabilities and governance to create scalable, cost-effective solutions for complex data challenges.

What industries can benefit from this approach?

While focused on title insurance, this video demonstrates batch inference patterns applicable to any industry processing large volumes of unstructured data like documents, images or forms on the Databricks Data and AI platform.

Full transcript

Hi everyone. Uh good morning. Thanks for coming and joining us today. Uh today myself Prabhu will be presenting a use case on data bricks we use in first American. We have a number of use cases. We use data bricks for uh data engineering AI and number of other ways and lakehouse and unity catalog. Uh today we're going to present one use case. Um uh and we trying to go over the we we'll have a high level overview of our company what group you are coming from and go over the problem and then go over the what our ML design and process looks like.
We go over the matrix and then finish with the lessons lantern questions. So about first American um how many of you have a experience of buying a home or that's good thank you and how many of you actually know first American as a name first American Financial Corporation. Thank you. Uh if whoever hasn't gone through this process probably title insurance may not kind of may not know but we'll go over the process. Uh first American uh main business is title insurance but we actually support a lot more than title insurance.
Uh you can see we deliver the certainty and trust with the power of the real estate transactions. That's our kind of a purpose of our company. Um so our major business is title insurance. We also have a settlement services. We provide um lender and uh servicing uh uh solutions uh which uh also part of that comes from where me and comes from. Another one is real estate data solutions. Uh that's basically if if you're applying for a loan uh or lender wants to check uh there's any problems with applying your loan or property itself they come to us for checking for the fraud or the valuations and some of the things to us.
Um another group we have a home warranty and we also have a trust. Um and not to mention we are 130 plus years uh old company. We've been there for a long time. We started small. We keep growing. uh we do more than a million plus uh title insurance uh every year we are number one number two depending on sometimes we go to number two we are number one so um but we also have 16,000 employees across the world um so we support in uh Australia Canada and US is our major business so about data and analytics group which where we both of us come from kind of I said in the previous slide we have the solutions provide for um uh mortgage companies and lenders and some other things.
Um so I'm going to go over that little bit more. So we provide solutions to again the other insurance companies, title companies, not only first American but other companies too. We provide to the prop tech, fintech, island gas, lawyers, realtors, I think a lot of company, different industries users even including Zillow and some other folks buy information from us. Um so we we basically um help them in a number of ways u either making the decisions for their their customers or helping them in data analytics uh some other aspects of it.
So I want to spend a little more time on this slide. So we are a data company within the first American. Uh so think about as we uh collect information from 3,000 plus counties and number of courouses. We buy information from uh tens of different data vendors. We collect all that information. And if you think about the complexity of our information we collect, each county has its own history and they are different from time to time and they have thousand types of events happen every county recording.
Uh so you can think of the complexity of this problem and so we actually take all that information make it easy for our customers just enhance and enrich information and provide them easy for consumption. So we are number one in number of data sets property information we called assessor when you go when uh if you own a home you have a tax assessment that happens um the basically that happens from there uh we have another big one in our document images this is a basically image when you do a transaction in the county when a deed if one of you probably bought a home you saw the deed that becomes a signed document that becomes image goes into county and comes as image to us we have images all the way going back to 18 late 1800s.
So some of these documents we have it but the even counties does not have it. So we actually have much deeper uh than them and across the different counties and we have 140 plus 550 million residential properties and all the uh stats and all the properties of the properties too. So how do we use this information to provide services to our customers? So we have um way of pro giving them through the websites and web portals.
We have very rich map- based uh portals that actually customer can come and look for either analytical insights or single property or transaction reports or um looking at the market view. There are number of ways that customer come into web portals and come get their information. We have our own valuation models like Zillow. We provide to lenders. uh we have APIs where loan origination system or some other customer messaging queue can actually integrate into our API and get the information from those APIs.
We do the millions of transactions every day. Um so we we also have used the little modern techniques where customer can come and subscribe to A3 buckets or some data share concept or we can push to them. Uh that's another way we actually share the information to the customers. Um the other one is called Bul data licensing where person the company can subscribe to intern nation or subscribe to state or a county and we actually update them on a regular basis.
And the last one is in here advanced data solutions where uh customer has a particular need where they have let's say 30,000 loans and they want to get a 20 piece of information for those 30,000 loans. We take those loans understand their data set and basically give them the information need back. So before we go into like a actual problem for today um we I'm going to touch on two terms uh here uh title insurance is a kind of as I said is a major business for like a first American um it's a it protects the owner as well as the lender it's basically you can read through that but it's is a when if there is a problems with the deed itself ownership leans or anything that happens on the uh on the transaction and the customer as a owner is protected as well as the lender is protected and that the two types of policies one is the owner policy another one is the lender policy owner policy protects the owner lender policy protects the lender um so the important concept for today is a starter it's also called a preer or a back um it's a preer policy or a back title policy um it's a think about this if the the transaction that happened years ago there is a title insurance policy generated we call the policy when you get a home and title insurance that becomes a policy.
Um so that previous policy is called a prayer or a starter. So we're going to go get to that why that's important and we are going to dig much deeper into what we did with that particular use case. So it's a I'm just going to touch on the little um deeper into the complexity of this title insurance. The reason it's a title insurance itself is a complex. It it used to take I think few weeks to months. I think it some cases might still take that we are much faster.
We are trying to get a living title insurance some kind of a concept where you can actually get insurance much faster. Um so I'm legal description is one of the complex ones in this documents. It's basically states what's a property defining a property. It could be as simple as address when you have address it or some cases you have something called SSR parcel number simple number. It can tell define your property. in some cases very complex where they could basically say okay this is the three miles from this cross-section then you found a house with a this tree and after that go 10 steps and you get a river you get a five steps from this and so they define that entire thing in ethical description right so the challenge with that is basically is that how do you make sure that passed on correctly whether that things changed I think someone said house the house may not be there anymore right so those are the things complications come in that's legal description wise.
Another one we going to touch on more more on is the any um title in we supply we provide we have to go back all the way to uh like a parent parent so we call the patent when uh in the early days of US formation uh when government sold the property to private owner it's called patent so if you're buying a home today the tomato part of different things you have to go all the way back to that patent to make sure there are no litigations easements or encroach ments or anything that happened over the time to make sure issue title insurance.
That's the reason it takes a long time. Uh there's a couple of them on the personal side. We basically you have a um situations like people bought it as a married couple and then one of them got divorced and one of them is not mentioned in the document that could get into complications. Sorry, I went other way around. Okay, this is just an example um where I was explaining before. Let's say there was a 1950s US government sold this ranch to a a organization or corporation and that company divided that into three different ranches sold one of them into 1975 and that to the builder or a developer and that guy basically developed the properties and sold in 2002 and 2015 that person who bought a sold it.
So if you're doing a transaction 2015 you have to go back all the way to 20 1950 to find out there were no issues in this particular piece. So the starter concept is basically is that okay in 2002 we have done this transaction we have done this all the research that required for that particular property I can depend on the 2002 property and start from 2002 to 2015 instead of going going back all the way to 1950. So that's a concept.
Um uh basically we it's easy to say than done because the this concept been there for a long time. Um a lot of them were images when at the end of the time and capturing information from these document images could be somewhere from 10 pages to hundreds of pages sometime and there are all combined information from 20 30 documents sometimes different things. There is no separation. They're not consistent. There is no uniformity.
Uh that was a challenge. So if if you had to send them to operations keying we um I think is going to more go more into this. Uh but if you have to send them to operations for keying we have to key two people has to key if the two persons doesn't match you or the third person has to key it and there's a QC process is a pretty expensive process um that used to happen. So take over. Yeah. Um I hope you guys can hear me.
Well first of all thank you for coming. I know there are a lot of choices out there but uh thank you for coming. I sound like an airplane employee but um anyways so um the majority of from here what we're going to focus on is the problem statement itself um and how we formulated the actual business problem into a machine learning problem and how we went about designing the actual solution itself right and how we utilized a combination of a good engineering platform like data bricks to solve the problem right so we touched upon um a little bit about um what a title policy is right and at a Hello.
Um what I'll tell you is it essentially ensures you're the owner or the lender, right? Um and the when when we when I went through when we went through this journey, right? So imagine an internal customer or an external customer coming and giving you 400 million pages or just document images, right? And you have no clue what is in these images. How do you go about understanding what kind of data is in it? Are these pages in scope or out of scope?
What are the critical elements I need to extract from this? um do I need to protect my customer information? Those are some of the key critical um requirements that we had to deal with, right, when we went into this project. So, I listed a few here. Um and um and I'll just go into the problems in part two, right? So, here's an example of a 1955 title insurance document, right? And you'll get all the way from 1955 to 1900s to 2000s, all kinds of varied documents.
It is an real unstructured image document and once you get this image document you actually want to extract first of all you want to find out if the page is in scope or not. What is the inscope page? It means that is it relevant for a title policy or not. The second thing is you want to understand what are the critical elements within this right for example you have who's the title insurance who's the owner who's the lender what's the property address what's the policy amount um and uh when was it effective and what's the legal description of the property here right um and I touched upon um this a little bit not all pages are in um uh in scope you do need to split the title policy because imagine if you have um a thousandpage document not all pages are actually in scope and I'll show that show that in a bit.
So you want to kind of separate out what I call like we out the junk pages out right um you want to identify the individual document types within the policy and uh finally you want to extract the critical fields and um redact the important customer information both for internal as well as external use and then integrate into title policy systems for downstream use. Okay, so here's an example of an out of scope page, right?
Um I don't know if it's really clear, but this customer gave us close to about 2,200 pages um in one single GIF file, right? So sorry. So in you can kind of imagine, right? Like a lot of these pages are out of scope. Like the next page, this is one of the they would embed this page right in middle of the middle of the document, right? So you want to weed this out. So that is that is the problem that you had that we had to face with.
So let's just move on. Um so then um the way we laid out the problem is we first understood the business uh need behind this right. So we understand that there's a title policy. It could be lender policy or it could be an owner policy. And then we also understood what are the subdocuments within each of this policy. Right? And it's important to understand why these are important because um schedule A, schedule B and endorsements.
All it means is schedule A says that here's the property, here are the different elements and here's what I'm going to insure. Schedule B if you have insurance, you know there's going to be exceptions and exclusions. That's what schedule B is for, right? Endorsements is something that changes the original policy, either the amount or the dates or something towards that effect, right? So, you notice the document title, there are three different I just put a sample of them.
There could be 10 different variations of the document title, right? Um, similarly, then you have uh owners policy. I think I clicked on. Okay, got it. Put it there. Maybe I should do it here this one. Yeah, let me do it. Yeah. So, sorry about that. Uh, so you have owners policy and um it has the same u subcomponents and um like a commitment. Commitment is essentially just a title insurance um company saying that I'm committing to give you a title provided you meet certain uh conditions.
Right? That's all it is. Uh I'm not going to go too much into this detail. You can quickly read it. Owner's policy just ensures the owner. Lender's policy ensures the lender. Um, and a loan policy is essentially for the lender. Commitment is justly ensuring the commitment itself. Um, I I briefly touched upon schedule A, schedule B and endorsements, right? Schedule A um is for the property itself. All the pointers, schedule B is exclusions and endorsements is for uh amendments to the policy itself.
Okay. Now u we frame the functional objectives for the machine learning use case. Right? So the first objective is um you're going to get a page which is um a document unstructured image. Next you want to determine if the page is in scope or not. The third thing if it's in scope then you want to extract the critical elements from the page. Then the fourth thing is once you extract it you want to redact certain important customer information as prescribed legal and then you want to integrate into downstream.
So you kind of have multiple objectives um um um as part of your problem statement here. Now traditionally if you come from a machine learning background the way you'll formulate this is um let's just say you have um a document is in scope or not you'll find okay let me frame it as a binary document classification task. Let me just tell you if it's in scope or not right then the second objective is what kind of document is this?
Is it a scheduleula? Is it a schedule B? Is it an owner's policy? Lenders policy. Right? So that was our second important challenge. Then once we determine it's in scope, then we have to get the entities out of this. This could be owner names, addresses, amounts, dates, policy numbers, right? So we had close to about 150 different fields that would pin the policy to be um to be very good starter, right? Um and then um I I talked about like we have to split the starter itself, right?
So this is like a start page classification task where you have let's say 10 pages but not all 10 pages are good right so you want to form the starter exactly where it belongs right and then um you want to redact the customer it's almost like an um object detection task where you want to redact the customer information from the u pipeline so how do we go about deciding how do we address this problem right so we talk about a little bit about the trade-offs here So traditionally we had uh machine learning pipelines for document classification.
We had pipelines for um document um document entity extraction. We had pipelines for document splitting as well um and redaction too. Right? But um when you have this kind of pipeline are we going to orchestrate this one uh single model after the other or how we or are we going to orchestrate in a batch? Right? That was the important decision for us. So we had some trade-offs and the second thing is we obviously had to retrain all these pipelines to fit into the title policy data right so these were all um obviously trained on some other data sets.
The second thing is um one of the key advantages we had is all the data standardization like address amounts and dates are all standardized right so that it fits well into a database but when it's not a document sometimes they'll say Jan 1st 2021 or something so we kind of say 1 2025 doing all of that was built for us right so um so that was what we had but we didn't go with this approach so we actually went um ahead and um wanted to orchestrate this as a batch pipeline and the second thing we wanted to orchestrate this as as a one-step inference pipeline so that we could accomplish all the tasks at once.
Right? So, and um one of the things that the biggest con of this is obviously we didn't have any development um effort for this and the only thing we knew that we had was um we had some batch inference that we were really successful. So which kind of hinging on that uh prior events um and accuracy cost was very important family because we were doing a large scale batch inference. So imagine if you're doing 400 million pages.
So when you run such a large inference cost becomes really important um and accuracy becomes a factor as well because you have to constantly uh keep checking the performance metrics at every gate. Right? So and we'll talk through those a little bit more. Right? So these are the kind of trade-offs that we went through when we kind of went through which approach should we take right should we use our existing pipeline that we already had and um kind of chain them or should we build something brand new right those were u kind of trade-offs that we looked at okay I'll say it's all about the training data and the reason I'll say that is because the way you formulate the training data will actually mean how you actually going to infer also right so if you have been in the machine learning world so the way we formulated the training data is pretty simple actually when once we understood the problem statement um so we kind of broke the tasks into three main tasks right let's determine if a page is in scope let's then determine what kind of page it is and let's then extract the elements out of this right after once we did this then let's just say we had a policy document come here right let's just in this case you have an eightpage document.
Our first step was to label the document as scope um and say if it's in or out, right? So that's our first label. And that would be our first prediction, right? Then our second prediction is let's figure out is it what kind of title it is, right? Is it a lender policy or no policy? And then let's just further break it down into a schedule A, schedule B, an endorsement, um or a commitment. At this point, right, you kind of know that in this particular case, 235 is a good sty good title policy and page 78 is a good um policy as well.
So, we we kind of throw out page four and page six because they're out of scope documents here. So, that's how we form the training uh data itself, right? Let's just see. Okay. Um and you kind of notice here I think it's hidden down below but um the policy in this case is um one two three uh and the pages are 235 right and the policy is at the end is seven and eight. So that's that's very important like how you formulate the training data because that's essentially how you're going to train the machine learning model so that um you have good inference right so we kind of went from a multi-objective ML system um to one single stage um large scale ML inference by framing all the subsequent task as an extraction task but not as individual um ML task right so let's just walk through um what are our different um items that we picked as part of this uh pipeline right so the first pipeline is it's pretty simple once you understand the business flow once you um formulate the training data so you have an image you kind of have an OCR you kind of OCR it then you have a unified training um uh data that is available to you um you train a large language model right um I won't give you any hints what that language model is uh Right?
And then you have some inference coming out of that, right? And then you kind of post-process the results, right? That's the simple pathway that we chose, right? And this was our first iteration that we did. Um and um why is this important is it's because it's important to kind of set up a very basic vanilla pipeline just to make sure that the batch inference itself is actually going to work, right? Um so once we had this but then let's just then we started to figure out how much is it going to cost me to run 400 million pages and continuously keep adding millions of pages right so this is broken down this is how we did the cost calculation right um and we felt that this is very effective because family you have two parts to it one is the OCR cost itself right and these are just sample costs just just to give you an idea but it gets you pretty close um so that's the first part right oh I'm going to OCR 8,000 pages cost me so much, right?
Um and then the actual cost comes in when you're actually doing the inference on the um inference, right? So here's how much a DBU cost for thousand for 10 pages, right? So you kind of have to work through uh different several layers and our journey on this particular it took it took us about a week to come up with this estimate but once we did this um our estimates for 400 million pages were pretty good, right? We were very comfortable that we were around the ballpark of whatever amount that we would going to spend on the engineering platform, it would be a good u uh good investment for us, right?
So that's the cost estimate now. So we had the training data, we had the cost estimate, right? So we kind of then figured out um what what data what platform to choose, right? And the primary reason when we went down this path, data bricks was an obvious choice for us primly because um when you run large scale inference, the GPU availability for running large language models is primal, right? And data bricks was the only cloud provider at that point who was giving us this this availability, right?
Um and also the engineering of the data bricks inference is very um I will say structured for both for development and for um engineering as well. Um we also had done a large scale inference jobs with uh other other image processing pipelines right so we were very successful with data bricks right so that's why we chose uh data bricks and the other key features that we actually included um one is obviously observability right so you kind of know how much GPU utilization is happening um scalability is very important for us probably when when we deploy the um the inference model for us to scale from A100 from one GPU to four GPUs it took it takes us less than 30 seconds, right?
So that is very important for us. So and databris was the only engineering platform that would help us scale that quickly um in and that's short uh time, right? Um I do want to call out both and Favor. They've been amazing customer support um and partners during the entire process. Um so we had great support from um a lot of the engineering leaders within data bricks as well as their customer and partner teams here. Okay.
Um so we actually um in terms of the language models we actually played around with quite a few. We played with um I won't name every every uh language model but we ended up with llama primly because we were able to it was tightly integrated into data bricks right that's the first and foremost and the second thing is we could fine-tune it. We did several iterations of both full weight fine tuning and then parameter efficient fine-tuning as well.
Um and then uh once we fine-tune we we notice that the results are more consistent and uh the number of hallucinations and the accuracy kept getting better uh through iterations, right? And we're also able to actually um scale um on an deployed on an A100 or an H100, right? So it was pretty easy for us uh once we deployed this. So those are our design choices. Um so in terms of our training pipeline right it's actually pretty simple I would say once we knew the entire landscape you have a blob storage you had um broken out our 400 million pages into several different containers right very divided very well organized that's the first step right then we had an OCR job that would continuously look at these blob containers and run the OCR our choice for OCR was Google Doc AI primly because we felt that it maintained the read order and also worked had good quality on the image itself.
Right? Then we fine- tuned the large language model llama in this and then basically stored up predictions in um in a delta table. Uh training model uh training input for the large language model was primary two things. It's pretty self-evident. One is the OCR the prompt input u prompt input with the field field value for the ground truth. Our parameters were um obviously it has good context length so we're not worried about that.
But the learning rate is where I felt data bricks really helped us to pin exactly which learning rate and this is pretty important primaryly because it tells you it's linear to the both the GPU uh compute time and your inference cost right so that's why it's very important to pin the um the um the learning rate here we ran it for five epox right so that is uh that's essentially where this is our training pipeline so it was pretty easy to train but then um when we started doing the inference here you'll notice is we actually don't stop at the ML predictions.
Uh so there's some post-processing happen and what the post-processing itself is um dates, addresses, um title company names, all of those address standardization, all that would happen in the post-process predictions, right? And then that would be sent down to the downstream applications for title exchange. Okay. So in terms of the streaming batch inference um I think um so the way um I kind of explained this a little bit right you have a big blob storage container very well divided then you have you stream the OCR text um and then you kind of run the AI query which is kind of your inference point um and then you store the results in in a delta table.
So the left side is a latency graph from directly from data bricks. And what you'll notice here is 50% of our results um pretty much finished within 5 seconds and 90% of results finished um around I think it's about 10 to 15 seconds. Sorry it's about 30 seconds right so that was our latency right but uh apart from latency what you notice on the right side is the throughput graph. So the throughput graph is very important finally because the way we measure throughput is per second right and that matters quite a bit.
So if you notice all the um all the responses are below are kind of around the 13-second mark right so what that means for us is in one um in uh 13 pages in 1 second that's what it translates to us so I'm able to process 13 pages in 1 second which kind of roughly translates to about 47,000 pages in 1 hour in 24 hours that means about over slightly over a million documents right on one GPU right and then you kind of scale up the number of GPUs and you're able to process that many pages u in in batch inference mode right so that is our story right from prom statement to framing the training problem and then setting up the batch inference um and kind of scaling it so that we could run this at large scale and you kind of once you set this up you have a pipeline where you can kind of run this in batch mode continuously okay I'm going to hand it off to probably we'll talk about more of the metrics and how we went through the other lessons learned as well.
Thank you. So I just want to touch again I think when we started with the data bricks early last year um there was only kind of using API you can fine tune only using APIs and you can inference only using endpoint. I think it was very early stages collaborated closely with the product engineering team able to do it pretty quickly and actually eventually AI query came in that simplified quite lot of a processing you don't need to do all that stuff yourself and I that was really really important for us to make it success so like was explaining um our training data set needed a quite lot of a thinking how you want to capture this information we have a set of experts in offshore we would basically who knows these documents would provide in a uh files for us to kind of use that for evaluation and training and we had to like it was talking about in scope out scope what the document type and the subtype and also the fields and everything.
Um as you can see here we went through the multiple iterations right basically we do this in all the projects too you start from somewhere number of documents we give that we do the evaluation because it's mostly classification problem who worked on AI ML classification problem big metric to use is F1 scores right I can basically balance between precision recall and um so there are different categories of information that we have to capture I think the like we call header information like a policy number and some other things then you have a land information or the property information you have address city and APN and some other things that comes up and one important part of the thing is I think once we do the first iteration we would get a feedback and say okay this is not working well then we pick and choose okay we need to another label another set of documents for this set of documents set of prof fields and also the set of features in a way so we give that back and we get the back and do the iteration we did like three times at the end I think we did around 40k plus pages to get to this accuracy Um so it's a the another thing is that I think human does some mistakes too.
You're not going to get 100%. So you do have some cases where it's only like 80s or 90 most of them are above 95 and one of the things we kind of mother touched on this is the second set of fields which is kind of basically on the person name but the similar concept. Um so the we also have a lot of this information we use our existing data sources for validation. You can think of LLM's always kind of talk about hallucination.
How do you make sure we don't have hallucination problem with these extractions, right? So one of the things we do is address is address, right? If the address was existing before I extract this address information, why don't I validate with my existing data sources? If it's there, I'm confident this is a valid address. Same thing with the date, same thing with the name. Sometimes there are number of things we validate with it. uh so that's how we get a confidence and the post-processing that actually helped us quite a bit on standardizing and validating to make sure we are confident and at the end of it I think touched on a little bit there too is there are fields that are not we are not confident they're really critical um need to go to human review so we basically send I think less than 10% of them actually goes to human review they actually correct it and give it back so this is another set of fields um kind of we have APNs I kind of talked about address the personal number um address I talked about legisription blob um so lessons learned I think mu touched on this also some of these concepts before uh defining the problem itself clearly right I think the one of the challenge I think probably most of the folks have is when we talk to product fix folks they want everything right that's the challenge right so basically the challenge is that they wanted this 175 180 fields we had to kind of work with them in multiple iterations say what do you need what do need exactly right don't ask for everything so we got to start somewhere and basically iterated on them we are actually started probably 75 plus fields here today 175 plus fields we enhance quite a bit um and every time they add a field now I have to go for a labeling I think they have to understand the cost too right so that's another part of it um touched on design choices up to some extent so ML objectives he touched on a slide right so for us cost was really important I think we we have exist existing platform does the real-time inference.
I think what was touching on you have a document classification um uh the splitting and also the extraction. So if you have to go to real-time extraction the cost would have been 10 15 times of the cost we did with the LLM approach. So that's definitely important and when we looked at the LLMs itself there was option of using prompt engineering with the closed models versus using the smaller LLMs with the finetuning and then using the pft and quantization to make them smaller to get the cost cost reduced.
So those are also considerations we have to part of go of part of the design. Um so another important aspect for us actually is this is a we are kind of data manufacturing company. We always say create, maintain, deliver. This is kind of our mantra for our business. Right? So one of the things for all of our pipelines is we want to capture information as is exists on the document. So what I think when I the train that model you can expect there is less hallucination as well as you can expect that to be on the document.
So when the when we give it to operators human operators to we don't want them to change any anything from the document. They want they want them to be king as is on the document. So it can be used for our training too. Um we talked about like storing all the predictions. Um so we talked highly about the integration. I think our pipelines even though we're talking ML we have a quite bit robust um pipelines that goes with the integration platforms all the way from data equation to ML process to post-processing to going to product and customer.
We have to integrate all this process with everyone right. Right. So I think some of the things we learned definitely part of this is I think as we transition from real time to this batch there are set of steps involved in real time uh we missed in the when we start implementing the batch we have to make sure this address translation happens. There is a validation happens for the names and there is a validation happens for the dates normally.
So we have to put all those things part of this listens learn. Um finally um so we at the end it kind of hits the customer right so basically what happens is we extract that all this information we load into kind of a delivery like a databases and then we have a platforms that actually can query and search and return back to the customers. You can see here basically we have search fields either I cannot read well myself here but I think address APN or sometimes they have a legal description lot block and some other things and as we going over the first few an example where you're trying to buy a home or something that particular home we identify that home one of those rectangles is one of those homes and we try to pull the policy for from the history and so okay this is the policy I found and if you don't find a policy for the particular home.
Now actually can see can I find a policy for a neighbor because if you think about these two properties for a parent has the same parent. So if I cannot find a policy for my myself can I find my neighbor and that can help me to reduce research. So we kind of use all all that the kind of technique to reduce our cost of uh title insurance. So this is a kind of this is how the policy document image looks like uh on the right hand side which is actually pretty big sometimes.
Um I think that's all for today.

Learn more about the Databricks Data and AI platform.