Millisecond Pricing: High-Velocity Feature Serving on the Lakebase

Summary

This video demonstrates how OnlineFS provides millisecond-latency feature serving for a UK motor insurance pricing engine handling hundreds of queries per second across over one million features.
Fennel enables instantaneous online feature updates from telematics events, eliminating the offline-to-online freshness problem and preventing training-serving skew between model training and production inference.
The architecture combines Databricks Lakebase, Tecton, and Fennel with four predictive models delivering sub-200ms latency across 16 compute slots, demonstrated live with a load test at 100 queries per second.

Millisecond Pricing: High-Velocity Feature Serving on the Lakebase

Fragmented architectures kill pricing precision. Real-time systems require millisecond latency across millions of features without training-serving skew. We replace slow, siloed pipelines with a high-velocity pricing engine built on the Databricks Lakebase, Tecton and Fennel with Genie and Apps. Technical Highlights: Millisecond serving: OnlineFS provides rapid online serving for millions of features to power complex pricing models. Rapid updates: Fennel enables instantaneous feature updates, allowing the online system to react to data shifts in real time. Unified foundation: Databricks Lakebase provides the high-performance backbone for governed, enterprise-scale feature retrieval at scale. Intelligent analytics: Databricks Genie and integrated dashboards deliver real-time visibility into pricing performance and financial risk. See how to build a scalable, autonomous blueprint for real-time decisioning that bridges massive historical datasets with sub-millisecond execution.
Talk By: Laurence Ryszka, Senior Solutions Architect, Databricks ; Rahul Pathak, Senior Specialist Solutions Engineer, Databricks ;

Chapters

FAQs

What is millisecond feature serving on the Databricks Data and AI platform?

Millisecond feature serving uses OnlineFS to retrieve features for pricing models in real time, enabling complex models to respond within sub-200ms. In this video, the approach powers a UK motor insurance pricing scorer handling hundreds of queries per second across over one million features.

How does Fennel prevent training-serving skew in a real-time pricing pipeline?

Fennel provides instantaneous feature updates so the online system reflects the latest data the moment it changes, such as a new telematics reading from a vehicle blackbox device. This keeps the features seen at training time consistent with those used at serving time, eliminating skew.

What role does Databricks Lakebase play as an online feature store?

Databricks Lakebase provides the high-performance backbone for governed, enterprise-scale feature retrieval, acting as the online feature store with branching and governance capabilities. It unifies the medallion architecture with the online serving layer for end-to-end consistency.

How does the fe.publish() command work in this feature pipeline?

The fe.publish() command is a single-command operation that synchronizes offline features to the online feature store. In this video it is shown as the mechanism that keeps the online system fresh after new data arrives in the medallion pipeline, replacing complex multi-step sync processes.

Full transcript

[00:08] Welcome everyone. Good morning. Uh we want to show you millisecond pricing. So how do you use leg base to serve features for your models? And I'm going to use example that is from insurance world. So essentially very fast motor
[00:21] pricing um serving in the UK market. Not sure what to do with this. So I'm going to let it hang for a little while on screen so everyone can see. I'm sure
[00:33] you read it all. So now quick introductions. I'll start. I'm Lawrence. I'm a solution architect based in London. I specialize in insurance and I built quite a lot of interesting stuff for insurance this day
[00:45] specifically and uh on business uh level and business value. So if you need anything on that please find me later. Rahul. >> Hey I'm Rahul. I'm specialist architect with brakes. I'm also based in London.
[00:57] same team as insurance but I'm more focused on technical side of data science machine learning and how actually you can deploy into production good cool so the agenda for today three questions
[01:10] three parts so first is does it actually work so we're going to discuss the business case and I'm going to show you the demo to see that it actually runs and you can ask questions about it later
[01:23] as well that's me now the second part will be done by Rahul. So that's the technical back end of it. So how is it possible? What's actually um um coming together to deliver the result that I'm
[01:36] starting with and then at the end we will have what about? So your questions uh please come uh with the questions happy to discuss deeper either on the insurance or on uh technical level.
[01:49] Cool. So before we actually jump into the slides I wanted you to meet John. So I'm going to tell you about this gentleman who's who is 19 years old and uh he
[02:01] wants to buy insurance for his car in the UK. So UK market we obviously penalize people for being young. Uh they get very very expensive quotes and very
[02:13] often what they do they install what they call blackbox. So some device uh in the car that watches um monitors your driving habits and reports back into uh
[02:25] the insurer so they can actually um uh react accordingly to what you do. So our John here is actually checking his quote. He's looking to renew his policy. This is like a standard datab bricks app
[02:39] uh where he can uh where I have um quote um quoting system. So I can check my quote. I will want you to remember this number though. We're going to come back to
[02:51] this. He's getting 5,000 with this actual telematics device on. So again, one more thing I'm going to show you. I
[03:03] wanted to show you the event. So our John is a little naughty. He's gotten his policy that he shouldn't be out after 11. He shouldn't be driving very fast, but he actually
[03:15] has taken his car. So, there is some blackbox event that I'm going to trigger now. So, the um insurer will find out from his box when it's um synchronized
[03:28] that John has been actually out and there are some events uh on his uh policy now that will affect our pricing system. So, now back to some theory. How
[03:40] many of you work in insurance actually? Not many some here I will give you very very high level overview of uh insurance pricing. So the way you actually approach it is customer asks for a quote
[03:54] we need to estimate the risk. So we built models before we then turn it into the price and then they accept it or leave. in insurance um um like motor insurance in
[04:08] the UK uh or in Europe it's usually very fast process completely automated so we build the models we need to calculate our risk so how likely the claim on the policy for this person and how much will
[04:20] it cost us to cover that claim and then all this together with some other enrichments we create the price for our client so that's the price we would the best actually make
[04:33] uh money on and we serve it automatically. Now some challenges have started over the last few years. So challenge number one the pricing works in milliseconds
[04:46] these days or in seconds. So we have with the aggregator so the websites that you can fill your detail scene and they give you prices from every single insurer insurers um insurer in the market you have about 1.6 six seconds to
[05:01] turn around with the price. If you don't give them the price in two seconds, you will get penalized. Your result will be down the bottom down the bottom of the list. Potentially your client will not even see it. So the speed, we must do it
[05:13] very very fast or we lose it. Now the challenge number two within that little little time we have to calculate our
[05:25] price very well. So that means we want to use many features. Something that was impossible only recently. People would maybe go thousands of features for the
[05:37] model. Now we're looking at millions. We're looking at multiple models running in parallel creating or drawing from this um um uh from the features. The more the more detailed the calculation,
[05:50] the better the price. And we still have to fit it within real time. 1 second under one second typically with all the added overhead overhead. So that's the challenge number two for us
[06:03] and the stakes. So what does it mean for us really? If you want to catch the rabbit, you need to be fast enough because if you don't price on time, if you don't do it fast enough, the
[06:15] rabbit's gone. So you don't get that quote at all. You won't get won't get that policy. You won't get the business. Now, if you don't calculate well enough, if you don't have your systems spot on,
[06:28] you will only catch a skinny rabbit. So, the skinny rabbit means you will basically be by be getting uh the risks, ensuring the risks that other people don't want. So, the policies that not
[06:43] necessarily will make you any money. So, speed gets the quote, skill gets the quote that you actually want.
[06:58] So now we have to multiply it by hundreds per second because that's the typical
[07:10] maybe more maybe less but roughly that is the level we usually want to uh operate on. So the call that we're going to simulate here or actually run for
[07:22] real. We have the quote request. We have our single endpoint. That's our pricing scorer. It runs four models. So if you work in insurance, it would look familiar to
[07:34] you. We have frequency. So so-called predicting models. Frequency and severity. So those are the ones that I mentioned before. How likely the claim? How expensive the claim? We have some
[07:46] more here for enrichment demand model. So how likely the clients to accept? How can we actually change that price to still be accepted and make it actually more profitable for us? And fraud, which is clear. In real scenario, people might
[07:59] run 10 of those models or 15. The more the better, not always, but generally we want this to be as rich as possible. Then the outcome of this models gets to
[08:11] our rating engine with which calculates actual price that we show to the client and on the way it hits my feature store online feature store. In my case I have
[08:24] 1 million features. So my table has 1 million rows. We have tested it up to about 60 which is the number roughly speaking of the people in the UK. That's what the client wanted to test
[08:36] and we want to do it at speed end to end under 200 millconds because you know there will be different overheads there will be different enrichments maybe you want to have um uh other systems that are connected and and and do some more
[08:49] stuff maybe call some API on the way 1.2 2 seconds can be actually quite a long time uh to do many many other things but if we can cut quite a lot of that it's better. So now
[09:03] how does it work under load? And I'm going to show you again another piece of my application where I can simulate
[09:15] random quotes that are running whoops. It's too small, too big. That are running again against my pricing system. So I cranked it up straight away to 100.
[09:27] 100 is just about as much I can support from the application. If I was to run it externally, obviously I can throw much more at it. And I can see my P50 is about 100. My P65 is 200 uh milliseconds
[09:43] or less than that. But what's the best of it? You can see here I'm only running 16 of my slots uh for the compute. So if if I was able to crank it higher, it would
[09:57] then scale up. We have way way more capacity to run it even faster. So I I did some testing before. 16 actually in this case is just about on the maximum
[10:09] it can um uh so 100 QPS is maximum 16 um slots can do. But from my tests when I was going slightly lower I was actually achieving
[10:21] even even faster um uh speeds. So in this case it's operating around maybe 100 milliseconds P95 and P99. It doesn't do many optimizations here. I
[10:34] just put together those models and deploy them at the end point and they just work so fast. So that's that's how it works. Now let's stop this. So back to my sli my slides
[10:47] we have done we have shown that we can actually calculate fast and we can calculate with detail because we throw all these features at our model at speed
[10:59] and it runs it runs very fast. But there is another challenge and that is the challenge that people find extremely difficult in the market and that is how do I keep my my online data store my
[11:11] online feature store in this case how do I keep it up to date data changes by minute. So our John when he was driving at night in most places most cases the systems
[11:24] will find about it pricing systems in a week maybe in a month people use external systems to feed features to the pricing systems and they often run on likes of Cosmos moving data from your
[11:37] offline to online is a challenge. So what happens here? You price the reality. You price um the the picture of the past where your
[11:49] rabbit was however often you update your system. Can be every month. Some people do it maybe daily, but I don't think anyone does it in minutes. So our challenge is to make it also fast
[12:03] and reactive to the changes that happen uh as we go. So again, back the last piece of the demo. Remember, I've actually triggered this
[12:16] update. So now it's read back my table. The behavior score has gone down. There is one speeding event, curfew breaches, and harsh braking on the policy now. So
[12:30] we can recalculate the price. Imagine John has come back home, decided to review it, and if everything goes well, in the time I was speaking, it's already updated my my online feature store. And
[12:42] as a matter of fact, it has done it much faster. It's about 30 seconds currently, and we expect it to be much much faster in the coming months. Uh Raf Rahul will tell you why. Uh now, what just happened
[12:56] in few minutes? millions of feature rows um that uh I'm feeding to my pricing system with four predictive models. By the way, if I was to scale it up to 10 or 15, it would not affect my speed. If
[13:09] I was to scale my my features to 60 million, very unlikely to to to have significant impact on speed. We are well well within uh what's possible. Uh so five models in fact four predictive and uh rating engine in one call.
[13:23] 150 milliseconds or less at hundreds of QPS. We can obviously crank it up higher. 100 QPS is roughly what the client wants us to test. Features update within minutes
[13:36] automatically. I don't absolutely do anything with it. I just drop them in my table u and then they automatically uh so my standard feature table offline feature table they automatically go to online store. Something I haven't
[13:48] actually spent time on today. You see it everywhere but I thought it was a good idea to mention point in time reproducible. Every quote is auditable in one word. We have unity. We we uh
[14:01] check everything. We do full inference recording. So we have absolutely full clarity for regulators or for any other reasons. Now beyond insurance you can use this type
[14:14] of solutions some mostly insurance or only insurance but you can use it in many other places. Obviously frauden payments where you have to react fast and you need to know what happened what someone has done somewhere else. Maybe you get some data from somewhere you
[14:26] need to have it very fast in your life system or the other side of the slide logistics. When you have some um automated planning of the roots someone has gone the wrong direction you can
[14:39] reoot on the fly and still have those savings rather than having them going around and burning fuel. So many other options. If you have any interesting let us know we can absolutely help you to build it. Now how it works technical
[14:53] side of it please over to Rahul. >> Thank you Lauren. So can you hear me? Is there right good okay so we just saw like how we can build these things on
[15:06] the UI but what about our data teams? How they going to know like what my text are going to look like? How I'm going to combine all the multiple systems together? The purpose of this slide is to give you a reference architecture. It's pretty common, pretty standard for
[15:19] all kind of solutions. Depend how you want to use it. Now on the left, the goal is to move from left to right. On the left, your data sources could be anything like batch real-time data coming from different applications. That's fine. You can use streaming
[15:32] injection tools. The purpose here is you use lakeflow spark declarative pipelines to ingest the data into medallion structure. On the top of that you can build your features using your data bras feature engineering libraries and then
[15:45] everything is like audited have a proper lineage and governed through unity catalog and on the top of that you can bring a single source of truth data for all the features that will be pushed for your downstream application that could
[15:57] be your model serving feature serving BI analytics so on so forth everything al together so this is like important architecture because you can not only use for insurance pricing thing but for
[16:09] any solution. Now we're going to see like how act how we can what will be the end to end life cycle of feature would look like. So this is a very important part because we're going to travel the journey from offline to online. So as I
[16:23] said earlier we ingesting data from different sources. So now we're going to ingest and compute our features in the lakehouse using lakeflow declarative pipelines where you can ingest transform
[16:35] compute your features. You can create your single sort of tour features stored into medallion structure which is in the lake house architecture. Uh then once you compute your features you find all the features will be recorded into sim simple like delta
[16:48] table. I'll say okay this is my offline feature store which could be used for any of the purpose any of the reasons but when you want to promote those features for a online feature store which we're going to cover in upcoming slides you do not have to create fancy
[17:01] custom logic or like build any bespoke application jobs. What you can do is just simply use one command fe.publish table which continuously synchronously ingest your data from offline feature
[17:13] stores to online feature store which is lakebased back online feature store. Then once your data is available you can use model serving with model serving. There's one important catch here we're going to cover in slides again but
[17:27] once you using the features you do not have to pass the whole feature spec while calling the model endpoint. All you can do is just provide a policy ID, customer ID, any unique ID and with the help of um feature lookup the bricks
[17:41] automatically joins in the back end and tries all the feature values for you so that you can serve it to the consumer applications and then you have like uh everything is governed through unity catalog and MLflow is keeping the track
[17:53] of all the application what's happening in the back end. Then from the data bricks building blocks you know we talk about a lot of stuff and we have a lot of vendors lot of tools
[18:05] services we use in our day-to-day life but how you can you're combining everything all together within one platform you have delta lake and unity catalog your data is stored in open formats in a data links you have unity catalog on top of it to govern
[18:18] everything you have data uh lakeflow declarative pipelines which you just saw just to inest curate your transformation your pipelines use data bricks feature engineering libraries which will help you to run feature engineering pipelines
[18:30] to prepare your features. Then you have MLflow. So once you develop once you develop your training set then what you do is like train your model log your model with the help of MLflow and MLflow
[18:42] will manage the life cycle of your whole model. Then you have model serving you can serve the model and then the important part here is lakebased. Now the online feature source we're going to discuss in upcoming slides is it's packed with lake based autoscaling
[18:56] capabilities where where storage and compute independently scale together. You don't have to worry about just for the storage and you know um compute part. They are like totally decoupled and scale independently. Then you have
[19:09] AI gateway which we can apply more cartrails governance on the models which you're deploying through data bricks. Then okay now how am I like f like I spoke
[19:22] about online fe offline feature store. So again as I said you do not have to create any fancy joints but the most important thing is like how you prepare your trading data. All the trading data is available are built with point in
[19:35] time joints. What mean is every transaction or every code you receive is enabled with a primary key unique identifier and a time stamp the time it happened. So once you train your model
[19:48] your your model will is not getting any unknown values which values which haven't happened yet. So it's better like you record all the time sense and everything so that your model can understand the past experiences and start learning and start making
[20:00] predictions in future. This brings two important features. One is like you will not be going to get any training and serving SKUs. You're going to get the same feature experience which you got into training side. You're going to get the same thing in production as well
[20:12] while you're serving in the real time. Second thing all the training sets everything is logged into unity catalog which maintains the end to end lineage which means you can actually keep a track of like where is data coming from how are they moving from left to right
[20:26] and you can keep track of like all the audits do quality checks everything all together now you have prepared your data you have an offline feature store in the table
[20:38] that's where the real game starts in the legacy application we have seen people are building like bespoke jobs and you They are being custom logics. But here with simply just one command fa.publish table and you are describing a mode the
[20:52] way you want to sync your data. Modes could be like continuous, snapshot or trigger. I have used here continuous because I want to pull all the incremental records whatever is being ingested into my offline feature store
[21:04] to be synced directly with my online feature store which is backed by lakebase. As you saw in the demo presented by uh Lawrence, he changed he made some
[21:16] changes on the codes. So with the help of data change data free changes capability of delta, every single transaction has been recorded into checkpoints and it helps to quickly publish into our uh lakebased online
[21:30] feature store. So that every transaction with it associated with the time stamp and primary keys it keep on checking what are the new time stamp we are receiving and all together it's merged and whether you're performing insert
[21:42] update and delete everything is recorded into your online feature store. So again as I said your offline feature store is a single source of truth. Whatever things you are whatever operations you're performing on top of it it's directly in sync with online features.
[21:54] So you don't have to worry about spinning up a separate or dedicated lakebased instance. Everything is backed internally by for you. uh like in so you have to focus on what instead of how and data bricks manage the whole underneath
[22:08] then lakebase you might be going to other sessions as well so just want to quickly touch on the lakebas here so as I said lakebase decouple storage and compute so that each individual components can scale independently what
[22:20] most important thing which I really like is branching it means that instead of directly hitting your production database you can create a you know branch like a stage branch of your actual data. You can test your schema.
[22:33] Uh if you want to make any changes on your features, you can test all of those uh logics al together by branching the database. And then after that once you're happy with that, you can actually merge it. So merge capabilities and
[22:45] progress going to be there in future as well. But for now, you can simply replace your existing scheas when you're done with your branching testing into directory production. So what we just saw like what Lawrence
[23:00] mentioned like we served more than a million features less than 20 millisecond latency and they're likeund20 query per second and why it's
[23:13] happening because of the lake base itself I think this one do we have it this one >> do we have a demo one the one you showed there >> the demo the one you Oh, you want to uh
[23:27] you want that pipeline? Yeah. >> Yeah. So, you see the pipeline here. So, I don't know if I I hope it's clearly visible. So, what you just saw in the UI, everything is actually happening in
[23:39] the back end into the database or into the data bricks. It's not like you're building everything in the UI just use data bricks app just to create everything all together in the UI from the user experience point of view. But everything you saw like it's left to
[23:51] right moving ingesting the data you're creating your offline feature store you're spinning up your online feature store and there's one thing to check like we have used four or five different models here is four which you can scale to like 10 15 models but the most
[24:05] important thing here is like you do not have to create separate pipelines for every model. What you can do is you can bring all this model together in one pipeline and use py function which will wrap them together which is going to hit
[24:18] the same online feature store without impacting any performance. And then you have uh the component on the right you're serving the model you can apply uh model monitoring and other activities
[24:30] on top of it. So building the features is not the only challenge here but how you maintain other things as well that's very crucial part here. So as I said it is backed by
[24:44] lakebased online feature store but we we were creating centralized registry and at the same time unity catalog is taking care of all your data which is fully governed have a proper lineage uh all
[24:56] the permission access control everything is there. You can simply you can simplify model scoring with automated feature lookup and compute means that you don't have to apply joins while you are passing your features or applying
[25:08] any features to extract the code from the model serving and it eliminates the training and serving skew which we just discussed a couple of slides ago because we have this one unified feature store already available. So it means that you
[25:20] don't have to worry about creating uh all the data transactions are timestamped. It means that each individual transaction or code is recorded as a unique unique row and you
[25:33] your model will not be hallucinated on unknown data which is not even available yet. Um and there so this is the part like we will
[25:45] show on the demo is they were like different models all together and we just use like pyang function to wrap everything together and once you're done okay you have this final model you can
[25:57] log into unity catalog and you can start doing testing once you're happy you can move that model from challenger to champion all of these things now people will be asking how you going to deploy so you can use data bricks asset bundles and
[26:11] you can get like all the MLOps link on the official documentation. It's a pretty much straightforward process. You have you install project and deps project will um create a basic skeleton. You can have dev, staging and production
[26:24] branch all together. Once you happy with the code you can move to staging. On the staging you can start running your unit test cases and you're happy you can move to the production. So that's the next part. You can take it from here and move it to the production. But combining all
[26:37] those different models into a one wrapper instead of creating a different pipeline or like separate pipelines you can combine it all together into one wrapper. That's a key theme of that. This is the benchmark results um we have
[26:50] tested for like other customers as well. This part is not covered in this demo but it's like general available benchmarks for everything. So our online feature store you can see P50 95 P95
[27:02] serve on 20 features more than 20 million ids and when we are serving 20 50k queries per second so it's it's super performant and we have we did some good acquisitions last year which going
[27:15] to take this numbers below 10 millisecond which is going to be sub-second latency going to be soon available. Um so you want to cover this like the code part
[27:28] >> sorry >> the code part you do made the changes and everything so okay so what Lawrence made made the changes on the UI so what happened is like we have claim data ingested we store into medallion we we
[27:40] made some computation on the feature we store in the golden tables and silver tables that is our offline feature store and then you sync it using publish table command and will be synced with the lake base and then next code sees the new
[27:53] risk score. So that will be treated as a past entry historical data for the model. So everything just happened less than 5 minutes end to end.
[28:07] And again I'm going back to the governance because that's super critical for any application. You have leanages end to end. You can have a look audit data quality checks and everything all together. And the most important thing
[28:19] to understand is like datalog managing your data, governing your data, maintaining the access control. On the top of that you are storing your machine learning models as well. And you can you know on the top of that it's going to
[28:32] store your inference logs. You can build your data quality dashboards for your model performance and everything. So all through the unity catalog which is baked in um demo we have seen like show on the UI
[28:45] side but the future of feature store how it's going to look like what we are aiming how it's going to look like now so today we have online feature store but I mentioned we are we acquired uh tecton so tecton is something which has
[28:59] the real time feature computation capability and serving capability which means that now you can serve those feature less than 10 millisecond which is going to be sub-second latency you can achieve it and we're going even
[29:12] further one one more level up by introducing fennel fennel is another product we acquired last year so fennel is it's more to boost our feature engineering capabilities so for example
[29:24] right now we doing all the feature engineering application in the batch fashion but in the future we wanted to make it happen in the real time so the moment you get the feature it automatically runs out all the feature engineering pipeline in real time and
[29:36] quickly within seconds it directly push you to the offline feature store and in sync with the lake base. Um the next one oh that's me again. Uh am I yeah so uh fennel actually will
[29:49] help us to achieve subsecond changes to the online feature store rather than multiseconds as it is now. But I think what what is also important here, we have just shown you how we apply all this great tech to actually resolve some
[30:03] real business problem that talks to the business that is understood by by business decision makers and we're building way more than that. So this is probably only interesting to the insurance people but we are implementing quite a lot of uh well building demos on
[30:17] on um specific business processes uh to show that you can run full business process end to end on data bricks or integrating uh with others. So what you have seen uh today was actually part of
[30:30] what I call pricing workbench. Uh so everything that we are showing is real. Uh it's implemented from ingestion through um um the offline feature store. So I have here modeling mart. I have
[30:42] model development. All these models are developed here. You can compare them. You can um control them via governance. And the piece that we were looking at is only small part of my demo. That's model deployment. And obviously since we
[30:54] haven't mentioned single agent in this presentation, we also have agents in this. Cool. I think that's all from us.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.