Benchmarks Don't Lie: Inside Lakebase Performance Engineering

Summary

Lakebase is Databricks' cloud-native OLTP database built on Postgres with disaggregated storage, and this video documents the performance engineering journey the team took over the past year to improve write throughput, read tail latency, and scalability under concurrent load.
The team used HammerDB TPC-C workload benchmarking to isolate bottlenecks across storage, compute, and network layers, with key optimizations including write-ahead log and safekeeper replication improvements, read tail latency reductions through page compaction, and parallel storage path optimization.
Competitive benchmarking results against EBS-based Postgres show Lakebase's latency and throughput advantages, and the team notes performance engineering is ongoing with further optimizations expected in the near term.

Benchmarks Don't Lie: Inside Lakebase Performance Engineering

Get an inside look at how the Lakebase engineering team designs benchmarks that reflect real-world workload patterns rather than synthetic best-case scenarios, and how we isolate variables across the storage, compute, and network layers to understand where time is actually spent. From there, we'll walk through results from our latest engineering work, covering throughput under concurrent load, latency across workload types, and the optimizations that moved the needle most. This is not a marketing benchmarks talk. It's an engineering post-mortem on what we tested, what surprised us, and what the numbers mean for how you architect and tune your own Lakebase workloads.
Talk By: David Wein, Principal Software Engineer, Databricks ;

Chapters

FAQs

What is Lakebase and how does its storage architecture differ from traditional databases?

Lakebase is Databricks' OLTP database built on Postgres with cloud-native disaggregated storage, separating compute from an object-store-based storage layer. This architecture enables cloud-native scalability for transactional workloads while remaining compatible with the Postgres ecosystem.

What benchmarking methodology did the Lakebase team use?

The team used HammerDB with a TPC-C workload, a standard industry benchmark simulating realistic OLTP transaction patterns rather than synthetic best-case scenarios. The methodology focused on isolating variables across the storage, compute, and network layers to understand exactly where time was being spent under concurrent load.

What were the most impactful performance optimizations described in this video?

Key optimizations included write-ahead log and safekeeper replication improvements, read tail latency reductions through page compaction, CPU utilization analysis to identify real bottlenecks, synchronous replication overhead reduction, and parallel storage path optimization. This video frames these as part of an ongoing performance journey, with more work planned.

How does Lakebase compare to EBS-based Postgres in benchmarks?

This video presents competitive benchmark results comparing Lakebase against EBS-based Postgres across latency and throughput dimensions, with the graph referenced from a Databricks Summit keynote. The engineering context built up throughout the video explains what the benchmark results mean and how the optimizations contributed to the performance improvements shown.

Full transcript

[00:09] All right. Uh hello everybody. Uh my name is David Weine. Uh thank you very much for coming on after lunch on the last day of the conference. Uh it's uh it's a lot of dedication to be here. So
[00:23] thank you very much. Uh I'm an engineer on the lakebase product and today I'm going to talk about benchmarking and performance. Really this is going to be a story on the performance journey that we've taken with lakebase uh over the
[00:37] last year since last uh summit. So um if you know the last day of the conference some of these slides look familiar forward-looking statements. Some of the stuff I'm going to show you is forwardlooking in terms of where we are
[00:50] on performance. So this is the safe harbor slide. Uh and then please do complete the surveys. Um they're very helpful and uh I would greatly appreciate uh seeing the feedback so we
[01:03] know what to do better next time. All right. So what this talk is um this is supposed to be an engineering talk and not just a marketing talk. All right. I do have some you know markety type
[01:15] slides at the end but mostly I want to walk through the process that we took to improve the right performance on lakebase. So I'm going to cover like what sort of the benchmarks are uh how
[01:27] we used evidence to find where the bottlenecks were and then the optimizations that we took. Um I'm going to you know try to give a lot more context. Um and then performance work is
[01:40] never done. This is kind of where we are right now. But uh you know after I'm done talking here, we're going to head back over to the office and keep going and uh hopefully we have a whole other set coming out in the next uh few weeks.
[01:52] This talk is also a little bit of an explanation of what this slide was. If you caught this uh during the keynote, Nikita's keynote, he showed a latency versus throughput graph for Lakebase verse some competitor systems. Uh I'm
[02:05] going to, you know, this is sort of going to be at the end of the talk. We're going to build up to what this slide really is and what it means. All right. So, uh let's start with a little bit of lakebased architecture
[02:18] recap uh just to make sure that we're grounded in what we are talking about and we're on the same page. So, I if you've been you've been here all week.
[02:30] If if you haven't heard this from us before 17 times already at this conference, we haven't done our job. So you know uh hopefully you've seen something like this before. Lakebase is an OOLTP database for transactions. So
[02:45] we are talking about high right throughput low latency transactions. It's fundamentally a Postgress database with a specialized storage system beneath it. Um and a real pitch on
[02:58] Lakebase is a unification of your data lake, your analytics and your operational system. Okay, so I'm going to go like I said supposed
[03:10] to be an engineering talk not a marketing talk so not too much stuff on those bullet points but let's just recap what this actual architecture is. I talked I mentioned just now that it it's a Postgress system with a customized
[03:24] storage layer beneath it. So um if you uh know Postgress at all you may you may be familiar with Postgress heaps which is the way the tables are stored. The data is stored in rows versus columns in
[03:38] a typical OLAP data lakeink. So in lakebase we actually use all the same Postgress data storage formats. It is Postgress. So the tables are Postgress heaps. They're Postgress rows. They if
[03:52] you were to pull a page and do a binary dump of a page it would look like a page in any you know open source Postgress system but rather than sticking it on local disks we decouple and disagregate
[04:05] the storage system. So um we have a three layers of storage effectively when you generate uh transactions inside your Postgress system rather than sticking the write ahead log which is the log of
[04:19] transactions in the database it's the log of the changes made rather than sticking that on like a block device like an EBS volume or just on a file system we send it off to a layer that we call safekeepers. Safekeepers are a set
[04:34] of computers in three different availability zones whose job it is to harden that log and persist it to disk giving you the D in acid. So the
[04:46] durability of transactions. Okay. So when you commit a transaction in Lakebase the log records are sent off to this safekeepers in multiple availability zones. multiple copies are
[04:58] persisted on disk and then the transaction is considered committed. The uh client can be acknowledged. Those safekeepers are just uh uh they're just um persisting the log. They then uh
[05:12] archive that log off to S3 or object store of your choice depending on your cloud. Uh and then they also ship it to another layer that we have called page servers. Those page servers take those log records and they reconstruct them
[05:26] into the actual Postgress pages. So you have you do an insert. You have some log record that Postgress generated that says put this data on this page at this spot. So the page server receives that
[05:40] log record says okay great pulls that page off of its cached copy on disk applies the change writes it back to all right that um that forms your Postgress pages those pages are then available to
[05:54] be read back to Postgress to serve queries uh if it's no longer in cache and then they're also archived off into the object store. So that's like the difference that that's the core storage
[06:07] architecture of lakebase. All right. Now um this is a data bricks conference. A lot of people here doing data science, data analytics. OLTP might be a little bit of a foreign beast, but
[06:20] let's just go through the life of an OOLTP transaction real quick. Um, so, uh, let's do, um, just something simple like, um, uh, update accounts set
[06:34] balance equals, uh, balance plus $1,000, right? So, you you deposit a,000 bucks. So, the first thing that has to happen is the database has to receive the query. So, the query comes from the
[06:46] client with the update statement. Database will parse that and plan it into a set of uh, steps that the execution engine takes. This is actually kind of the same way uh OLAP queries work, right? Um it will then execute the
[07:00] plan going through what it needs to do. Uh it will um touch uh different rows. It will read pages from cache or from disk in order to find those rows. It
[07:12] will generate log records describing the changes that it's making. it will write that log record out. Um and then that is the the in our case sending them to
[07:24] safekeepers. We need to wait for the uh log you know commit time we have to wait for the log records to be durable so there's some time to uh commit and then we return
[07:36] results back to the client. So it's pretty simple. Receive a query, parse plan, execute, do some IO as necessary, finding rows and updating rows, generate change records, send change records to disk, then return to the client that you
[07:50] got it. All right. So when we look at performance, it's kind of like how much time is being spent in each of these steps, right? And with uh transactions and OLTP, it's oftentimes less about the
[08:04] CPU and what the CPU is doing, what it's busy with. It's usually what is the transaction waiting for? What are the things that are causing these various steps to actually pause execution and
[08:17] generate latency? We'll come back to those weight events later or those those weight things later. Um but just wanted to set that up.
[08:32] Okay, now more into performance benchmarks. So, um our performance benchmarking though we run you know kind of a variety of different micro tests. We really like uh something called Hammer DB and TROx C. So, who here has
[08:46] heard of TPCC? A lot of folks. Okay. TPCC is like the standard uh OLTP accepted benchmark going back to I think
[08:59] the early 80s and since then it is what transactional databases have used to compare performance against each other and um the thing is TPCC to do an actual
[09:12] TPCC result you need to have audited runs and there's all these rules and it's actually an extremely complex and expensive process and most people don't do it anymore. Instead, the TPC, the
[09:24] Transaction Processing Council, sponsors something called Hammer DB, which is an open-source alternative. And instead of TPCC, they have TROX Cy.
[09:37] Um the the term TPCC is actually copyright reserved for only audited results, but uh they say no, use TRXY. It's a really good approximation just like there's uh TPCH, there's TROCH. Um
[09:52] so, uh this is what we use. It's basically TPCC. It's a warehouse order entry, delivery, and processing system. And it's it's been around for so long because people understand how it works.
[10:04] It does a pretty good job of showing an OLTP workload. Um, and it's also something that translates well when talking to customers, talking between vendors, things like that. So that
[10:17] that's what we do. All right. So what's our first thing? So we got to work on performance afterlast dice. All right. And uh we found that
[10:29] our numbers down at like the really low end like one uh vcpu were not bad. That's actually pretty good. That's better than a lot of other systems. It's 24,000. Uh this is the the
[10:41] number that you see for TROxy or TPCC is called new orders per minute. The transaction workload runs a bunch of different transactions. It runs a transaction order status. Uh I forget
[10:55] the name of the other ones, but the most important one is uh new orders. How many orders can be placed? And so it measures the new orders per minute that were run. So this is 24,000 new orders per minute.
[11:08] It's actually pretty good for one CPU. And then it scaled pretty well from 1 to four CPUs up to 76,000 orders per minute. But that was all right. Like we're like cool. Uh so let's let's
[11:21] multiply the CPUs by four times. Give it 4x the system. Nope. No more. Uh went up to 24 vCPUs. We got a little more. But clearly this is not this is not where we wanted to be. All right.
[11:37] So to understand what's going on, we built a benchmark service. So um this is something that we have internally. We set up a um sort of a workflow system that can run a variety of workloads,
[11:51] compute sizes, concurrency levels, data sizes and things like that. Um we built it actually so it can run across a number of different Postgress cloud providers. So if I want to compare
[12:05] LakeBase to one of our uh competitors, actually I can just choose in my little guey that I want to do this one against say um uh Aurora or I want to do it
[12:17] against Lakebase. I can pick a region. I can pick a size and it will run all these tests and give me give me a nice report. So um not only does it uh run a
[12:29] workload, but it also stores the results for us, which is very important for seeing things over time. Um and if you're doing database benchmarking internally and trying to to make the
[12:41] database better, having those numbers in isolation doesn't really do you any good. Like if I just like if somebody said, "Hey, David, we ran the test and we got like uh we we got 78,000 on 16
[12:55] vCPUs." I'm like, "Great." Like, so what what the heck does that mean? Like there's abs I can't do anything with that. So wired into this entire benchmark system, we also are scraping uh metrics from the system. And we
[13:09] actually do this from SQL. So I can, you know, scrape them against uh the different providers as well, so I understand how they work. But we query, we collect a bunch of PG statistics.
[13:21] Um we just do some baseline latency probing with select ones just to see how the system is just responding to pings. Um we capture Postgress weight events which I'll talk about in detail. We
[13:34] capture um the different lakebase has a metrics API you can pull some metrics from. So we capture all that um and that all just goes in a database along with the results. All right. So now we can have uh when we have a number we can go
[13:47] and look and have an explanation for why we hit that number. All right. So onto our journey of making this thing better. So uh Postgress has something called full page writes and we knew that that
[13:59] is sort of a historic problem for Postgress scaling. All right. So with a full page write basically after every checkpoint in Postgress when you write it to a page the first time you touch a page you put the entire Postgress page
[14:13] content into the log not so like if that that example where I'm updating an account and I just have one little update record there that should be like I don't know 100 bytes 200 bytes we'll
[14:25] say hey this is the first right to that page since the checkpoint we have to put the entire 8 kilobytes into that log record. So it massively bloat the log. Um and you know in these workloads we
[14:39] know it can cause like 15 times inflation of the wall traffic. But with our architecture we don't actually need to do this. So what what is this whole thing about anyway? Like why what is
[14:52] this full page rights? Why does it need to be there? And why don't we need it to be there? It's there to protect against something called a torn page. So, um, databases have page sizes, right? The,
[15:04] um, Postgress's page size is 8 kilobytes. Um, you can actually compile Postgress to have a variety of different page sizes, but no one ever does that. Everyone runs 8 kilobytes. So, Postgress is effectively 8 kilobyte page. However,
[15:18] disks do not have 8 kilobyte sectors. They typically have 512 byt sectors. So when you write a page to a disk, you write that 8K to a disk, it's going and writing uh 16 different sectors to fill
[15:33] 8K. And if your machine crashes while it's write like writing one of those pages, you might have three sectors from the new page and uh 13
[15:45] sectors from the old page. And when you go to read that back from disk, you'll get what's called a torn page. You'll get something that's half and half. And you cannot recover from that like you can't just apply the insert record and
[15:58] recover from that. So that's why Postgress when it uh after a checkpoint it writes the full image of the page. That means that during recovery even if
[16:10] it reads a garbage page from disk the first log record that modifies that page will have a clean copy of it in case there was a torn page it can apply the log record and it guarantees durability and recovery. If the Postgress has a
[16:24] flag to turn full page rights off, if you turn it off, you are typically risking data corruption if the machine crashes. All right, that's how that works. However, we don't actually need to do that because if you go back to
[16:39] boop, I'll just go back a few slides. Go back to here. We're not writing POS pages on disk. We're sending wall records to a different wall service. and
[16:51] we have a wall materialization service and we take care of all of that in an entirely different way where it is impossible to get torn pages. So therefore we don't need to ston. All
[17:03] right. So uh so we suspected we had a problem with full page writes. We looked at those benchmark runs. As I said they capture all the Postgress statistics in them. We looked at a Postgress statistics table
[17:17] called PG stats wall. We looked at all the other stuff and we found we were writing 58 on average 58 kilobytes of data per transaction. All right. And we were doing, you know,
[17:29] let's see, in that 78,000 uh new orders per minute, like couple hundred thousand uh transactions a minute. That's a lot of data at 58 kilobytes. And that 94% of that was
[17:42] actually the full page images and not just the changes. Okay. Um uh the other thing was that 28% of the records actually carried a full page. So that was bad. All right. So the first thing
[17:55] we did, we're like, okay, what's got we got to fix this. We don't need to do it. So the obvious thing is like, hey, just turn off full page rights in Postgress and you're good to go. Well, there was kind of a catch.
[18:09] See that storage system I talked about. I said we don't write Postgress pages to disk on Postgress. I talked a little bit about this whole pipeline. I just showed the sign the slide. We write wall
[18:21] records to a safekeeper service and then that safekeeper service sends those down to page servers. Those page servers apply them to the Postgress page and then that is how we have a Postgress
[18:33] page available to read back if we need it. All right, the thing is it doesn't apply every log record as soon as it gets it. That would be very inefficient. It kind of batches them up and applies
[18:46] them when it feels like it. Um and so it was kind of saying the storage system was going hey we got this actually really convenient thing happening the database keeps sending us all these full page copies so this is awesome like if I
[19:01] I don't have to do all this application of log records all the time to rebuild Postgress pages I'm still getting these full pages from the database sweet easy um but if we turned off full pages those
[19:13] wouldn't be happening and now we would be building up long chains of log records in in the storage system and all of our reads would become slower. All right. So
[19:25] we we did what we called image push down or full page push down and we taught the storage system to more deterministically generate full pages based on the log records rather than just relying on them
[19:38] coming from the database server itself. All right. So what that gave us slack stupid slack um what that gave us is basically more
[19:51] predictable compaction of full pages down in the storage system. It improved our read tail latency. It did not it meant we did no longer needed to send full pages off the server and it in
[20:03] drastically improved our right performance. So, uh, what we did is we were actually able to come up with a way to apply this to every single lakebased system without the customer needing to
[20:15] reboot or anything. It just rolled out seamlessly. We rolled it out through our storage system. And um, on our benchmarks, we got 5x better write throughput. We reduced our tail latency for reads by 2x and we reduced the
[20:30] traffic that was getting sent to the safekeeper significantly. So that was awesome in the benchmark but just want to point some real world benefits. These are some dashboards that we had from a couple of customer use cases. This first
[20:43] one here uh on my right uh my right is a sync table use case. So this is in lakebased customers doing sync table and they're doing continuous
[20:56] sync and before we enabled that change on their system they were getting uh I don't know 19,000 rows per second and then we flipped that change on and they got over 60,000 rows per second. So this
[21:10] like made a this made a significant difference for real customer workload. Um the other slide um the one that's on uh my left uh this is a production AI
[21:23] lab um at a very uh very high-profile customer and this is one of their systems during the day and this is transactions per second that they're processing and we flipped the uh we
[21:36] flipped it on there's a thin blue line when we flipped the switch but basically you can see that they're now getting they were able to get significantly more transaction actions done. It actually generated much less log traffic and so
[21:50] that was the real world impact of that change. Okay. Now, how did that show up in our benchmarks? Awesome. We've got that done. We went from 78,000 up to almost 200,000 new orders per minute.
[22:04] So, that was two and a half uh times improvement. That was super awesome. Um and we're all very happy. It was great. So um the next step is that we didn't
[22:16] really get as much of a gain though going from 16 to 24 vcpus right so like from um like we we we're scaling uh we
[22:28] had a huge scale up from 4 to 16 but then from 16 to 20 not that much or 16 to 24 not that much. So what's the next layer doing performance work it's always onion peeling you're taking layers off.
[22:41] All right. So, we we solved that one problem. Now, we have to see where we're losing our time. So, on an OOLTP system, you're or really any data any like
[22:53] actually probably any any program that's running, you're either on the CPU or you're waiting for something, right? Um and in an OOLTP system, you spend most of your time waiting. the there's um the
[23:06] CPU work that's done um for the most part isn't done like churning through data. It's done to kind of find a specific thing or or coordinate or wait or or pull the little data and you spend
[23:19] most of your time waiting. So like the time on the CPU is like parsing, planning, executing, you're formatting results. You want as much of that actually happening as possible. But in
[23:31] reality of if you look at the wall clock you're spending most of your time doing network round trip uh blocking on locks waiting for writes reads or writes from disk things like that.
[23:45] So how do you track that? Well Postgress has a system called weight events. Um, and anytime your Postgress backend is blocked on something or waiting for something, it sets a wait event and
[23:57] says, um, you know, I'm waiting for say a page lock or a transaction lock or something like that. And you see those in PG lock. And if you look in PG stat activity, there's a weight event column
[24:10] that tells you what you're currently waiting for. Okay. Um we enhanced this system in lakebase to actually track this uh on a metrics basis in the system
[24:23] with uh with counting the number of times we hit each weight and the time to the microscond that those weights took. We also added a bunch of lakebased specific weight events into the system
[24:35] so we could see what was happening and we made that available through SQL. And so if you're running lakebase and you're really curious what's going on in terms of weights, you can run uh a
[24:47] couple SQL commands. So we have something called uh neon backend weight report um which you can give a Postgress PID to and like this example will give you a 30- second report of everything that backend waited for over the last 30
[25:01] seconds. All right. And so you know it this one particular it was doing some prefetches. It was waiting for the clients. it was writing to disk and had some lock contention. So you can see the
[25:13] number of weights, the weight time and the milliseconds per weight. You can see that for the entire system or specific backends, but we integrated collection of that to our benchmark service and it's actually being integrated to our
[25:26] new uh lakebased observability product that's coming as well. So back to the life of an OLTP transaction, but looking at it as weights. Um most of the the pars and
[25:39] plan that's on CPU, no weights. Uh execution. Um you may be doing some IO, you may be taking some locks. Uh reading from disk. Well, that's obviously uh you're you're looking at cache or you're
[25:53] fetching from storage. So again, you may have lightweight locks to coordinate on cache. you may have IO. Um the biggest weight is usually um waiting for the commit to become durable. It has to go
[26:05] to three a persisted to disk. And that weight event in lakebase is you know it's uh not intuitive but it's called IPC syncrep. It stands for interprocess
[26:17] communication synchronous replication. That's where that comes from. So that's the time that's spent there and then you have to send time sending results back to the client. So we created a profile of weight events
[26:30] and we saw on our 16 vCPU run uh before we did that optimization we spent most of our time throttling on the log and then uh 34% of our time in commit which
[26:43] is kind of where you want to be actually a lot of your time in commit but after we fixed that we got rid of the log throttling and we sp spending the majority of our time in something called LFC lock like And we had this LWLSN
[26:58] lock. So we're spending 50% of the transaction in these locks. That's that's no good. We also showed we had some graphs internally that we use. And this is
[27:10] showing kind of the weight event stack and the different weight events as you increase the number of users in the database. The big thick dark green line is this LW lock on something called LFC.
[27:24] And so you could see as we increased the users that just completely dominated and took all the time in the system. So next step partition hotlocks. So what
[27:36] the heck's an LFC and why do I care? So um Postgress has um uh or let me back up for a second. In a OLTP database like Postgress, you want you have to have
[27:48] data buffered into your cache in order to operate on it. So pages are read from disk. They go into uh a cache in Postgress. It's called shared buffers and then it's operated on there.
[28:01] Postgress uh shared buffers are presently fixed at boot time which means you have to configure the number that you have when you boot the system. And then a database usually that's where you want most of your memory. Uh however and
[28:14] it's fixed at boot time. So it really needs to be set to the size of compute that you have. But our compute is autoscaling and it doesn't have a fixed memory footprint. It scales up and down
[28:27] without having to reboot. So we can't really boot up and tune shared buffers appropriately. So what we do is we have a fixed number of shared buffers and then outside of those we use something
[28:39] called local file cache or LFC which is some local NVMe storage that we scale up and down with compute. And so when we read remotely from page servers, it goes
[28:51] into this local NVMe cache and then it shuffles in and out of shared buffers. And as the compute uh increases and decreases, as it autoscales, we autoscale this LFC.
[29:03] Um, all right. So this LFC lock is the way that we protect multiple Postgress backends or users from colliding with each other as they move pages in and out
[29:17] of this LFC. All right. And that was dominating our time. So what we did is we partitioned this. There was another lock there that was in called LWLSN was to do with some other important caching
[29:30] related to our storage. Both of these were easily partitionable. So we partitioned these into multiple different locks and we were able to reduce the lock time there from 44% to 22%. And now we're back doing uh the
[29:44] majority of our time on what we want waiting where we want to wait which is waiting for SQL from the client and waiting for the the logged disk. And that pushed us from 198K to 268K.
[29:58] So that's a pretty good improvement. All right. But uh you might be like, well um that's also still sounds kind of inefficient. You're doing you still had all that lock contention there like
[30:11] 20ome% of the time. That's still not good. And you're still shuffling data between these shared buffers and this LFC. So um that's another opportunity to improve. All right. So, um, this is what
[30:24] we're currently working on and we've got we're we're deploying right now, which is to more appropriately tune our shared buffers to a better size based on your min CU. We have optimized all those code
[30:37] paths that go between shared buffers and LFC to maximize the actual effective caching and reduce the locking. And then we've improved the whole way that we're managing virtual memory on the machine.
[30:50] um we run inside of a microVM, not not a container. Containers are not security boundaries. We have to run in a microVM. And so we have to do a lot of tricks with what's called huge pages and OS page tables in order to have more
[31:03] efficient large memory in these virtual machines. So after we tuned the memory there, we went up to 614,000 new orders per minute.
[31:16] and um and our lock contention was down to less than 5% of the profile. All right, so where does Lakebase stand today with
[31:28] all these changes in this journey we've been on? So these are our latest measurements. This is on the 16 vCPU machine and this is sort of that that slide that that Nikita showed at the
[31:40] keynote but just in a different form. So we started there on the bottom 77,800. And an important thing with these OOLTP numbers is it's not just how many
[31:53] transactions, but at what latency. What latency did you get? And in our lab, we look at um this slide is showing the P50 latency. I'll show you a better latency on the next one, but we say look, it's
[32:06] an OOLTP transaction. Anything that has P95 latency greater than 20 milliseconds does not count because you can you can kind of tweak these things and I can
[32:18] like get another 20% throughput if I crank it so that my latency goes up to 200 milliseconds but that's not OLTP. Um, so LTP we're talking like 20 milliseconds or less, right? So here we
[32:32] started our P50 latency was 8 milliseconds and we got 77800. Um, uh, one of our one of our data bricks's friendly competitors um, also
[32:44] launched a a Postgress service after Lakebase was announced. Their Postgress service uses a more traditional EBS volume, so a block device in the cloud. It doesn't have any of the disagregated
[32:56] compute and storage. Uh we benchmarked it. Um and uh they they got uh 195. That was their I I did a bunch of runs. The best I could get out of that system at 16 vCPUs was 195 or we'll round it to
[33:11] 196 at 15 milliseconds. >> The latency is that the select one? >> No, that is the transaction time for the whole new order. So that's the new order transaction time to place an order in
[33:24] the system. All right. Um a what we call cloud gen 2 another vendor that was kind of the first one to introduce disagregated compute and storage and have a similar sort of
[33:37] architecture but the one that's maybe 12 or 13 years old. Um they they have much better throughput. uh they got up to 444,000 at 16.73
[33:50] and on our system which I have the forward-looking thing because we're still deploying these changes. We're at 614,000 at 5.2 milliseconds. All right, you'll be like, well that's that's
[34:02] awesome. That's but that's your P50. What about the tail latency? We don't care about P50. We care about what like the tail latency is on these transactions. Well, actually the funny thing is if you start looking at tail latencies, those other systems cannot
[34:16] perform as well. Like that's the P50, but if you look at the P95 latency and you say, you know what, we want our P95 less than 20 milliseconds. Well, so let's see what kind of results
[34:29] you get when you say, I want P95 to be 10 milliseconds. P95 at 10 milliseconds. Well, um, our first system where we started, it didn't go down that much because it wasn't, it was pretty crappy
[34:42] to begin with. Um, cloud, uh, the, uh, Gen Z service, the one that launched a little bit after we did using a very old architecture. They went from 195 to down to about 122,000 at 10 millisecond P95.
[34:58] the um sort of pioneering disagregated architecture went from 444 down to 280 when you look at P95 latency. All right?
[35:10] And ours stayed at 614 because even though our P50 was 5.2, the P95 was only 9.5. So, we feel like on this size system,
[35:24] we're in pretty good shape right now once we roll these out in terms of this benchmark. Now, I keep saying 16 vCPU. You're like, what does that like mean? Like, does that is that a meaningful thing? Um, if you're used to AWS
[35:37] computes, that's a 4x large. Um, if you're used to say like Aurora serverless, that's a 64 ACU system. Um, that's uh 64 CUS in Lake Base. Um I
[35:50] think yeah 64 CUS in lake base. It's what we think is like P90 to P95 biggest system that is actually out there in reality. You can go much larger but
[36:03] those are less common like again probably P90 P95 of actual customer sizes that they run will be this size or smaller. Now we still want to do better on the upper end and we're working on
[36:16] that. And the other thing because I do not want to mislead you is that this is a cached workload which means that the working set that I'm working against
[36:28] here like the the the warehouses the orders that I'm doing transactions on those fit within my RAM. I'm still writing my transactions to disk but I don't have to do a bunch of random reads all the way through the stack to process
[36:41] the transactions. The working set sits sits in memory. So for working sets in memory on a very common, fairly large um system, we're doing great. So what's
[36:53] left for us to do? All right. Um I really want to get better on more read heavy working sets. So where you're reading now mixing your your working set is much larger than
[37:06] cache and as the part of the transaction you have to keep hitting the disk. Um, that's kind of the next thing that we're going to tackle and improve on that. The latency right now in Lakebase is
[37:18] actually pretty good. Like I'm really happy with it. It's like around a millisecond or less to read a page, but we're not getting enough eyeops. So, it's not scaling as well on that chart. So, we're definitely going to go to work
[37:30] on that just like we went to work on this. Um, that whole Postgress shared buffers are fixed at boot time thing, that's got to change. So um we actually um we're working with the Postgress
[37:43] community in open source to get that checked in for the next version of Postgress. Um there's a principal engineer in open source at Microsoft that had been working on a patch for that in Postgress. Um the reviewer of
[37:56] the patch is one of the co-founders of Neon and so is on our lakebased team and um one of the engineers that's driving that at lakebases is here at the conference. The group of us all met up at the Postgress committers conference
[38:08] last month and it's all being publicly discussed. Patches are being exchanged on the Postgress mailing list and that should hopefully get into Postgress 20 but we'll deliver it on lakebase ahead
[38:20] of that. And then we're also doing more work on that transaction commit pipeline uh getting even higher throughput out of our wall system and our wall proposers. So, um, that's sort of the story I
[38:34] wanted to tell. I know it's like pretty deep on some of the internals here, but I just wanted to show you like the type of work that we're doing, how we're going about it, and the way we are investing in lakebased fundamentals.
[38:46] like this conference you heard an awful lot about like really cool things that we're doing with LTAP with lakebased integration with the lake all these you know great things that unlocks a lot of
[38:59] value and that we can do with data in the object store but I didn't want you to think that we weren't working on the core transaction stuff as well we're doing all the fundamental work to make this a blazingly fast transactional
[39:13] database and um that's the uh that's that. So, thank you very much. Um, thanks for coming. I hope you had a great conference.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.