Serving Data at Scale without Redis: Why Lakebase Works

Summary

Superhuman replaced Redis with Lakebase to serve online data for its productivity suite, tuning the platform to handle 50,000 requests per second with read replicas and connection pooling.
The migration enforced strict data freshness SLAs using Change Data Feed-driven syncs and carefully configured query timeouts to maintain production reliability.
Switching from Redis to Lakebase delivered significant cost reductions and reduced operational complexity by eliminating the need for a separate caching stack alongside the data lake.

Serving Data at Scale without Redis: Why Lakebase Works

Online data serving at scale requires managing separate Redis or DynamoDB stacks alongside the data lake, adding latency, infrastructure complexity, and operational overhead. Superhuman evaluated multiple approaches and discovered Lakebase could consolidate this architecture while maintaining strict SLAs and significantly reducing costs.
This talk covers how Superhuman modeled entities, tuned Lakebase capacity and read replicas for 50,000 requests per second, enforced freshness with Change Data Feed-driven syncs, and optimized connection pooling and query timeouts. See production code examples, configuration patterns, and before-and-after metrics comparing Redis and Lakebase for latency, maintenance burden, and cost.
🤝

Chapters

FAQs

Why did Superhuman replace Redis with Lakebase for online data serving?

Superhuman found that Redis lacked the visibility and created operational complexity when running alongside their data lake. They evaluated multiple approaches and discovered Lakebase could consolidate the architecture while maintaining strict SLAs and significantly reducing costs.

How did Superhuman tune Lakebase to handle 50,000 requests per second?

Superhuman tuned Lakebase capacity and read replicas to meet the throughput requirement. They also optimized connection pooling and query timeouts to ensure reliable performance under production load.

What is reverse ETL and why does Superhuman use it?

Reverse ETL is the practice of using data from the data lake to improve the product itself rather than just for reporting. Superhuman uses this concept to power use cases like adaptive growth and personalization across its suite of productivity apps.

What production challenges did Superhuman encounter when moving to Lakebase?

Superhuman encountered challenges with proxy configuration, connection management, and setting appropriate query timeouts in production. The video walks through how they addressed each of these with specific configuration patterns and code examples.

Full transcript

[00:09] Hey there. Let me ask you a question. How many of you, please raise the hands who have used Redis to serve data online for in production? Okay. Okay, quite quite a few people. Okay, and how many of you have used Lake Base?
[00:25] Okay. Well, that kind of makes sense because Redis is a very well-known and mature technology and it's been around for years in the industry and it's been a default um answer when the question of serving data online came.
[00:40] But at Superhuman, we decided to challenge this status quo and spoiler, Redis was not the answer. My name is Michael and today we're talking about Lake Base. This is a standard forward-looking statement uh legal disclosure basically
[00:58] stating that all uh all that we say here is at best effort and there are no guarantees, but it's pretty standard for all of the sessions. Yeah, let me tell you a bit more about myself. Um my name is Michael Kobaliev. I'm a software engineer at Superhuman,
[01:15] formerly Grammarly. I've been uh for around 7 years uh in the industry with focus on back end, databases, data stores, lately data in general and a fact fun fact about me, I am practicing a handstand. Hopefully,
[01:31] we'll be able to do it soon. Well, we'll see. Yeah, I joined Grammarly around 3 years ago and last year we acquired Coda and Superhuman Mail and changed our corporate name from uh Grammarly to Superhuman to reflect the
[01:47] change from single product company Grammarly proofreader to multiple product productivity suite of apps and agents that now includes uh Grammarly proof reader, Coda Docs, Superhuman mail, and uh Grammarly goal. And the uh project that
[02:04] I'm presenting you today actually powers the growth funnel for all of them. So, today we will talk about the use cases for serving data online, our setup, how we approach this problem, and our challenges and learnings uh from the
[02:21] way that when we changed from uh Lake base to uh from Redis to Lake base. So, let's get started. Why serve data online? Actually, uh serving data online is a not a new idea. It's very well-known in the industry under the term reverse ETL,
[02:39] and um it kind of makes sense. A lot of effort is put into building a high-quality data sets in our data lakes. We replicate application databases, we import data from external sources. We have tons of ETLs
[02:56] processing logic, all to have useful data uh in the end to draw insights from. But, a logical question is why not use this data in the product itself to improve the product itself? And that's what we do.
[03:12] Uh our main and most important use case for serving data online is adaptive growth funnel. So, for example, we want to show users um some usage-based special offers, such as free trials, discounts, or promotions based on what
[03:29] users have used uh in the past, or if they use specific features. Or alternatively, we might might want to personalize the upgrade UI or upgrade copy, uh also based on some usage behavior of users.
[03:44] Another use case is to uh improve the product itself, the product features itself. And for example, now that we have this uh sidebar with agents, we might want to play around with ordering those based on usage user persona, what user have used
[04:00] in the past, and things like that. A more general use case uh is to just feed the historical user context for uh ML models and LLMs. For example, we have a a proofreading model um that requires a language to be
[04:17] passed, and the language detection algorithm falls short in some cases, for example, when the text is short. In these cases, we might want to use the historically preferred language uh of the user, and it actually improves the quality of suggestions.
[04:34] So, uh with those use cases, um the question is um how do we do them? I mean, the um the requirements here are totally different from analytical ones. The load is much larger, and uh latency needs to be quite
[04:49] mhm well lower than for analytical use cases. So, how do we do it? And uh um let me tell you more about how we did it at first. Our first attempt was actually Feast based solution. Feast is an feature
[05:05] store open-source library. Uh and the team developed solution on top of DynamoDB. We had a Python based HTTP server, and separately, we also had uh jobs in Databricks that wrote data directly to
[05:21] the store. So, to sync the schema between writing and reading, we also had a common registry stored on the S3 bucket. The team quickly realized that the performance is not enough, so DynamoDB acceleration layer, DAX, was added on
[05:37] top of DynamoDB. And it actually required us to fork Feast because Feast did not offer um support for DAX out of the box, but it was also not enough. So, in the second iteration of the service, the team used Redis as a data store and
[05:54] Java-based gRPC server to serve the features. The system was quite high load. Um it served up to 70 to 100,000 requests per second with a P99 latency of around 10 milliseconds.
[06:09] At least for Redis. I'm not talking about DynamoDB here. Uh but it also had multiple drawbacks, and the main one being its complexity. Uh whenever we had an incident, it felt like we need to connect multiple dots from different parts of the world to
[06:25] understand what's happening. Um it was very hard to understand where the data originates, where how it is written to the store, and where it is actually consumed. In general, we had really poor visibility into the data, and uh for example,
[06:41] uh quite recently we had an incident where our Redis cluster ran out of space, and it was so hard to find what job actually consumed so much space. And it turned out it was not a job, it was just an ad hoc notebook run by some user. So, um it was quite hard to
[07:00] get around in the system. So, um we with the team last year, we um gathered and started discussing how we can make the system better. And this is where Lakehouse comes in. Lakehouse was introduced 1 year ago at
[07:16] Databricks AI Summit in June last year, and was in um um public preview since June. And around October, we decided to give it a try. It was still in preview, but we thought that it might be worth it. Because we planned
[07:33] the system to be quite simple. We would want to have just one Lakehouse instance. We would load data via synced tables and managed pipelines sync pipelines to Lake Base. And then we would query the data directly
[07:48] from Lake Base from the client services. Um this system offered multiple advantages. And we got the managed sync pipelines out of the box, and the system looked very simple. And actually
[08:06] ball park calculations showed that it would be also quite cost-effective. Here we have a comparison for serving around 100 GB of data at around 50K RPS. With Redis, with AWS Elasticache Redis,
[08:21] we would pay around $1,200 per month for a single node. Probably for high availability, for high load system, we would want to have a cluster. So, it would actually double or triple the cost for read replicas, for
[08:37] more nodes that we have. And for Lake Base, the actual cost actually differs based on the usage and can vary from 100 to 800 RPS, $800 per month. And for 50K RPS, it
[08:53] would be around $500 that we load tested. Also, a note here that those are list prices, those are those are public prices. They do not include any discounts or reservations that you might have. For example, Lake Base now has a 50%
[09:09] discount till the end of the year. Um are there any problems with this plan? Of course. One does not simply write a high load system and hope that everything works out of the box just from the first try. So, we did have a
[09:26] lot of challenges and learned a lot along the way. So, let me share some of them. Um the first one is that we actually needed a proxy. It turned out that our client services do not have JDBC dependencies.
[09:42] Uh most of them do not have their own databases. And uh obviously they do not have Databricks SDK to uh manage authentication with Databricks. So, to um abstract away this complexity from our client teams, we built a small proxy
[10:00] service in Java that would connect via JDBC to Lakehouse and expose HTTP API uh for services inside of our private Kubernetes cluster, so that teams do not need to handle um this complexity on
[10:15] their own. Ideally, we would want to have uh a solution from Databricks for that. We have tried using uh feature serving end points, but they didn't solve the problem of um authentication token rotation. So, we ended up building
[10:30] our own. Another problem was discovered on the load testing phase of the service. As I mentioned, the uh system was going to be quite high load, so we did extensive load testing be- before going live.
[10:46] And uh during one of the tests, we noticed um big latency spikes for connection acquisition time. For So, here on the graph, we see um uh we see uh Hikari connection pool connection connection
[11:01] acquisition time. Hikari is the default choice for Java Spring applications. And with yellow, we see maximum uh acquisition time and with uh green, average connection acquisition time. So, we see this huge spike of 600
[11:17] milliseconds, and it coincided with the time when uh a batch of new note uh pods started. Uh it turned out that uh Lakehouse didn't handle well when we tried to open uh uh dozens of new connections at the same time and would uh uh uh struggle to open
[11:36] all of them all at once. So, how we uh how we solve this problem? We configured quite an aggressive pool pre-warm up strategy. Um a bot needs to have at least five active running connections before it can accept
[11:53] the traffic. And it was it and it would smooth out the connection acquisitions in time and LakeBase would not struggle and actually that we didn't have any problems with connection acquisitions after that at all.
[12:09] Another problem was with hanging connections. Um here on the graph with we see LakeBase query latency. All of our queries that we do are primary key lookups using um index. So, nothing fancy. And the P99 latency that we see with yellow line here is 4
[12:26] milliseconds. It's very good. But from time to time, we see that the maximum latency for those lookups is around 200 milliseconds. So, some of the queries is actually less than 1% of the queries, actually less than 0.1% of the
[12:42] queries, they um are resolved within 200 milliseconds. And unfortunately, we do not know uh we didn't find the root cause for that. It could be on the client side, it could be on the LakeBase side, networking. We didn't find the root
[12:58] cause and we worked around it in the following way. We configured a hard 20-millisecond timeout for the query. And if query cannot be resolved within this time, uh we would just retry this query. And
[13:13] it actually helped. On the next graph, we see the maximum database time with green color. We still see those 200-millisecond spikes. But the maximum HTTP time, this is the time that clients actually wait for the
[13:29] data to be served. The it is 30 milliseconds most of the time. And that means that the second query and that was retried after the first 20 millisecond failure was was succeed succeeded under than
[13:45] 10 milliseconds. So, the end-to-end time the end-to-end maximum time for serving those entries was under 30 milliseconds. And the P99 latency is obviously much better. It's around 10 milliseconds just as for Redis.
[14:05] So, the key takeaways from this journey. First, scale is hard. Even such simple problems as primary key lookups with index can get really hard and I spent quite a lot of time trying to dig deeper and solve them and it was definitely an interesting journey.
[14:22] Um secondly, managed infrastructure is more robust. Yes, managed infrastructure can have its downsides and it might be not as flexible, but in this case, the fact that we consolidated a lot of pieces of the of the service in one place in Databricks
[14:39] offered us very good tools for lineage, for debugging, and for understanding what is happening under the hood and it actually saves time and sleep during incidents, especially in fast-moving companies where everything breaks and it was definitely
[14:57] uh worth it. And finally, lake base was the answer for us. It was reliable and performant enough. It required much less maintenance. It was a more cost-effective solution that it was no-brainer for us to switch.
[15:13] And it doesn't mean that Redis is a bad technology and shouldn't be used, but it just means that there are use cases as I showed before that actually can be service like base with much better terms. With this, I thank you for your
[15:29] attention. If you have any questions, follow up questions after the talk, please contact me at this email. And here is a link to my LinkedIn. Please connect. And please complete your journeys
[15:44] service. Complete your service after the talk. Your feedback is very welcome.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.