Tracing the Path of a Row Through a GPU-Enabled Query Engine on the Grace-Blackwell Architecture

Summary

• GPU Query Optimization: This video traces how rows flow through GPU-enabled query engines on the Grace-Blackwell architecture, revealing performance optimization techniques.
• Advanced Hardware Integration: Learn how Databricks leverages cutting-edge hardware like NVIDIA Grace-Blackwell processors to accelerate query execution on the Databricks Data and AI platform.
• Performance Deep Dive: Discover the technical details of how modern GPU architectures improve query performance and enable new possibilities for data processing.

Tracing the Path of a Row Through a GPU-Enabled Query Engine on the Grace-Blackwell Architecture

Talk by Clemens Lutz, Senior Developer Technology Engineer, NVIDIA ; Thomas Graves, Principal Systems Software Engineer, NVIDIA
Grace-Blackwell is NVIDIA’s most recent GPU system architecture. It addresses a key concern of query engines: fast data access. In this video, we will take a close look at how GPUs can accelerate data analytics by tracing how a row flows through a GPU-enabled query engine. Query engines read large data from CPU memory or from disk. On Blackwell GPUs, a query engine can rely on hardware-accelerated decompression of compact formats. The Grace-Blackwell system takes data access performance even further, by reading data at up to 450 GB/s across its CPU to GPU interconnect. We demonstrate full end-to-end SQL query acceleration using GPUs in a prototype query engine using industry standard benchmark queries. We compare the results to existing CPU solutions.Using Apache Spark™ and the RAPIDS Accelerator for Apache Spark, we demonstrate the impact GPU acceleration has on the performance of SQL queries at the 100TB scale using NDS, a suite that simulates real-world business scenarios.
Here's more to explore:
Databricks named a leader in the 2024 Gartner® Magic Quadrant™for Cloud DBMS: https://www.databricks.com/resources/analyst-paper/databricks-named-leader-by-gartner
An open, unified approach to your data, BI and AI workloads: https://www.databricks.com/product/databricks-sql
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

FAQs

What is the Grace-Blackwell architecture?

This video explores the Grace-Blackwell architecture, a modern GPU processor design that Databricks uses to accelerate query execution on the Databricks Data and AI platform.

How do GPU-enabled query engines improve performance?

According to this video, GPU-enabled query engines significantly accelerate query execution by leveraging parallel processing capabilities of modern GPU hardware.

How do I choose between CPU, single-GPU, and distributed GPU training?

CPU training suits small datasets and simple models. Single-GPU works for moderate models and data. Distributed GPU training handles large models and datasets. The video shows tradeoffs: CPU is simplest, single-GPU adds acceleration, distributed GPU scales to massive workloads.

How do I serve CPU and GPU models on the same platform?

Databricks model serving on the Data and AI platform supports both CPU and GPU instances. The platform automatically routes requests appropriately, enabling organizations to optimize costs by using GPU only for models that need it.

What are the single-node alternatives to Spark?

For single-node analytics, alternatives include traditional SQL databases, DuckDB, and specialized engines. However, Spark with GPU acceleration provides superior scalability while maintaining the SQL interface familiar to analysts.

Full transcript

Right, good morning everybody. Um, welcome to our talk. So, we're going to be talking about tracing the path of a row through a GPU enabled query engine on the Grace Blackwell architecture. I work at NVIDIA on the Spark team. I've been there about six years. Prior to that, I worked at Yahoo, Verizon on their big data platform team doing all sorts of big data stuff. Hadoop, Mapuce, Yarn, Storm, Kafka, Spark. Uh I've been doing Spark stuff a long time.
Been a Apache committer and PMC member since about 2013. So I'm going to hand it over to my colleague who'll introduce himself and then get us started. Thank you. I'm also at Nvidia. I'm a sen a senior developer technology engineer uh working on database stuff. Um yeah before that uh I was a PhD student at T Berlin uh also working on very similar topics uh back then that was on a power 9 um Volta platform where I was working also within VLink between these two platforms. uh and yeah was publishing at Sigmod and some of the other database conferences back in the day.
With that I'll dive right in. So the premise for today's talk is that you all are interested in processing large amounts of data and you know these data can be very big in the hundreds of gigabytes terabytes maybe upwards and typically um GPUs would be a very good architecture to process data there because GPUs from the hardware architecture perspective are very highly data parallel. They have massive amounts of memory bandwidth in the terabytes per second and they also feature a very high compute throughput to match.
The question is now how can we actually scale to these large amounts of data that interest us using this kind of GPU architecture. And there are some systems out there that uh can do data processing on GPUs. And the typical architecture uh for such a a GPU enabled database management system would look something like this where we have our big quantity of data and we're storing that alongside the CPU because that's where we have the space to store it either in uh some terabytes of uh CPU memory or on the disk also alongside the CPU.
But then during query execution, the system will transfer that data ad hoc um to the GPU where it then actually performs the execution of the individual operators. And the underlying assumption being that we don't have enough GPU memory to store all of this data um in GPU memory. Of course, we can now also say we don't just do this on a single GPU, but we scale it out to many GPUs. Uh maybe we also use the CPU alongside the GPUs uh to do that scale up processing.
And then we need to connect these devices with some kind of uh interconnect fabric uh prefers preferably something that's fast enough to actually saturate these GPUs. And I want to convince you in this talk that in both of these scenarios. So when we are running on just a single device or a single node but also in the scaleout case uh Grace Blackwall offers a unique advantage in terms of how fast we can execute queries on large data sets.
So going forward in this talk, our agenda is going to be that we're first going to look briefly at the Grace Backwell hardware architecture and after that we're going to look at some um measurement results and the architecture of a prototype query engine that we're building at NVIDIA to showcase how to use these kinds of grace backwell systems. And the results that I'm going to be showing is going to be on a single GPU.
But of course, we can also scale out using Grace Blackwell to a cluster setup. And that's what is going to show you uh with the Rapids accelerator for Apache Spark. And he will also then show you even bigger data sets than I can show on a single GPU on the cluster setup. So let's dive into the Grace Blackwell architecture. Just to give a very brief overview of how a node looks like in Grace Blackwell. A node will usually consist of uh two Blackwell GPUs and these are connected to each other with uh NVLink and they're also connected with NVLink to a Grace CPU.
So this is an in-house design also from Nvidia. All of these processors have some local memory attached and the processors uh the three processors all also are within a cache coherence domain. That means we can basically take a pointer on any of these uh CPUs or GPUs and dreference that pointer to anywhere in that coherence domain. and get that data and the hardware will take care of fetching it for us which is very nice when we're writing software on these platforms.
But for the performance aspect, I want to highlight three key features of Grace Blackwell. The first being that these Blackwell GPUs have a very high amount of GPU memory. So we're talking now about 192 gigabytes per blackwell GPU which is significantly higher than uh previous GPU architectures had. So for ampair hopper we're usually talking about 80 gigabytes. So this is more than twice the capacity and that means we can fit larger data directly with the GPU.
The second advantage is the CTOC interconnect. So this is in vlink core to core between the blackwell GPUs and the grace CPU and that gives us much higher bandwidth than competing technologies. So we're talking about 450 gigabytes per second uh in total and that's compared to something like 128 gigabytes per second for PCI Express 5. So this is now four times the bandwidth of uh the competing technology. And a third key advantage for data processing is that blackwell GPUs have on their die a uh hardware component for uh decompression.
So we call this the uh decompress engine and that is capable of decompressing data streams for heavy weight compression algorithms in parallel at up to 600 gigabytes per second of output throughput. Um, and so yeah, that's a pretty high bandwidth and it supports uh several different uh popular compression formats. So these include snappy, LZ4, uh, gzip deflate and some others as well. So these three key advantages for Grace Blackwell make it for us very interesting to build uh systems uh database management systems for the this platform and that's what I want to show you by tracing the path of a row uh to familiarize familiarize ourselves with this prototype query engine that we're building.
So just taking a very simple example to give an outline of how the system processes a query. Uh this is just a very simple select statement that takes a table, executes a predicates uh on that table and then does a a group by of on a key and does a a summation of another column. And what this might look like in the execution plan in our query uh execution engine is that we're taking the data, we're feeding that to a read operator that passes its data to a filter.
The filter then uh passes its data to the aggregation which then and finally we write out that results to a result table. But let's take a look at the single row in the output and see how that comes to be. So that row is going to be written to the output table by the riot. But that riot takes it from a hash table. So that row is originating from a single entry in that hash table which is being created by our aggregate uh um operator.
So this aggregate is summing up uh multiple columns from its input and um creating that h that row in the hash table and the aggregate is taking its input from the filter uh which is filtering out some columns from the read. Right? So this is how data is flowing through the system. But the interesting part now is when we look at just some ballpark estimates for what kind of performance we might expect uh for these individual operators, we're going to see that the read is uh you know maybe 100 gigabytes per second in that ball ballpark where the other operators are going to be much faster.
Right? So that means our potential bottleneck is usually going to be that read operator in our system. And that means we need to take a much closer look at that read. And that's where what we're going to do now. So I want to show you some actual benchmark results as we go along. Um this is going to be using the uh NDSH benchmark from Nvidia. NDSH is a derivative of the well-known TPCH benchmark used in industry. Um, and what we're going to be doing is executing the same 22 queries uh from TPCH and executing those in memory in our uh query engine and then computing a geo mean uh throughput over these 22 queries.
Uh but I also want to emphasize that because we're not executing the full TPCH benchmark, these results are obviously not comparable to results published on the TPC website. Right? So these are just that these 22 queries being executed individually. Um I'm going to be showing these uh benchmark numbers on two platforms that are being hidden here by the transcript. So I'm just going to read this out loud. Um so the one is our prototype query engine that we're we've built in house that's running on a uh GB200 uh system which is a Grace Blackwell system that has 480 g gigabytes of CPU memory attached to it.
Um and it's going to be running on a single GPU. Uh I'm going to be comparing these results uh to a state-of-the-art CPU baseline that's going to be DOCTB in the most recent version running on a single CPU namely an Intel Emerald Rapids platform which has a terabyte of memory attached. All right, let's dive into these numbers. And so just looking at this read operator again, uh what's going on with the read? Well, we can now due to this advantage of the grace uh blackwell architecture store uh sign significant amount of data directly in GPU memory and that means our read operator is just going to take that data project out the columns that we actually need because we're storing it in a columner format and then um feeding that data into the next operator which which is our filter.
So the read itself isn't doing that much at this point. And when we look at the measurements, what we can see here is uh for the audience in the back, the dark green bar here is an A100 GPU uh which is an older generation and we can fit scale factor 80 directly into GPU memory on that GPU. So we can execute you know some data in GPU memory also for these kinds of GPUs and of course we can do the very same thing on a grace black wall right we can just fit scale factor 80 in GPU memory there and both of these GPUs are going to get some pretty good numbers uh compared to a state-of-the-art baseline because GPUs have very high uh memory bandwidth however the interesting part is now when we start scaling up our data to larger and larger sizes as we go.
So here we can already scale up to scale factor 200 which is more than a doubling in GPU memory on uh Grace Blackwell and that kind of data isn't going to fit on these older GPUs. Um it will execute on the CPU as well uh just with lower performance but you know we're not happy yet. We want to scale higher. So let's scale to a larger size. And we can do that by using the second advantage of the grace backwell platform which is this NVLink C2C interconnect to the CPU which means we can now store the data in CPU memory.
Um meaning we have more of that and can scale to a larger scale factor. In this case it's going to be scale factor 500. And what the read operator is now doing is it's taking that data in CPU memory, projecting out the columns, which isn't a lot of work. This is just operating on some metadata. Um, so we're not actually touching the data at this point. But then we're going to transfer those columns that we need for query execution across the interconnect to the GPU, store them temporarily in uh GPU memory, and execute the query from there.
And the interesting part here is when we compare the dark green bar which is now also a black wall GPU just connected with PC express instead of in VLINK. Uh we're getting three times the um the amount of performance compared to this other black wall GPU using the Grace Blackwell platform. Both of these are getting better performance than our state-of-the-art CPU baseline. Bill um now let's scale factor 500 but we can scale higher.
So let's go to the third advantage of Grace Blackwell which is the decompress engine. With the decompress engine we can store our data in a compressed format which means we can store even more memory uh or even more data in memory alongside the CPU and our read operator is going to look slightly different now. So what we're doing is we're taking our uh projected columns and transferring that data again, but this time it's compressed data that we're transferring and on the GPU we're going to use our hardware decompress engine to decompress that data and then we can access it in a decompressed format from the other operators.
So it just adds an additional step there as well. And this time um I'm going to be comparing uh a Grace Blackwell system to itself um just in two different software configurations. The one being that we're using a softwarebased decompression on the Blackwell GPU which is a dark green bar. The light green bar is using the hardware decompress engine. Now we're scaling now from scale factor 500 previously to scale factor 1000 in CPU memory.
Um, and we're getting twice the performance when using the hardware decompression versus the uh softwareb decompression. And both of these scenarios are going to be faster than our baseline which is now also using compression because we couldn't even fit the scale factor into one tab of memory. Um I do want to point out though that these numbers uh especially for the decompression engine I I just recently got them like uh yesterday.
So these are very very early numbers. We're still working to improve them and you can expect better performance uh going forward as well in future presentations. Before I hand over to, I just want to wrap up my part of the talk uh and have some conclusions here. So the first being that GPUs as a hardware architecture are generally well suited to compute uh data and specifically have them embedded in database management systems.
That's a win for us. Um but there was always this challenge of accessing large amounts of data and this is where really Grace Blackwell shines because we're now getting three advantages of that platform that we can take advantage of in our system which is the larger memory capacity scaling getting us a scale factor 200 right in GPU memory. Um then the grace black wall and vlink which gets us to 500 scale factor 500 using the CPU memory uh with a very high performance.
And then the decompress engine is the third advantage where we can then store even larger scale factors also in memory without even having to fall back to disk and still get really good performance and uh yeah scalability right there. So, you know, taken together these three features, um, really enabled us to build these high performance query engines using GPUs and, uh, get some really nice numbers there. With that, I'm going to hand.
He's going to show you scaling up to even bigger data. Thank you. So, I'm going to go over Rapids Accelerator for Apache Spark. And so this is how we can scale to larger data like mentioned. So what is the rapids accelerator for Apache Spark? So this is a plugin for Apache Spark that basically accelerates data processing using the GPU. Um so our main goals here when we started this project was to make it as seamless as possible for users.
So we didn't want them to have to make code changes and we wanted to go to where they were at. So lots of people have different versions of Spark. So we wanted them to be able to use our plugin on any of the versions or as many as possible. So how did we accomplish this? So we ended up having to make changes to Spark to allow this. So we made some changes for Colner processing. We added a plug-in point into the uh planner to allow us to plug in to be able to uh change operations to run on the GPU.
And then we also added GPU scheduling to Spark. really it's resource aware but GPU is one of the resources you can schedule. So that allowed us basically to just um create a jar that you just download the jar, turn on a config and then you're off and running. So you can run and use the rapids accelerator uh to accelerate your jobs. So in this picture here we have our data analytics applications. So if you're running Apache Spark and the other requirement is you have to be using either the SQL or the dataf frame API.
So those are the ones that are going through the catalyst planner and optimizer and as long as you're using those we can seamlessly um transition your jobs to run on the GPU when needed or when possible. Um so as this picture is kind of showing we support uh various of the platforms and CSPs. So data bricks AWS GCP uh will run on any of those. Um, under the covers are the Spark plugin actually uses the library called Rapids, which is another Nvidia library.
And within there, it has a component called CUDF, which is CUDA data frames. And that's the layer that actually knows how to do the ETL operations on the GPU. So, it's got the C++ and CUDA code. So, we end up using that. All right. So, to look a little bit closer at a query. So, here's a group by aggregate. And we have both the dataf frame and SQL API. And so what does spark do with this? So when you execute this, it has creates a data frame and then it'll go through the planner, right?
So it creates a logical plan first and then it'll do optimizations on that plan and then you'll get a physical plan out. This physical plan is what would actually execute. So if we look at that in more detail, here's the CPU version, right? So you have hash aggregates, projects, broadcast, hash join, and so forth. And so that's the code that would actually execute and produce your results. So how does that look with the Spark Rapids uh accelerator?
So we have no code changes over here. So code's exactly the same. But when you get to that physical plan stage, it actually iterates. And our that's where our plug-in comes in. And so we go through and we look at each of the operations that you're performing and determine if we can replace it and do it on the GPU and if we can we'll replace it and if there's any transitions needed. So the CPU is running in row format.
Uh GPU is running in columner. So if we need to do any transitions we'll insert those for you as well. And all this happens without you having to do anything. So here's an example of a GPU plan. You'll see the whole thing is actually running on the GPU. And so where it was a CPU hash aggregate, we now have a GPU hash aggregate and a GPU checked and so forth. And so again, the user didn't change any code here and it will just run on the GPU when possible.
If there's something we don't support on the GPU yet, it'll just keep the CPU operation there uh and deal with the transitions for you. Okay, so let's look at this actually running on Grace Blackwell. And so we're using NDS here, which is a TPCDS- like benchmark. So it's similar to the NDSH, but it's TPCDS based. Um, so in our setup here, we had eight CPU nodes. They're Emerald Rapids CPUs with 56 cores, 112 threads.
There's two sockets on each node. Um, we had two terabytes of host memory, 400 gigabits per second network and eight NVMEs set up in RAID zero. On the GPU side, we had eight Grace Blackwells. So, the Grace has 72 cores. The Blackwell or we had one B200 per exeutor basically per node, two terabytes of memory, uh, 400 Gbits per second on the network and eight MV eight NVMEs in RAID zero as well. For storage, we're using Luster and we're getting about 50 gig gigabytes per second and we're using Spark 352.
So, this is kind of the glossy brochure. The NVL 72 is one of these Grace Blackwell nodes. It's got a ton of 72 B200s on it, 36 graces. So, we obviously weren't using this whole machine in the benchmark. We're only using eight, but um it's got full NVL link uh between the B200's and the interconnect uh between grass grace and Blackwell like had mentioned. Okay, so I'm going to dive into one of the interesting components.
So with all this hardware and NVL links, we can really take advantage of it in what's called the Spark shuffle. So it's a phase where you're transferring data between the maps and reduce sides. And so Spark has a shuffle manager in it. Uh we have a custom shuffle manager called the rapid shuffle manager. Um and you can set it up in two different modes. So we have the multi-threaded mode and this is for if you don't have any fancy hardware NVLink or anything like that, you can just use this and it will still accelerate shuffles.
Then the UCX mode is what UCX is an open source component that you can go download and it can take advantage of the NVLink or Infiniband or and it's actually smart enough to use what it think is the most efficient one if you have them multiple set up and under the covers the UCX layer is what's end up calling into CUDA for like NVLink. So Grace Blackwell would use NVLink. So looking a little bit more here at the shuffle manager.
So this is the multi-threaded mode. So you have a task computing data and normally at the end of it on spark it writes that data to disk and our multi-threaded mode also writes to disk but it will do it in parallel with multiple threads. So normal spark would do all this serial serially. So you get a serialized compressed right serialized compressed right and so forth. Uh the multi-threaded mode lets you do this in parallel and that serialize and compress is really expensive.
So this can make a big difference um in how long this takes. We also have the advantage of we're processing on the GPU. So a lot of times we'll have idle CPU cycles to where we can really this can be efficient and not block the actual data processing. Okay. So what does this look like when we actually run a benchmark? So this is the configuration we used. So on the CPU side um some interesting things is executives we have 30 32 cores per exeutor.
We used all the cores on the nodes. So you'll see the shuffle partitions there is 1,792. This is basically to make that shuffle most efficient on the CPU side. We used all the processors. Um you also notice that the executive memory is 256 gigs. So that's heat memory. And since we're on the CPU, we use a lot of heap. If you compare that to the GB200 using the multi-threaded shuffle here, some of the major differences are so we use 16 cores per exeutor.
Uh that's one GB 200 per exeutor. Basically, uh our heap size is smaller because we're doing more off heap and using GPU memory for things. We just left the shuffle partitions at the 200, which is the default for Spark. Uh the spark plugins is where you actually enable our spark rapids plugin. The max partition bytes here is we're reading a little bit more data from our input. So if you're reading from K uh because the GPU really likes bigger batches and so we can get more uh efficiency out of reading using bigger batches.
So the pin pool is a very fast memory pool for transferring data between CPU and GPU. So we configured that at 128 gigs and then we have spill storage size at 200 gigs. So basically if we can keep any spills in memory that's more efficient. The concurrent GPU tasks uh is how many tasks can actually run on the GPU at once. So here we have 16 cores on an exeutor. That means that you can run 16 normal tasks. Um and those can all be starting up and processing data.
But then when it runs to goes to run a GPU operation, the concurrent t concurrent GPU tasks will limit it. And mostly that's to because there's a limited amount of memory on the GPU. Uh and so we start out with this static config. And we've actually added more logic to make that more dynamic. So you don't have to tune this as much. So then the shuffle manager here is enabled. And then we configure the shuffle manager to have 32 threads.
So there's more configuration here. We're working on making that easier for users. You don't have to configure all these, but this is what we used for this benchmark. Okay. So, our results. So, the CPU versus the GB200 using the multi-threaded shuffle, we ran it with three different data sizes. So, so we have three ter terabytes, 30 terabytes, and 100 terabytes. At three terabytes, we had three three and a halfx speed up.
30 terabytes we had 5x speed up and at 100 terabytes we had a 5. 3x speed up. So you notice this is pretty linear. Uh both the CPU and GPU side uh were pretty linear speed up. Okay. So now let's look at the UCX shuffle. So both spark shuffle and our multi-threaded shuffle write to disk uh between the stages. So here when our task runs uh and goes to write data we have a spill framework and basically this is a caching framework.
So it knows how to uh or it will try to keep things in memory when possible. So it can cache things in GPU memory. Uh if it won't fit there it'll fall back to CPU memory. And then finally, if it won't fit there, it'll it'll write to disk. Obviously, we want to try to not to write to disk because that has more overhead. And so on the right side, that's the executive one where we're actually doing a map and writing the data.
It'll write to this spill framework. When that finishes, the next stage over here on executive 2 shows that uh reader will go through and try to pull the data. What happens then is we end up copying the data from the spill framework into a GPU buffer um copy buffer and then it goes across UCX and in Grace Blackwell case it'll use NVLink uh over to a GPU buffer on the other side and then the task will read it right from that GPU buffer.
So it's a very fast path. We don't hit disk at all uh and we get a lot of efficiency out of that. Okay. So the configuration we used for the benchmark here. So it's a lot like the previous one where we enable the plugin uh have same number of cores again heap size is smaller here. We did tune the shuffle partitions a little bit for the 100 terabyte. Um it was such large data we tried to keep it this smaller per task.
So we configured that to have more partitions to split it more. Uh similar sizes in pin pool size. The spill storage again we increased at the 100 terabyte level to try to keep from writing to disk to keep it most efficient. Um shuffle manager is the same. And then the other thing here is the shuffle mode. So we changed it to be UCX from the instead of the multi-threaded shuffle. And then UCX has this other parameter we had to set basically because we were using Java uh to disable some error signals.
Okay, our results here. So this is comparing the CPU versus the GB200 with multi-threaded to GP200 with NVLink. So at the 3 TB scale, we went from 3 and 1/2x speed up to 4. 1. At the 30 terabyte scale, we went from 5x speed up to 6 1/2x. And then at the 100 terabyte scale, we went from 5. 3x to 5. 5x. So you'll notice the 100 terabyte wasn't quite as good, didn't linearly scale. Uh that's because we ended up having to spill to disk.
Uh and we're actively working on that to make that better. But this kind of shows how NVLink and that fast transfer can really make a difference. To look at that a little closer. So here are some of the queries uh at the 30 terabyte scale that shuffled a lot. So some of these are shuffling one tab of data. You can see that some of them got up to like 3x faster. I think the query 23 here, part one and part two, uh, can really show that.
So, it can show it can make a big difference having that fast NVL link to there to transfer data. Okay. So, what if you don't have a Grace Blackwell and the newer hardware yet? So, this is just a quick benchmark to show you how Spark Rapids could help you. Now, so this is running NDS and EC2 at a 10 terabyte scale. And here on the CPU side, we used eight M7 GD8 extra large. On the GPU side, we use G6 8x large nodes, eight of them again.
Um, and we used S3 for storage. So, our NDS data 10 ter and we're only using 10 terabytes in this case, but it was stored in S3 and we're using Spark 351. The configuration is kind of similar to before. So obviously change the size of things based on the nodes we used. So we have 16 executive cores on the CPU side and the executive memory is 32 gigs of heat. On the GPU side uh again uh we have eight exeutors so 16 cores each one using it was an L4 is what it's using.
We enable the plugin. Partitions are the same. We have a pin pool size. Again, that's the fast memory to use to transfer data. And 32 gigs of spill storage. Here, we're only using three concurrent GPU tasks because the L4 has a lot less memory than the B200. And again, we enable this the shuffle manager. And so our results here, excuse me. So this is 10 terabyte scale. Uh so overall we got a 4. 6x speed up. Again this is on EC2 nodes running Spark standalone, but it shows you what you can get now.
Um and a lot of people the L4s are a lot less expensive and they're more available than some of the bigger GPUs. Okay. So you can go try this out yourself if you want. So here's some links to the Spark Rapids documentation. There's, excuse me, there's a collab notebook you can go try and then we have documentation on our user tools. Those tools can be used to help you decide if your workload's a good fit for the GPU.
There's also tools on tuning and getting started and uh debugging there. And that's all I had.

Learn more about the Databricks Data and AI platform.