GPU-Accelerated Operations Research: Production-Grade Optimization with NVIDIA cuOpt on Databricks

Summary

NVIDIA cuOpt is an open-source, GPU-accelerated operations research solver designed to tackle NP-hard optimization problems—such as vehicle routing, shift scheduling, and production allocation—that grow exponentially harder as constraints increase.
Only 50% of supply chain organizations currently perform optimization, and this video shows how running cuOpt on Databricks serverless GPU clusters can reduce solve times from hours to minutes for large-scale linear, mixed-integer, and routing models.
The integrated solution manages optimization workloads end-to-end on Databricks: parameterizing and submitting jobs to serverless GPUs, governing data with Unity Catalog and Delta tables, and using Genie Code to generate cuOpt solutions automatically.

GPU-Accelerated Operations Research: Production-Grade Optimization with NVIDIA cuOpt on Databricks

Enterprises face a surge of optimization challenges—workforce scheduling, supply chain planning, resource allocation, etc. that are computationally explosive as scale and constraints grow. Learn to solve these problems with Databricks using NVIDIA cuOpt on serverless GPUs, with large-scale linear, mixed-integer, and routing models in minutes instead of hours. We will walk through end-to-end architectures on the Databricks Lakehouse—parameterizing and submitting optimization jobs to serverless GPU clusters, managing data with Unity Catalog and Delta tables, and orchestrating workloads alongside existing ML pipelines. Learn from real-world examples I.e. shift scheduling, production allocation, and route optimization with performance benchmarks and cost benefits compared to traditional CPU solvers. Leave with practical patterns for modeling constraints, validating feasibility, and knowing when GPU-accelerated optimization can drive measurable gains in efficiency and delivery performance.
Talk By: James Maki, Sr. Solution Architect, NVIDIA ; Josh Melton, Delivery Solutions Architect, Databricks ;

Chapters

FAQs

What is NVIDIA cuOpt and what types of problems does it solve?

NVIDIA cuOpt is an open-source, GPU-accelerated operations research solver that handles NP-hard optimization problems such as vehicle routing, shift scheduling, supply chain planning, and resource allocation. It is designed to find optimal or near-optimal solutions within tight latency budgets where traditional CPU solvers would take hours.

Why is operations research optimization so computationally difficult?

Optimization problems are NP-hard, meaning the decision space grows exponentially as additional options or constraints are added. Organizations also face tight latency requirements—solutions must be found quickly because business processes cannot wait—and constraint density continues to increase as real-world operations grow in complexity.

How does running cuOpt on Databricks improve deployment and governance?

Databricks provides serverless GPU clusters for running cuOpt workloads, eliminating infrastructure setup overhead and ensuring environment consistency. Unity Catalog and Delta tables govern the input and output data, while LakeFlow orchestrates cuOpt jobs alongside existing ML pipelines, and Genie Code can automatically generate optimization solutions.

What is the business opportunity for GPU-accelerated optimization?

Based on data presented in this video, only about 50% of supply chain organizations currently perform any kind of optimization, with 39% planning to adopt it within two years. Approximately 75% of Global 500 enterprises expect to implement some form of decision intelligence by 2026, spanning linear programming, mixed-integer programming, and vehicle routing.

Full transcript

[00:07] Well, thank you everybody for joining. Really appreciate it. Know we're late in the conference, probably some of the last stuff you'll do. So, appreciate you spending some time with us. Um, my name is Josh Melton. I'm a delivery solution architect for Data Bricks. Um, and I'm here with uh James Mackey.
[00:19] >> I'm James. I'm a solutions architect on the media entertainment team at NVIDIA. Uh, Josh and I got connected because we like to solve hard problems. We want to solve them faster and we like to focus on problems that help make the world run more efficiently, help save companies
[00:32] money. Um, so we'll be talking about operations, research, agents, skills, uh, several different topics today. We're excited. Thanks for being here.
[00:45] >> Um, so as part of this, uh, I'm not a lawyer, but forward-looking statement,formational purposes only, etc. Um, and make sure you uh fill out the survey so we get some feedback and and make sure this is helpful um as possible
[00:58] for you next year. So with that, uh, James will kick things off. Thanks. All right, we'll start with an overview of what Nvidia Co-op is. This is our open-source operations research
[01:10] solver. It's GPU accelerated. And first off, uh, there's tremendous opportunity in this space today. We think this will continue to improve. So um based on the data we see only about
[01:23] 50% of supply chain organizations are actually doing any kind of optimization today. We expect that to increase um and 39% plan to adopt optimization in the next two years. Um we're also seeing
[01:37] that about 75% of the global 500 enterprises are expecting to implement some sort of decision intelligence by 2026. And this is a mix of uh like linear programming, mixed integer programming, uh vehicle routing and AI
[01:50] to augment or automate very high stakes business critical decisions. Um
[02:03] all right, why is optimization hard? Uh there's a couple different components that go into this. Let me update my notes. Uh the first is just the problem complexity. Uh so there's these are NP hard problems. As you add additional
[02:15] options or constraints, the decision space continues to grow exponentially and becomes harder and harder to solve. Um we're also trying to balance performance and profitability. So we typically have
[02:28] pretty tight latency budgets. Like we need to solve and find an optimal solution fast because the business can't sit idle and wait for these solutions to to solve. um and the the constraint density continues to grow and grow.
[02:41] There's more and more constraints being added um that we need to work around. Um scalability is also a challenge. Um so it's not enough to just solve a problem once. We need to be able to rapidly resolve. Um and we also need to be able
[02:54] to scale between different regions and product lines um as as the complexity continues to grow. Um, back to the ability to resolve like real-time decision-making and being able to adapt
[03:06] and resolve again is very important. Um, to to be able to take a close to optimal solution when the business conditions have changed and resolve um to get back to a an optimal solution. And all of
[03:20] this puts a lot of computational burden. Um, so you end up with these very hard computational problems. Uh and so that's why we're going to be talking about GPU accelerating uh operations research and
[03:32] GPU accelerating these solving these solver algorithms. Um these are all reasons why companies struggle implementing optimization today. And it's also good news because every time we make incremental improvements in
[03:46] solving capabilities like we push the frontiers of what's possible which is super exciting. All right. Co-op covers three major pillars of operations research. The
[03:59] first one being linear programming. Um so this is when we're modeling a continuous decision. We also support mixed integer programming where you have integer constraints like you need to hire uh three employees or four employees because hiring two 2.7
[04:13] employees wouldn't make any sense. So you don't want a floating point uh decision value there. Um and we also support vehicle routing problems which Josh will dive much deeper into. Um, so if you have pickup and drop off
[04:26] locations and a fleet of different vehicles with different capabilities and different ranges that they can travel, um, all of that can be taken into account and we can create an optimal plan.
[04:42] Here's some of the results we're seeing across these different categories. Um, again, co-opted is open source. The code is available on GitHub. With mixed inure programming, um, we're seeing up to 60x improvements on some of these benchmarks like simple rows. Um, for linear
[04:56] programming, we consistently see around a 70x speed up. Um, we measure that on benchmarks like the Middleman uh benchmarks. And then for vehicle routing, we see up to 240x improvements. Um, we use benchmarks like Lyric for
[05:11] measuring those.
[05:23] All right, we'll talk a little bit about route optimization. Um, in this workflow, we're going to use NVIDIA co-op to minimize vehicle routing costs by finding the most efficient routes. Um, so we're going to feed in uh
[05:35] fleet data, task data, like what what needs to be picked up and where it needs to go. Um, information about our our cost matrices. So the distance between the various um pick up and drop off
[05:48] locations um and the what you know our ultimate goal here is to just dramatically cut costs, fuel consumption, the miles that need to be driven um manh hours, all those sorts of uh factors that we want
[06:01] to optimize and and make sure that we are addressing.
[06:20] All right. Um, so our by using GPUs, we achieve massive speedups over CPUbased linear programming solvers and we also consistently outperform uh their solutions. So this is comparing to commercial VRP
[06:32] solvers. We can get better results in a fraction of the time. Um, and co-opt has achieved world record performance on some of these benchmarks. So like MIPLIB for mixed integer programming, Middleman
[06:44] for large scale linear programming and the the Garing and Homeberger uh benchmarks for vehicle routing co-ops designed to scale and integrate with lots of other players in the
[06:56] ecosystem. Um so things like Ample, Pulp, Pyomo, and Scypi. And the whole goal is to give you the flexibility to make decisions in real time or optimize for cost and do largecale batch jobs that could could
[07:10] take longer but um allow you to run considerably cheaper. All right, I'll hand things over to Josh. Cool.
[07:22] All right, so um sort of as James said, uh I might flip back to that. Um, as James said, you know, Coop is really intended to allow you to provide value to your businesses and make the world
[07:34] run more efficiently. Um, so as I as I mentioned, I'm a delivery solution architect with data bricks and I work with our industrial customers to help them run some of the largest supply chains in the world. So when I see slides like this from James, 60 70x
[07:47] speedups, this is super exciting to me. But I also have a policy with vendor benchmarks, whether it's Nvidia or data bricks or LLM benchmarks, right? what if they don't fit like that on my data, right? Trust but verify when it comes to
[08:00] vendor benchmarks. Um, and so I said, um, what are the most challenging things for GPUs? Right? It makes sense that GPUs work at really large scale. Maybe I'll try a smaller problem just to see if I can kind of
[08:12] find an edge case, right? And then mixed integer programming uh was only only 60x faster. So I'll use that instead of like routing optimization. And so uh I ran a benchmark. I also used like very uh
[08:24] pessimistic pricing for GPUs and I was just trying my best. Can I find the edge or corner case where GPUs are not the most performant before I come up on stage and say K-OP is the best and uh basically that you know not a huge
[08:36] surprise but KOP is the gold bar across the top and despite my best efforts it was just like wildly faster than all of the CPUbased um solvers that I tried. So we can see here we've got the objective value this is the thing that we're
[08:49] minimizing. So in this case is an inventory optimization benchmark and you can see only two of the libraries actually found the optimal solution um and one of which was the gold bar of co-opt. We can also see the solve time
[09:02] is dramatically improved with co-opt um which again it makes sense right is solving it in a little over 3 seconds whereas the the best uh CPU library solved it in about 90 seconds. um so over a minute and a half and and if
[09:15] that's not enough right it's great to have sort of real-time decision- making um in terms of user experience but that also translates to lower cost right when you're paying per minute in the cloud um your cost is correlated to solver time
[09:28] so even when you you price in really aggressively for for GPU uh price per minute co-opt is several times faster um or I should say several times cheaper um again conservatively and then depending on the benchmark you're running 30 40 50
[09:41] times um faster Um, and that only improves with scale. So, to see how to get started with co-opt, it is uh it is actually a very
[09:53] quick demo. Um, it it makes it pretty straightforward to get started with this stuff. So, we're going to go over a route optimization example and kind of follow that from start to finish. Um, I'm going to connect to serverless GPUs on data bricks and if you want to try
[10:06] this out, um, you know, feel free to run that in your environment. Our data bricks free edition now supports serverless GPUs. So that's a great option if you want to try that out. Um and and again uh get started with co-opt. So how this kind of works is
[10:18] we're going to import co-opt import QDF. So that's the the sort of GPU libraries for this stuff. We'll set up our cost matrix. So here we'll just set up a very basic hypothetical scenario. So how long does it take to get from point A to
[10:30] point B, point A to point C, etc. We'll then set up the data model. So this is that slide James showed earlier where you're setting up the basically the solver configuration, the problem statement. So we'll let it know that we
[10:42] have four locations here. We've got two vehicles to use and we have three deliveries to make um or orders to make. We'll fulfill orders at these locations. So just points one, two, and three. We'll add the cost matrix and the
[10:54] transit time matrix, which in this simple example we're just treating as the same thing. We'll let it know um any settings we want to apply. So in this case, we'll say only think for up to 5 seconds about the optimal solution. And
[11:06] then we'll say routing.solve. And we've got our first um example of how to do a vehicle routing optimization problem on co-opt. So fairly straightforward to get
[11:18] going. And while this is very trivial, obviously this exact same code you can pretty much copy and paste and run 10,000 uh uh delivery optimization um on an A10 GPU. Um so that is the principle
[11:34] and if you want to try out any of this stuff, I've got a QR code here that takes you to a GitHub repo that has the hello world example. It also has the inventory optimization benchmark that I ran. feel free to take that try it out on your data and um you can kind of
[11:47] figure out what works best for you. Um if you are going to do that though there are going to be some operational challenges. So in the next section I'm
[11:59] going to talk a little bit about how data bricks can help you operationalize your operations research. >> Yeah. Yeah. Yeah. Of course.
[12:13] >> For sure. And I'll show this QR code. uh at least one more time too so you'll have more opportunities. Um so if you are going to take this benchmark and try it out on your own data or start applying operations research as as James said only about 50%
[12:26] of supply chain organizations have done this. So it's a pretty big opportunity and we're going to run into some operational challenges such as experimentation. So when you're trying out all these libraries and again don't
[12:38] just take our word for it. try out the CPU, try out the GPU, try out different libraries, different configurations of each library. But you're going to want to track all of those different permutations and how they map to the output metrics, right? Which one found
[12:50] the optimal configuration? How long did it take? What's the solve time? Um, and of course, the solve time kind of directly translates to cost in a cloud environment. So, the first challenge that we want to address is experimentation. How do we choose what's
[13:03] the best fit for our solution or our problem statement, I should say. The second challenge that we're going to come across is deployment. So, how do we track environments consistently across our development environment? What are
[13:16] we, you know, the GPU we're using for development versus the batch deployment that James mentioned versus the real-time API deployment. All of these things are really difficult to manage. Um, you know, environment management in Python in general is always really
[13:29] difficult. So, part two, problem statement two is deployment. And then the final thing is governance. The third statement is governing things like access control, lineage, auditing, all
[13:41] of these are super important. So these solvers are going to be put in the hands of operations research analysts or maybe even uh AI agents in order to help them make decisions. And whether those are the right decisions or the wrong decisions, we want to have an audit
[13:54] trail of why they made those decisions. Why would why did we publish the solver with this configuration? Um because these things have real world tangible impacts. And if you're familiar with machine
[14:06] learning or MLOps that those three problem statements sound super familiar, right? Experimentation. You try all these different libraries, all these hyperparameters, and you track the metrics for each sort of configuration. There's also the deployment thing,
[14:19] right? Tracking dependencies, how do you do things consistently across a batch versus real-time interface? And finally, the governance, right? Access, control, auditing, and lineage. All the same sort of deal. So while the math of of
[14:31] operations research and some of these solver problems is very very different, they solve fundamentally different problems, the operational problems behind the scenes are actually really really similar.
[14:43] And so what I found with some of our larger customers running really big supply chains and solving these sorts of problems at scale is that even it sounds a little awkward but we can actually use the same principles that we've used for MLOps for over a decade to have really
[14:57] robust uh automated processes for experimentation deployment and governance of operations research solvers. So whether you're deploying things or developing things from a notebook or whether you've got like a
[15:09] local spark connect sort of interface, you're running an automated cloud job, you can test out all these configurations. You can use MLflow to track all the configurations and their metrics. Once you've done that, you can decide this is the best configuration,
[15:21] deploy that whether in batch or real time and then have governance through the MLflow uh Unity catalog model registry and have immediate access to uh access control, auditing and lineage.
[15:34] So here's that QR code one more time. Same one um as before and we will flip back to this. But essentially just to show we kind of started with this hello world example,
[15:46] right? We connected to serverless GPU, we set up our data um and then set up this problem and said routing.solve, right? In order to operationalize this, we're going to do something that looks super
[15:58] super similar, right? And this is actually borderline cut and paste the same code. So for this benchmark that I ran, uh what we're doing is importing our library. We're setting up the uh the
[16:10] cost matrix, we're setting up the order locations and assigning the cost matrix to co-opt. We're setting that time limit and then we're saying routing.solve. So it's the
[16:22] exact borderline the exact same code with a little bit of boiler plate attached to operationalize this get to solve those three problems that we mentioned. we can subclass what's called an MLflow
[16:34] Python model. And the idea here is that we can take any arbitrary Python code and treat it like we would a machine learning library. So these couple of lines basically is just this part, right? We we initialize a class or define a class that
[16:47] subclasses Python model. In our initialization, we can add configurations. So things like a time limit. Um and then our solve uh our solver is going to be run in the predict function.
[16:59] And again, it sounds a little awkward, but really once we've implemented this, it's really, really straightforward to operationalize things. We can initialize our solver and then we say solver.predict and we've got route
[17:11] optimization. In order to solve this experimentation problem, we're going to start an MLFlow run just like we would in our normal MLOps pipeline. We can log parameters same as other MLflow objects. So that's
[17:23] going to be your library. What scenario am I solving for? um what configurations did I pass? We'll log the metrics. So that's going to be things like the objective function value or the solve time
[17:36] and then we'll log the model which is to say the solver and that is what actually produces the benchmark screen that we we saw here. So you kind of have a leaderboard built in again just like our MLOps process to show you know co-opt is
[17:50] the gold bar, CPUbased solver is the blue bar. We tried some other configurations down here. Um and we can really clearly compare like this is why we chose this solver. Um and it ties back to some some of the source data that we used um to to measure the the
[18:04] optimal way to configure this. So that's the experimentation problem solved. And the same approach actually solves the deployment problem. So when we log the model or the solver, we can log it
[18:17] with the pip requirements that are required for co-opt or whatever solver we decide to use. And that's problem number two. And then problem number three again we kind of get for free because we can use the
[18:29] MLflow Unity catalog model registry. So we can actually um access control govern and audit uh lineage. We get all that stuff uh right out of the model registry just like our machine learning models.
[18:46] So what this leads to we can even see I'll just click through. Um you know now we have a a function or a model. We've got our solver alongside all of our other data assets right here. We can control again who has access to it. We get lineage out of the box. Um uh we get
[19:01] traceability like right back to the the job that created it. And then what's more, the deployment process becomes super simple. So for those batch use cases, all we really have to do is say I want this version
[19:15] maybe like production version or champion version of this model or solver. We load that and then we say solver.predict just like we would with a machine learning model.
[19:29] Similarly, we can use that same source artifact to deploy to a real-time API. So here I'm using the data bricks SDK to create a model serving endpoint and I let it know that it's going to be on a small GPU
[19:42] scale to zero is enabled and this allows us to use the same interface to deploy these in this case route optimization solvers. So here we can see the results of one of those batch solves. I've read in my data from a table all the
[19:55] deliveries that I need to make. I've passed that through my solver like we saw in that notebook those three lines where you load the model you know solver.predict predict and I save those results to a table. In this case, a lakebased Postgress table. That
[20:07] Postgress table is the source of truth for where my routes where my trucks are going to deliver routes to or deliver orders or packages to. So, that's the batch bit. But what if we have, you
[20:19] know, scheduled um pickups, right? Like returns pickups for all of our customers kind of scattered across the place. And uh, you know, one of those pickups is kind of our trucks is driving right by it. it seems kind of silly to uh to
[20:31] schedule another batch process, right? We might need a real-time solve to say this is the optimal route to kind of tack that order return onto. So, for the purposes of this demo, I'm just going to start clicking through and we will see
[20:44] actually in real time we've got a route optimization solver that's going to take those and kind of snap them to the existing routes. So, we're kind of tying together the batch route optimization interface with a real-time route
[20:56] optimization interface. And we can see um again in the same way that we solve those three problems we get experimentation deployment and governance in the same interface and allows us to solve batch in real time in the same way.
[21:10] So again very very very similar or analogous to the problems that we are trying to solve with machine learning. Um and again everything is in this QR code. You'll have one more one more chance for that. So all that being said
[21:24] this is like a pretty technical problem still right? Right? We did the hello world example and I'm sure you guys have all that memorized, but like in the real world it's even more complicated, right? Like we have a very simple route
[21:36] optimization problem, but in the real world you're going to have capacity constraints and you're going to have time windows. Like if you're Door Dash and you want to make deliveries, it has to be within an hour of pickup or whatever, right? There's labor requirements or compliance requirements
[21:48] where you can't have a shift that lasts 30 hours of of driving, things like that. And so the real world gets gets complicated. Um, and the the fake demo I set up is arguably already complicated enough.
[22:01] So, the good news is AI is democratizing these technical skill sets, right? Everybody's knows Genie. Um, we've heard that a thousand times, I'm sure, this week. Um, back to the bad news. AI models are
[22:13] smart, but they don't always have the right context to deliver this stuff, right? So you can tell really really smart AI to write you your co-op um route optimization solver but it might not have the right context. Where does
[22:25] it find the data? Um do you use the Python APIs like we saw or do you run your like self-host your own co-op server these kinds of things. So we need to add the right context to make sure Genie code quad codeex whatever it is
[22:38] omnient um you know knows how you like to run your solvers. So, this is a different link. Um, you can also find it by searching NVIDIA agent skills. Um, but it is a link to the repo that has
[22:51] very extensive AI agent skills that are going to allow your favorite coding agent to implement co-optively. You know, eventually it might might get there with trial and error, but this is
[23:03] a super helpful way to kind of supercharge your development process for co-opt um and solvers in general. So, um, the live AI coding is always the
[23:18] nerve-wracking part. We'll see how it goes, but hypothetically should be very easy. Um, so I can say I've basically I've taken my skills um from the the co-opt um NVIDIA repo and I can say something
[23:32] like show me how to do route optimization and it is going to look up those skills. I basically just prompted Genie code
[23:44] before and I said, "Hey, I like co-op for my optimization problems." Um, and then I cloned that repo from Nvidia and said, "Hey, Genie Code, make sure you add these skills to your skills folder." So, you can kind of prompt your way to
[23:56] getting these skills loaded. It's not like super super challenging or sophisticated. We can see it it failed a couple of times. It loaded some co-op skills. Um, and so by adding these skills and again saying use the Python
[24:09] APIs for co-opt, we're going to pretty quickly get this um this basic route optimization problem implemented. and we can add those more complex real world world constraints to co-op that I mentioned. So, while it's thinking about
[24:22] that, I just want to also add um to get you guys going quicker. Uh there's a a serverless GPU. Obviously, that's going to be the the most effective, fastest way to get started with this. I've also done something in the background called
[24:35] creating a base environment. And this base environment is essentially just taking a shortcut instead. You can always run a pip install at the top of the notebook like you might be familiar with, but this kind of shortcuts that and caches those libraries so that you
[24:48] can reuse this consistently across all of your developers. Speeds things up a little bit um etc. So we can see uh Genie Code kind of hashed that out. It uh it doesn't actually know that I already have co-opt installed. So I will
[25:03] remove that except all click run and cross my fingers that this works. We can see though it's set up a very similar problem. Right? Here's a little bit more complex cost matrix. Again, you just read a normal table and then
[25:16] convert it to a to the CUDA data frame interface. Um it's setting up that that data model adding the cost matrix. It's going to do a little bit extra. So it added a capacity constraint. Um it added those time windows that I mentioned. And
[25:29] then it is going to say routing.solve and um you know ultimately will then get the uh the successful output. So, if you're looking to optimize your supply chains, your route optimization, um
[25:41] staffing optimization, all sorts of problems like this, um I would recommend just getting your skills loaded into Genie Code or your favorite coding agent and um prompting your way through it. And serverless GPUs and base environments can be super helpful.
[25:59] So, with that, um I guess in conclusion, um you know, co-opt is the best library that I've found for these sort of solver problems, right? route optimization, mixed integer programming, all these kinds of things. These are really great problems to solve because they have
[26:11] tangible, measurable business outcomes tied to them, right? So like a couple seconds of infrastructure cost can get you a a measurable 20% increase in route efficiency, which is typically
[26:24] significantly higher than the cost of the infrastructure. Um, and in order to operationalize these things, you can get started really quickly with skills from NVIDIA and with serverless GPUs, base environments, and ML flow from data
[26:36] bricks. And that is the summary of our talk.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.