Learn to Program Not Write Prompts with DSPy

Summary

DSPy Framework: DSPy enables programmatic GenAI application development using modules, signatures, and typed interfaces instead of manual prompt engineering.
Automated Optimization: Use state-of-the-art algorithms to optimize prompts and weights against evaluation datasets, treating GenAI like machine learning.
Production-Ready Applications: Deploy DSPy applications with MLflow, enabling proper testing, maintenance, and version control for GenAI systems.

Learn to Program Not Write Prompts with DSPy

Talk by Austin Choi, Delivery Solutions Architect, Databricks
Writing prompts for our GenAI applications is long, tedious, and unmaintainable. A proper software development lifecycle requires proper testing and maintenance, something incredibly difficult to do on a block of text. Our current prompt engineering best practices have largely been manual trial and error, testing which of our prompts work well in certain situations. This process worsens as our prompts become more complex, adding multiple tasks and functionality within one long singular prompt. Enter DSPy, your PROGRAMATIC way of building GenAI Applications. Learn how DSPy allows you to modularize your prompt into modules and enforce typing through signatures. Then, utilize state of the art algorithms to optimize the prompts and weights against your evaluation datasets, just like machine learning! We will compare DSPy to a restaurant to help illustrate and demo DSPy’s capabilities. It's time to start programming, rather than prompting, again!
Databricks Named a Leader in the 2025 Gartner® Magic Quadrant™ for Data Science and Machine Learning Platforms: https://www.databricks.com/blog/databricks-named-leader-2025-gartner-magic-quadrant-data-science-and-machine-learning
Build and deploy quality AI agent systems: https://www.databricks.com/product/artificial-intelligence
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements

Chapters

FAQs

What is DSPy and how does it differ from prompt engineering?

DSPy lets you program GenAI applications using modular components and typed signatures instead of managing blocks of text. This enables proper software development practices like testing and maintenance.

How does DSPy optimize prompts?

DSPy uses state-of-the-art algorithms to automatically optimize prompts and model weights against your evaluation datasets, similar to how machine learning trains models.

Can you use DSPy in production?

Yes. DSPy applications deploy via MLflow, enabling version control, monitoring, and maintenance—the same practices used for traditional software systems.

What is the restaurant analogy?

The restaurant analogy illustrates how DSPy modularizes your GenAI application into components (like ingredients, recipes, and servers), each optimizable and testable independently.

Full transcript

`0:00` Introduction and context

All right, for those of you who have their headphones on, can I get a thumbs up? Can you hear me? Fantastic. All right, we're going to get started. Uh, if you have any technical issues, please uh wave down someone um up front or just kind of put your hand up if you have any technical issues. Um, with that, so let's get started. So, we have only 20 minutes to get through this, but today's session is about learning to program, not write problems with DSPI.
We're going to use a restaurant example to kind of articulate how you're going to use DSPI to um declaratively program your Genai applications. And we'll go through an example of that as well. Um, but first, our forward-looking statement. You're going to see this a lot, so I'm not going to read through this. And a little bit about myself. I'm a delivery solutions architect from Data Bricks. Uh, I work in startups and emerging enterprises from Los Angeles and I have a huge interest in my cats and video games.
So, fun facts. So, all that to say, um, as we all know and as you've probably interfaced with like chat, GBT, claude, and so forth, writing prompts is extremely tedious and very unsustainable, right? You don't want to be managing an entire block of text, which I'm sure some of you have already tried to do, um, to get your large language model to do exactly what you want it to do. So, it's also hard to identify where in that prompt something is going wrong, right?

`1:05` Problems with manual prompts

It's it could be this portion of your text, it could be this portion of your text, who knows? Um, but all that to say, you're ultimately just doing this by trial and error, right? If you were writing a system prompt for like your other framework, I'm not going to name drop, but um and you're trying to figure out which system prompt works the best. You're doing no shot, few shots, self-consistency, all these different prompt strategies, and you're just trying to figure out if it works.
That's not also good practice. Um and it's also incredibly difficult to modularize, right? um you have to maintain essentially a library of prompts that you're going to send to your different applications, but at the end of the day, it's still text, right? Like what are you going to do with text? So, all that to say, that's why we're here to talk about DSP. DSPI 3. 0 will be announced there's a three DSpai 3. 0 session tomorrow from the creator of DSP himself.

`2:22` What is DSPy

So if you want to learn more about DSPI 3, please register for that session. But this will kind of but I'm not going to cover that. We're just going to walk you through on how to use DSPI to then program your Genai application. So what is DSPI? , sorry if I can't pronounce his last name still, but, he's the one who created it. Came out of. um he was under mate uh our CTO and a lot of a lot of his inspiration comes from PyTorch and neural network abstractions.
So as you look through the DSPI examples, you'll actually start to see some familiar patterns from those frameworks within DSPI. It's a declarative uh framework. A lot of people first used to use DSP before structured output came out for from OpenAI. But ultimately it uses typing. It uses reusable modules to develop and code large language model abstractions that you understand instead of manual strings and you can iterate much more quickly with that structured code instead of again blocks of text that we've all probably have worked with at some point uh in your development.
It's lightweight and futurep proof and by what I mean by that is that it's pure Python. You're not here waiting for an integration to come out for your particular framework. It's designed to accept really any Python library that you want to use. It's you don't have to wait till that library is compatible with other with whatever framework specific typing that comes out. Um it's lightweight. There's not a lot of different um dependencies which is probably music to some Python developers ears and it's very reproducible.
Okay, so all that to say, let's look at some code and stop looking at slides, but it's really as simple as uh defining a class. So the first thing here, if you can see the laser, no. Okay, I'm going to try and point to it. But this first line right here is you actually setting the large language model itself, right? You don't have to change anything else if you want to use any other provider. you just change it to like bedrock forward slash whatever your model provider is and the and the rest of your uh DSPI code will work because in the back end it uses light LLM that uh handles all that uh conversion for you.

`4:14` Example DSPy workflow

Um so again if you don't want to use data bricks you can use open AI you can use anthropic you can use whatever and you you don't have to change or install any other packages you just need to change that line and you'll be able to use that new model. This right here this class is essentially what would be your prompt but you're defining a signature that's describing how you want the model to accomplish the task. So for this one, we're trying to classify emotions.
And I'm expecting a specific input called sentence, which I want it to be a string. And the outputs that I want is a literal with these uh strings. So DSPI knows to take this and then force the input to be a string and the output to be one of these emotions. So and then if I need to run it, there's these last three. the actual input, the sentence itself. The um this is us assigning it to a pre-built module DSpi. predict.
Uh DSpi. predict is one of their is one of the pre-built ones to actually just interact with the large language model. And then finally, you run classify sentence equal sentence. Um the output is I didn't put it up here, but the output will then come out to be your sadness, joy, love, whatever, whatever the case might be. And you can use the output like classify sentiment to access the value that came out. So all this to say to help you better like visualize and understand how to develop something with the Spie, we're going to um think of this as a restaurant.
It's like taking a recipe and then putting it giving it to your cooks to uh create a brand new dish. So you're trying to define your ingredients like you saw with that DSPI signature. You're trying to uh define what your inputs are and what the final dish is supposed to be, right? What's the final output? And then you got to define your goal so that the so that the uh class so that you know what the class is supposed to do.

`6:24` DSPy restaurant analogy

Try them out together, figure out what you want to do with it, and then and then discover uh your new dish. So you'll be using DSPI signatures as you saw a preview of in that first uh code snippet. uh that's your recipe on how you want the large language model to handle that use case. And then the module itself, there's pre-built one like I said, but you're you are also allowed or DSP lets you build your own DSPI module um which you'll see in my later session at 4M if you want to join that one.
Um where it's a much more um detailed version on how to do that. But you can actually just create it by initializing and then having a predict method to put not only different modules together, but your own Python logic. You don't have to use only DSPI code. You can use mostly Python code and then call a DSp. predict to u complete everything, right? So you like do some ETL first or something like that. Um so these are just some example modules out there, but there's still new ones coming out.
Uh the PI 3. 0 has a few more coming out as well, but these are the main ones that uh you can use. And for those of you in the back who can't see, there's predict, chain of thought, react, and best of end. React is what you would use for agents. So if you are familiar with function calling, there's the reasoning and the action part. And that's kind of what that enables for your language models. Okay. So let's go through the step-by-step process that we talked about uh earlier.
Your first step is to define the ingredients. So your ingredient is what model are you going to use? what are you going to use for this classific what do you want to use for this classific uhation task so like I said earlier you can define any multiple configurations from any provider and then you can also interchange them depending on the model module right if you have a uh very specific use ca uh sorry very specific signature that's only doing like text extraction maybe use llama 8 billion or if you have a like image classifier and you need to use a multimodal model then you should maybe configure your anthropic cloud 4 and then use that for that part and you can use that all in one uh DSPI program.

`8:13` Defining ingredients and goals

Okay, so I kind of went over this step two is defining your goal and your end dish. So let's go back to this example. What were what were we trying to do? We're trying to classify a emotion within some kind of string string text, right? So we need to use the DSPI signature to actually declare the inputs and outputs of these of it so that we know what we're looking for and we know what the inputs are and what the outputs are.
So you have to make sure any kind of typing that is in the typing library essentially you can use it could be a boolean it can be an integer it can be a float but just keep that in mind and if you want to use custom uh typing you can always use pideantic um base type also came out from NDS PI 3. 0 know where if you're familiar with uh paidantic base model it's pretty much the exact same thing but you can create your own uh own like that and pass that in as a type so that way you can also do some data validation before it uh you put in an input or get an output so one thing that I haven't really mentioned is the iteration step of this right so we especially when developing generative AI applications you want to go through you want to iterate quickly you want to make sure what you had created and the model that you're using is performing up to the task, right?

`10:06` Testing and modules

So you it can be it's really all you need to do is pass it into something as simple as this. And but if you don't if you feel like the predict is actually not working too great, but you want to use this large language model, maybe consider using chain of thought. Um if you don't know what chain of thought is, um please Google it. I don't have time to talk about that but you probably know what it is. Uh but chain of thought does help gives it some extra steps extra reasoning to help with that particular use case.
So you can go through um some other modules very quickly to in order to uh accomplish that uh use case. Okay. So kind of covered all this and we'll keep iterating. So here's another examp so this here's another example where you maybe you've decided okay I don't actually don't like that large language models performance I'm actually going to switch it to openai's GPT 4. 1 I'll just reconfigure it to that mo that model and I don't have to change anything else you see the class is pretty much the same thing um and you can just keep reiterating iterating it over and over again with minimal code change to make to really understand what's impacting your gen AI application the most So once you're satisfied, you can use datab bricks's mlflow integration, dspi. autolog to do mflow tracing.

`11:09` Iteration and optimization

You can also do uh and then you can uh register it, log it, deploy it all on data bricks. Of course, that's going to be the pitch. But um you would use mlflow to um kind of track different versions of your agent. um make sure that you can do governance on it through Unity catalog and use the infrastructure data bricks provides on the model serving infrastructure to actually deploy it. So this is simple I don't have the actual deployment code in here but this is just to get it into uh Unity catalog.
So altogether this is what the entire code looks like. Um so with the first step again let's just to kind of like walk it back. The first one is we define what model we want to use and then we define the signature itself right here to understand how we want the large language model to uh accomplish the task that we're defining. Then we do a quick test, maybe do another test. Maybe we want a stronger large language model or maybe you know if I think what there was like a new model that came out like a few days ago.

`12:21` Deployment with MLflow

Um you want to test that one just change this and then you can change it and then it'll work against this same signature. So and then when you really like it you can use it down way down below here. So that's really it. Now you know how to use DSPI. That's all you need to do in order to start using this framework. You don't need to know any other classes. You don't need to know any other uh functions. That's really the core of how you would use uh DSPI.

`13:06` Resources and Q&A

And if you need if you want to see a more advanced or other features of DSPI, there's a 3. 0 session tomorrow and then there's a my session, the multi- aent session at 4:00 later today. So, please join the community. is very active. I've never seen a um a creator so active on his Discord, but he's literally replying to every single person. He's um so uh feel free to join him, join us on Discord, contribute to the repo, and this is the official documentation on the left right here.
So that's it. Um we got six minutes left. So if there's any questions, I will have to come up to you and repeat your questions. So if you have any questions, raise your hand. But otherwise, that concludes the uh that concludes the session. Thanks all.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.