Sponsored by: Fireworks AI | Own Your AI: Beat Frontier Models With Your Own Data

Summary

Fireworks AI makes the case that companies should own rather than rent AI intelligence by fine-tuning open-source models on proprietary data, turning internal data into a compounding competitive moat instead of depending on closed model APIs.
This video busts three myths about training open models: that closed models are always superior, that fine-tuning requires a dedicated ML team, and that large labeled datasets are needed—drawing on real production examples from clients including Cursor, Uber, and Notion.
Fireworks uses a data flywheel approach—LLM-as-judge for automatic training data generation—and its Fire Optimizer auto-tunes workload-specific inference; the platform serves over 35 trillion tokens per day, combining training and inference in one fast-iteration cycle.

Chapters

FAQs

Why should companies train their own models instead of using closed-source AI APIs?

According to this video, closed-source model APIs create dependency risk—providers can downgrade or change models without notice—and they cannot leverage the unique domain expertise and proprietary data that differentiate one business from its competitors. Fireworks AI argues that owning a fine-tuned model means treating intelligence as company IP and building a moat from internal data.

What are the three myths about open-source model training that Fireworks AI addresses?

This video addresses three common myths: first, that closed-source models are always better than open-source alternatives (disproved by recent open-source quality trajectories); second, that fine-tuning is too hard without a dedicated ML team; and third, that successful training requires massive labeled datasets. Fireworks AI offers techniques like LLM-as-judge to generate training data automatically from production outputs.

What is a data flywheel and how does it help with model fine-tuning?

A data flywheel in this context uses an LLM-as-judge to automatically evaluate production model outputs and generate labeled training data without human annotation at scale. As the model is used in production, it generates more examples, which feed back into training, compounding the model's performance advantage over time using the organization's own data.

What is Fireworks Fire Optimizer and what does it do?

Fire Optimizer is a Fireworks AI capability that automatically tunes inference for specific workloads, optimizing latency and throughput for a given model and use case. This video presents it as part of Fireworks' unified training-and-inference platform, which allows companies to iterate quickly from fine-tuning to production serving without switching tools.

Full transcript

[00:07] First off, thanks everyone for for joining. Um I'm going to I'm going to talk about open source models and training post-training open source models and how we do that at Fireworks. So, for a little bit of context, I work at Fireworks AI. I'm the head of applied AI
[00:20] there, so basically forward deployed engineering working with companies like, you know, Cursor Recognition, as well as like large enterprises Uber, Notion, etc. Super excited to talk to you guys today and then after after this
[00:32] session, I'll be here, so please come on over and we can also have like a more personal conversation. Um so, at Fireworks, we've had this vision of like owning your AI rather than renting your AI, right? And what that means is that
[00:44] with open source models, you can basically customize them, specialize them for your specific use cases, right? Whether that is customizing them because you want like lower latency for your product so that your users are happier
[00:57] with it or that is using your own data and the wealth of your data to fine-tune the models to push the frontier on really, really complex use cases that the close source
[01:09] providers haven't really cracked, right? So, we're all about specialized intelligence and that's why a lot of our philosophy philosophy is all about owning your AI, right? Um as I said, you know, I think we we have a vision that, you know, we use close source models
[01:23] internally as well. They have their place, they're super, super strong, but when it comes to companies really differentiating, it really is all about owning the intelligence rather than renting the intelligence, right? And it's all about the model being thought
[01:36] of as your product and as your IP because as you bring in that data that really is, you know, this is this is not the first time you'll hear it from someone, but the data is your moat, right? Like the subject matter expertise
[01:48] and all of the data that you've accumulated inside your businesses, that is what really differentiates you and your business from whoever else and all of the competitors, right? So, hopefully today I'll convince everybody to stop
[02:01] renting and start owning. Maybe show of hands, who here uses open source models already? Yeah? Okay, half. Cool. So, some of the newest ones like GLM, Kimiko 2.6, Deep
[02:14] Seek, all of those, yeah? Okay, nice. Love it. So, on Fireworks we've been working with companies as I said from, you know, the the most innovative AI natives like Cursor all the way to huge Fortune 500
[02:29] e-commerce companies, right? And what we've seen time and time again is that the most innovative of these companies are really owning their intelligence by training models on their own data, right? And they train models on their
[02:41] own data not just for cost reasons, although that can be a valid a valid reason, right? But really to push the quality of the model inside their use case, right? So, some work that we've
[02:54] done recently and there's a an excellent Spotify podcast of our of our Dima, who's one of our he's our CTO and one of our co-founders, talking with Federico Cassano, who is the head of
[03:07] research at Cursor, how we trained Composer 2 and Composer 2.5. That's super interesting work that we've done, global scale infrastructure in order to do these RL rollouts. That's of course like some of our most sophisticated work, really pushing the frontier in
[03:20] terms of like coding models, and and really hitting that Pareto Pareto point between like latency and quality, right? Like, how you get a model which is blazing fast, but it's just as good as an Opus 4.8. And then as
[03:34] you move to the right, right? There's Gen Spark, another big clients of ours. We've done a deep research as well as like slide generation work with them. There's Vercel on coding, and then the last one which I worked on directly is a very large e-commerce company where we
[03:47] used SFT, supervised fine-tuning, that I'll talk a little bit about in a bit, Um, and RL in order to bring down uh the the cost while also increasing the the uh the quality by around 15%.
[04:00] So, a lot a lot of work from, you know, the the startups all the way to the large enterprises, those that are owning their AI see a ton of benefit. Now, what I'm going to what I'm going to going to talk about now is there's this um
[04:13] misconception, let's say, in the industry, right? That open-source models are not as good. And the second one is that fine-tuning and post-training is super difficult, right? People think post-training and they
[04:26] might think, you know, oh, Cursor, they have a ton of like PhD engineers, Fireworks AI, they have a ton of like X PyTorch engineers, and that's true, right? But we've we've worked on developing a product that allows any company to really start training without
[04:39] needing the level of expertise that you would have needed a couple of years ago, right? So, I'm going to dispel a couple of myths, right? The first one is the idea that um open-source models are worse than closed-source models. I think, you know,
[04:53] the the delta has it's it's almost like a uh a a race, right? Which is, you know, Anthropic, OpenAI, they release their new model, and then, you know, Nemo Tron Ultra from Nvidia comes out, GLM 5.2, they got out like literally
[05:05] yesterday. Um, so, these these slides are outdated because this is Gemini 2.6, uh but GLM 5.2 that came out yesterday, it's neck and neck with Opus 4.8, right? And time and time again, we're seeing
[05:17] all of these different companies who before, even a year ago when I started Fireworks, they were like, you know, open-source is okay, but it's just not good enough. Now everything is changing. And now a lot of like closed-source providers, they're doing things like, you know, basically uh only giving you
[05:30] access to a certain model for certain questions, and then automatically downgrading you to a lesser model, right? Which you can imagine that can be very complicated for developers. You can imagine if you're a company that has a
[05:42] specific product, and then all of a sudden, you know, the AI labs release a product to compete directly with you, then you're really at the mercy of these AI labs, right? But if you're using open source and really training with your own data, then it's a completely different
[05:55] different ballgame. And it's all about specialized intelligence. So, first myth, you know, open source just as good as closed source. Second myth, right? Fine-tuning is super hard. Everybody, a show of hands if you guys have heard of
[06:07] like reinforcement learning here and there. It's like the the new hot topic. Uh but I wouldn't say new but it's a a hot topic nowadays in Silicon Valley. And look, I'm not going to lie to you. RL is hard. Like it really is. But it's especially hard if you're doing
[06:20] everything from the ground up yourself, right? And what we've done at Fireworks is we basically built a platform that allows you, depending on like your level of sophistication, to use the right tools for the right job, right? So, all
[06:33] the way on the right-hand side, you see we have a training agent. So, here you can start already with things like SFT. You basically talk with your agent, tell it more or less what the use case is. It'll it'll find you the the the proper
[06:46] hyper parameters for this training job. It'll run the training job for you. And then you can get a fine-tuned model at the end, right? And then as you move to uh more control and more complexity, we have a product called managed training.
[06:58] Managed training effectively is for ML and AI engineers who know SFT, know DPO, which is preference optimization, know these different methods, but they don't want to they don't want to handle the complexities of infrastructure, the
[07:11] complexities of writing their own uh loss functions, etc., right? So, that's our managed training uh uh product. And then the last one is training API. Our training API is really built for the most sophisticated of teams. This allows teams to really like
[07:25] write their own custom loss curves, integrate to all of your internal systems, because at the end of the day the specialized intelligence is that one that you train inside your own systems, right? Using your own harness, using your own data. So, really built this
[07:38] platform for basically the entire audience. And once you get the right tool, it's so much easier than it was maybe a year ago, maybe 2 years ago, right? Where you would have had to hire multiple ML engineers to stand up the
[07:51] infrastructure, maintain it, and then iterate. So, it's all really about like that developer velocity and hitting that virtuous cycle that I call, which is um um more data, right? Better product,
[08:04] better model, and and then the iteration continues. The third The second myth that that we start to talk about here is that we I at least me, right? I grew up in a in a world with Databricks, right?
[08:16] Where everything was you need massive labeled data sets, right? You need a ginormous data set in order to train a model because when you put it out in the wild, the data is out of sample, it's going to be really bad. Maybe you you uh
[08:29] you it's it's uh overfit when you do when you use your training data, etc. etc. But now more and more you start to realize that these companies, right? They have You guys already have production traffic. right? You already have users. All of these users are
[08:42] already generating that data flywheel. Um and by using things like LLMs, LLM as a judge, you can automate a lot of these evals, and you can get to these proxies that are super super good. So, once you
[08:54] get to automated evals or a version of automated evals, right? And you can start setting this like model leaderboard, that becomes your own training signal, right? And this is the self-improving flywheel that I was talking about before, right? Like this
[09:06] is the vision that we're looking for, which is you have a better model, you have a better product, you have more users, those users generate more data, and then you have a better model again, right? And this continues, right? You trained on We have GenSpark as the
[09:18] example, right? They trained originally with us on Chem K2.5, right? Um actually Chem K2, I'm lying. They did Chem K2 multiple months ago. New model release, Chem K2.5 came out. They don't have to start from scratch. They already have
[09:30] the flywheel, right? They just update to the new model, let it rip, and they have the the of a better model. Chem K2.6 out, same. Kimiko 2.7, which literally got released maybe like Friday, we got it on our platform, which is also
[09:43] insane. Like I mean, side note, but like the pace at which open source models are being released and like the quality of them is just like absolutely insane. But you get the idea, right? Like you want a infrastructure platform that allows you
[09:56] to iterate really, really quick and to keep pace with how fast innovation is happening to the market, right? So this is the third the third myth that I want to dispel. That you really don't need massive labeled data sets. You you kind
[10:08] of already have them, which is your own user data. So I talked about three different myths, right? One is are closed source models always better than open source models? And the reality
[10:21] is they're really not, right? And nowadays as well, I have a one of our co-founders has a really good saying, which is like you don't need a bazooka to kill a mosquito, right? And by that he means not all you use cases require,
[10:34] you know, Claude Mythos or Fable 5 or or Fable or Mythos 4 or Opus 4.8, right? A lot of use cases are hey, I have intent classification, I want to route different support tickets
[10:46] to a different type of like customer support agent, whatever it is. You can use pretty small open source models for that, train them on your data and I bet you will get the same level of quality or better than whatever the largest model is. Plus you'll get latencies that
[10:59] are I don't know, 200 milliseconds instead of multiple seconds and then don't even talk about cost, right? Like literally orders of magnitude, right? So it's all about really picking the right tool for the right job depending on the on the complexity of your task and then
[11:12] also having models that are already so good start to help you in the more difficult tasks on those agentic use cases. I was talking before with a gentleman over here before we started about some of our work with the coding
[11:24] companies and I can tell you outside of like the large coding companies like Cursor Recognition, etc. Smaller very, very successful coding companies are now working directly with us because they realize that they can actually build their harness with a
[11:37] model like Chem 2.7 and reach effectively the same level of quality than they would with a much larger, you know, uh, Sonnet or Opus or Gemini 3 at like a fraction of a cost and they have so much
[11:49] more control over it, right? So, second one already talked about it, right? But every team at any stage can start training with your data today. I'll be here afterwards if you're interested, happy to chat in detail about any of our
[12:02] products. And then the third one, it's all about that virtual cycle, right? Let your data compound. Your flywheel is already spinning. You have your own production data and ultimately, like, let it rip, right? Let it rip and, uh,
[12:15] reap the benefits. Now, I talked about these different myths. I do want to take a step back to talk a little bit of Fireworks. How many people here have heard about Fireworks, know about Fireworks?
[12:27] Okay, roughly half. Okay, love it. Um, so, Fireworks we started around 3 years ago. Our co-founding team were the ones that basically scaled up PyTorch at Meta. And then a couple of people that started Vertex AI at Google as well. At
[12:41] this point, we are, I would say, in the top five largest token fact, quote unquote, token factories in the world. Maybe already in the numbers in in number three. We process more than, this is actually a little bit outdated, more
[12:54] than like 35 trillion tokens per day, right? And more than 280,000 requests per second. To put that into perspective, that is more requests per second that Google Search receives. Of course, Google Search is different,
[13:06] but whatever, but you get the point, right? Like, we're at a scale now that we're rivaling the closed-source models. We have a ton of different customers, good investors, whatnot, but what really
[13:18] makes us special, I think, is like we have the tech, we have the engineering talent as well, but we're still ultimately a startup that wants to work like human to human, right? Our forward deployed engineers, um our field engineers literally work day in day out
[13:32] with all of our all of our uh accounts and work super super closely there, so that we build this specialized intelligence together. Um there's a ton of different, you know, benefits about Fireworks. Uh we started really around just like inference, fast
[13:45] inference was our bread and butter for the first first couple of years, and it still is. We're really really fast, right? Um when I start like I talked today mostly about fine-tuning and post-training, but another part of um
[13:59] specialized intelligence is all about how you customize inference for your use case, right? And what that means, I call it workload optimization. What that means is, let's say let's grab two different very different use cases,
[14:11] right? Like one might be the search use case that I talked about before. Nobody wants, you know, if you have an LLM inside your search bar, let's say you're at Uber or Amazon or whatever whatever uh e-commerce company, and you're you're
[14:24] using that to get better results, you don't want your search to take 10 seconds, right? That's just going to cause a ton of churn, right? Customers are going to hate that, and you're going to lose them. You need your search to be maybe 1 second, maybe 2 seconds tops,
[14:37] which means that inside that pipeline, whatever LLM that you're using in order to get better results, that has to be 150 milliseconds, 200 milliseconds, really really fast, right? Now that use case is completely different to the use
[14:49] cases of, for example, agentic use cases, coding companies and their use cases where latency might not be as important, right? But it's all about quality. So workload optimization um and really this like customization
[15:01] with a fire optimizer, it's all about finding the optimal configuration for your use case, but for you guys to not have to worry about like the headache that can be infrastructure, and we will just create like the optimal deployment
[15:14] shape for your use case. And then, of course, you know, we work with a lot of like very large enterprises, so, you know, enterprise grade, you name it. Uh uh GDPR, ISO, um what's the other one? HIPAA, you
[15:26] know, all of these different ones. They're top of mind. The other thing I'll talk about a lot a bit as well as like reliability and uptime is so so critical for us, right? And we have a ton of different ways to do this through using partnerships with a ton of
[15:39] different cloud providers, having multi-region deployments where if one deployment fails, we automatically route all of the all of the traffic to the one that is working on. And we've partnered with the best open source labs out there, Quan, Kimmy,
[15:52] Nvidia, Nimatron, Mistral, you name it. So, I'm going to I'm going to leave you with one thing, which is the power of having training and inference in a single platform, right? I sound like a broken
[16:05] record when I say it's all about that data flywheel, right? But it really is, right? It's how do we remove all of the friction so that your teams can build great magical AI products really really
[16:18] fast at like the fastest iteration possible, right? The reality is we all know this, you know, you have to iterate. You have to test things empirically. Sometimes things don't work and you have to test another another out. But when you have something that allows you to iterate quickly, that's
[16:31] when you're let's call it not TTFT as in terms of time to first token, but TTVT if that's something. First time to value is what I call it, right? Wow, I messed that up really really bad. But anyways, you get my point, right? Reduce the iteration
[16:44] reduce the friction for iteration, you'll get to value, you'll get to production much much quicker. And what we built is a platform to enable to do that exactly. Going from training to inference to production as fast as possible. So, that's about it.
[16:58] I'll This is a slide around what I was talking about like workload optimization. Happy to nerd out after this about workload optimization if you're interested in learning about quantization, specter decoding, disaggregated serving, you name it.
[17:10] But for now, that's all I have for you guys and thank you so much for spending the last 20 minutes with me. If you're interested in open source and Fireworks, I'll be here. Let me know. Uh super excited to learn about what you're building.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.