Agent Harnesses vs Models: Building Enterprise AI Systems

Summary

A single 'Hello' to an agent like Claude Code consumes over 30,000 tokens because the harness — system prompt, tools, memory, context, and plugins — is what makes modern AI agents powerful, not the model itself.
Nicholas Pelaez outlines an emerging four-layer stack for enterprise agent systems: context at the foundation, skills for workflow reuse, collaboration mechanisms for teams, and governance for production deployment.
Storing traces, model calls, prompts, and reusable skills in Unity Catalog and Databricks AI Gateway future-proofs AI investments by enabling model swaps without losing institutional knowledge.

Agent Harnesses vs Models: Building Enterprise AI Systems

A simple Hello to Claude Code uses 30,000 tokens. Why? Because the harness (system prompt, tools, memory, context, plugins) is what makes the agent magical, not the model alone. This talk reveals the emerging stack for agent systems: context at the foundation, skills for workflow reuse, collaboration mechanisms for teams, and governance for enterprise deployment.
Nicholas Pelaez walks through the economics and architecture of frontier models versus open-source alternatives, showing when to use each. Frontier models excel at open-ended exploration with large context windows, while well-curated open-source deployments handle repetitive workflows at fraction of the cost. Learn how to future-proof your AI investments by storing traces, model calls, prompts, and skills in Unity Catalog and Databricks AI Gateway, so you can swap models and patterns as the landscape evolves without losing institutional knowledge.
🤝

Chapters

FAQs

What is an agent harness and why does it matter more than the underlying model?

An agent harness is everything surrounding a language model — the system prompt, tools, memory, context, and plugins — that transforms a raw model into a useful AI agent. As demonstrated in this video, a simple 'Hello' to Claude Code consumes over 30,000 tokens because the harness provides the structure that makes the agent effective, not the two tokens of the actual message.

When should enterprises choose frontier models versus open-source models for agents?

Frontier models with large context windows excel at open-ended exploration tasks where flexibility and breadth are critical. Well-curated open-source deployments are better suited for repetitive, predictable workflows where cost and stability matter, since they can run at a fraction of the cost of frontier alternatives.

How can organizations future-proof their AI investments as models evolve rapidly?

By storing traces, model calls, prompts, and reusable skills in Unity Catalog and routing requests through Databricks AI Gateway, organizations build institutional knowledge that is portable across model generations. This approach decouples the value of your agent system from any single model provider, so swapping models does not mean losing accumulated patterns and workflows.

What are the four layers of the emerging enterprise agent stack?

According to this video, the stack has context at the foundation, then skills for workflow reuse, collaboration mechanisms that let teams share and build on each other's patterns, and governance for enterprise deployment. Each layer builds on the previous one to move from a raw model to a governed, production-ready agent system.

Full transcript

[00:19] Welcome back to Summit Live. Uh so excited to have 160 countries here with tens of thousands of people all over the world. And right now, one of my favorites of all the world is Nicholas. We are on the same team and we've traveled around and had many good times
[00:34] and adventures together. So, Nicholas, welcome. Thank you so much, Ari. And and it's been a pleasure. Honestly, I've been super fortunate to work with Ari for the last 3 years. It's been a great time. Today is a kind of an interesting talk. So, we're going to be talking about how kind of things are going from Claude
[00:50] Code and Codex to kind of open source. We're going to talk about agent harnesses. And we're going to talk about kind of the entire system, not just the models. Now, to kind of kick us off, Ari, I have a question for you. And I And I love asking this to people. If you say,
[01:06] just "Hello" to a harness, let's say like Claude Code, Uh-huh. if you were to guess how many tokens this request would go, and you know, "Hello" is roughly about two tokens. Okay. If you were to guess, and don't be shy, like how how many tokens do you think Claude Code actually uses to answer
[01:23] "Hello"? Uh six, seven. No, no, I would Sorry. Bad joke. Maybe most of the world doesn't get that, but I don't know. Seriously, like maybe a dozen or so. So, the actual answer is somewhere north of 30,000. Wow, that's a lot. A lot of money. At first, people might say like, "Wow,
[01:39] that's shocking." Like, what what's wrong with Claude Code? But, you know, the classic saying that this is really a feature, not a bug, right? Two of those were "Hello." The other 30,000 plus tokens are the harness that includes things like the system prompt, the tools, memory, context, plugins.
[01:56] Everything that really makes Claude Code and Codex feel so magical compared to kind of the days when we were just using the the raw models. And I'll kind of tell you a story of of how I came to notice this. So,
[02:12] uh in March of this year, I was super super fortunate. I got to go to NVIDIA GTC. Awesome. the That's a big conference? It's a is is a pretty big conference. Uh and I got the the honor of waking up at 2:30 in the morning to go stand out li- outside in line for
[02:30] 5 hours. Again, for the privilege of paying full price for a 5090, which for those of you that don't know what a 5090 is, it's the best consumer graphics card that you can kind of buy right now. So, I'm all excited. I get home, I plug in my 5090,
[02:46] I boot up my inference machine, and I at that time I'd been using a lot of Claude Code. And so, I pointed Claude Code at a Quen model, an open-source model, that fits on consumer hardware. All excited, I type "Hello." And I watch
[03:03] my inference server just ticking away and ticking away ticking away, and finally when it's done, it comes back and it says that that request, and this was a few months ago, took 21,200 tokens. Wow. This was eye-opening to me for a few reasons. One,
[03:20] when you use the frontier models a lot, you kind of get a little spoiled. And you know, we're very fortunate at Databricks, we do have access to the frontier models. You know, we're we're able to use these things at kind of full steam. However, local models, their context
[03:36] windows are much smaller, right? And so, whereas you might have a million context window in an Opus, my local model at best had 100,000 tokens, and you don't even want to use all of those because generally the open-source smaller models suffer a little bit more from what we
[03:51] call context rot, which is that their performance degrades the more that you fill that context window. So, the reason this was so eye-opening to me was that I had just said hello, and I filled up 25% of my useful window. Yeah. Like, huh?
[04:08] I have to kind of rethink how I I think about these things. And and by the way, like, that's just for you. Like, one of the themes I hear as people are trying to scale up is multiply that by an organization that has 30 or 300 engineers, that is where it starts to add up real quick with
[04:24] money. And so, yeah, exactly. And and as kind of organizations are thinking like, okay, maybe we start looking towards these open-source models, open harnesses, you have to really consider that it's really not just the model. There is so much to the harness that helps make it
[04:41] really really a lot more useful to you. And and a good analogy I've heard is that the model is actually just the engine. Mhm. Right? The harness is what makes the whole car. And really what that harness is doing is it it's it's giving you the ability to really pull a lot more context and do it
[04:56] in a very structured way. So, there's kind of a stack that we see emerging, right? Now, at the top of it is context. Context is always king, right? It's what makes the models perform way better, right? If you go and you just tell a model like, "Hey, tell
[05:11] me how to make a cookie." It doesn't even know what kind of cookie you want. So, how do you expect it to give you a really high-quality answer, right? Now, the trick is that getting the right context is not super simple. And that's where the harnesses have come and put a
[05:27] lot of time and effort, and that's why Claude Code and Codex feel so magical because it, when you ask it something, knows where to reach things, where to kind of pull structured data from, how to pull it. And as we use these more,
[05:44] especially across enterprises, we're figuring out that we're doing a lot of the same workflows. Right. But if we never teach the model what those workflows are, then how do we expect it to actually do those reliably really well over time? Well, then the concept of skills kind of
[06:00] came about and became more abundant, right? And so now you can maybe figure out a workflow and save that as a skill and I can share that with you. And now if you import that skill to your cloud code, you have saved the maybe days of trial and error that I had to suffer through.
[06:16] You can now do the same thing in maybe 10 minutes. Yeah, and then you and I work well together. We've actually collaborated on really, really cool projects for another time, but then you have three people, then you have 50 people and that's where like that whole collaboration just becomes so much more important and
[06:32] that's And that's Yeah. Nobody works in a silo, right? We all kind of work on teams, whether you're a small organization, a large one. We all rely on other people to kind of help, right? And so that ability to share skills and to kind of dictate workflows and help you learn from what I'm doing
[06:48] is another enormous part of the kind of harness system. And then there's also uh an idea of permissions, right? What can this model actually go and and and reach? So all of these together are how we're going from a model to a proper kind of harness, right?
[07:05] Yeah. I have Databricks one on my mobile phone. I demo a lot. So without that security, I could say what is your What's my salary or something? compensation. Right. So governance is key. It is. Cuz you you can imagine, right? Especially if you're a large
[07:20] organization, it doesn't take a very big event of kind of spilling out information that you shouldn't for everyone to, you know, put the brakes on everything. So Yeah, and context. One business term, Databricks our fiscal year starts what? February 1st? Yeah, it's very confusing. Other companies may not, but that's the
[07:36] context and that's why it's so valuable to get that accurate insights. Exactly. Exactly. Now we talked a lot about harnesses. we talked a little bit about open models. But a lot of people then ask, okay, well, you know, why would we use frontier models? Well, the answer is honestly, like you still need them for a lot of things. They're incredibly
[07:52] useful. And so, we're kind of seeing that there's there's different use cases for each. So, for example, if you have an open-ended problem, you're going to go research something, you don't know what the path is. The frontier models have a lot of smarts, right? And so, they can
[08:08] actually go, they can do a lot of critical thinking, they can call a lot of things, keep a lot of context. Very, very useful when the path is not trodden, right? But, when you have a really well-laid-out path, you might be able to get away with a much smaller open-source model.
[08:25] Which then goes to the next one, which is cost and scale, right? If you're doing the same thing over and over and over, then and you're you want to scale that out, you might not need a frontier model. You might be better off with a very well-curated set of skills and tools. And then you, if you have a
[08:41] robust governance and and a kind of a evaluation stack, can figure out that, hey, you know what? If I swap this frontier model out with the slight open-source model, our performance is almost the same, but our cost may go down a hundredfold. Yeah. Now, governance is a little bit tougher, right? If you want to pull open-source
[08:57] models, you have to deal with your own governance unless you use something like Databricks' AI Gateway. Enterprise uh frontier models, they do a fantastic job of guaranteeing the enterprise-level security. And then the last one is stability. So, you know, this is probably not new to
[09:14] many people. Uh last week, you know, very famously, there was a model that got pulled back. Yes. It makes you you know, if you're building your business on top of these things, it it's a little bit kind of disconcerting to know that you're not in control of your whole stack. So, if that
[09:29] is a very critical part of your enterprise, then you might be more willing to put the time and effort in to the open-source models that you can own and guarantee how it gets deployed, what the uptime is, and be very stable and consistent. And before you go on, this is like my
[09:45] favorite slide so far of the day. Brilliant and simple. You helped come up with this. I It's like I have I have to give credit, right? So, I didn't make these slides by hand, right? I also use an open-source harness. And so, now again, one of the beautiful
[10:03] things of how things have been evolving is that you can really use and leverage the work of others. So, for example, if you all watching, you want to go and Google a GitHub repo, beautiful HTML slide templates, you'll find exactly the template that I use for these slides, and you can go
[10:20] ahead and make your own, plug it into your own harness, and make something similar to this. Sweet. Yeah. It's been very satisfying. And so, what I kind of want to like leave people with, like what what do you what should you be thinking about like from here going forward, right?
[10:35] The model, the harness, that's not really what's important to you and your company. Or your work, even if you're a solo entrepreneur. It's really the patterns, the workflows, the work that you're doing, that's what matters to you. Now, how you store and
[10:53] save all that information is really what's going to make the difference in how you're able to deploy that in the next 6 months, right? So, if you are not storing anything, you don't have this nice kind of like harness that's all tied up tied together, you are just kind
[11:09] of blasting away, and all of your knowledge and all of the work that you're doing is just kind of going off to the ether, it's going to be hard to compound on that. But, if you have a platform, or you have at least like some kind of rigid structure, where you're storing things like all of your traces, you're
[11:25] storing all of your model calls, your prompts, what skills, what tools were used, then whatever this AI world evolves in the next 6 months, which none of us really know, you're going to be way better primed to use that knowledge,
[11:41] that workflow that you are etching into as your company or your enterprise or even your solo person, and actually build on the next layer of AI that we all know is going to be ridiculously cool, but we just nobody knows what it's going to actually be. So, Harness and Databricks helps you
[11:57] future-proof as things will absolutely pop in and out of existence. Exactly. And so, for us, you know, we're super fortunate on the Databricks side, you know, everything that we do at work, we run it through our own AI gateway, all of our traces, all of those kind of
[12:13] really important information that dictates what it is we're actually doing is all stored in Unity Catalog. So, if in 6 months from now, I want to use a an open-source model, which we also host, by the way, Yeah. and I want to replicate some of these patterns, I'm good to go. All of my
[12:28] data, all of my traces, everything, all that information is there. And so, I'm primed for the future. Awesome. Well, super super duper cool. Um I think I think that's a wrap then. Any any final words of wisdom? Yeah, experiment, play, learn. Like,
[12:44] that that is the only way to really stay on top of these things. And to be honest, it's never been a more fun time to do that. So, just get out there and try stuff. Awesome. Well, Nicholas, thanks so much, and we're going to break to Allison in the room. Thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.