Coding Agents with Open Models: Best Practices for Harnesses, Sandboxing and Safety

Summary

Hybrid Model Stacks: Optimal AI systems combine open and closed models to reduce token costs while preserving high performance across diverse coding and reasoning tasks.
Meta-Harness Architecture: Standardizing operational primitives like sub-agent coordination and context compaction across multiple harnesses decouples developer workflows from specific underlying models.
Centralized Sandboxing: Open models require uniform sandboxing policies and safeguard mechanisms to prevent unsafe execution and sandbox escapes that closed platforms mitigate natively.

Coding Agents with Open Models: Best Practices for Harnesses, Sandboxing and Safety

Talk by Co-founder, Ollama
Open models work well with coding agents when you pair them with a meta-harness, consistent sandboxing and full trace logging. This video explains how to use open models such as GLM, Kimi, DeepSeek and Nvidia Nemotron with coding-agent harnesses, how to choose a model, and how to stay safe.
The Ollama co-founder describes why a mix of open and closed models tends to be the best configuration, citing lower cost, control over privacy and geography, and access to weights and traces. He mentions AT&T shifting 40% of its tokens to open models and roughly 30x growth in tokens per developer. He introduces the meta-harness as a sixth layer that standardizes web search, sub-agent coordination, context compaction and sandboxing across harnesses, then covers auto mode and safeguard models. He also explains why providers quantize the same open model differently, so teams should monitor cache hit rate, tool-call success and thinking time.

Chapters

FAQs

Why should teams use both open and closed models instead of choosing one?

Hybrid stacks reduce token costs while preserving performance for different tasks. This video explains how combining open and closed models optimizes efficiency across diverse coding and reasoning workloads.

What is a meta-harness architecture and what problems does it solve?

A meta-harness standardizes common operational primitives like sub-agent coordination and context compaction across multiple models and harnesses. This decouples developer workflows from specific underlying models and tools.

What additional safety measures do open models require versus closed platforms?

Open models lack built-in safety classifiers, so teams must implement centralized sandboxing, auto-mode safeguards, and content moderation policies. This video covers uniform safety protocols essential for open-model deployment.

What observability and debugging benefits do open models provide?

Open models offer full access to raw thought traces and weights, enabling security auditing and customized developer loops. This video demonstrates how transparency enables better debugging and detailed reasoning verification.

Full transcript

`00:00` Introduction

Hello everybody. Um, thank you so much for having me. It's um, really exciting to be here and yeah, let's jump right in. Um, and funny enough, the Olama name in some ways a lot of people ask where the name comes from. Interestingly, it just happened to be a name that had a domain available and we were able to get like the GitHub name for it. And um, I think Olas kind of found its own character. And what isn't what is Olama?
It's kind of an alpaca, kind of a llama. Um, this is a llama. Um, there'll be pictures of a llama throughout the deck, uh, you know, sprinkled through. Today, I want to dive talk a little bit about using open models in this incredible wave of coding agents that are obviously going to solve many use cases beyond just coding. Um, and what role open models have played along the way. um and why they're interesting to think about in this new age of, you know, harnesses, metah harnesses, and ultimately this kind of unstoppable cravings for tokens that developers and businesses at large um are are undergoing right now.

`01:04` Why use open models with coding agents?

Um but why why open models in that world? Um how to think about choosing a model and there's so many open models. Um there's been one almost every week this whole summer like a frontier level or near frontier level open model. Um how to think about using them with harnesses. And then lastly some touch points I've learned along the way along how to around how to stay safe with open models. Um because as you know you just get them as weights on you know somewhere like hugging faced and so how do you actually you know think about security and observability and safety uh with open models.
Okay, so quick background about me. Um, co-founder of Lama. Um, the project's been around for about three years. It's since evolved into a cloud service for hosting the largest open models. Um, back when Olama started, there were a lot of models in the like 7 billion to like maybe 70 billion parameter range like the original Llama models. They've gone much bigger. They now come in sizes of trillions of parameters. So, we had to stand up a cloud service to help with that.
Um, it's a large install base. We've collected a lot of information and feedback from developers around how to be productive with open models, but also with businesses and it's a little known fact that businesses are adopting open models incredibly fast and I'll talk a bit more about that and you've probably seen that all over the news in the last few months. Um, and you know, we've obviously been taken along for a ride there, too.
Um, and lastly, you know, we like I mentioned, we've since gone from like a lot of people know Alama as like an open source tool for running models on your Mac or maybe your server, but it's we actually have a a cloud service that's really exploded since the beginning of the year. And so, we've gotten to learn a lot of the know-hows of how do you run these things? How do you but not just like how do you run the inference, but how do you actually plug them into harnesses and be productive with open models.
This is a graph. Just speaking of open models, like I I took this the other day um and it's a snapshot of the different model families that are running on OAM's cloud. Like this is kind of the token volume um you know magnitude and then also split by like the model families. And as you can see it's predominantly these like frontier models um from China but there's al also a large percent that are you know emerging from US and European model providers like the Neotron models for example from Nvidia.
Um but these these models you know it's really started at the beginning of this year as you can see like the lines pretty much invisible before in 2025. This is you know the the first half of 2026. Um it's mostly these frontier level uh coding agentic models a lot of them from China that have really taken off this year. And so but if you look back to the history of open models like how did we get here? Well, a lot of it started with Llama and obviously in 2023.

`03:36` Timeline of open model capabilities from 2023 to 2026

Um, I think the first Llama model was in March if I remember right. That definitely Llama 2 was in July because it was the same day O Lama launched. It's also my mom's birthday so I remember that day very clearly. Um, and then you've kind of saw like capabilities and uh both capability um but also different modalities and use cases be layered on through the years. And it's been like an incredible three years. Like we saw tool calling come out in early 2024.
Everyone remembers the deepseek moment that like crashed the stock market. Um because we had a model that was very capable and very efficient. Um and of course that was kind of our first open model moment. Like many people had no idea what open models were until this moment and all of a sudden it was all over the news. Um in March in 2025 we really saw the emergence of models that could do egentic tool calling and multimodalities.
So if you think about like a modern model you're using for agentic coding, those didn't even exist um in the open model world like the capabilities weren't even there um until uh late last year. So has it's only been like a year since GPOSS came out and that was really the first model that had like an agent looped programmed into it which was awesome because I there was Quen 3 as well so it's about the same time but I feel like GB2SS was was by far the highest performing one.
Um, and then this year we really saw the models size up. And so the Kimmy K2. 5, GLM5, you had these massive models that are agentic in capability. They're multimodal. And it was like finally there was a way you could run pretty hard coding tasks with open models. Um, and you know, the cloud code moment happened over the holidays and then these these open models were fast to to come out. Um, and I'd say a lot of it's really culminated to like last week when GLM 53 and 53 Flash came out when you had these models that are like cheaper and more capable than similar closed models.
And in some eval capable I think you saw like the Kimmy K3 model was better for webdev versus the you know any other model. And then for some of the cyber evals, the new GLM53 model is really good, which we'll come back to in the safety section. But I'd say out of all these moments, like it kind of felt like deepseek was really it. Like it was the biggest all of a sudden moment. Um, but I think if you look at enterprises and customers and developers adopting open models, like right now is the biggest moment by far and we're feeling it. you know, you have these models that at least, you know, from a cost performance standpoint are cheaper and and that's matched with the fact that like token usage by developer is just exploding like you know I have a graph I don't have it here but of like the how many tokens a developer is using on average and it's like 30x since the start of the year on an average week and so you have this problem now where you know open models are great for many reasons I mean one is obviously they're much more capable and even sometimes more capable and But, you know, you can see it all over the news is that they're really a solution to cost problems that have been introduced by some of the really large models.

`06:51` Enterprise adoption and token costs: why teams shift to open models

And for many businesses, they're realizing you don't need a frontier model for every single task within a business. Um, I think we saw that AT&T shifted 40% of their tokens to open models. And that's matched with the fact that there's like an open model every 10 days from uh, you know, a myriad of different model labs. some in the US, some in China, some in Europe. Um, you know, we saw the Muse models from Meta are no longer Llama, but the Muse models are now, um, you know, Muse Glimmer was open source and there's also a commitment for, you know, the variant of the Muse Spark model being open sourced as well.
So, that's really exciting. Um, so we're here, like we're finally at a place, you know, I've kind of been waiting three years since working on O Lama for this moment. Um, but adopting open models isn't a panacea, right? Just because they're cheaper doesn't mean there aren't other considerations. The good news is there's a lot of tooling stepping up to help. The reason a business adopts open models, I mean, the first one is cost, right?

`07:44` Open vs. closed models: cost, control and owning the learning loop

If you think about why would you even reach for an open model, you know, and we're really realist about this um on my team with Lama, like I'm using open models every day, but the reality is is a mix of open and closed models tend to be the best possible configuration because for a lot of tasks, you can just use open models and save a bunch of costs. um you have a much tighter control over where the model's running and privacy and security and that you really get to control where the model is, the geography, what hardware it's running on, whether it's in your environment, a trusted cloud partner.
And then lastly, the cool thing about open models is you have full access to them. Often they're licensed in like MIT or MIT variant licenses. And so you really have access to like all the traces, you have access to all the weights, you can kind of do anything you want with them. Um, and what this really translates to, and you've probably saw Satia's tweet about like owning the learning loop, is that open models are a really good way for you to own the learning loop and for teams to do that with the traces that emitted from these models and to actually go and build really a custom stack for their AI.
Um, versus kind of using a stack out of the box from some of the Frontier model labs. The hard part is there's a lot of them and I think the easy part of this exercise is that you there's really only a subset of models that are truly cost performance you know at the frontier from a cross performance standpoint and are actually you know very productive at solving coding tasks namely it's kind of the Kimmy the GLM the deepseeks neotron this small set of models that are you know anywhere from like 500 billion parameters to two trillion in size you know we started with Alama running like 4B models on your Mac, which was really exciting to for chat and rag use cases, but for these much more difficult coding tasks, um you need the big models.
And you know, I think over time that'll change, but right now there's kind of a really small subset that are really um able to solve the kind of problems you can solve with like an opus or a sonnet level model. Um and in some cases even kind of the fable frontier level models. Okay, so that's some background on open models. How do you get started with them? And this is, you know, where it's quite daunting because like like I said, you might get an inference API from somewhere or you might be able to get the weights off a hugging face.

`09:49` Getting started: plugging open models into agent harnesses

But how do you actually get up and running? And generally the first thing every single developer we talk to says, I want to take this model and I want to plug it into the harness. I'm already using the coding agent. Maybe it's a personal assistant like OpenClaw or Hermes more popularly now. Um maybe it's like an editor and IDE like VS Code. Um there's a whole bunch of new harnesses. I think the thing that's a constant here is that this layer, the stack is constantly changing.
Um, but this tends to be the first step to like using an open model is to like take the tool you're using and bring in an open model. And so this is kind of the five layer cake if you think about like what it takes to get an open model running. It's very similar to the five, it's basically the five layer cake that you know Jensen talks about which is you have like an agent harness, you have the models, you have like the infrastructure which is generally just running the inference.
Um you have the chips which is like Nvidia, AMD, um you know new players like Cerebras and you have you know all the power uh that that gets consumed from that. Um and so this stack doesn't really change from the open models to closed models. Um, and you know, if you think about it, most of the time, you know, to be successful with open models, you have to think about the whole stack, right? You don't really think about chips and data center that much unless you're running your own.
Um, but it's a very similar stack. It's not like it this dramatically changes. Well, so now what's the problem? And so it ends up being able to use any harness with any model um actually comes with a lot of issues. And they're mostly good problems to solve, but they're nonetheless problems that anyone adopting open models and kind of building their own five layer cake needs to figure out. Um, here are some examples of it.
Web search, for example, how do you actually bring up-to-date knowledge from the internet into your context window? How do you manage sub agents and coordination? There's a lot of things behind the scene that the harness needs from the model to power this. How do you do context compaction? Um, sandboxing, auto mode for safety. um where does your traces and memory go when you're switching between different harnesses? And lastly, everyone has their favorite set of tools they use to automate their work.
How do you bring that uh into your favorite harnesses? And the thing I've learned from talking to tons of different enterprises adopting open models is that they're not running one harness. They're running like five or six harnesses and they're even building their own harness. And so, how do you take care of all these problems? And it's really not a problem where you want to like redo it every time you're trying a new tool.

`12:21` What is a meta-harness? The sixth layer

And so there's the six layer cake. And I think, you know, this is more of a a way to think about it. I don't know if it's fully worth a uh an extra layer in the cake u in the sense of maybe you can fit it into the model. Maybe it's part of the harness, but I think there's this new concept of the meta harness, which I think Omnigen, the Domin team really coined. And this is where if you think about those common items that are the same between every harness, that's this kind of new layer that's formed.
And it gives you the portability of any model you want, but also with any harness. And this is especially important for open models because the models are changing so often. There's a new one coming out every week. And then with the harnesses, because there's not one prescribed harness, you want to be able to or your team needs to be able to access different harnesses with the same kind of stack below that. So, enter the sixth uh layer and much bigger cake, much tastier.
I think this is a really powerful primitive that, you know, is really emerging now because of this sudden shift to open models in the market. Um, and that's an example of like switching from, you know, a a closed model, one harness to another one. But there are many other ways to do this. You could have two harnesses running at the same time. You could have a harness that's designed for a specific task. Uh, and or you could even just build your own harness and have it integrated.
Some of the things that you know come with this six layer that I've been thinking a lot about because of what customers have been telling us, what developers have been saying is how can you use it not just as an opportunity to be, you know, to buy flexibility and control and portability between harnesses and models, but how do you make sure you stay safe? And one of the challenge with open models is that they don't come with a lot of the included benefits that the API platforms and co and platforms and uh agent platforms from the closed models come with.

`14:12` Safety gaps with open models: sandboxing, auto mode, moderation

Here are some examples. You know for sandboxing for example you might have you know the codeex harness comes with a sandbox and then you'll have like cloud agents with codecs or claude like all that kind of just works out of the box. Whereas with open models you kind of just get weights and and inference. um for auto mode. Like everyone here probably uses auto mode at this point. It's the default in most of the harnesses and it's a heck of a lot less work than saying yes, allow, allow, allow.
No one enjoys that. Um and so the closed models come with this awesome safety classifier system that make that possible and it just doesn't come with open models. So a lot of people resort to bypass permissions which isn't a great experience or they kind of like jerryrig the auto mode to just pass or they just end up asking get getting asked you know for permission every single command that's no fun. Um content moderation is obviously included which may not be important on an individual use case but if you're thinking about a team at a business you really need to make sure that you're filtering content going in and out of the LM.
Um this is included with some open model inference providers. it's probably not the one you want or it's not something you're expecting. Um, generally this doesn't come out of the box. And I'd say the most important one is this idea of like now that you're running different harnesses, you're accomplishing different tasks, where does all that data go? And for a lot of the closed model providers, they provide you tools to capture that, to go back in history, to log it, um, to potentially even train a model on top of it.
Um, although less and less now. And for open models, you're kind of left just dealing with that yourself. And I think it's the most powerful piece. But let's start with sandboxing. I think sandboxing is exciting because it allows you to kind of let an agent run free and you can kind of trust that it's not going to do anything outside of its boundaries. Um, but the challenge there is that every harness has a different sandboxing technology.

`15:52` How to get consistent sandboxing across harnesses

It has different perimeter of what it covers. And then lastly, like they have different policies out of the box. You end up with this really fragmented experience. The ideal case you're going for is how do you have one set of policies, a consistent experience local to cloud. And I think this is really exciting from a standpoint of how does a developer build something locally, then like push that to the cloud so their agent can run when they close their laptop.
That's an experience you get with like the closed models, but for open models, there isn't really technology for that right now. Um, before this, I was at a company called Docker for four years and like that's all they did was like making things portable. I think there's a ton of opportunities to explore there if you're building in the space. Um, and then lastly is that the open models are actually becoming frontier level at security tasks such that eventually could lead to escaping sandboxes.
And so this is an especially important problem as teams adopt open models because models like GLM53 um that's actually what they're really good at. And so I think at the end of the day and it's true also you know the mythos and fable models. So I think it's a really big opportunity if you're building in the space or you're building tooling. Auto mode is such a convenient piece of technology right like you don't have to think about approvals anymore.

`17:18` Auto mode and safeguard models

You just say let it go you know go grab your coffee go take a nap come back and your work's done. But it's actually a really hard problem and it generally relies on a a welltuned model. Um you can use a large LM for this for now. like you could take you know an LM as a judge approach and kind of we built that into a llama right now if you use a llama as a provider um it'll do this with cloud with auto mode with codecs and with a few other harnesses um but there's actually a big opportunity to solve this problem for all open models and you can see on the right this is a diagram from openai that actually released a safeguard model um in the past year that just focuses on this problem and I think it's a really exciting use case and what's so cool about open models um that I'll talk about in a is that you can actually see their traces.
So it allows security teams and policy teams to actually analyze like why was a request allowed or denied because you can see the whole chain of thought and that was a really exciting uh kind of decision and and feature when they open source this model. So I think we already have some models that can do this in the wild. I think there are going to be newer ones and more efficient ones. Um but this is a really exciting space to watch and also build in if you're looking to build tooling in the open model space.
And the last one I'll say is logging and traces are such a valuable piece of data, right? And what's cool about open models is the entire trace is available like every single thought process. And you probably saw some of the thinking of the new open models. Like some of them are like super strange. It's almost like caveman speak. Um but you can actually see this and work with it. And you can actually like like you know even inject like h like thoughts into the model's process.

`18:32` Using full thought traces for security auditing

There's a lot you can do here. But I think from a security standpoint what this means is you can see everything an an agent did why it did it introspect that edit that and customize that for your team. Where we've seen this become really valuable is for developer teams to be able to say customize a harness around the traces that they've already collected. And for security teams being able to audit each agent action, what it did, why it did it.
If you think about like what are security teams a lot of them really worried about, it's are agents going to take action on a system or collect data or send it somewhere um that is against their policy. And so being able to access this is super important. I think there are quite a few tools emerging to solve this problem, but it's a really exciting space as well because there's so much the volume of data coming out of these agent coding agent models is just incredibly massive.
It's only growing. This is one we've learned. So, Alama's cloud, we basically pull a bunch of different inference providers and neo clouds together into one kind of experience for adopting open models. And I think as you go on your journey of adopting open models, one of the challenges you'll see is that it's not like open and anthropic where you might have one maybe two providers for the model. Like you might have like anthropics API and like bedrock in Google cloud.

`19:38` Monitoring inference providers and quantization differences

So maybe three. Um there are like 30 providers for a given open model. And one of the things we've learned is they do not all run the model the same. They quantize it differently. Um just because it's the same weights doesn't mean it's going to be the same tokens as well because it can be on different hardware. it can be running in different conditions. Um, generally it's because it's it's not the it's the same base weights, but it's been quantized.
And I think one of the important musthaves when you're running open models is to really not just look at the tokens, but actually monitor the performance and make sure that you're hitting cache, you know, your cache rate, your tool calls are succeeding, um, which can be a sign of how well the model's running from a quality standpoint. How long is the model thinking? These are things that we've learned, we've had to monitor for on behalf of our users and developers, but it's something as you adopt open models that become a really important uh performance characteristics beyond even just the data going in and out of the model, which is something you generally didn't really have to think about at least as an individual developer um or a small team with the closed model providers.
Uh, I hope this was insightful and interesting on like how to get up and running with open models, how to think about using them with a meta harness. Now, meta harnesses can really unlock moving to open models with harnesses, switching between harnesses. Um, and I think it's a great next step, just check out these open models, try the latest ones. The new GLM models are incredible. Um, obviously the Nvidia models from a US model standpoint are just awesome.
Also really awesome with coding agents. Um, I've been using the Kimmy model for like a lot of creative tasks, which is like a 2. 8 trillion parameter. Um, and always make sure to run models uh with a great sandbox. Awesome. Thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.