Database Branching: Evolutionary Development with Lakebase

Summary

Kevin Hartman from Databricks and Pramod Sadalage from ThoughtWorks revisit evolutionary database design principles first developed around 2003, showing how Lakebase's copy-on-write branching technology updates those practices for the AI era.
Database branching provides three foundational pillars for modern development: self-sufficiency for independent work, innovation through fearless experimentation without breaking colleagues, and collaboration through governance-aware branches that integrate with Git and CI/CD workflows.
The expand-transition-contract refactoring pattern enables safe schema changes in production, and the same guide rails applied to human developers must also govern AI agents that interact with databases at higher speed and volume.

Database Branching: Evolutionary Development with Lakebase

Evolutionary database design principles pioneered in 2003 are being revolutionized by Lakebase's copy-on-write branching technology. Rather than developers contending for access to a single shared dev database, each team member gets an instant, full-fidelity clone in roughly one second with zero initial storage. This enables three fundamental pillars: self-sufficiency to work independently, innovation through experimentation without fear of breaking others, and collaboration through governance-aware branches.
Join Kevin Hartman from Databricks and Pramod Sadalage from Thoughtworks as they reframe evolutionary database design for the AI era. See how database branching transforms the developer loop with paired Git and database branches, version-controlled migrations, and CI/CD integration. Learn the expand-transition-contract refactoring pattern for schema changes in production, and how agents require the same guide rails and practices that modern development teams enforce through code.
🤝

Chapters

FAQs

What is evolutionary database design and how does Lakebase change it?

Evolutionary database design is a practice developed by Pramod Sadalage that treats database schemas as living artifacts that change incrementally alongside application code. Lakebase's instant copy-on-write branching changes implementation by giving every developer their own isolated database clone in roughly one second, removing the contention of a shared development database that has historically slowed evolutionary practices.

What are the three pillars of evolutionary database design with Lakebase?

The three pillars are self-sufficiency — each developer can work independently without waiting for a shared environment — innovation through fearless experimentation without risking colleagues' work, and collaboration through branches that respect governance and integrate with code review and CI/CD workflows. Together they transform the developer loop for database-backed feature development.

What is the expand-transition-contract pattern for database refactoring?

The expand-transition-contract pattern is a technique for making schema changes safely in production: first expand the schema to support both old and new shapes, then transition application code to use the new shape, then contract by removing the old columns or tables once migration is complete. This avoids downtime and breaking changes when refactoring live databases with active users.

Why do AI agents need the same database development practices as human developers?

AI agents interact with databases just as developers do, but they operate faster and at higher volume, amplifying the consequences of mistakes. Practices like branched isolation, version-controlled migrations, and defined guide rails ensure agents work safely without corrupting production data or interfering with each other or with human developers.

Full transcript

[00:07] Well, thank you all for coming to our session today. Uh I'd like to introduce myself. My name is Kevin Hartman. I work at Databricks. I've been at Databricks for 2 years now. In my role, I work with our consulting partners Censei. So, I was uh in a past life uh as myself in consulting. Um a lot of my
[00:25] work was in software software development. Uh now I work with partners um in helping them build their practices. Uh I'd like to introduce you Pramod Sadalage who uh is actually working for a a partner of ours, ThoughtWorks. And I reached out to Pramod about 3
[00:41] months ago um because when I was back in the days when I was a software developer, there were some experiences that I learned and some practices that I was developing. And I recall all of those moments and the things I was going through. Um and then I found out here at
[00:58] Databricks we have this product and it's called Lake Base. And it has branching in it. And it also it it took a little a few months before I was like, "Wait a minute. There's something here that we can connect back from when
[01:13] we were doing, you know, code development and uh doing feature development and aligning our code uh now with live data data uh database branches. And I reached out to Pramod to revisit and look at some of the practices that he had uh developed
[01:30] um in 2006. So, Pramod literally wrote the book on database refactoring. Um and he's been a thought leader um for many many collaborator with with many of our thought leaders in the Agile development space, wrote several books. Um I'd like to introduce Pramod Sadalage, everybody.
[01:46] All right. Thank you, Kevin. Uh so, so this talk is basically based on the concepts that were developed long time back, but the concepts how we implement that has changed over a period of time because of new technology, because of new capabilities, and that kind of stuff. And like Kevin mentioned, lake
[02:02] base is a new technology from Databricks, and it totally changes the way you think about like evolutionary design, evolutionary development, and the improving the dev loop cycle in like this make a feature, test a feature, and deploy it in the production environment.
[02:18] It kind of improves that stuff. So, over a period of time in this in this particular deck, we'll take you through this whole notion of what evolutionary database means, design means, and that kind of stuff, and how lake base kind of improves that loop, right? So, in this three pillars of
[02:33] evolutionary design, this article that I wrote with Martin Fowler back in 2003, I think, and refreshed it in 2010, and a book around that, and that kind of stuff, the three big things that we talked about in that space was this notion of self-sufficiency.
[02:48] As a developer or as a data person or whatever, you need to be self-sufficient. Like how you provision instance for yourself, how you like test within that instance, how you like can change the schema, design it, and that kind of stuff over a period of time without breaking others, right? So,
[03:05] that's important thing here. That leads to better innovation because now I can experiment in my own space, I can do things, I can try different things, and that kind of stuff without breaking others, so leads to better innovation. And the third bit is it allows you to collaborate. If you're not breaking
[03:20] other's schema, other's code, and that kind of stuff, the fear factor goes down. And invite someone else to your desk, to your Zoom session, to your like Slack session, Slack, whatever, and then collaborate with them to come up with a better solution, right? Because now you
[03:36] can innovate, now you can change, you have that self-sufficient cycle. So, this flywheel effect is what lake base enables, and we'll in this session we'll talk about that uh throughout the session on how it helps you and that kind of stuff. Perfect. Okay. So, just to sort of frame
[03:52] where we're going, quick agenda. uh We're going to talk a little bit about little more about evolutionary design, current state so we can frame, and then what Lake Base really helps us redefine and then both from two standpoints. There's an operational model, and then there's also the dev
[04:08] loop. All right, there's two really important facets there. Um and then, what are the new practices that really enable the team? What are some of the new team conventions? Because this actually, you know, we can invent we can create some new new new thematic approaches now that we have a different
[04:24] technology platform that will enable those. And then talk a little bit about how agents need to be tied to these practices, too, so they're not going awry and doing the wrong things. And then we'll reframe, we'll kind of close with all of the new 11 design
[04:41] evolutionary design practices. There were seven before, and now there are 11 for 2026. All right. So, we'll go into a little bit of framing for evolutionary database design. Yeah, so in the concept of evolutionary design, like one basic question people
[04:57] ask is, "Hey, why should I even do evolutionary design? Why can't I just design everything up front?" And if we have been in the industry for like so long, and we know requirements change, your competition changes, so you have to change your software, or a lot of these kinds of things happen surrounding you,
[05:13] which forces you to change, right? So, now because change is there inevitable, might I as well get better at managing that change, right? So, the continuous evolution comes from that concept of like, "Hey, you build something, MVP, V1, beta, alpha, or you call it
[05:28] whatever." Then it matures over a period of time, and during that time you have to evolve, right? So, this continuous evolution is what is one of the notion of evolutionary design. The other part of this is, "Hey, just like how you treat your code, like it sits in Git,
[05:44] it's version control, nobody can deploy directly to production, it has to go through a cycle. All these practices that you apply on the code side also should be applied on the database and data side, right? So, your database should be version controlled. Your database changes should be version controlled. If you have any like seed
[06:01] data, reference data, or data fixes, or whatever that is, all of that should be in version controlled. And you only deploy through version control deployments, not by hand like someone just like connects and runs a deployment script and that kind of stuff. So, take that away. Now, what that does is
[06:18] basically gives you auditability, gives you lineage of these changes that happened, allows other people to say what other things were committed, can I make a change of that, or can I fork that and do things with it and that kind of stuff. That's the 2026 shift here
[06:33] that we are talking about, right? You used to do this in code, now you can do this in the database also. Very cool. So, um why would we want to adopt this approach? Well, there's um four primary concerns as to why you'd want to
[06:51] um Yeah, so again going back to this notion of like if you had like let's say a shared database that you're working with, a dev database where everyone is developing against. So, anytime you change anything in that, like one person, one developer is working on a feature, they change something, they
[07:07] linearly break someone else because you probably rename the table, probably split a table, split a column, did something, and you break the rest of your team. And the rest of your team is now screaming at you, which does two things. One, slows them down, and two, introduce fear in you because now it's
[07:24] next time you try to do something like this, that fear stops you from innovation, right? So, these two things is what you want to conquer. So, by eliminating those bottlenecks, you're doing that. The third thing also is because you have a shared database, you're always working with one gatekeeper. Like many a times, the DBA
[07:41] is the gatekeeper. So, that's why many devs don't like the DBA because they are like controlling access, controlling change and that kind of stuff. So we want to reduce that bottleneck, right? The other thing is ability to test this. So you want to test this as earlier as possible. So shift left in testing is a
[07:58] thing that you want to do. That's why shift left makes sense in this so that you can test in your own version, own own schema, own whatever stuff. So you want to do that. The other is this notion of because now we are liberating the DBA from this gatekeeper
[08:14] role, now the DBA can actually put their knowledge for better design, better solutions, working with all the developers, pairing with them to come up with a better solution. So these are the four reasons why you kind of want to like implement this approach.
[08:29] So now, so Pramod, how would we implement evolutionary design? What are some of the practices that we currently have? Yeah, so I think the rigorous notion of like you don't just branch the code, you can also branch the database now, right?
[08:44] So paired branching. My branch number 1 2 3 also has equivalent branch in the database called 1 2 3, right? So that those two things go together. Now I can run my code, change my database within that branch, do things with that. So that paired notion of branching really
[09:00] helps you. The other notion is version control everything. We talked about this in the previous slide where you want to version control your database changes, you want to version control everything. This notion of migrations which is like I'm going from V1 to V2, what is the
[09:16] delta between V1 and V2 is the concept of migrations. So you can take those migrations and apply to whatever future environments like downstream environments, then you upgrade that environment to be whatever it is. So if you version control those migrations, you can automatically upgrade any
[09:31] environment downstream, right? So version controlling is a big thing. Similar to how code is part of the CI cycle, the continuous integration cycle, you want to bring database changes also into the continuous integration cycle. Because if you are making local changes, if they don't work on the main line,
[09:48] then it's 100% chance that deployment will fail, right? So, if it goes through a CI cycle and the artifact that comes out of the CI cycle, if it passes, then you know for a fact that whatever you did is the right thing that you did. Because whatever has happened works on
[10:04] my machine is no longer a thing, right? Because it's working somewhere else now on the CI instance. Similarly, like if you are if someone is reviewing your code at the end, they also should be reviewing your database changes, right? So, you can make the diff between your PR, what you branched and what you're
[10:21] committing, make the diff there so that people can see what you changed, right? And similarly, every database change that you make is a structured refactoring. So, in any refactoring, the thing is you're semantically you should be correct. Nobody likes to lose production data. So, when you make
[10:37] changes, make sure your data is the staying the same. You can change the structure, but the data is still there, right? And similarly, at the end of it, you're inheriting governance, like Unity Catalog is there, so you can whatever governance you have put structures in there is coming back into that. And at
[10:54] the same time on the developer side, everything is version controlled. So, you can exactly point out what was changed, when was it changed, if there is a reference to some like Jira ticket or something, all of that can be traced back to a requirement. Yes, and that's going to Unity Catalog
[11:10] piece to this, uh you could inherit controls, uh put controls, policy on the top-level branch and then branches that follow, uh and have those cascade through. And then you could also put masking rules if certain elements uh need to be uh restricted uh
[11:26] on certain branches. Okay, so actually let's let's talk a little bit about the current state uh of development. Um what is there What's happening now? Yeah, so the current state is like basically our contention is always on the dev dev environment. Like you have a
[11:42] shared space where people are modifying their contending for review from the DBAs. There are all kinds of stuff. The other scenario where developers automatically whenever they run into a roadblock, they try to find a way around it. And the way around it they generally find is by trying to do mocks, trying to
[11:58] run your stuff in like a in-memory databases and things like that. So what that does is creates complexity. Some of them also run like Docker. Like let's say you're running Postgres, you can run in a Docker instance on your own machine and that kind of stuff. Now you have like a fleet of Docker instances running
[12:13] all over the place. So there's no way to control what version you're running and that kind of stuff. This The other other side of this is the DBA who's at the receiving end of all of these changes trying to review, make sure they are meet standards, they make sure they are there's design cohesion between all
[12:31] the changes that are happening. So they are also like sitting there gatekeeping. One being hated by by the developers and the other having to do a lot of work in a queue, which kind of slows down the rate of change also. Perfect, perfect. So those are current blockers, things that impede our
[12:47] progress to We we work around them. We have been working around them. Um but they still exist. Uh with uh Lake Base and Lake Base branching uh we have actually have the opportunity to redefine the operating model and
[13:03] eliminate uh eliminate or reduce like those bottlenecks that we've all had to deal with um for the past I don't know, 25 years. Um in the way that we can do this with the operating model as if if you think about it, there's two sides to this
[13:19] um for the team member and for uh the people that uh support the team member. And that is now I can really do very quick and instant provisioning around what I need what I need access to. All right, databases can are now very very
[13:35] easy with Lake Base to provision immediately. Um and then branch immediately. So, normally the setup would be I have a database. I put my controls around it, and then I can now provision from there with branches. Uh also within Lake Base, and you may
[13:50] have heard the announcement from keynotes, is that there's now built-in disaster recovery. Um I that was a concern that, you know, you might have to deal with for some environments. Uh that is now taken for taken care of for you with with Lake Base. And then as I mentioned just previous
[14:07] around the security and the governance and the policy management, there's a unified plane for that, which is Unity Catalog. So, I can set that stuff up once, and then as a DBA or some platform engineer could help define what those what those permissions are going to look
[14:23] like. Yeah, talking a little bit if you can go back one slide, Kevin. Thank you. So, talking a little bit about this provisioning and instance management Docker we talked about this fleet of Docker containers and all that stuff. All that just goes away now. At the command line, the developer can just
[14:39] say, "Hey, provision me a branch." And suddenly they have an instance of the branch. And all the security policies, all the versions, like, "Hey, I'm running on Postgres 12 versus other versus some somebody else. A patch has been applied here. A patch has not been applied here." All of that just goes
[14:54] away because the instance is coming from the same place, right? And if the base instance gets upgraded or whatever, everything just gets upgraded by itself. So, all of the DBA maintenance of these things goes away, frees up the DBA to do value-added work.
[15:09] Thanks. Yes, and Pramod has done a lot of DBA work in his past. So, speaking from experience. Yeah. Um so, so there's the operating um model opportunity to redevelop that and we also now have an
[15:26] opportunity to rewrite how we approach the developer loop. And this is where things get really interesting and more concrete. When I think about what I do in a dev loop, the very first thing I have to do is I kind of have to set up my environment. So this is sort of sprint zero activity. I have to set up some
[15:41] scaffolding for everything and I'm going to walk you through what this looks like leveraging this plugin that I've been working on and building that takes some of these elements and allow you to use them through an extension through VS
[15:59] Code. This is works with VS Code and with Cursor and I'll share links at the end. So I'm going to walk you through the elements of the scaffolding of your initial project, what it looks like when you claim a branch now because it's it's a little different. Implementing a
[16:14] change in the dev loop and how that looks, opening a PR. And then doing a review and a merge. All right, so I'm going to go through and quickly walk through Great, it's all it's all working. I'll walk you through. This is the extension
[16:31] VS Code or Cursor. And what I'm doing, I'm actually starting a very fresh project and I want it to be a Python project. It's going to be self-hosted. I select my Lake Base instance and I create a GitHub instance and all of these things are I'm being
[16:47] walked through to create a project for me. So it's going through all the mechanics behind the scenes, provisioning my database instance and all of the other things I need to get going. Okay, so once I have my project and I'm
[17:04] working then on a feature. Just took me out of a Pardon me. Actually, I'll stay here since we're going on the going into the next one anyway. Okay. So, the next thing I'd want to do
[17:20] now is I'm working on um I I I want to be work I want to work on a feature is I need to claim a a branch. What does that mean? Well, uh if I've got uh my my VS Code cursor, I can go and create a branch from staging. Staging now is
[17:37] a construct that I have defined in staging pairs uh a uh a Git branch with my database branch. And I'm creating a feature branch from there. So, we can look at the model of I've got a main which could be your
[17:54] prod, and then I have a staging branch which I want to commit into because I want to do a release. Uh and right now what I'm doing is I'd like to create a feature branch from uh staging. And you could see here that I've got this combined construct. I've got both a
[18:12] uh Git branch and a and a Lake base branch together. And at least initially, maybe I think my mic is going down. Is it? Is it just me? I've got um no schema differences. So, you can see in that last page that I have no a a a fresh schema to work with. Um I'm going to actually skip over
[18:30] implement change and I'm going to do that live with you all, but I want to show you what the uh opening a PR and doing a review of that PR can look now.
[18:46] All right. So, here I am and I've got a a feature that I'm now ready to commit up and merge eventually into my staging branch. And what I'm doing is I'm saying, "Okay, I've completed this dashboard. It's ready for V1. It's ready for review." And like um
[19:02] I'd like to do uh a PR and then get that into the staging environment. So, that maybe uh uh some a product owner or someone could look at staging and review what I've done um as I work. So here I am, I'm now doing a PR request and this is should look familiar with get.
[19:20] Right now looks like there's no conflicts and it's running through on my CI server. It's actually running all the integration tests that I had written directly against the database right there. So in that moment too, as part of that PR, I also
[19:36] cut a fresh branch from the environment I'm going to merge into. So it cut a fresh branch of staging. And unfortunately it just it just it I'm sorry. There was a piece there where you can see the schema branch as well if you caught that.
[19:51] And and show you the schema depths. So normally you're when you're doing a PR review you're actually just you know you're just reviewing the code changes and looking at notes there, but now you have an opportunity to to to review the schema and the schema changes that are happening. And this brings your DBA
[20:07] as another PR reviewer. Right, that's not typical, right? That's just sort of a a new way that we can empower reviews is having everything contained in code. And let's look at the final stage which is the merge.
[20:25] So once I am done with a PR and it's been reviewed and accepted, the next thing I'd want to do is merge that in and up into my parent branch. So the way that this paradigm works is I'm always merging upward into the branch that I came from.
[20:41] And you can see there that there was there were two main tiers in there. There was a prod tier and a staging tier. Yeah, one of the good things this particular plugin does is abstracts this notion of the developer remembering I need to
[20:58] branch code, I need to branch database, then remembering this in the head. The plugin does that automatically. It creates this abstraction so that the developer has to remember the higher level stuff. I just branch, it does both of the things in the back for you, right? And when you are merging, it
[21:13] shows both of the things back together at the same time. So, what you are doing is taking conventions and encoding them into a plugin so that everyone follows the same thing. And later on, we'll talk about agents and you'll see why this is useful. And so, in this plugin too, in this
[21:29] view, there's you don't you can also be self-contained uh within. You don't necessarily have to go to the Git UI, but you can stay here and also look at your schema changes. Um this particular schema change is comparing
[21:45] um now with what your staging looks like with production. So, staging is now reflecting your feature commit. Um but production doesn't have it yet, so it sort of gives you the opportunity now to accumulate some changes in staging. And then once you're done with a sprint or
[22:01] two sprints or however many, then you can migrate those things up into your prod line. So, the interesting thing about that, too, is that before I merge into prod, the same methodology and approach follows. I'm going to take a fresh cut of prod as a branch as a
[22:18] database branch, apply all the tests that I had accumulated on the and run through all the live integration tests cuz they they can be live now with with with a branch of of your production data, you have basically your production data.
[22:34] And but you can have a branch, so you can safely operate with it without affecting production. So, you know when you do your final migrations, you've now done an exact clone of what you had before you merge into it. Yeah, because we always run into the
[22:50] situation of you build a or develop a migration, you accumulate this over two or three or four sprints or however many, and at the end when you're deploying it to production, you have this fear, "Hey, this size in dev or this size in stage is not as big as production. Will this migration run in
[23:05] time? Will this migration run properly? All this stuff can be answered by taking a branch of production, running the migration, if everything runs fine, then you basically destroy that branch and you deploy it to production, right? So, you get this confidence that this works
[23:21] in production also. Okay. So, we're going to go into now I skipped over this implemented change piece, which I think is sort of the highlight. Um and I'm going to sort of frame with what I've been showing you behind the scenes, what I've been working with. It's this I It's this pretend app. It's
[23:38] something called Stock Flow, right? And so, what I'm doing is I'm doing custom app development for an organization. They didn't want to purchase uh of a commercial uh um uh WMS, so they wanted something simple designed. So, this is custom-built. It's
[23:55] you know, being implemented with uh with Python and Fast API, React. And uh the features that we're implementing are listed. I've only accomplished the first two so far. Um I can uh you you've seen a little bit of it in the what I was
[24:10] doing, but now I found in my V1, I have a problem. My V1 that I have developed, it did some encoding of one of the fields. And like, oops, I put the batch and serial number together in this tracking code, this inventory code. And it makes
[24:27] it really, really hard for me to do filtering and searching against it. So, I'd like to extract that out. All right, I want to actually do something called uh split column refactoring, um which is actually in Pramod's book from back in the day. And I want to take it out, split those things up, but I
[24:43] want to de-risk what I'm doing. I want to try this out on a branch and and implement the change with safety. So, I'm going to demo what that looks like.
[25:00] And I did test this, so it should work. Just before. Here is my VS code. Um I think that I can maximize this guy. All right. So, you can see here I've got my tiers. I got prod, I've got staging,
[25:16] and in here is my next feature to do the split tracking. Um and this is ready to go. But what I what I decided to do for this demonstration is to actually take another branch from my feature branch
[25:31] and throw a little experiment together, so I can show you exactly what is happening um with the with the work that is going to be uh migrated and put into into staging. Um see. So,
[25:55] So, what I've got here is I've got this state of what the application looks like today. You can see I've got this inventory code tracking code, and I've got some things that are that are embedded in here. I've got actually the location code, too. It's like duplicated. And I've got um a serial uh and a batch like combined.
[26:11] Like, all right. I want to extract that out. Here's the way things look like today. And so, as I'm designing my change, what I want to do is I'm going to be very careful. Um I want to be able to do things in phases. And the first thing I would want to do is to split those things out and
[26:27] make sure that they work and then do testing. So, this is a schema migration that's that's happening right now. It's through Alembic. Um and I'm stopping right after the first level of migration. And these are all scripts that get run against the branch. So, I'm doing stuff against what would be live data eventually. It's actually a copy of
[26:44] the live data currently. Um and then I'm taking those numbers and I'm going to extracting them out. You can see in the UI now, this is uh this is in a transition state. I might actually keep this UI for a little while until I'm really confident that this refactoring is going to work. Yeah, and
[26:59] one one point I want to add here is when you are implementing any refactoring, there are three states of refactoring. It's the actual change you make, that's the beginning when you are actually introducing a column, introducing a table, or whatever. And you know that when you have an app running in production for years, there are all
[27:15] kinds of people talking to the database and you can't just change the table structure willy-nilly, right? Because everything else will break on the side. So, there is this middle period known as a transition state where the old as well as new kind of live together for some time. Like in this situation that Kevin
[27:31] is showing, the tracking code is still there, the batch number is there, and the serial number is. The new apps that know about this new column will go to new data. The old apps that have not yet migrated, like your extract scripts, your ETL jobs, or whatever that is, is still talking to the old column. And
[27:47] over a period of time, the ETL will now start looking at the new columns. And all the dependency has been gone out, then you take out the old stuff. That's the scaffold when you remove the scaffold or the contraction phase, right? So, expand, transition, and contract. That's what Kevin is showing. Yes. And then when you're satisfied that
[28:03] now this I've done both validation through scripting or tooling automation, but also visual verification, uh I can then finally evolve the schema into my final state. Um and that is then what my UI will look
[28:19] like uh in the last last moment. You can see on the top two that as I'm moving through these refactorings, uh this is a a product of the of the of the plugin. There's a a way you can you can see what those things look like from uh your
[28:35] database changes, too. And you can see now the state of the final final UI. column has been removed, the inventory code, and two new columns have been added, yeah. So, I'll actually will go in. Um so, this is actually it's implemented in this new feature branch. Uh, I can see
[28:52] the final state of that, too. I've got, uh, so I really, uh, excited for everyone to start looking at this plug-in and installing it because I think it's really, uh, helping to define the shape of what things can look like when you're doing both code
[29:07] and data database changes in one in one unit. And I and I can see what those changes look like. Here's my my source code changes sort of as a view. Uh, I've got different tracking in ways to look at this and this is showing me what my schema is at a very high-level summary,
[29:22] what's added and what's being removed compared to where my branch came from. Um, I can see more detailed reviews of that, too. Uh, I could, you know, go in and I could see in my in my my branches here I've got the code assets. Uh, I've got
[29:38] the what the database will have. The database has a change as well. That's the stock table that I'm changing. Um, I can also click on that and review that and see what that looks like compared uh, to what staging is. Staging has, uh, the inventory code still. It's going to
[29:54] be removed. Um, my feature branch has that removed plus the two additions. So really really helpful for, uh, for developers to see this in this combined view. Okay.
[30:12] Let's get back into the presentation and talk about what this means uh, for the team. So how does this enable, uh, the team, uh, promote? Yeah, so one is it helps you create isolated environment so you're not stepping on each other. So the moment
[30:28] when Kevin showed I created a branch, you are in your own branch, own private space branch. You can do whatever you want, mess up things, whatever. Nobody else is going to get affected. So that fear of breaking others is gone now, which enables innovation, right? Which which enables you to do destructive
[30:44] testing, which enables you to try different things. Say, I want to implement it this way, let me try. If it doesn't work, you can get rid of that branch, get a new branch, and start fresh again. Just like what you do in code. Like many a times when you make a branch in code, you try a bunch of
[30:59] things, you get into a state where like it's all messed up now. I don't want to do this anymore, to revert it back or whatever. You just do a reset and get back fresh, and then start again, right? So, you should be able to do the same with the database also. This is what that allows. It also allows you to do
[31:15] rapid bug resolution. Like for example, many a times you get bugs in production that you cannot reproduce in dev because the data doesn't exist, the shape of the data doesn't exist. So, now you can get the right code and the branch from production, run both of them
[31:30] together, and see where the bug is, fix the bug, then bring that code back to your main line, bring the change you made in the database side back to your main line, and then deploy it back into production. So, that's how that does you makes you help bug resolution. Like I talked about a lot about experimentation
[31:47] already, and the governance is a big thing here because many a times when people say, "Hey, you can branch production." Suddenly, they're like, "Oh my god, am I going to be able to see all like production data and all this stuff?" No, the governance that get applied on Unity Catalog carries through. So, if there are some things
[32:03] that you cannot see because you're a dev, same applies to the copy you made from or the branch you made from production, right? So, that kind of rules still apply. So, it's it's a safe space to try that out. Yes. So, a question may come up of what about uh
[32:19] there is a masked column that I can't see, but I actually have to implement a change there. I'm a developer. I need to implement the change there. Well, uh you you could have a service principal that is running in CI which might have access to that field. So, you still have that opportunity to try whatever you need to
[32:34] try, even though you can't see that column yourself, your service principal might. Okay? Um So, this is also an opportunity for us to look at a new model uh for how you can envision your branches. Yeah. So, so here the question comes up
[32:51] like, what is my main branch? Is it my Is production my main branch or is something else my main branch? Like, what is my UAT branch? What is my QA branch? Because some of these branches are long-running branches because a QA environment stays running for some time because QA people put some specific data
[33:08] in there. They want to test like various varieties of uh let's say policies or let's say something else and that kind of stuff. So, you want to start thinking about what are the types of conventions you want to maintain? Like we are showing a main branch here as a production branch and other stuff is
[33:23] flowing off of that. You could even have a main branch where it's schema only for your purposes and a branch could be a production branch that is longer living and then the main branch just carry schema and seed data or like reference data and things like that. So, there are various techniques and uh that kind of
[33:39] stuff conventions. I and Kevin wrote a three-part blog series on this. Links to that will be at the end of this slide that talks about these various conventions that you can follow. Like, one of this is showing here but there are two or three more that you can try it. Again, depending on your situation,
[33:55] you may want to come up with like different conventions of how you do this. All right. Uh so, we talked about the reset that is available, right? This where you have a moment to redefine how we operate and leverage the database now
[34:12] that the database can branch. And we can all sort of think of of this for a while and say, "Oh, this is actually makes a lot of sense." But if we're going to do stuff with agents, um agents need practices, too. So, so those have to be defined um
[34:28] uh if if if not, uh they're going to be just like junior developers, right? They don't have those guide rails, they don't have those practices. So, without uh a methodology or a framework, um they're they're not going to do uh
[34:44] what they might think they're doing, right? So, they're they'll all produce code, uh they'll apply the migrations, but then um they create these maybe technically correct, but then unmaintainable systems. Right? That's what they'll agents will do without proper guide
[35:00] rails to to focus them. Um what they really need to do is operate inside of a workflow, just like how we do, right? And respect what we just talked about with this tier type pilot topology. Um use some patterns, name patterns, refactorings. Uh and then also, what they probably won't do unless
[35:17] you really guide them, is a live write live code against a live database branch. Right? If you don't give that opportunity to an agent, they're not going to know that they should be doing those things into course them into writing TDD
[35:32] with live integration tests instead of mock objects. Um so, how do you constrain them? Um by by creating something that allows them to operate within those guide rails. Yeah, so we got like last 3 minutes, probably go to the last slide and take
[35:48] questions, live questions. Uh Yep. So, these will be for you to we check out the blogs. Uh we talk about these 11 practices there. Um some links that um and things that you could do, you know, starting your Monday morning uh after that is to go
[36:04] and try some of these things out. Yeah. Yeah, so what we wanted to say here, yeah, if you go back to like home, maybe Thursday, Friday, take the weekend off, Monday when you go to work, try these these things. They are ordered by priority, so maybe start with
[36:19] experimentation, and then keep going that way so that you get to enable these practices within your teams. And then, uh just so for context, I talked a little about the plug-in and what that how it operates. It operates against these primitives that are
[36:35] available uh in a CLI. There's a kit um that the plug-in operates against. And the links there is allows you to do this follow this construct. The links there are here. This is the link to the plug-in that I shared with you that we walked
[36:51] through. And then, uh behind the scenes is an app dev kit. Uh the app dev kit enforces that those primitives in those structures in the workflow. Yeah, and that's me, Pramod, and Kevin. Thank you for your time.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.