Engineering Velocity Under Constraints: How Lakebase Database Branching Helped Us Deliver Faster

Summary

HTEC used Lakebase Database Branching to accelerate delivery of G2 Nimbus, a multi-year cloud transformation of a 25-year-old financial system under strict PCI DSS constraints that require network-level isolation between development, QA, staging, and production environments.
Lakebase Database Branching gave the team a git-flow equivalent for databases, enabling branch-per-feature development, safe schema evolution, QA parallelism with multiple test environments from a single branch, and safe analysis of production data without touching live systems.
Before Lakebase branching, parity testing between the old and new system under PCI DSS network segregation and per-customer tenant isolation in a private data center was a highly manual and tedious process for every change.

Engineering Velocity Under Constraints: How Lakebase Database Branching Helped Us Deliver Faster

In this video, HTEC shares how Lakebase Database Branching accelerated delivery of G2 Nimbus, a multi-year cloud transformation under strict PCI DSS constraints with parallel client migrations. We'll show how the database shifted from bottleneck to first-class SDLC citizen, enabling branch-per-feature work, safe schema evolution, and faster hotfixes without compromising compliance. You'll leave with concrete best practices and how to bring the same productivity gains to your own regulated, high-complexity programs.
Talk By: Darko Todorovic, CTO, HTEC ;

Chapters

FAQs

What is G2 Risk Solutions and what migration challenge did HTEC face?

G2 Risk Solutions is a provider of bankruptcy risk intelligence used by large banks in the US, operating on a 25-year-old system that required migration to a modern platform. HTEC had to execute this migration under strict PCI DSS constraints with zero-downtime for end customers and per-customer tenant isolation in a private data center.

What is Lakebase Database Branching and how did it help HTEC?

Lakebase Database Branching provides a git-flow equivalent for databases, allowing teams to create isolated branch-per-feature environments with production-like data. For HTEC, it enabled safe schema evolution, parallel QA environments, and the ability to analyze production data without any risk to live systems.

Why was database parity testing so difficult before Lakebase branching?

PCI DSS requires strict network segregation between development, QA, staging, and production environments, which breaks standard git branching workflows. Combined with per-customer tenant isolation in a private data center, parity testing between the old and new system required extensive manual effort for every schema change.

What future use cases does this video mention for Lakebase branching?

This video points to agentic development teams as the next frontier for Lakebase branching, where multiple AI agents working in parallel on different features each get their own isolated database branch. This enables safe, concurrent autonomous development at scale without risk of branch conflicts or exposure of production data.

Full transcript

[00:07] Hi everyone. Uh my name is Darko Todorovic. I'm a CTO of HTEC Group. And I want to talk today a little bit about our journey with a company called the G2 Risk Solutions.
[00:21] Uh where we partner up to basically um do the transformation from an very old system that is 25 years old to a new
[00:35] platform that should enable G2 Risk Solution to actually deliver a modern and better service to their customers. So, but before we go into that, I just want to say a little bit about HTEC.
[00:48] HTEC is a global digital engineering company. We are providing services in engineering space end-to-end, meaning from the strategy design to build out and then deployment and operations as
[01:01] well. And we are focused mainly on providing the services in the full stack now of AI from the silicon to the identic applications.
[01:14] On the other hand, our partner G2 is a company that is provider of bankruptcy risk intelligence solutions. And uh they deliver predictive insights and
[01:26] monitoring of bankruptcy events. So, basically uh big uh institutions like big banks in in US are the customers of
[01:40] G2. And this space is very specific in a way that we cannot when we are doing the migration from the old system to the new system it's very important that the
[01:54] companies that their users do not feel any change and experience any downtimes. So, what are the constraints that we had
[02:06] in this very very strict environment? This is PCI DSS security. And it's isolated by the design. So, it's very important to understand that
[02:18] we have a strict network segregation on even on the network layer the between the development QA stage environment and production environment. So, any paradigm that you know that
[02:33] you have in the regular software development where you have standardized Git branching is actually falling down when you have this strict environment. And probably some of you in the audience
[02:45] here are coming from that industry so that you pretty much understand what what challenges we have faced. The other thing which is very important here is that every customer has a
[03:00] separate tenant and everything is hosted in the private data center of G2 Risk Solutions. So, for us implementing
[03:12] any changes then also doing the parity testing between the new and the old system was very tedious job. So, on in that regard,
[03:25] we introduced the lake base branching which gives us some of the very obvious benefits and you have probably listened about lake base and lake base branching is a feature. So, we
[03:37] have isolated experimentation. Uh we have point in time reproducibility so we can make basically automatically snapshots and make so we can do the
[03:49] regression uh uh we have the possibility to do the parallel development inside of one of the environments, which was very hard to do previously, because if you have only the
[04:03] dev environment, or you have a production environment, which we are going to touch base a little bit later, uh it's very hard to have a team of five people, because you need to have five
[04:15] different uh coherent environments in which they are developing and testing, and then moving the code to the production. So, um imagine if you have a couple of teams. Luckily, this is not a huge
[04:28] environment a huge project at this point, but similar similar engagements have shown us that this type of regulated environments are actually not efficient
[04:40] in terms of uh delivering the software, then also uh going into the production and doing the operations on the production environment. So, uh we can leverage the atomic and safe deployments, and better definitely
[04:54] collaboration, and most importantly, full auditability of the whole process. So, these are the all of the things that uh uh uh lake base branching is allowing us.
[05:06] And basically, you can look at it as a way that you can branch the schema, and also get the data uh from the even production or dev environment uh in in one
[05:20] space, uh and uh it's very similar to the Git flow. So, but we couldn't actually uh use all of these features. So, we focused on a couple of
[05:34] uh paradigms and a couple of features that we wanted to make sure that we can utilize from the lake base branching. And this is QA parallelism, safe uh development and debugging, easier deployments,
[05:46] uh and hot fixes and secure operational access. So, we'll deep dive a little bit more into QA parallelism. So, basically, as you can see, for each of the features that we have in the base,
[05:59] we are able to create a separate environment and a separate schema uh and every QA could do their testing in uh separated environment that also has
[06:13] the latest data, whether it's from the development branch in this particular uh case, or that could also be in the operations case from the uh from the uh
[06:26] production. So, here is the uh very quick demo that we are going to hopefully run.
[06:45] As you can see, this is a mock-up application here. We just wanted to show the concept. So, right now, we have configured to run from the production branch.
[07:02] And we can create couple of new branches very easily. So, we have feature A, feature B, feature C branches.
[07:40] So, this was a production. Take take closer look on what's in the right hand side of the screen. So, you will see the differences in feature branches.
[08:09] And then we are focusing on this feature here. Uh delete.
[09:18] So, basically as you have seen, uh the other solution that we have, apart from the QA parallelism where we can introduce uh testing for each of the branches much easier in a safe environment, we have
[09:31] also the other solution, and this is allowing us to have safer development and debugging, and basically to have easier hot fixes. So, in the development from production
[09:43] branches we can create a branch that mirrors the exact state of the production data, which is really important for G2 when we are doing the parity testing.
[09:55] Uh also, we have atomic deployments. This is allowing us not to pile up a lot of fixes inside of the one environment because it previously it was very hard to build one environment for each of the
[10:08] developers. You have to have separate development environment. And uh this is allowing us to have very small atomic deployments and to also have a possibility to do a fast
[10:20] rollback. The third solution, which is also very important for the operations in G2 Risk Solutions, is that we can have the read-only branches. With the read-only branches, we can
[10:34] query and analyze production data without compromising the data that is sitting at that particular moment on the production. This is allowing operations team of G2 to monitor
[10:46] at all times how the data is flowing, what is happening with the on the daily basis without compromising any of the PCI DSS principles.
[10:58] So, in a nutshell, the constraints that we had in in this environment, a highly regulated environment, allow us to basically use them
[11:11] as a design driver. Not as something that was impeding the way how we were delivering the solution, but we used Lake Base branching feature to actually allow us to be much faster and
[11:23] to produce results that are safer to debug with a production-like data. So, what's even more important is that we
[11:35] have secure and auditable access without expanding compromising permissions. The next step in evolution, this is something that we didn't yet do, but it's something that we are
[11:47] currently working on is that imagine now that instead of having a team of five, you have a team of five where each of those has five other agents. So, you have 25
[12:00] developers to say that are going to use these features to develop faster inside of the one environment and also to help operations do
[12:12] much efficient work in production environment as well. So, our goal So, the next the next goal for us is to implement the harness and there is also
[12:26] what we heard in a couple of lectures during this uh uh couple of days is that there are harnesses that we can purposefully use to help us spin up the agents very fast,
[12:39] but also use them in an environment with PCI DSS compliance in mind. And that would be it. Thank you very much.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.