How to Bridge the Gap Between No-Code and Pro-Code with Lakeflow Designer

Summary

Lakeflow Designer is a generally available visual, drag-and-drop pipeline builder on Databricks that lets SQL analysts and business users create production-grade medallion ETL pipelines without writing code, powered by the same engine as Lakeflow Spark Declarative Pipelines.
A built-in GenAI assistant can auto-generate joins using Genie Ontology, add calculated columns from natural language descriptions, and apply multilingual text classification — enabling business analysts to perform transformations that previously required a data engineering ticket.
Lakeflow Designer coexists with pro-code Python extensions and integrates with Lakeflow Connect for ingestion and Jobs for scheduling, enabling seamless collaboration between business analysts and data engineers on the same pipelines.

How to Bridge the Gap Between No-Code and Pro-Code with Lakeflow Designer

Data engineering has a bottleneck: there are too many pipelines to build, not enough engineers to build them. Lakeflow Designer changes the equation by giving SQL analysts and citizen developers a visual, no-code interface to create production-grade ETL pipelines, backed by the same engine that powers Lakeflow Spark Declarative Pipelines (SDP). this video explores the full Lakeflow spectrum. We'll start with Lakeflow Designer's drag-and-drop canvas and GenAI assistant, showing how a business analyst can build a medallion pipeline in minutes without writing code. Then we'll show how a data engineer extends that same pipeline with custom Python logic, data quality expectations stored in Unity Catalog, and Data Quality Monitoring with anomaly detection. We'll demonstrate how Lakeflow Connect ingests from sources like Salesforce and PostgreSQL, how SDPs handle the transformation layer, and how Jobs orchestrate. You'll see how visual and code-based interfaces coexist, enabling collaboration.
Talk By: Germaine Ong, Senior Solutions Engineer, Databricks ; Nan Chulpaibul, Senior Solutions Engineer, Databricks ;

Chapters

FAQs

What is Lakeflow Designer and how does it work?

Lakeflow Designer is a generally available visual, no-code pipeline builder on the Databricks Data and AI platform that gives SQL analysts and business users a drag-and-drop canvas to create production-grade ETL pipelines. It is powered by the same engine as Lakeflow Spark Declarative Pipelines, so pipelines built visually run with the same reliability and performance as hand-coded ones.

How does AI assist pipeline building in Lakeflow Designer?

A built-in GenAI assistant in Lakeflow Designer can auto-generate join logic using Genie Ontology, add new calculated columns from natural language descriptions, and apply ML-based transformations such as multilingual sentiment classification. This lets business analysts perform data transformations that would otherwise require opening a ticket to the data engineering team.

Can Lakeflow Designer pipelines be extended with Python code?

Yes — Lakeflow Designer is designed for no-code and pro-code collaboration, so a data engineer can extend any pipeline built in the visual canvas with custom Python logic, data quality expectations stored in Unity Catalog, and anomaly detection monitoring. Both teams work on the same pipeline without duplicating work or maintaining separate codebases.

What is the Lakeflow platform on Databricks?

Lakeflow is Databricks' end-to-end data engineering suite that includes Lakeflow Connect for managed data ingestion, Lakeflow Spark Declarative Pipelines for transformation, Lakeflow Designer for visual pipeline building, and Lakeflow Jobs for scheduling and orchestration. Together these components cover the full data engineering lifecycle from ingestion to productionized delivery.

Full transcript

[00:10] Hi everyone, welcome to room uh 305 and uh my name is Nan and today with me is Germaine and we will be sharing about how to bridge the gap between um pro code and no code with legflow designer.
[00:22] So before I get started maybe I just want to tell you um a story about my customer. So a few weeks back I I met I talked to my customer and they shared a story that I feel you might be familiar
[00:35] with. So um they were having like business teams that are using Excel every day, right? And their business teams um they know what data they need, what data they want for the business.
[00:47] They also know like the business rules behind these data sets that they want to apply. uh but the problem is that they can't really apply it by themselves. They can't turn it into a usable data
[01:00] sets themselves. So what happened is that every time they want to um create a new data sets, they need to open an a ticket to the data engineering team, right? And then over
[01:14] time um the tickets just keep piling up and their data engineers will pull into this kind of like simple data preparation task right instead of doing like a high value um task that only data
[01:27] engineers can do. So what makes them want to change the process? So in um Thailand, so my customer is based in Thailand and the customer is one of the bank. So um they
[01:40] were having um a lot of demands for new regulations and also new reports right so that's why they need to change the process right but uh so we were thinking
[01:52] about like how should we change the process and the thing is the problem is not like one team right um because business analysts they are really good at understanding the business themselves
[02:04] and then data engineers they are also really good at building the pipelines So the problem here is the handoff between the teams right and also in fact
[02:16] um analysts can lose up to 80% of their time to insights to this kind of like hands off and dependencies between the team. So that's why today we are here to share about how to bridge the gap
[02:28] between no code and pro code using legflow designer. So um let's start with the foundation. Um no matter what you are doing whether
[02:40] it's Excel or whether it's geni you need a fresh data you need a good data you need a clean data and reliable data right and that's where leg flow legflow comes in. So, Leglow is
[02:54] an end-to-end data engineering suites on data bricks data intelligence platform and with leg flow it has three main capabilities. The first one is legflow connect which
[03:08] is used for ingesting the data and then the next one is spark declarative pipelines which is used to transform your data and then the last one is legfl which is for um orchestration.
[03:23] So let's start with um legflow connect. So with legflow connect um it lets you ingest the data into one place which is the data intelligence platform right and
[03:35] legflow is uh electro connect is a built-in and also point andclick UI where you can ingest your data from SAS applications like your Salesforce or workday and also databases like my SQL
[03:49] Microsoft SQL or even streaming sources. Um, leg flow connect is fully integrated with unity catalog which means that you get um unified governance uh by default.
[04:03] So once you ingested your data into one place uh we can now transform it for the business and that's where spark declarative pipelines come in. So um
[04:15] spark declarative pipelines uh the idea is pretty simple. It's a declarative pipelines right? So you just need to define declare what data set you want what transformations you want and then
[04:28] spark declarative pipelines will take care of uh the much of the how right so for example how to read the data how to actually transform the data so on and so forth so what this means is you have more time
[04:41] to spend on the logic of your uh business and you don't need to worry about the underlying uh heavy lifting of how to implement this and that and um with spark declarative pipelines
[04:55] um it's also open source which means you can run it on data bricks or you can also run it on prem right so um with spark declarative pipelines your code will be a lot more simpler a lot more concise as well
[05:11] and then the last step would be to orchestrate it so how do we make it run by itself and that's where leg flow jobs come in so legflow jobs uh provides you with that orchestration capabilities. So you
[05:24] can um orchestrate your task, make it run sequentially, make it run in parallel. Um and then it also comes with uh that end toend observability. So you can make sure that hey does your pip did
[05:37] your pipeline run every day? Did it succeed? Uh make sure that they run um efficiently and reliably.
[05:51] Um but so far we have been talking about um everything that is in the data engineering world right but as you as everyone here know if we are working on analytics projects is always a collaboration between business analyst
[06:03] and the data analyst right so the data analyst they are really good at uh building uh reliable scalable pipelines in production meanwhile the business analyst they are also really uh they
[06:15] have a really good understanding of the business but then both of them they live in a separate world right maybe the data engineers right now is using legflow right legflow jobs legflow uh connect uh
[06:29] connect um or even uh spark declarative pipelines on the other hand the business analyst they are using excels and spreadsheets
[06:41] they need to connect these two teams together right and obviously data bricks we're not the first one to notice this needs. And so that's why there are also other tools out there that are already
[06:53] um implementing this, right? And a lot of tools here are really loved by business analyst because they are so easy to use. They are they also don't need a coding um knowledge in order to
[07:06] get started. But then we still hear some challenges from customer um and these are the three main challenges that we heard. So the first one is siloed workflows.
[07:19] Uh what do I mean by this? So let's say an analyst they build a pipelines or some business rules in one tool and then they need to pass this rule to the data engineers to reimplement it on another
[07:32] tools maybe on data bricks right what this means is you are basically implementing re-implementing like the logic twice and that's siloed workflows. The second one is production headaches.
[07:46] So um these tool they do not live in the data intelligence platform. So they don't have access to unified governance. They don't have access to um your lineage, your audit trails, right? And
[07:58] you also lose that unity catalog. And then last but not least is um AI productivity becomes quite limited because AI in these tools they um they
[08:11] don't have access to your lineage your business semantic um and also your genie ontology right and if you listen to the keynotes yesterday um genie ontology is basically the context map that datab
[08:24] bricks built uh based on your assets that you have built in datab bricks. So what it means is that AI can help you to generate those general syntax or
[08:36] troubleshooting but they cannot really understand they don't really understand your business. So that's why um we've what if we are thinking that what if data intelligence can combine these two world together
[08:49] right what if the business analysts and data engineers can use the same tool and then can just remove that hands off together
[09:02] so that's where leflow designer comes in right so with leflow designer and it's GA right now um it gives you a canvas where you can drag and drop transformations S and even easier if you don't want to drag and drop at all you
[09:15] can just describe it in natural language right in the text box and then AI will help you to build the pipeline automatically right so you don't even need to really drag and drop the transformations
[09:29] um
[09:41] the canvas will transform into an actual pi spark that is ready to be transformed right so let's say you are a business analyst and then you build the pipelines using this visual ETL tool or use the um
[09:56] AI to describe it in natural language once you build it you can check it into git and then hand over to data engineers right data engineers can review the code they can open the pull request and they
[10:08] can also orchestrate it just like any other leg flow jobs that they have built by themsel.
[10:20] So um here are the three points that we feel like we have solved right and it's um the first one is sharing and collaborations have become drastically simplified because now analysts can use legflow designer and data engineers they
[10:33] can also use legflow connect spark declarative pipelines and legflow jobs right so there's no more handoff between different systems right and there's also no more rewrites
[10:45] the second one is that there is a built uh in built-in path to production. Uh what do I mean by that? Because every pipelines that are built in the designer, they will create uh pi spark
[11:00] spark declarative pipelines under the hood which means that you can ship it directly into production, right? Just check it into git, review it and then deploy it into production. So everyone basically use leg flow and then um it
[11:14] just provides a more seamless experience and you can go to production faster. Last but not least is AI becomes more effective because legflow designer is
[11:26] built in natively into data bricks data intelligence platform. And what this give you is that the AI can actually tap into your lineage, your business semantics, your genie ontology, so on and so forth. and then it can really
[11:39] understand your business and then help you uh do your work more productively or even find insights uh for you. So um just um this is probably like the
[11:51] cherry on top of it. So let's say the analyst manage to you know build that pipelines by themsel visually and then deploy data engineers deploy it into production. The next question that comes is probably about data quality. So how
[12:05] do we ensure the data quality that the analysts have built right because one day maybe the upstream
[12:17] we maintain it right because um the data sets can increase uh and then there's also like limited people to maintain. So we need to find a scalable way to ensure that the data is still um up to
[12:31] standard. So in order to do this there is um data quality monitoring feature. So it is a built-in features um native to unity catalog right and basically
[12:43] what this does is it tracks the health of your data over time right so there are two main capabilities the first one is anomaly detections. So what anomaly
[12:55] detection does is it will help to learn the pattern of your data over time and then help you to find like a critical findings. So for example let's say for
[13:07] um completeness right if your data arrive like 1 million rows every day and then today it suddenly arrived like 100 rows so it will be flagged as being like incomplete
[13:19] or freshness. So let's say you have data coming in at 9:00 am every day and then suddenly today 10:00 a.m. there's no data coming in yet. So um anomaly will also flag it as um being not fresh.
[13:33] Um so why anomaly detection? Because normally you would have to go in and set per table threshold for this kind of anomaly detection rules. But then with this it's just one toggle at the schema
[13:45] level and then it will apply to all of the tables in that schema. So it's a lot easier for you to monitor um anomaly detection at scale. Um the second capabilities that it
[13:57] provides is data profiling. So with data profiling basically um data rigs will help to track the data distribution for you over time right. So let's say um for
[14:09] example your average your percentage now so on and so forth right and all of these are just like a few clicks of a button and then you get a dashboard that you can um customize as well and monitor and also set up uh alerts rules.
[14:24] So with this I just want to hand over to Jermaine to show you the actual demo of Leglow as well as data quality. Well, thank you Nan for walking us through the comprehensive suite of services in Lake Flow. Now, I was taking
[14:37] a look at the audience over here. Some of you are nodding your heads and some of you having like a blank look on your face and some of you are like, well, I've heard this before. So, the demo today with Lake Flow Designer penders to
[14:49] every one of you here today. So let us put everything together that we have learned um onto a drag and drop canvas and we take a closer look into how lickflow designer can enable just anyone to build data pipelines and achieve
[15:01] faster time to insights on their data. So let me just toggle us towards the demo page and in today's session we'll be meeting Susan. So Susan is a vice president of customer experience at the retail company Trenco and Susan needs to
[15:15] defend a 4 million retention budget at the next board meeting. She has a one-to-one meeting with her CFO to align on their strategy in three hours. And in this strategy alignment meeting, she needs to answer three questions. Who are
[15:28] our most valuable customers? Which of them are signaling that they are about to leave? And why? And if there is one thing that Susan can fix with her 4 million retention budget, what would that be? So now there's a few things
[15:41] going on over here. She needs to look into the customers. She needs to understand the tables, the reviews that the customers are giving uh on their orders and what is the sentiment that they are signaling. So now imagine
[15:54] you're Susan. You go into the datab bricks intelligence platform and you toggle into your catalog. All right. So now Susan goes into her catalog. She sees that okay I have this schema. I go into it and I have a bunch of tables.
[16:08] Now Susan knows that all right the segmentation that I have to be doing has to do with the customer's table. So she goes into the customer table and immediately she gets an overview of all the information. She clicks into sample
[16:21] data and she can see that she has the customer ID the customer names what loyalty tier they belong in and most importantly what's their lifetime value. So if we go back to the first question that Susan needs to answer who are our
[16:34] most valuable customers? Well, she gets an inkling, you know, with the lifetime value over here, there is some form of segmentation that she can do to put them into categories to study the value that these customers are having.
[16:47] On top of that, there is also the orders table. So the orders table gives information on what are the orders that are being made by which customer, what's the order status, is it completed, is it in progress, what's the shipping method,
[17:00] and finally the total amount of each orders. On top of the customers and the orders table, Susan knows that she also have another data set that's coming in into her Google Drive that's being shared to
[17:12] her on a routine basis. And this Google Drive comes in the form of a CSV file. So she has information on the customer, the orders and also the comments that were being given by the customers when
[17:24] they received their orders. Now earlier Nen mentioned about the automated data quality checks that are made available on Unity catalog tables. This is where we can see it in action.
[17:36] So there are two components to the data quality monitoring feature. The first one is anomaly detection. Under anomaly detection, Susan can see that okay my table is healthy. And why is it healthy? because both the commit freshness and
[17:48] completeness are expected. So just for a recap uh commit freshness actually tracks the historical commits to a table and if let's say there is some unusual um activity in terms of the commit for
[18:00] example let's say you're expecting the table to be committed at 9:00 a.m. every day and on the second day you see that by 12 p.m. there hasn't been a new comet um data quality monitoring checks will flag it out as an anomaly and say okay
[18:12] the comet is not fresh. As for completeness, um it studies the historical patterns of all the number of rows that will be added to the table on a 24-hour basis. If it falls below this threshold, it will be flagged as
[18:24] incomplete. So now Susan understands that with the data quality monitoring checks, the data is ready for her to consume. So let's get to the exciting part. We'll go back to our canvas and
[18:37] let let us all get started on a blank canvas. So I'll start a new visual data prep and I will click on source to bring in some data sources. The first one that I will be bringing in is from the Unity catalog table. Let's just go into it and
[18:52] bring in the customers data. The second data source that we'll be bringing in is also similarly from the same catalog and schema is the orders table. And finally the third data source
[19:05] that we'll be bringing in is the Google Drive CSV file that we saw earlier. So I'm going to scroll down and you can see that there are a bunch of external sources that are supported by Leglow Designer. I'm just going to go ahead and tap on the Google Drive one. Before that
[19:17] I am going to copy the link of that Google Drive over here and then I will just pop it in
[19:31] and CSV type and then load it. I'm going to rename this as reviews and I'm just going to bring in the customer scenario over to the same blank canvas so we are on the same page.
[19:55] All right. So now we see that we have three data sources in the canvas and I'm just going to click on one of them. So Susan gets a idea of a preview of her tables. she gets to see okay these are all the information that I have now and
[20:08] she has an inkling you know like we have to join the tables together somehow so for myself I'm a trained data engineer so it comes very very intuitive to me I have a orders table I have a reviews table I'm just going to join them on order ID and when I get the results I'll
[20:21] join them together with the customer table and we get a full table data set but Susan doesn't really know about all of this right so she's just going to make use of genie code over here and just say um can you join the three
[20:33] tables for me and Genie code will fire ahead. Um, Genie code is enabled by Genieontology that has been revealed by Ali in the keynote session this morning and yesterday and it runs through this to get an understanding of um, all the
[20:46] information that you have within your Unity catalog and you can see that it is running an operator. So the operator does two things. So um, Susan goes in to look at the first one. She clicks on it and you can see that it is joining the
[20:59] orders and reviews on the order ID. And what it's going to do next is to join the results of that with the customer table. So it is exactly what I would have done as a data engineer but Susan
[21:11] can do it now but herself with the help of just genie code. So I'm going to accept all the changes that has been done and just going to clean this up a little bit so the arrows don't intersect with one another. And she has a full data set available to
[21:24] her. Now I'm just going to drag this up and you can see that the output of this table has the order ID, the customer information, um the total amount, the channel, the comments, all of the
[21:36] information are now aggregated in a single table. So we are actually at a stage where we are ready to answer Susan's first question that she has, which is who are our most valuable customers? She's ready to do a customer
[21:48] segmentation on the combined data set. So what I'm going to do is I will use Genie code for that back to my previous canvas and just pop the question
[22:06] over here. So what Genie is going to do is to add a new column called LTV segment that buckers the customers based on their lifetime value. So these are definitions that Susan has made uh being in a customer experience team. So the top 10% should identify them as
[22:18] champions. The next 20% are loyalists, next 40% are casual, and the rest of the customers will be trial customers. So, I'm just going to pop that in and let Genie Code run behind the scenes to get
[22:31] that results out for us. While that is processing, I'm going to just doubleclick into this output and look into one additional feature. So, I'll click on all of this and hit the open
[22:44] selection details. So you can see that you're able to get um an understanding of the output table even as you're on the canvas and building out the pipeline and you can see that the analysis is based on sample
[22:57] data. So over here you can set the amount of rows that you want the preview to be scanning so that it doesn't take so much time for the previews to come out and once you're ready you can actually switch it to the full data set
[23:09] if you want uh a full view into the insights that are being generated by LakeFlow Designer. and I will just click into the LTV segmentation piece. I'm just going to bring this away
[23:22] and accept the changes. Okay, so this LTV segmentation piece, I'm just going to pull this out and take a look into the uh SQL query
[23:34] that has been written. So I understand SQL but if let's say in the lens of a Susan she comes in here she sees okay I have a percent rank um over 0.9 looks like the top 10% then I have the next
[23:46] 20% and then the next 40% and finally else everyone is in trial kind of looks about right but not not not not so sure right so she can go into the output over here just to inspect the LTV segment
[23:58] that has been generated just to make sure that Genie is giving her exactly the results that she wants too. All right. So now Susan is able to answer our first question who are our most important customers and that will
[24:10] be all the customers that fall into the champion segment and she's ready to go through a series of additional steps. Next what she's going to do is she will translate the comments. So earlier we
[24:24] saw that the comments in the reviews they're coming in as different languages. So AI will not be able to classify it as efficiently if it's in different languages. Uh but what AI can help us to do is to first translate the
[24:37] comments into English. And what will happen is we'll tell it to add it as a new column. Some of the comments are in these different languages. Translate it into exactly uh exactly English and then classify them into one of these
[24:50] categories. So um Susan has the business understanding of her data, the business understanding of her customers. So she knows that there are a few categories that are usually leading to customer churn. So some examples are like shipping, is it product quality,
[25:03] support, price or others if it can't fall within each of these categories. So she would provide this to genie code and genie code will go through a thinking process to understand um the query that
[25:15] has been given to uh by Susan and write out the operators required for this operation. Okay. So now it is done. We're just going to take a look at what Genie provided us with. The first one is
[25:29] to use the AI translate AI function to save it as a new row called uh translated comment and a column to translate is comment and into a target language of English. The next step after
[25:41] the translation is done is to perform a classification step and that is to classify the created column into one of these labels. And I'll accept the changes and I'll take a look at what uh
[25:53] Genie has done. So you can see the translated column over here. Um all the comments have been translated to English and they have been classified into buckets of reasons. So is it support, is it shipping, is it pricing, is it other
[26:06] reasons or product quality. This has been done for Susan. So she now has not only the segmentation of her customers, but she also have an understanding of why um certain comments were made about
[26:18] their orders. Now, we are just really, really close to helping Susan get what she needs for her meeting. And that will be one final step. Okay, I'm just going to copy this and
[26:30] proceed to the last step. So what we are doing in this step is that we need to understand who are the
[26:42] detractors because in terms of the NPS net promoter score in a retail setting detractors are much more likely to churn compared to customers who are passive or customers who are happy. So we are going to filter them with NPS score of less
[26:55] than or equals to six. So this means that the comment that they have given as a review is negative. And then we'll group by the segment of the customers. Are they uh trial customers? Are they loyalists? Are they champions and the
[27:07] reason for churn and for each group we will show what's the total lifetime value which is at stake and also the number of surveys. So we knows that um how many people are contributing to the specific reason to churn and I will sort
[27:19] this result by the total lifetime value. So that sounds quite a mouthful, quite a lot of steps. Um and the reason why we we um actually crafted the prompt to genie as as such starting from very
[27:32] vague non-specific to something that is very specific is to show that in link flow designer no matter what level you are no matter how familiar you are with uh data engineering operations, business semantics, Genie code will handle that
[27:45] for you because everything is powered underneath by Genieontology. So now I'm just going to hit um the boxes and take a look at the results. The first one is it's going to filter the detractors. So we will keep the rows where the score is six or less and we'll
[27:58] exclude the rest. And the next one is we will aggregate by the segment. So the total lifetime value, the number of surveys and we will group them by the same. And then finally we'll sort all
[28:10] the results by LTV lifetime value. And I will accept the changes. So the final results output over here that we see looks something like this. So we have an LTV segment of casual and the reason is
[28:22] for shipping and this is the total lifetime value that is at stake and the number of contributing surveys. So again this is just a snippet of the entire data set. It's just for us to um get a preview of the output while we are
[28:35] developing the lakeflow um designer pipeline and this exactly lends us into what Susan needs for her review. So she knows that okay in a specific LTV segment what's the reason that's leading
[28:48] to them giving a negative comment and what is the total lifetime value that is at stake for her. So if we go back initially to the questions over here if there is one thing that she
[29:00] can fix what would it be and that would be the reason contributing to the highest amount of total lifetime value um that leads to this uh customer churn. So now that we have all of this information that we need, we previewed
[29:13] all of the steps and we know that this is the um output that we desire. What we need to do is to productionize it. We need to write this to an output table, run it on the full data set so we get a clear and complete picture of what's
[29:26] going on. So I'm going to just pull an output table over here and I'll just write it into a table called LTV go results. put it into the catalog
[29:39] and you will see an overwrite button instead of run because I've just ran this before this session. So um I'm just going to show the table output that has been done earlier and you can see that
[29:51] there are a few items over here and the top three of them all points towards shipping. So no matter if the customer is a casual customer uh whether they are loyalist or whether they are champion the associated reason for them to leave a negative comment on the orders is
[30:04] because of shipping and we can see the total lifetime value that is at stake here if if Susan does not work on the shipping issues that all of these customers are facing and all of these add up to up to 1.5 to 1.6 million worth
[30:17] of revenue for them and these are the total number of surveys that has been attributed to um this data set. So now it is very clear for Susan if there is one thing that she needs to fix and she
[30:29] needs to talk to her CFO about to bring up in the board meeting there will be shipping and this is where the money will be going into. So now I'm just going to go back to the prep over here and we can see that there is a schedule
[30:42] button. So when we have written this end to end lakeflow designer pipeline and we want to make it productionized and we want to run it on the schedule what we can do is just hit on hit the schedule button give it an interval let's say
[30:54] Susan wants this data updated every day she can put it as a daily schedule and she can just create this pipeline and once this pipeline is created this is how it will look like it will run it for
[31:06] the first time when I ran it manually and it will run on a schedule so every day Susan gets an updated information on what is the most likely reason for customers to be detractors and what should she be working on and once she
[31:19] implemented the resolution to the customer churn um she's able to make use of the same pipeline to review the efficacies of her strategies. So Susan did not just build um something scrappy
[31:32] and put something together in time for her meeting to get all the insights that she needs. What she shipped over here is a production ready lakeflow designer pipeline. And if I click into the run, you'll be able to see that everything is
[31:44] written in a Python notebook and it is packaged as a flow job and it runs on a schedule. She did not need the help of a data engineer. No data analysts were involved. She can be self-sufficient in
[31:56] this entire journey. The data engineers can now focus on preparing the data such that she can be enabled to do these processes herself and everyone focuses on higher value added work for the organization and improving faster time
[32:08] to insights. So that is all we have for the LakeFlow designer demo and um uh my name is Jermaine and I'm a senior solutions engineer based in Singapore. Together
[32:20] with me is my co-speaker Nan. Uh we have our LinkedIn up there in on the screen. Thank you so much for listening to us and we hope this can help you um hasten and and and speed up the process of getting insights for your organization.
[32:32] So if you have any questions uh we'll just be hanging around up here for a while. Uh feel free to come over and talk to us. Yeah. Thank you. >> Thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.