No Time for the Dad Bod: Automating Life with AI and Databricks
Summary
- Event-driven architecture: This video demonstrates building AI-driven solutions using event-driven agent design and scalable automation patterns.
- Real-world applications: Examples include research agents for podcasts and multi-agent systems for meal planning—all powered by real-time data.
- Databricks technologies: Delta Lake and DSPy streamline reliable data management and generative AI workflow development.
No Time for the Dad Bod: Automating Life with AI and Databricks

Talk by Sean Falconer, AI Entrepreneur in Residence, Confluent
Life as a father, tech leader, and fitness enthusiast demands efficiency. To reclaim my time, I’ve built AI-driven solutions that automate everyday tasks—from research agents that prep for podcasts to multi-agent systems that plan meals—all powered by real-time data and automation. this video dives into the technical foundations of these solutions, focusing on event-driven agent design and scalable patterns for robust AI systems. You’ll discover how Databricks technologies like Delta Lake, for reliable and scalable data management, and DSPy, for streamlining the development of generative AI workflows, empower seamless decision-making and deliver actionable insights. Through detailed architecture diagrams and a live demo, I’ll showcase how to design systems that process data in motion to tackle complex, real-world problems. Whether you’re an engineer, architect, or data scientist, you’ll leave with practical strategies to integrate AI-driven automation into your workflows.
Databricks Named a Leader in the 2025 Gartner® Magic Quadrant™ for Data Science and Machine Learning Platforms: https://www.databricks.com/blog/databricks-named-leader-2025-gartner-magic-quadrant-data-science-and-machine-learning
Build and deploy quality AI agent systems: https://www.databricks.com/product/artificial-intelligence
See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements
FAQs
What is event-driven agent design?
Event-driven agents respond to real-time triggers rather than fixed schedules, enabling more responsive and efficient automation systems.
How does Delta Lake support automation at scale?
Delta Lake provides reliable data management with ACID compliance, ensuring agents process data accurately even at scale and can recover from failures.
What is DSPy?
DSPy is a framework for streamlining generative AI workflow development, enabling easier agent creation without manually crafting prompts.
What practical examples does this video show?
The video demonstrates research agents that gather information for podcast prep and multi-agent systems that plan meals based on preferences and real-time data.
Full transcript
Welcome everyone. this is No Time for the Dadbot. Automating life with AI and data bricks. How many people how many people here are parents? A good amount. Uh so when I put this talk together I had all of you in mind but even if you're not a parent I think we could all use and benefit essentially by having more time available to us and that's really what this talk is about. So about 15 years ago I left the sort of comfort of academics.
I was doing a posttock to become the CTO and founder of a startup. And I remember thinking vividly at that time that when I moved into the real world I was like this is the real world. This is real work. I now know what it means to be busy because I'm doing things in the real world. It's no longer this academic sort of uh you know high flutin type of stuff where I have lots of free time. And then fast forward to today and now I realize that I was extremely naive because now I have two of the most demanding bosses that you'll ever have.
And I'm not talking about, the CEO and founder of Confluent. I'm talking about toddlers. Um, you know, they are very, very demanding and they don't really care what your day looks like because essentially the only thing in their mind is their priorities and what they want. They're they're, you know, slowly sucking the life out of me. They're very cute. I love them very much, but they're killing me at the same time. My day started at 4:30 a. m. today because my that was when my daughter woke up screaming this morning.
So, every day feels like a sprint. It's like go go go until I basically pass out on the co couch at the end of the day. So, I, you know, remember the luxurious days when I was the CTO and founder of a company and I really only had to care about myself and my business. So as we get older and we take on more and more responsibility, whether that is at a as a parent or our careers or sort of combination, it becomes harder and harder essentially find time for things like our health and fitness and prioritize things that maybe are personal to us.
We end up essentially often prioritizing our kids and our careers over our own, you know, personal health, our own personal fitness, which makes a ton of sense. But essentially, hence we end up in a situation where we end up with the dad bond. So I wanted to figure out how could I essentially help myself balance this equation because this becomes very unsustainable where you're putting so much time into your family, so much time into your career, inevitably you have to give up something.
However, over the last couple years, we've had a tremendous amount of success using AI in the workplace to basically increase employee efficiency. you know people are able to depending on your job increase your efficiency by maybe two three four five times x uh at times for different types of of work. So what I wanted to do was figure out not only how can I be a more efficient employee but how can I be a more efficient dad and how can I be a more efficient husband essentially and use some of these techniques that we're finding success in the workplace with my home life.
So that's really what this talk about is about is how can we use AI to essentially make our family life more efficient and all this stuff st started pretty innocent for me you know initially it started with just prompting essentially using check EPT so my kids go to preschool and every week they have to bring a showand tell item based on the letter of the week and my son's been going there for like two and a half years now and some you know when you're on the 14th time of trying to figure out a showand tell item for the letter and it's 6:00 in the morning.
Sometimes I need a little help. So I started using essentially chat GPT as an extension of my dad brain to help me brainstorm these types of things and it works really really well for this kind of stuff. So I do started leveraging this all the time. And then additionally I started using it as a way things like as a way to help my son explore his own imagination. You know he thinks that when he grows up and you know maybe this will be happen but he wants to be a superhero when he grows up. you know, that's the job he wants.
And in his mind, he wants to be the superhero named Rainbow Racer and that can shoot trains out of his arms apparently. And essentially, we used and a and I helped him craft the prompt to essentially create the image of what was in his head. And then we went from there to start taking essentially his ideas and creating his bedtime stories where we can create these, you know, highly personal dynamic stories for him where he's the protagonist and the hero and they're based on his ideas.
It was a great way one to encourage him to go to bed uh so I could have more time at the end of the day and make him excited about you know reading and storytelling crafting the story and kind of learn how to prompt AI. So that's where these things started but of course I wanted to take it up a notch. wanted to essentially offload more and more sort of responsibilities that are become a hindrance in my home life so I could carve out more space and time to prioritize things like making the right food choices and exercising and things like that.
So that really leads us to agents. So agents, I'm sure there's a million talks about agents. Agents is all the rave right now. Agents are the new app. SAS is dead. It's all about agents and agents are very very exciting from the idea that we could offload these large asynacous chunks of work where historically maybe a person was in or multiple people were involved in that process. So I wanted to figure out how can I leverage agents in my home life to automate some of this stuff.
On top of that, I've been working in the intersection of data, AI, knowledge engineering for my entire career originally as an academic, even doing my company really, really early sort of conversational AI systems. So, I've been building in production sort of enterprisegrade AI for a long time. And even though these products that I was building for my home life, I'm probably going to be the only user. uh they're basically be fine if they were demoare.
I wanted to take things that I talk to customers at Confluent every day about and what are the right technologies that you need to actually build this stuff in production systems and apply that to these essentially projects I was building on my own so I could use those as a way to demonstrate some of these ideas I talked to customers about. So because actually building agents as exciting as they all are is actually really really complicated.
You know, a single agent has a tremendous amount of potential dep potential dependencies where you have data systems that you're relying on tools. You have a stochcastic model that's making decisions on your behalf. You have feedback loops. You anything that's in production, you're going to have a certain amount of observability. You want logging, tracing, all the sort of things that you do with any sort of piece of software that you're putting in production that's userf facing.
And really, you end up with a large distributed system problem. And realistically, people aren't going to build a singular monolithic agent to solve all problems. What they're going to do is essentially decompose a problem into smaller subpros where these smaller sub problems are essentially encoded into purpose-built agents and you have a multi- aent system where you have multiple agents working together to solve a complex problem.
Just like we have multiple people in the workplace work together as a team to solve complex problems. And as these systems scale and you add more and more agents and more and more tools and more and more data, you essentially end up with this very complex labyrinth of interdependencies and a quadratic explosion of dependencies. And this becomes very difficult to scale in a request response pointto-point uh approach where you're just making essentially API calls between each of these because if one of these agents decides to hallucinate or produce an output that's not what you're expecting, the whole thing is going to end up in a really really bad situation.
Now luckily we have seen similar problems in the past. So if we go back essentially back to the era of when we broke up the monolith some people are still doing that but like 15 years ago we realized that essentially a monolith becomes a barrier to scale because you have to scale continually scale the resources available on the server. So people split up the monolith essentially into microservices and that way you could decompose it.
You could have different teams work on different parts of it and you can scale the resources for each of the microservices independently. However, you end up with a similar set of quadratic explosion of dependencies if you're using essentially a request response model for this. So now microser A that's sending data to microser D needs to know a lot about the resources available in D otherwise it could potentially overload it, send too much data to it, take it down and take the entire graph down.
So the way that people solved this problem was through event- driven architecture where instead of having a quadratic number of dependencies, we end up with a linear number of dependencies where everything's loosely coupled. All microservices essentially just producing consuming events and we have an event broker essentially responsible for durably uh delivering those messages on each side. And this way we can have multiple consumers.
Microser A doesn't really need to know anything about downstream consumers. And this is extremely scalable. This is how some of the largest systems in the world like Netflix and Uber essentially run with thousands and thousands of these microservices. And really I think the great gift of the world that technology like Apache Kofka gave us was this decoupling these decoupling of services decoupling of teams. And this same architectural pattern that we've been using so successfully for scaling microservices should be essentially the same architectural pattern that we use when we start to think about scaling and deploying multi- aent systems because realistically what is an agent from a deployment standpoint it's really just a microser with a brain it's a microser that has an LLM that has some additional context but you're going to deploy it essentially as a microser multi- aent system is a bunch of microservices all need to talk to each other So what we need to be doing essentially is building event- driven multi- aent systems where agents become just producers and consumers of these events.
So you get a loosely coupled system and get act exactly once delivery of the messages and there's a lot of other nice properties where you can basically rewind the tape replay these things. So I can do essentially you know dark deployment of an agent AB test an agent test a new version of the agent all against live traffic without side effects. And when we combine that essentially with something like Mosaic AI where we have a very robust AI platform for building, deploying and hosting agents as well as governing, we can essentially build extremely robust, durable, fault tolerant enterprisegrade multi-agent system.
So I wanted to essentially take that idea and apply it to my own life. So I've done this a number of times for a bunch of different projects, but the one we're going to talk about is building a meal planning agent. So every week it becomes rather stressful at times to figure out what we're going to essentially feed ourselves and our kids. And when you have little kids that go to bed at 7, you don't have a lot of time at the end of the day to figure out what you're going to eat.
And if you don't have a plan, what you end up doing is eating a lot of garbage or takeout or cooking separate meals. None of it is a good solution. it becomes stressful, not ideal for trying to find extra time. So, it's much better to essentially have a plan in place. So, I wanted to solve this problem where I wanted to be able to take essentially our kids preferences for food, our own preferences for food, any hard requirements like my daughter can't eat fish, number of meals that we need, any recent meals so that I'm not just repeating the same meals over and over again, throw it into this AI magic box and essentially create a weekly meal plan with our recipes and everything like that.
That's the goal. So the way that I did this was I decomposed essentially our AI magic box into a bunch of different multi- aents that are purpose-built. So the first one is our child meal planning agent. I'll go into details about what's involved there. Adult meal planning agent is responsible for building an adult meal plan. A share preferences agent. So basically the the responsibility is to combine these into a single holistic meal plan.
And then finally an agent that's responsible for taking this blob of text converting into an output that we can actually render on a screen. I used a couple of different agentic patterns to support this. So the first one, most of the agents are react agent design pattern where this is basically a tool calling agent. You go through a thinking loop where it's figuring out which tools to call, performs those tool calls, gets back the additional context loops until it essentially generates a final response.
And I also use the reflection agent for combining the meal plans where the first agent is generating the combined meal plan and the second agent essentially judges it and evaluates it and then you go in a loop until you create something good or you hit the threshold of the loop. So bringing this all together a little bit hard to see but the I built a simple front end and back end using uh Nex. js and MongoDB to host this.
This is essentially where I store the settings, family preferences, meals, and when a request comes in, I'm going to send that essentially to the database. And I'm using Confluent Cloud's manage Kafka and Flink for interention communication. So this is basically the backbone of that event-driven messaging system for agents. And then I'm using Mosaic AI for the agent and model hosting. So when a request comes in essentially I use a manage connector to bring that data that request from MongoDB automatically into a Kafka topic called the meal plan request and then from there I'm syncing that through a manage connector down into this micros service which is this child meal planning agent and essentially this agent has a specific role where it's an expert in designing nutritious meals that toddlers love and there's a bunch of other specification here.
So that's that its job and it has a bunch of access to tools like kids preferences, hard requirements, recent meals. So it's going to use those dynamically and essentially create a meal plan that fits all this uh profile. Oh, there's also bias towards meals that you can prepare in like under 20 minutes. And then in parallel, completely decoupled from the child meal planning agent, I also run the adult meal planning agent where its role essentially is to be an expert in designing high protein, low glycemic, low carb dinners for couples so that we can, you know, make good healthy meal uh choices during the week.
It also has access to tools like recent meals. Both of those agents spit out their output to another Kafka topic, the adult child meal plan. And then I essentially I use Apache Flink to com to write a a Flink job to combine those streams. So this Flink job is essentially running continuously joining those two streams. Doesn't matter when they arrive. Essentially I'm just joining on the request ID. Pretty simple stuff.
Once we have that combined meal plan, I send it over to the shared meal plan agent which is using this reflection pattern. And there essentially there's one agent that's going to generate response. The other J agent is essentially evaluating and grading the quality of the recommended meals on taste, variety, nutritional value. So the goal is essentially just keep looping until you create something that's actually good and fits all the specifications I have.
And then that writes the output to the complete meal plan. I sync that down to the output the format output agent. The format out page agent is just going to take this blob of text essentially split it up into the ingredients lists, the meal plans for the week, the shopping list, and so forth. And then I send that all back to where I can surface it on the front end. So what this looks like essentially is something like this where I get the meal plan for the week, all the meals, the adult child versions and the ingredients and then there's also a recipe list uh as well as the instructions for actually cooking this.
And we've been using this for about six or seven months now. With this, we were essentially able to take uh something that was often like a chaotic situation, led to bad choices, cost us a lot of money because you're you're you're you know, you're substituting essentially door dash delivery for food sometimes in a scramble rather than having a plan. And I was able to do it essentially using these sort of production enterprisegrade techniques where we're using event-driven architecture to address things like state persistence, durable execution, coordination, replay-ability, all really really important for building multi- aent systems in the enterprise and then mosaic AI and testing tracing uh model and agent serving.
Is this overkill for a single user of meal planning agent? Absolutely. it is uh absolutely overkill but it really shows and demonstrates what it is the combination of technology that you really need to bring to bear to actually put this stuff into production and I would say 99% of actually building successful multi- aent systems is about practical computer science software engineering that you know many of us in this room have been doing for most of your career and it's only has a small percentage to do with actually the AI models because building on top of AI models is much different than building models itself Well, there we go.
I think we have a few minutes uh for any questions if we have it.
Learn more about the Databricks Data and AI platform.
The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.