Sponsored by: Google Cloud | Unleash the power of Gemini for Databricks

Summary

Agentic AI integration: This video shows how to integrate Google's Gemini models into the Databricks Data and AI platform for building intelligent applications.
Production-grade security: Learn how Vertex AI delivers managed ML infrastructure with fine-grained governance and security for scalable GenAI applications.
Model selection strategy: Discover how to balance model performance, cost, and accuracy to choose the right Gemini model for your specific use case.

FAQs

How do you integrate Gemini models with Databricks?

This video covers using Vertex AI, Google's managed ML platform, to connect Gemini models directly into your Databricks workflows. You can access Gemini models through Vertex AI connectors that integrate with the Databricks Data and AI platform, enabling end-to-end AI applications built on your data.

What percentage of organizations struggle with GenAI integration?

According to a Salesforce survey cited in this video, approximately 60% of organizations polled did not know how to effectively integrate generative AI within their applications. This video addresses that gap by demonstrating practical integration approaches.

Should I build machine learning on my data platform or use a separate ML platform?

This video explores integrating Google Cloud's Gemini models with Databricks Data and AI platform, showing how organizations can leverage best-of-breed AI models within a unified data platform. The integration enables ML workflows that stay on the data platform while accessing the latest frontier models.

Which cloud platform is best for production machine learning?

This video shows how Databricks on Google Cloud integrates Gemini models for production machine learning workloads. The combination delivers the security, governance, and scalability required for enterprise ML while accessing Google's advanced AI models.

What are the key customer pain points for production AI?

The video identifies three main challenges: managing and scaling operations with production-grade security across applications; choosing the right model based on use case while balancing performance and cost; and ensuring content is grounded and factual, especially for user-facing applications.

Full transcript

Hello everyone. I'm a uh ML engineer part of the applied AI engineering team at Google cloud. Uh and today we'll be talking more about Gemini and how to integrate Gemini within data bricks ecosystem itself. uh we uh recently there was a poll conducted by Salesforce that showed that almost 60% of the organizations who were pulled did not know how to integrate generative AI within their applications. Uh and what we are uh what we aim to do is essentially reduce the time and reduce the effort needed by organizations to uh integrate uh all kinds of generative AI applications uh with uh any sort of models itself.
Right. But specifically we'll be talking more about uh data bricks and specifically uh Gemini family of models from Google cloud. Uh as you know we have uh been working with data bricks quite closely for a long time and we have invested in building out the entire integration with data bricks and Google cloud starting uh obviously from the analytics platform and data bricks uh but recently we've also started building integrations and connectors with vertex AI which is our managed ML platform and specifically around Gemini and of course it also brings you the uh richness of the open cloud ecosystem with Google Cloud uh access to compute, access to identity management, access to in region uh machine learning processing for all sorts of large language models.
So what are we hearing specifically from customers? What are their pain points? So the first thing that we're hearing is how do we effectively manage and scale our operations across uh our applications and actually bring in production grade uh security and production discipline itself. Uh second is how do we choose the right model uh based on a specific use case but also balance out the performance and the cost depending on our use case.
And finally, how do we ensure that our content is always grounded, factual, and is accurate uh especially for userfacing applications. So that's where we bring in Vert. Ex AI. Vert. Ex AI is a managed ML platform within Google cloud which gives you across the stack access to uh starting from accelerators with GPUs and TPUs uh moving on to models where you can access definitely Gemini. uh along with Gemini all sorts of thirdparty model providers and then of course moving on to the uh application platform where uh as you know these days lots of organizations are building agents, search uh engines, chat bots and so on.
So what's specifically new? Uh recently we announced the Gemini 2. 5 family. Uh Gemini 2. 5 family are thinking models with advanced reasoning. uh they show thoughts for step-by-step transparency and show uh complete thoughts and are able to solve complex tasks very easily. Specifically, Gemini 2. 5 Pro is our state-of-the-art model uh for coding, advanced reasoning and any sort of complex task. Uh we also launched Gemini 2.
5 flash which is our workhouse workhost model which is used for uh all sorts of everyday applications. Uh we also have two new cool features that we introduced. The first one is called Vert. ex AI model optimizer. Think of it as a smart load balancer uh where if you don't want to decide based on your incoming query which model to use uh we will automatically route it to the uh different backends. So if a query is too complex, we will get it answered by Gemini 2.
5 Pro. And if it's simple enough and easy enough for a normal LLM to answer, then it'll we'll get it answered by our workhouse which is Gemini 2. 5 flash. And finally, we also went into preview with live API. You uh most of you may have seen this as project which allows you to do realtime communication within LLM including with all sorts of modalities especially with video, audio and including uh computer screen sharing itself.
Uh so let's dive a bit more deeper into Gemini and specifically what's uh uh really good about it. So uh Gemini 2. 5 Pro is our latest uh state-of-the-art model. It scores in the highest number one in terms of most benchmarks mainly around reasoning which is humanity's last exam. We have it on science on GPQA diamond uh around mathematics on the AM benchmark. uh on coding on their ADA polyglot and agentic coding on sweepbench.
So across all these uh super complex benchmarks, Gemini 2. 5 Pro scores number one and it is number one even on uh LM arena which gets uh real time user feedback in in the models. Uh and why is it specifically important for organizations? uh along with uh getting the best pro performance uh Gemini 2. 5 Pro also offers you the best uh cost. Uh so if you look at the parareo principle for the models starting from left to right the cost increases as you can see Gemini is on the paredo frontier for most models which means for the amount of performance it gives you which is the highest on your y-axis it also costs the cheapest amongst all flagship models.
So Gemini 2. 5 leads on all sorts of uh Pareto principle across all models. Uh if you look at long context which is a very unique differentiator for Gemini uh we can see that across very large contexts including like up to 1 million tokens Gemini's performance remains relatively the same and even at the 1 millionth context performance Gemini uh still outperforms most models in the market and as you can see we have been investing quite a lot in order to make these efficiencies possible uh starting from when We started with Gemini 1.
0 Ultra uh to Gemini 2. 5 Pro, our current state-of-the-art. Uh you can see that the cost has reduced but also the performance has increased drastically in just a year. So we have been investing heavily in this and Gemini is a super awesome model and we would love for all of you to try it out. So uh this is this slide gives you a summary of essentially everything we talked about, but just a couple of things to call out.
Uh specifically in Gemini 2. 5 Pro, we also have a new feature called thinking budget which allows you to control the amount of reasoning a model does. So if you have super complex tasks like coding across a very large code base, you can actually increase the thinking budget so that the model thinks more before it answers. Uh we also have a new model for Gemini 2. 5 Pro called Gemini 2. 5 Pro Deepthink which is our best performing model yet and we have it in private preview for select customers.
Uh Gemini 2. 5 flash is our workhorse model. It allows you to get maximum performance at a fraction of the price and at a fraction of the latency with super performant uh performance across the board. Uh and of course we have the live API and we're also going to be launching a private preview for computer use. So just moving on uh yeah so how do you actually get this started within data bricks itself right? So within mosaic ml you can configure Gemini 2.
5 pro within vertex AI or even my favorite model which is Gemini 2. 5 flash. uh you can provide your own GCP project and you can set up the Mosaic ML uh serving endpoint and this is how it should look like. Uh and the way to use it would be uh within your notebook you can just use the standard ML flow client uh and when you specify the endpoint you just specify the endpoint parameter we just set up earlier and then you're able to do uh send the API request in a standard OpenAI SDK format.
Uh so it's fairly simple to use and fairly easy to get started with. Uh but let's actually look at a more complex implementation of how this would look like. Right? So uh we're going to be building a datab bricks data science agent which allows us to query uh you know data bricks tables which allows us to get the schema which allows us to analyze data which we query from data bricks uh and finally train a machine learning model and deploy it and serve it all with uh just an agent.
So literally no code being written overall right so let's take a look at how it's going to be built. We're going to be using Google ADK which is Google agent development kit and uh for all the reasons listed here which is it's very easy to set up it's super modular easy to test locally we provide native multimodal support and it's ready to deploy with a single flag right so we provide out of the box uh deployment options for ADK but let's take a look at what exactly we're building right so we're going to be building a multi- aent system for data science uh and our data science agent would have a database agent which essentially is our interface with data bricks.
It it's able to you know generate data bricks SQL execute the data brick SQL and get the results back. We then have an analysis agent which is a code executor through which we are able to generate visualizations and plots and finally we have an ML agent which is able to run machine learning experiments including training model serving it and deploying it itself. So let's actually take a look at how it looks like. Uh so I'm going to be using the Kaggle Titanic data set.
So uh for folks who may be from the data science background, you may be familiar with the Kaggle competition for the Titanic uh data set. So we'll be using that uh in order to essentially see how it looks like. So the data set essentially consists of various uh passenger ids the different features on the passengers. So if we just take a quick look at how it looks like uh you have the passenger ID, you have the class where they're seated, you have their names, you have their age, you have their details.
Along with that, you have this column called survive which essentially indicates whether they would survive a plane crash or not, right? And that's essentially what we're trying to set up in this case uh a model in this case. So let's go ahead and quickly open up a new session with agent development kit. Uh so just to show off hands, how many of you have used Google agent development kit before? Anyone? Okay, we have a few people in the room.
That's great. Uh so let's get started with Right. So I'm just going to say uh what can you uh help me with? So uh essentially what it's doing now is setting up the session connection between uh our uh Vert. ex AI uh deployment to data bricks. So once we get the session, we are actually able to run queries on top of it. Right? So as you can see, it tells that it's a data science agent. It's able to query, retrieve, uh train models, pre-process models.
So let's actually take a look at it, right? So I'm going to start off with some easy questions. So I'm going to say uh list all the tables in the cluster in the catalog. Uh this is a fairly simple uh question, but let's see how it performs, right? uh as you can see it actually found the correct uh data set which is the Titanic data set here and found the two tables which is my training data set and testing data set. Right?
So let's dig a bit more deeper into this. So I'm going to say uh use the uh train data set u and list the schema and a few sample rows from it you know. So let's uh actually observe how data bricks how ADK is actually able to make the connection. So as you can see it actually found the schema for the table and now it actually makes a call to call data bricks agent which is my sub agent here right. So my primary root agent actually uh hand handed off the control to my call data bricks agent and my call data bricks agent in the back end is now uh passing this and it gives a question which is list a few sample rows and once that is done uh you can see the response here that it actually was able to um not just execute the query but also format it in a nice little table and give me an explanation of how it actually did that.
Right. So let's take this one step further because ultimately this is just querying simple tables which is fairly simple. So let's take it a bit step further and let's run some analysis based queries. Right? So I'm going to say calculate uh the total percentage uh calculate the uh percentage of men and women passengers. Uh, give me a pie chart for it. Give me a pie chart for it. So now what I essentially wanted to do is do the calculation within data bricks uh get the results and then I wanted to call my Vert.
Ex AI code interpreter which is a sandboxed Python code execution environment in order to render and generate these plots in real time. Right? So as you can see the first step was uh to calculate the total number of uh people and then it actually generated the SQL query and it gave me the answer as 314 female passengers and 577 male passengers. Right? So it actually got the exact percentages as you can see and now it's calling the uh uh code executor which is my analytics agent which the handoff was done to and hopefully in a couple of seconds we should see a very nice pie chart being generated.
So fingers crossed the demo a demo gods are on my side. Let's see. Yeah. All right. That's great. So as you can see it was actually able to uh get the data, interpret it, write code in order to generate the plot and then actually render the plot within my user interface as well. Right? So that's cool. And then we're going to take it one step further because now that we have the data set, we can also train models on top of this.
Right? So uh let's take it a step further and train a model uh on top of this. So I'm going to ask the agent to uh can you train uh random forest model uh with the train data set. Uh ignore all the categorical features. Yeah. So I'm just going to tell it to create a random forest uh model in order to predict the survival. Uh and then essentially uh let's see what happens. Right? So as you can see this is the question that went into the agent which is uh train a random forest model.
The target variable is survive. Use the following numerical features which is passenger ID, class, age, fair and all these features in order to predict whether they will survive or not. Right? So u yeah let's give it a couple of seconds and see what happens uh and you can see that the intermediate result is here uh and that it it was actually able to train the model right uh so as you can see the features the target variables used the categorical features and the mlflow uh experiment ID so let's actually go and see if it ran so I'm just going to go into my mlflow UI and you you can see the experiment ran 40 seconds ago.
Uh it trained a sklearn model from ADK and then uh it has 73% accuracy which is great for a model that just you know has literally no data and an agent that has absolutely no feature at all. Right? 75% accuracy on average is great. So now that I have the model trained, let's actually go and deploy this, right? Cuz at the end of the day, having a model is not uh enough. we actually need to deploy it and serve it in production as well.
So let me go and do this. Can you register and deploy the model uh as I don't know we'll just give it some name ADK uh RF test deploy demo you know so that's my model name or my endpoint name that I wanted to uh that I wanted to reply as. As you can see, the argument was done and then the model is there. Uh, okay. All right. Yes, please use the run ID. All right. Some hiccups, but that's fine. Uh, hopefully it's able to pick up my experiment and still uh uh deploy it.
But in the meantime, I'm just going to show you how it looks like. So once it's actually deployed uh the endpoint would look something like this. Uh if I go to my catalog uh within this here I have my Titanic data set. Oh actually if I just come back here uh the model was actually registered and deployed as you can see. So we'll just give it a second for it to formulate the response. But uh ADK RF test deployed demo endpoint was done.
So if I come here, I'm actually able to open here and this is the model that was deployed. As you can see, there's two versions of the model. Uh the model versions was deployed. If I'm going to open a deployment, I'm actually able to see it. So now uh within MLflow, I can actually utilize this model uh within data bricks directly in order to do all sorts of predictions on top of my data. Yeah. So that's a short but sweet demo in terms of how complex workflows you can actually model with Gemini and specifically Gemini 2.
5 Pro in this case in order to achieve super complex tasks fairly easily out of the box. Yeah, thank you so much. Uh I'm opening up if you have any questions feel free to uh raise your hands and happy to answer them. Yeah, thank you.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.