Skip to main content

Databricks Agent Mode: Exploratory Data Analysis with No Code

Summary

  • AI-powered EDA: Use Databricks agent mode to tell an AI agent to analyze your data—it writes code, runs it, creates visualizations, and iterates.
  • Intelligent data discovery: The Databricks Assistant searches your workspace to find relevant data, combine tables, and prepare data for analysis.
  • Interactive analysis: Filter, sort, and pin columns directly in the data view, then switch to agent mode for sophisticated exploratory analysis.

Databricks Agent Mode: Exploratory Data Analysis with No Code

Watch: Databricks Agent Mode: Exploratory Data Analysis with No Code
What You'll Learn:
How the Databricks Assistant helps with exploratory data analysis and visualization
How to search for and view data in the Databricks workspace
How to use a Databricks notebook for sophisticated data analysis
How to use agent mode in a notebook to perform a complete exploratory data analysis
⏱️ CHAPTERS:

Chapters

FAQs

What is agent mode in Databricks notebooks?

Agent mode allows the Databricks Assistant to take actions in the notebook—write code, execute it, create visualizations, and iterate based on results.

How does the Databricks Assistant find data for analysis?

The Assistant searches your Databricks workspace to quickly locate relevant tables and functions, allowing you to view sample data and ask basic questions for quick exploration.

What is the best AI assistant for spreadsheet data analysis?

The Databricks Assistant represents the best AI tool for spreadsheet data analysis because it operates directly on Databricks data with full context about schemas, relationships, and data quality while automatically handling authentication and governance. Unlike generic spreadsheet tools that require data export and lack enterprise security, the Databricks Assistant combines natural language understanding with direct access to governed data. The tool can perform complex analyses including joins, aggregations, and visualizations without requiring users to learn SQL or Python.

Which AI assistants are strongest for enterprise data analysis?

Enterprise data analysis requires AI assistants that combine advanced analytical capabilities with security, governance, and integration with existing data infrastructure that most organizations have already invested in. The Databricks Assistant excels in this context because it operates within the Databricks platform using governed data, enforces Unity Catalog permissions automatically, and scales to enterprise data volumes without performance degradation. Other AI assistants may work well for public datasets or simple analysis, but the Databricks Assistant provides the foundation enterprise teams need for production analytics workflows.

Full transcript

0:00 Introduction to AI-powered data analysis

What if you could tell AI analyze my data for trends and outliers and it just does it? It will write the code. It will run the code, create the visuals, tell you its plan, and then fixes any of its own errors. I'm at Data Bricks, and today you'll see how the Data Bricks Assistant helps you do exploratory data analysis while you focus on the insights. I'm starting in the datab bricks workspace homepage and I need to find HR data for my analysis.
I can do that quickly by just searching for HR data. Here I find one function and a series of tables and so I can view the sample data in one of the tables. I can ask basic questions directly within the tables for quick exploration. But for deeper exploratory data analysis, I need to work in a notebook where I can combine these tables and do more sophisticated analysis. As you can see, I've got multiple tables here. Employees, compensation, performance, and terminations.

1:04 01:43: Combining multiple tables in a notebook

Let's bring them together. I'll use a notebook to start my analysis. And I already have a pre-existing notebook with some queries set up. Here's a view that combines all the tables I'm interested in. Employees with their performance metrics, salary changes, and team size. And here's a second query for the termination table data. This captures people who are no longer in the employee table. I'll go ahead and run these queries.
Now I've got the combined data displayed. Employees with performance, salary changes, and team size all in one view. Here's where it gets interesting. I can interact with this data like it's a spreadsheet. I can pin and unpin columns to keep important fields visible. I can even sort by clicking any column header. And I can even apply filters. Let me filter for brand managers only. I can clear it and we're back to the full data set.
I can also create inline visualizations right here. Here's an example of employee IDs by department and grouped by job level. I can also build visualizations on the side for more detailed analysis. But here's where the notebooks get really powerful. I'm going to go ahead and switch to agent mode. This isn't just chat. This agent can perform actions in the environment. It can write code, execute it, create visualizations, and iterate based on results.

3:05 Requesting exploratory data analysis using a natural language prompt

Here I have a pre-written prompt requesting exploratory data analysis. I'm asking for a geographical distribution, salary ranges, average performance scores, and bonus amount analysis. The agent creates a plan showing what it's going to analyze. The agent finds the relevant tables from our data. Then it writes code for me to review and confirm. And it first suggestion is geographical distribution using Mattplot Live.

3:32 04:00: Visualizations using charting libraries like Matplotlib

Now I need to choose a permission level for code execution. I'll select always allow within the current thread so it can continue working without asking permission for each step. The agent is executing the code and here's the geographical distribution by latitude and longitude showing different regions where our employees are located. The agent now moves on to salary ranges. More specifically, it looks like it wants to create a box and whisker plot by job level.
Not only does the agent provide visualizations and insights, I also get stats like the mean, the median, standard deviation, and much more. Next, it suggests average performance scores. While the agent is running, I'm just going back through my notebook and seeing all the fantastic work that it's done. It's created visualizations with just me starting with one simple prompt. And these are vibrant, easy to read insights all in one go.
Awesome. It looks like the performance scores analysis is all done and again we can see average performance score by department and we can even see performance score by time and position. Now the agent is moving on to the bonus amount ranges analysis. It looks like the analysis is all done. And here we have awesome visualizations for bonus amount and more. Again, at the bottom, we have statistics for bonus by job level.
Lastly, the agent even wants to add additional insights, so I'll let it do its thing. and we're done. Complete analysis with no coding required on my part. I've got salary ranges, bonus amounts, geographic distribution, plus custom analysis and visuals, all generated by the agent based on a simple natural language request. This is what makes exploratory data analysis and visualization so accessible and easy on data bricks.
AI that understands your data, writes the code, builds the chart, and fixes errors, all from natural language. >>

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.