Skip to main content

How to use Meta Conversions API on Databricks to activate first-party data

Summary

  • First-party activation: Send high-quality conversion events directly from your Databricks Lakehouse to Meta Conversions API to improve ad targeting and recover lost signals.
  • Turnkey setup: Deploy Meta Conversions API from the Databricks Marketplace with a pre-built app that securely manages credentials and test events in your workspace.
  • Scale with PySpark: Leverage PySpark UDTFs to handle high-volume data transmission while maintaining governance and security of the Databricks Data and AI platform.

How to use Meta Conversions API on Databricks to activate first-party data

Watch: How to use Meta Conversions API on Databricks to activate first-party data
This video demonstrates how to seamlessly connect your Databricks Lakehouse to Meta to improve ad measurement and campaign performance through the Meta Conversions API integration.
Benefits:

  • Turnkey activation: Discover and deploy Meta Conversions API directly from the Databricks Marketplace via a pre-built notebook app.
  • Resilient signals: Combat signal loss and improve match rates by sending high-quality, first-party data securely from your Lakehouse.
  • Built for scale: Leverages PySpark UDTFs to handle high-volume data transmission with the governance and security of Databricks.

Learn more about Meta Conversions API on Databricks: https://www.databricks.com/blog/activate-first-party-data-meta-conversions-api-databricks
Chapters:

Chapters

FAQs

What is the Meta Conversions API and why does it matter for advertisers?

The Meta Conversions API allows you to send conversion events from your own infrastructure directly to Meta's ad delivery engine. It helps recover signal loss and improve ad targeting by giving Meta high-quality, first-party data about customer actions.

How does the Databricks Marketplace integration make Meta Conversions API easier to use?

The Marketplace listing includes a setup assistant app that handles credential storage securely using Databricks secrets and walks you through connecting your Pixel ID and access token. Everything stays in your workspace—nothing leaves your environment.

What data needs to be configured before sending conversion events to Meta?

You need two pieces of information from Meta Ads Manager: your Pixel ID (the identifier for your ad account's tracking) and an access token (a secure key that allows Databricks to send events on your behalf).

Why is PySpark UDTF technology used for this integration?

PySpark UDTFs handle the high volume of conversion data transmission at scale while maintaining full governance and security within the Databricks Data and AI platform.

Full transcript

0:00 Introduction to Meta CAPI on Databricks

Hello, I'm a solutions architect here at Databricks. Today I'm going to show you how to leverage our new Meta Conversions API app to get you started sending your conversion events from the Databricks Lakehouse directly to Meta Ads Manager. If you run ads on Meta platforms like Facebook or Instagram, this is how you get better signal back to their ad delivery engine. Think of it this way, the more high-quality data Meta has about your conversions, the smarter its ad targeting gets.
This connector makes it simple to send that data straight from your Lakehouse. Before we jump in, this is the Meta Conversions API listing on the Databricks Marketplace. This tutorial assumes that an admin has already clicked get access and installed the listing into your workspace. When that is done, the listing will show that it's been installed and it comes with two things, a sample data set with realistic conversion events and a companion app that walks you through everything else.

0:34 Marketplace Listing & Installation and Setup Assistant

So, let's take a look at this app. This is the companion app. It is a lightweight Databricks app that runs right inside your workspace. Think of it as a setup assistant. The app knows who you are Everything it does from storing credentials to deploying notebooks and creating jobs happens securely in your Databricks environment. Nothing leaves your workspace. The first thing you'll be prompted to do is to attach the Marketplace catalog.
If the app is able to automatically detect the catalog, it fills this in for you. If not, no worries. You can follow the instructions here to copy the proper proper catalog name and paste it back in the app. Now, let's get connected to Meta. All you need are two pieces of information from your Meta Ads Manager, your pixel ID and your access token. If you've ever set up the Meta Pixel on a website before, you've seen this pixel ID.

1:55 Creating a Meta CAPI connection for your Pixel

It's just the identifier for your ad accounts tracking. The access token is like a secure key that lets Databricks send events on your behalf. If you're not sure where to find this information, we've linked Meta's documentation right here in the app to help get you started. But basically, both can be found within your Meta Ads Manager. Once you've entered in that info, we store this information securely using Databricks secrets, never in text and never leaving your environment.
Now that we've added this information, let's test the connection. The app quickly sends a test event to Meta's servers to make sure everything is valid before we save anything. No guessing. You'll know right away if something is off. The connection is now verified and we've confirmed we've copied the correct information. And boom, you've successfully connected Databricks to your Meta Ads Manager for CAPI delivery. Now that the connection is set up, you have a few options for how to leverage it.

3:04 Quick Start: Sending and Verifying Test Events

First is the quick start. It uses sample data that came with the Marketplace listing to send test events to Meta right now. Second is the deploy notebook option. This is a great option if you want to make quick adjustments, attach your own data for testing, or just to see how the process works under the hood at your own pace. Third, we have set up a job. If you already have your conversions data ready to go, this lets you configure the column mappings Meta requires and set up a recurring job to keep the data flowing on a schedule from Databricks directly to Meta.
Today, we'll walk through the quick start end to end and we'll take a quick look at the deploy notebook and setup job options. So, let's start with the quick start. Notice the sample data location is already pre-filled. This is the sample data that came along with the marketplace listing. You don't need to configure anything additional. If you have one, you can also enter an event test code here. This is a code for Meta's Events Manager that lets you verify events have arrived without affecting your live ad data.
It's great for a first run, and if you have Ads Manager open alongside this, you'll start seeing events popping in in about 30 minutes after delivery. Now, let's send off the request. Behind the scenes, the app is reading the sample data, mapping each row to Meta's required format, hashing personal information like email and phone number for privacy, and sending everything to Meta servers in batches. And here are the results.
Let me walk you through what we're seeing here. The events sent denotes how many rows Meta has accepted. If any events failed, that will also appear. The number of batches that it took to send events will also be logged. We have an example here of if there is an error. So, you'll see when events failed, there will be a tally of the number of events that have failed, as well as a detailed error message to help you and your team figure out what happened.
Now, let's hop back to it. Now that we've completed the quick start, let's see these events directly in Events Manager. Here we have a screen share from a colleague of mine who has access to the Meta Manager. You could see that he's going to data set and pixels, and we're confirming that the data set is receiving events. Once you click in, you'll be able to see the events that have been sent over the past few days. You could see here that we have plenty of events surf being surfaced daily and a variety of different types.

5:30 Accessing Delivered Data in Meta Events Manager

And that's pretty much it. Now that we've seen the flow of sending sample data all the way to Meta Events Manager, so now let's take a look at the other options that we have available to us. Let's take a look at deploy notebook. In this section, you'll see a few fields and optional test event code just like we saw before and a workspace path where the notebook will be saved. You can edit this um to the directory of your choosing.

6:14 Advanced Setup: Customizing Notebooks and Recurring Jobs

Once you feel satisfied, you can click deploy to workspace and it will provide you a link of exactly where this notebook is. Once you click into it, you'll be able to see a full documented and detailed Databricks notebook that will walk you through leveraging the UDTF and provide you options to customize everything from the column mappings all the way to how you define your tables to deliver to Meta. Now, last but not least, let's explore what it looks like to set up a job.
In here, just like before with the other examples, you have the opportunity to specify an event test code. Then, you can specify the specific conversion data location. Right now, by default, we have the sample data set specified here. As you can see, this field is connected to your Unity Catalog. So, it makes it very easy for you to search, browse, and select the table that makes the most sense for your use case. But, for now, let's put it back that table to the Metacappy parameters.
We make it easy for you by denoting required parameters with an asterisk, and also providing any mandatory transformations with a lock. That way, you know that you aren't missing anything. Once you've specified the parameters that are relevant to you, you can fill in some additional details to schedule this job. You can have it triggered manually, daily, or weekly, and there are a lot of event settings that you can further customize on once you have rated this job.
If you do not want to use the application to create the job for you, you also have the option to export the requirements in a JSON that you can easily copy into your own repository, and launch the job via CLI or API instead. Here, we will create the job. Once the job is successful, you'll get a check mark and a link to access the job. Let's take a look and see what that looks like. Within the interface, you will see the Metacappy job under this title.
The notebook will be attached, along with the relevant parameters that you have specified. Finally, there are key descriptions, and you can further customize the job as you see fit within the UI. To things out, the Meta CAPI app also s- records the deployed resources that you've have. So, to make it easy for you and your team to revisit any jobs that you have or notebooks that you have created via the application. So, you have a one-stop shop for your Meta CAPI connections.
With that, that brings us to the end of this tutorial. So, to do a little quick recap of what we did, we started in the marketplace listing, then we connected to Meta with a guided wizard. This allowed us to leverage the connector with no code required. We sent some conversion events using sample data all under a minute. And we also saw how to deploy a customizable notebook for more control and how to set an automated job within this application.
All of this runs securely inside your Databricks environment. Your data stays governed through Unity Catalog, credentials are stored in Databricks secret, and everything is auditable. If you want to try it yourself, search Meta Conversions API within the marketplace, and you can be up and running in minutes. Thank you for tuning in.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.