From Dashboards to Data Products: MBC Shahid's App-Driven Revolution

Summary

MBC Shahid, the world's leading Arabic streaming service with 35 million monthly active users, 9 billion player events, and $370 million in annual revenue, evolved from static dashboards to a curated data product culture built on Databricks Apps, Model Serving, and Unity Catalog.
The Enigma app delivers conversational analytics — combining predictive show performance, real-time sentiment analysis, and voice-of-customer insights — to non-technical media buyers and leadership teams via an architecture of AI Gateway, agents, Genie, and vector search.
A GenAI video pipeline called Spectra Studio automates viral social media reel creation by identifying high-impact frames and flagging sensitive content, while the serverless app-driven architecture is saving MBC Shahid thousands of hours across the business.

From Dashboards to Data Products: MBC Shahid's App-Driven Revolution

MBC Shahid, the world’s leading Arabic streaming service serving tens of millions of users, has evolved beyond static dashboards to a curated Data Product culture. In this video, discover how we leveraged Databricks Apps, Model Serving and Unity Catalog to operationalize AI for non-technical teams. We will showcase two game-changing internal apps: “Enigma,” which empowers media buyers and leadership teams with predictive show performance, an actionable, interactive real-time voice of the customer and sentiment analysis experience, and a GenAI social media tool that automates viral reel creation by identifying high-impact frames and flagging sensitive content. Join us to learn how adopting a serverless, app-driven architecture is saving thousands of hours, merging the worlds of traditional tabular data, clickstream events, qualitative customer feedback and video data, all on Databricks Lakehouse, while ensuring governance, and delivering massive business value in record time.
Talk By: Mehdi Wissad, Senior Solutions Engineer, Databricks ; Phillip Badenhorst, Director of Data, AI and Strategy, MBC Shahid ;

Chapters

FAQs

What is Enigma and how does it work?

Enigma is MBC Shahid's conversational analytics platform built on the Databricks Data and AI platform, designed to give media buyers and leadership predictive show performance insights and real-time voice-of-customer sentiment analysis. Its architecture uses an AI Gateway, AI agents, Genie for data exploration, and vector search to deliver quantitative and qualitative per-episode insights.

How does MBC Shahid use AI to automate social media content creation?

MBC Shahid built Spectra Studio, a GenAI video AI pipeline that identifies high-impact frames from existing content and flags sensitive material to automate the creation of viral social media reels. This reduces the manual effort required for content repurposing and helps the social team move significantly faster.

What Databricks products power MBC Shahid's data platform?

MBC Shahid's data products run on Databricks Apps, Model Serving, and Unity Catalog for governance. The architecture also includes an AI Gateway, AI agents, Genie spaces, and vector search to power conversational and predictive analytics across the business.

What business results did MBC Shahid achieve with its app-driven data strategy?

MBC Shahid reports saving thousands of hours by adopting a serverless, app-driven architecture that unifies tabular data, clickstream events, customer feedback, and video data on a single lakehouse. The team can deliver MVPs in one to two weeks, enabling rapid business value across media buying, leadership, and social media functions.

Full transcript

[00:08] Uh good afternoon everybody. I just want to check you can hear me right. Uh I'm Philip. Um this is Medi. Uh we'll be taking you through MBC's journey from dashboards to data products.
[00:24] >> Uh yeah. Uh for those that are you of you that are not familiar, NBC Shahed is basically the Arabic version of Netflix. So we're the biggest streaming platform in the Mina region, both from a paid subscription and a free subscription or a free service. Uh where we go up to
[00:38] about 35 million monthly active users. Um as you can imagine, we do collect quite a substantial amount of data from these people. Um, so we do about 2.5 billion hours of viewing per year, 9
[00:50] billion player events, and about $370 million in revenue each year. I'm going to take you through the journey that we went through in the last couple of years, sticking to the media theme theme of essentially moving
[01:03] through it in seasons. Uh, we started off as most I think of us and and most companies essentially moving a traditional data warehouse into a modern lakehouse architecture. Season one for
[01:17] us was just basically replicating what we had in the data warehouse doing advanced BI and analytics but in the end of the day this was primarily structured data into newsletters and dynamic
[01:29] dashboards. There there was one big decision that we made at this time or one big framework that we adopted which has translated very well into the um the season where
[01:43] we are now 3 four years later down the line. Um and that was to adapt the concept of essentially building data bricks. Uh you can think of each of these colors as a use case. Um and we
[01:55] adapted this principle of building both vertically and horizontally at the same time. So every use case that we built had to reach real world value uh because we all wanted to stick around for a couple of years and to do that we had to
[02:07] actually deliver value in the first few years. Uh but we also built out horizontally. Every brick that we built had to go into the next use case and help us move faster and pick up velocity as we go. Back in the day, uh this would
[02:20] have been things like deploying a data bricks feature store and making sure if you do one calculation, one ML model output, that output gets saved in a non or unambiguous way and is usable for the next next person down the line. Uh in
[02:33] today's world, this would be things like skills for the agents uh to be able to essentially reuse this across the business and pick up velocity as you go instead of picking up technical depth and just creating more maintenance down
[02:45] the line. uh we season two for us um I think same as most people we would have gone down to essentially moving from structured
[02:57] data into unstructured data into event streams at this point schemas became more ambiguous it changed if if a developer change the tracking on one of our apps uh so we have about nine different app major apps platforms as
[03:11] you can imagine on the OTT side you have smart TVs you have gaming consoles mobile apps there's quite a few places that developers can track events into your platform. Uh this did give us very fast evolving schemas. Uh it changed the
[03:24] world a little bit for us. Uh but again on the lakehouse this was quite easy to handle in the end of the day and we evolved the team at this point from being essentially a very advanced BI team reporting on structured data into
[03:37] being a team that had actual impact on a customer level. Uh it was at this point that we set up our customer 360. We set up audience integrations to I think about 13 different paid media platforms and our own CRM system. This allowed us
[03:51] to retarget customers with all the data we had in our lakehouse. Whether it's churn prediction, content clustering, basic RFM scoring, all of that became available in a no code way to integrate into all of these platforms allowing our
[04:04] marketing teams to retarget customers across the region. Uh we also developed a service that does push notifications and emails. Everybody likes getting down to individual customer levels instead of
[04:16] cohorts. Um, so we send about 500 million push notifications and emails to customers with individualized recommendations. Essentially tracking your journey as you view content through our platform and
[04:28] making sure we manage that journey. Uh, obviously to reduce churn and increase our customer lifetime value. Uh, this brings us to season 3. Medi will shortly cover a bit of the
[04:40] architecture behind these. Uh but this is the the point where we are now where we've moved past being a BI analytics function into uh data bricks apps powered function where we've completely
[04:52] removed the limitations of what we can do uh to a large extent based on the capability we have received from data bricks and the data bricks app specifically we have been able to evolve into building products for the business.
[05:06] Um a couple of these ones we will show to you now. one being Enigma. So this is a conversational analytics platform which is powered by all of the data we have on the left hand side being the structured data unstructured data from
[05:18] season 1 and two as well as free text customer comments. So for the first time we are actually able to do qualitative analytics and qualitative insights at scale. We we are not talking about samples of a few hundred people done by
[05:30] a consultancy. We are tracking tens of thousands of comments per title or per series for every single show that we have. Um, and these are available to our internal people now to use and analyze
[05:43] and ask questions of in a conversational way, in a natural language way and get that merged up with the quantitative data we had from the first couple of seasons. Uh, we'll go into a couple of views on that now. Uh, just before we
[05:56] get to that, I'll just cover the structure we've put in place to develop these apps. uh as we have evolved past this function that did analytics into more of a sort of a development world we have created
[06:10] what we call FTE pods um so this takes a bit of the ownership of these projects and these apps away from us to some extent uh it puts a business sponsor in charge so we solve an actual problem
[06:22] that these departments have instead of coming up with a problem from our side that we think we want to solve we would get a problem from them to solve we'd have a pot lead that owns delivery of these apps. Um, and then we'd have these
[06:34] teams filled out with with between two and four engineers. We have migrated a lot of our analytics specialists and data engineers into AI engineers. Um, and then we still fill in with more data engineers as required.
[06:46] These are fairly short projects. So, we're looking at a week to two weeks just to get MVPs out and basically a couple of months to get working products out that has real impact in the business. Um we'll go through a couple of those
[06:59] apps that we've built now. Um the first one being Enigma. So this is where we're bringing quantitative plus qualitative plus context. Uh which is giving us very very rich insights compared to what we
[07:11] could do a couple of years ago. So this is an example of questions being asked of a title. Uh all the numbers are fake just for um confidential
[07:23] confidentiality purposes. But essentially you can go a few steps past sentiment analysis which was done quite some time ago or has been a have been capable to be done quite some time ago
[07:35] and go into much more detail. So you can let's get this running. You can go a couple of steps further. So we use this for example to understand what are resonating from our shows with people what are not resonating where we are
[07:48] missing the mark um and what people want to see more of. So this would go down to like a character level, a scene level, episode level because all of this is linked back to the actual performance data. Um, and
[08:02] now powered by Genie, you are able to actually ask questions like in season or episode between episode 8 and episode 9, our drop off rates of customers has increased compared to the previous episodes. What happened? That will then
[08:14] look up the free text customer comments from Twitter, our own internal shorts platform, Facebook, etc. and give you an analysis of what people are saying. Why did they drop off between those two episodes? What changed
[08:28] uh go I'm going to we'll cover the architecture now from each side. >> Yeah, thank you very much. So I met with solution architect working closely with MBC team. So basically Philip was
[08:42] mentioning earlier the FD motion. So I worked closely with those FDEs. So they went they met business they talked to them lived lived their life saw what is
[08:55] the burden in their day-to-day and try to solve this with actually uh real agents in production. So in order to bring agent outside from the data bricks workspace uh uh we were working closely
[09:09] to build apps uh such as this one. So as I was as it was was explained by Philillip uh it's an app that is linked to like different resources in datab
[09:21] bricks and the the big picture is like a custom datab bricks apps but what is behind the scene is like a full governance. So we have unity catalog we
[09:33] have AI gateway so the the user ask a question uh will get only answers of only the scope that he has access in the lakehouse. So you can see here like a full picture. There is AI gateway as
[09:46] like the first layer. The second layer is the agent. So for the agent we choose like LLMs. We have been trying several uh balancing everything regarding cost versus performance. There we tried with
[10:00] the Gemini models with open AI models on tropic models. Uh we had everything by doing experimentation. So we have like uh down in the slide MLflow training and
[10:13] inference table. So we were doing some evaluation based on what is the quality of the answer, what is the cost of the answer so that we can have the right balance in term of like how good is the
[10:25] answer. You notice that we have been plugging the agent with different capabilities. So first of all we start with vector search where uh MBC are collecting data from different sources
[10:38] from uh Facebook, Tik Tok like all the platforms. So once you have this ingested creating a vector search and using uh AI queries you are able to create really powerful data set that
[10:51] afterwards you can just like query saying for example what are the top 10 comments regarding this show and you will have the answer. The other break inside the agents is like Genie. I think
[11:03] almost all of you did experiment at least on how good is Genie with tabular data. So all this intelligence is actually mapped and tried with the agent so that you get actually the best output with a nice app that you can give to
[11:16] business. It's not only data that is sitting in data bricks but it's actually serving other people uh using the the app itself. So over to you Philip for the next app. Thanks.
[11:33] >> So, one of the limitations we ran in quite quickly when we started doing this is so we most of our analytics is based in PowerBI at the moment is getting to a level of what we can actually display or what we want to display um is much more
[11:46] dynamic than that than that would be allowed in traditional BI tools. Um so let's control that from here. So we started using Genie code as well as our internal data and enigmas analytics
[12:00] to build out completely interactive views. Uh this removed the limitation that you have with BI of a couple of different visuals that you can implement. Um and it created a much more interactive and honestly better looking
[12:12] interface that we use to re-engage our internal consumers in the business. Um so desperate for example just picking one title this is a combination of internal episodic decay data uh just
[12:24] dummy data for the time being um mixed with sentiment from what customers are actually saying about the title. This show if we do a season 3 for example this will feed back directly into the greenlighting process of do we do a
[12:37] season 3 based on the performance and what people are saying. Do they actually want a season 3 or do they think we've kind of lived out the title? And then what characters are resonating with people? Who do they want to see more of, less of? Um, and how do they feel about
[12:51] the pacing of the show? Is it too fast? Is it too slow? Is it drawn out? It gives us much better insights if you just look at some of the examples there as opposed to just saying this show had a lot of acquisitions. It had high engagement. That's great, but how does
[13:04] that actually help us decide what to do next? Um, so this gave us really good in input into what actually drives the performance as opposed to just saying the title have performed.
[13:19] >> Yeah. Yeah. And actually for this specific app you have this kind of architecture. So basically what you saw earlier is just the HTML but the HTML is actually embedded in the app. And one other important thing is that we have
[13:33] like a text box for the end user. Here we is just asking simply I want a dashboard an interactive dashboard giving me a 360 view report of my engagement and it's actually triggering
[13:45] an agent behind the scene that is connected with uh first a well curated genie space that gathers all the data regarding engagement. So we can think of like comments enriched actually with the
[14:00] sentiment analysis with like what are the topic discussing those comments and once you have this ready it's almost natural because you have an orchestrator agent that first generates a series of
[14:12] question saying that okay in order to build a 360 report of the engagement I first need to check the engagement data set and ask this specific question regarding for example the last two months on this specific show. It will
[14:24] generate for example five six question. I will then gather all the answers and send them back to my orchestration layer and the orchestration layer is able to
[14:36] collect all these KPIs and metric and then it becomes naturally to generate an HTML with the KPIs with the pictures that you saw earlier that are stored in
[14:49] a datab bricks volume and then rendered in the app. So we are giving to the end user an experience instead of going to the BI analyst asking them to build some custom dashboard it's quite easy to now
[15:01] just with prompt with the rate data in place to have like an end toend view similar to what you saw earlier. So last one I think is like the most impressive so far because now we have been talking
[15:14] about structured data and structured data but the next app will be talking about Philip over to you. Thanks. >> I think one of the use cases we had for this initially or the reason we built
[15:26] this, I think we can all I'm probably not alone in saying we're kind of sick of building PowerPoints. Um, and AI is much better at building HTML than building PowerPoint. So, it helped us replace a lot of the decks we had to build by just generating really nice
[15:39] interactive visuals that also came across much more impressive than just another deck. Um, so Spectra Studio is what we're working on at the moment, which is a new tool we've built. Uh as Medi mentioned previously we worked
[15:51] primarily with different sets of kind of traditional data as well as in freex data. Um but we are in we are a media business we work on video. Um so we didn't want to stop at that. Um so we've built this app. This is B built on data
[16:04] bricks built on the lakehouse as a data bricks app that essentially takes the titles we have flowing in. Um so we'll use an example from friends here but mostly in our world it's Arabic. So it works across all the different dialects and languages and does an analysis of
[16:18] the titles as it comes in. Uh this allows us to do one pass through of a video. Um as you can imagine we have quite a lot of long running shows coming through. Uh which can get quite expensive if you process all of that with AI and you do that over and over
[16:30] again. Um so we do one pass through that helps us generate shorts. We have a Tik Tok like feature in our app. Uh we do compliance with this as well uh to make sure there's no scenes or things that
[16:42] would not be acceptable in the regions that we operate. Uh we do subtitling um we do content metadata metadata generation all with one pass through that that significantly lowers the total cost of ownership of running these kind
[16:54] of models. Um and then I'll jump through to the actual video uh or the architecture and then we'll go through the video. >> Yeah. So I don't want to take a lot of time but like we had like FDs go into
[17:06] business right they noticed that the team to generate a specific short first there is like a friend uh episode you have to watch it fully to know which part you want to take but yeah maybe
[17:18] friends it's I want I will enjoy personally watching it but if I had to do short generation for like a full season or for like seven seasons and for each of the season I need to create a specific short that might be very time
[17:32] consuming but now we are really solving this problem with AI so we are bringing value with AI and not like some random cool AI project we are shortening the time of delivering of those shorts so
[17:44] actually just to give you an idea what is the stack I don't want to go in detail I will leave the video explain a bit what is the user experience but it's actually uh user input a video with for example a full episode it's stored in a
[17:57] datab bricks volume right And this volume has uh get we get the metadata from whisper. So it's a speech to text model. We get all the transcription.
[18:09] Once we have the transcription if you give this to AI it's instant. It will get tell you for example from this minute to this minute it's like a dramatic kind of scene and you can already take the video
[18:23] cut it into pieces and then have open CV to get uh because uh I don't know how familiar you are with shorts but shorts are like a specific format so it's not a wide format it's really rectangular so
[18:37] if I want to cut the scene I really need the faces uh exactly in the center so I need this tracking. But once I have all these pieces, I can really quickly generate this kind of uh 40 second, 30
[18:50] second videos, then save them, bring it back to the user and let the user get the feedback from it. So I will leave you now with the video. Uh it's
[19:20] So the platform essentially ingests all of the assets that we have on our system. Um and then process this with LLMs and creates these metadata fields that you are seeing here which gives us much richer metadata that feeds into or can feed into our recommendations engine
[19:32] as well as our decision- making. uh compared to just capturing a couple of genre and kind of manually capturing a couple of points as the titles are loaded in the system.
[19:48] As you can see, it then generates all of these different clips. Uh this is based on a criteria that you set. So we have titles that obviously we have comedy, we have drama, but for different titles you might want to create different kind of shorts. Um, this runs via data bricks
[20:04] and then it creates and identifies these kind of iconic scenes for you. As Medi mentioned, the biggest challenge in this is actually going from a widecreen contrast ratio to a typical phone screen which is vertical videos. So you have to
[20:16] identify the scene and then be able to get the right part of that scene at the right point in time. This is never perfect. Um, that's why you can see those little dots that allows you to skip between the to manually select
[20:28] between the different people that have been identified in the scene. Um, because otherwise you'll try generating these things over and over again trying to get it perfect, but you start from scratch every time. So, we've built this functionality that allows you to control
[20:40] the interface as well. Um, as you can see, we'll then reframe it, create this, create the episode or the short clip that you will see shortly. Uh, we also do compliance of this. uh we we operate in a huge amount of different
[20:52] regions around the world. Different regions have different rules as to what you can and cannot do and can and cannot show. Um so this allows us to manage our risk internally by immediately flagging anything both in the shorts that we generate as well in the as in the long
[21:05] form and making sure everything we have is compliant which avoids kind of issues for us as we broadcast these titles. Um we also do subtitling in quite a few different languages. This is in the same
[21:17] sense as in the previous um clipping or reframing section. You are able to edit these and tweak them and move them around um to keep the human in the loop and make sure you don't end up in a situation where you keep redoing things
[21:29] but you have to start from scratch. You get you are actually moving towards the end state quite quickly. Then
[21:42] >> of my life goes home. >> That's what I'm thinking. Should I use my invisibility to fight crime or for evil? >> Cool. Yeah. And then as you can see, this is quite good at identifying the
[21:54] right scenes, the iconic pivot scene from earlier as well as the right scenes in this episode that really do resonate with people. Um, and as as many mentioned, this was a a project done with a team by sitting with that team as
[22:06] an FTE team as a partner saying, what's the ch the challenges you have in the business? um and then sending basically a special forces team in to say go solve that problem for them. In Ramadan we publish around 50 titles per day, 50
[22:18] episodes every single day. Um so as you can imagine you can't have people sitting, you can't even literally watch through 50 episodes on your own per day. Um and this allows us to take that 50 episodes, process it, caption it, get shorts in all in the same day in a very
[22:30] short period of time. Thank you very much. I just want to highlight my team as well that these are the heroes that all the did all the actual work behind all of these ones and thank you Mie. Thank you. >> Thank you very much. >> Cool. Thanks.
[22:45] >> Um, >> yeah, we just made it. If there's any questions, you're welcome to come chat to us after um and have a discussion. Thanks.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.