Sponsored by: Exa | Perfect Search for a World of Agents

Summary

AI agents will run approximately 1,000 times more searches than all humans combined within a few years — a shift that demands a purpose-built search engine for agents rather than adapting keyword-optimized engines designed for human browsing behavior.
Exa is a neural search engine for AI agents that uses transformer-based models to retrieve semantically relevant, complete, and structured results — serving over 5,000 companies and 4,000 developers including coding agents, go-to-market tools, and financial analysis platforms.
Agents need fundamentally different search outputs than humans do: structured, extractable data ready for reasoning rather than a list of links to click, and Exa is designed to return exactly this format with the precision and speed that autonomous agent workflows require.

Chapters

FAQs

Why do AI agents need a different kind of search engine than humans do?

Human search engines are optimized for keyword queries and ranked link lists designed to help a person click, read, and synthesize information themselves, but AI agents need search results that are structured, complete, and immediately usable for reasoning without any human post-processing. This video explains that agents running at scale — coding agents, financial analysis agents, go-to-market tools — need answers, not links, and they need those answers to be accurate the first time because there is no human to catch errors. Exa is built from the ground up to serve this use case with transformer-based neural search rather than keyword ranking.

How many searches will AI agents run compared to humans?

According to this video, the number of searches from AI agents will reach approximately 1,000 times the total number of human searches within a few years — a shift the speaker describes as the most dramatic change in the history of information retrieval. Humans currently run about 15 billion searches per day, and Exa's projections suggest agents will surpass that number before the end of 2026, then continue growing at roughly 10x per year as autonomous agent use cases proliferate. This scale shift is what makes building a search engine specifically for agents, rather than adapting human search tools, a necessary investment.

Who is using Exa and for what kinds of AI agent use cases?

Exa serves over 5,000 companies and 4,000 developers across a range of agentic use cases, as described in this video. Coding agents like Cursor use Exa to look up technical documentation and news during autonomous coding tasks. Go-to-market agents — including tools built on HubSpot — use Exa to find qualified lists of companies matching any criteria, where the speaker says Exa is state-of-the-art for precision. Financial companies including banks, hedge funds, and private equity firms use Exa to supply high-quality external data to their financial analysis agents.

What makes Exa more accurate than a generic web search API for agents?

Exa uses a transformer-based neural search architecture trained from scratch to understand the semantic meaning of a query and return results that are genuinely relevant rather than keyword-matched, which is how traditional search engines operate. This video explains that because human search behavior shaped the training data for most web indexes, existing APIs are optimized for click patterns and engagement signals that are irrelevant to what an AI agent actually needs. Exa's retrieval is designed to handle complex, multi-part agentic queries and return structured data that agents can consume directly without additional parsing.

Full transcript

[00:07] Well, I my intention is to make this the most exciting talk. Uh, and I'm very excited to tell you guys about perfect search for AI agents. Designing a nextgen search engine for agents. AI agents, not humans. You might be wondering what the hell does that mean? I will tell you. Now, I'm going to say a
[00:20] lot of crazy things in this talk. Um, it's it's all true. Um, and yeah, first I'll tell you about uh you know, what is EXA? uh and how we got there and then where we're going. So,
[00:33] okay, if you take away anything from this uh conversation, it is that the number of searches from AI agents will be a thousand times more than the number of searches from humans in a few years. That's a huge statement and it's true.
[00:46] Um this shows that the past 30 years of searches from humans and agents. Obviously, 30 years ago there were no agents. Um there were only human searching. Nowadays, you know, humans search on average 15 billion times a day. agents actually we predict by end
[00:58] of 2026 we'll search more than that. So first year that that happens and then you know it's roughly 10xing every year. So it's been crazy to see that um that's just what happens with exponential trends.
[01:11] Exa is a search engine for AI agents. So we organize the world's information uh and then serve it to all sorts of agents. You know we serve companies like cursor coding agents like cursor cognition. Um what do they use this for?
[01:23] Well, you know, if you're talking a cursor and at some point it needs to the agents like thinking writing code and then it's like I need to look up stuff from the web, it'll go use Exa under the hood. Um whether it's for the latest technical documentation or um or even
[01:35] news. Um we serve go to market agents like ton hundreds of go to market agents like HubSpot for example. So when sellers need to find lists of companies to sell to, we are by far state-of-the-art at finding lists of companies that match any criteria. We
[01:47] serve a bunch of financial agents. I was in New York a couple weeks ago and literally every financial company whether it's banks, hedge funds, private equity, they're all building agents and they need high quality uh data. So EXO provides that to them. We serve 5,000
[01:59] over 5,000 companies now and over 4,000 developers. Um and we recently raised our series C so $250 million and that means we're going to grow a lot faster. My plan and I'm CEO so I try to make everything uh run really fast. Okay. Um
[02:14] okay so that's the what of Exa. But what about the why of Exa? is like, you know, I've been doing this for 5 years. You know, why devote a life to this problem? To me, I think this is like the most important and neglected problem. Uh, and
[02:27] the problem I see is like misinformed anarchy. Um, is one way of saying it. Um, what does that mean? Well, you know, we have the world's information on the internet, right? This is this is a visual depiction of the internet. You know, we have uh, you know, p uh
[02:40] pictures, we have uh company pages, we have news articles, we have Reddit post, tweets, we have everything on the internet, right? And it's really really really big. Um this is this shows you a few thousand pages. The internet's
[02:52] actually trillions of pages. So a million times this. Imagine a thousand times this that way and a thousand times that way. That's the internet. It's very big. Um and like it contains all the world's knowledge or basically you know most of the world's information. Um and
[03:05] you're you guys must have like a true understanding of the world because it's all there, right? Obviously not. um you don't have a true understanding of the world and the reason is because our tools for navigating this massive amount of information are very limited, right?
[03:18] So we have social media is one way to navigate the world's information, the internet's information. Well, that's very biased and you only see what the algorithm shows you. You know, we have the news which is biased and very incomplete just like, you know, like CNN
[03:30] and stuff like that. And then we have search engines and search engines are pretty good, but they're certainly not perfect. Uh and so why are search engines not perfect? Well, look, look at some examples. So, this is my favorite example. If you search Google, Google's
[03:43] a great search engine, but if you search Google shirts without stripes, you get shirts with stripes. I always thought this was strange. Like, why is it doing that? Well, it's because Google is not built to be like a perfect search engine that gives you exactly what you ask for. It's kind of giving you some
[03:55] recommendations. It's a recommendation engine. Um, you could see this more with some other examples. I was in Singapore recently, so that's why this I use this example. everyone in Singapore who works on AI search and any blog post or research paper they've written. I would
[04:07] love to to have you know make to get the results for this query because we're trying to hire. Um but like you know you don't expect Google to give you a database response of all the people in Singapore who work on AI search right. Um so it's not organizing the world's
[04:20] information or even more importantly like if I want to understand the world because you know I want to know what's the most important US news across all media today. I'm not going to get that from Google. I know I'm not going to get that. It's just not built to be that kind of search engine.
[04:33] So there's all this information out there and you can't understand it. And so basically every that means that everyone walks away walks around in the world with incomplete understanding of what's going on. So sometimes when I'm walking on the street I look at people
[04:45] and I'm like they have no idea what's going on. Also I have no idea what's going on. No one knows what's going on in the world. Like there's too much going on. Like think about even just like in AI right now. Like do you guys feel like you have a deep understanding of what's going on in a way that's like all the things that are relevant to you?
[04:57] Like obviously yeah you're shaking your head. Um obviously not right. So that's what misinformed anarchy is. Uh why is it anarchy? Well, I do think if we continue on this path, it will be anarchy. And and why is that? Well, we
[05:09] all feel the acceleration that's happening right now in AI. Like everything's getting way more powerful. Like we have like godle level intelligence almost. And you know, this is a depiction of San Francisco in 2035 where our technology continues to
[05:21] accelerate, but our information tools do not continue to accelerate because then like you know, individuals have no idea what's going on. We make bad political decisions. you know, there's centralized power. Who knows? All the bad things happen when people don't understand
[05:33] what's going on. This is my view of the world. I think it's correct. I want to fix it. Okay, that's the problem. Uh what's the solution? The solution or one big solution is perfect search. Uh meaning if you could figure out an algorithm so that you take all the
[05:46] world's information and you could like no matter what query you give it, any of the example queries I gave, it just works perfectly. So when you ask for all the people in Singapore who work on AI search, uh it gives you all 342 of them. If you want to find all the US news
[05:59] that's like most popular today, it gives you exactly like the top 10 articles like across all media. Like you just have a deep understanding of the world. That's what perfect search means. And so we set out 5 years ago to build
[06:11] that. This is the first day of Exa 2021 is the selfie. Um and um yeah, we were you know a bunch of kids honestly like were thinking like okay we're going to spend years of our lives uh trying to
[06:23] solve this problem and to build perfect search better than Google which is kind of crazy at the time. Um and uh yeah it was a research problem right so like and we had to do crazy things in order to make this work. So like you know the
[06:35] first thing we do we did Y cominator we raised a couple million dollars and then we spent half of it on a GPU cluster which our YC partners were like you're insane but we thought that was necessary in order to do research. So you know to do crazy things requires crazy you know
[06:47] uh beliefs and uh we basically did research for like a couple years um into new ways of doing search. So 2021 it was like trans you know GB3 had recently come out so transformers were getting really good and Google felt so dumb in
[07:00] comparison. So the idea was like what if we could use the same technology is transformers and apply it to search and so we did a lot of research. These are just experiments that we ran on uh these are graphs of like loss functions and one of these was actually a breakthrough in retrieval that to this day no one has
[07:12] else has ever done or I've never seen any evidence of people doing it. So we we've done a lot of really novel research over the years. Um the high level idea that of Exa has always been like you have the web and
[07:24] like old school search engines they basically do more like keyword matching. So you type in like shirts without stripes it'll it'll see shirts and stripes and they give you shirts with like stripes, right? like a shirt that has the word a web page that has the
[07:36] word shirt and stripes will match that query shirts without stripes but it's not really capturing the meaning right but when you want shirts without stripes you need to understand the meaning of the sentence and for that embeddings are a lot better um so embedding is basically like instead of turning a
[07:48] document into a set of words you turn it into a set of numbers that capture like the semantic meaning of it and X has always been like super embedding like we are like like on the frontier of embedding research and and we do crazy stuff here and that's basically what
[08:01] allows us to do like to handle searches even better than Google, right? Okay. Uh well, as we were building our search engine, we finally we launched our first thing in November 2022. Uh we called it we were called Metaphor at the time. Um and it got a
[08:14] lot of excitement on Twitter. People were like able to search things like, you know, find me all the people in in this space and get like a list of people which is very different from Google. It was one of the first times is in 2022 that like people were able to find information in a new way beyond Google.
[08:26] Um and then a couple weeks later this happened. So Chashb came out. Um, and that was a very different way of finding information. And so like San Francisco at the time, if you remember, was like felt like this. The Exa team at the time
[08:38] felt like this. We were like, what is the purpose of search anymore? Um, and then, you know, a few uh a couple months later, uh, as we were thinking about the future of our company because like the world's information was so new with with AI tools, someone asked us for
[08:51] an API uh like API access to our search engine. This is the first time someone asked me that and I was like, why are they asking me for an API? This is actually a picture of of when they asked it. Um, and then we quickly realized that like, oh wait, search is extremely
[09:04] important in this world. In fact, it's probably more important because these AI tools like GB4 at the time, they don't they don't have access to the world's information. Um, why is that? Well, G like GB4, GB5, they're very small. Like I mean, they're giant, but like in
[09:17] comparison to the internet, they're tiny. Um, the internet is freaking huge and constantly changing, right? Like so no matter what, these LLMs, they need to connect to the world's information. They need a search engine. And that was a really interesting insight. Um, so when you connect an AI tool to a search
[09:30] engine like like Exa, in this case, it's using Exo under the hood. Now, it could actually handle the the response. Then there there were more interesting things we were thinking about at the time like, wait, okay, so these AIs need a search engine, but do they need the same search engine? Should we just
[09:42] connect them to Google? And very quickly, we realized, no, you don't want to just connect them to Google. Well, first of all, Google doesn't have doesn't expose an API, so you can't do that. But, you know, you could scrape Google and serve it to the AIs, and people do that. But like, we're like, that's not optimal. Why? Because when
[09:54] humans search, they look like this, right? They're lazy creatures. They type in a few keywords, they get a few links. Google was optimized for this creature. When you're trying to serve this AI world, these AI products, right? Like things like imagine cursor or like right
[10:06] like in your head. Imagine cursor. That's what an AI uh is. It's like this crazy thing that could gobble up information extremely fast. Like it would be crazy if the same search engine for lazy humans was optimal for this creature. And there actually like 20 different like major ways why search
[10:20] optimized for humans and search optimized for AI are very different. For example, like a search for AI is like you want to be able to handle complex queries because the AI can instead of a few keywords, it types in like like the queries we get are like like paragraph long often like very complex queries. Al
[10:32] also like an AI can gobble up not like one or two links, it could like gobble up a thousand documents. So it wants like as much information as possible. So X is designed to return as much information as you possibly need, right?
[10:44] So there's these are just two of like literally 20 different ways that AIs and humans are different. Um anyway, so we we we in 2023 we built the first API uh for for AIS, first search API for AIs. That was a really weird uh thing at the
[10:56] time. Investors were like, "What are you doing?" Um but we knew it would grow fast. Um and anyway, skipping to 2026, our search engine is obviously a lot more complex than it was in 2022. We built out a huge number of things, everything that an AI system possibly
[11:09] needs. The team is a lot bigger now. This is actually half the team. Uh since this photo a few months ago, we've doubled. So, we're growing extremely fast. Uh and and that's what brings us here today. Okay. Now you know how we got here, what the problem is. Okay, now
[11:22] um going forward a little bit, well before I tell you where we're going, I'll just tell you like the current the present. So the present right now state of Exa is where the highest quality information for uh for AI agents, which is pretty cool. Um and a lot of companies, you know, use us and and like
[11:35] us and if any of you are building AI agents, would love to to chat. Um so what are things you can do with X right now? You can make very complex queries and it just works. So like every startup funded by YC working on AI, give me their batch and status. That's pretty
[11:47] cool. Um, you can make this as complex as you want and it'll just work. It might take a couple minutes, uh, but it'll do like a crazy amount of of research, tons of searches under the hood and give you a big list. So for any asynchronous use case where you just
[11:59] need like literally all the information, this is great for you. On the flip side, so that's a few minutes. On the flip side, we have on the flip side of the spectrum, we have the fastest search API in the world. So faster than Google. So we have a sub 200 millisecond search
[12:11] engine and it feels like that. It's extremely fast. It's too fast for humans, right? Well, we're not serving humans. We're serving AIS and AIS um like for example if you know we have customers who are building voice agents and when you're when you're talking to a voice agent it uh has to do a lot of
[12:23] processing so it doesn't want to wait a second for the search it needs to have the search extremely fast so for any real-time application this is very valuable other things we have are like you know
[12:35] extremely basically we save people money this is actually a saving money graph what does that mean why is it saving money well if we're if you're serving uh any LLM it uh you're charged based on tokens used, right? And so if it's consuming information from the web,
[12:48] well, you don't need all the tokens in this web page for your AI. You just need the most important ones. So we put a lot of research like like over a year of research into token extraction. Like how do you extract only the most valuable tokens kind of like Google snippets but
[13:01] on steroids. Um and so that's great. People want to save money. There's a compute crunch right now. That's good. Uh and then lastly, like we we've recently released this thing called agent. One way you can think about it is like we basically turn the whole web
[13:13] into structured data that you could get uh like whatever data you need. So for example, yeah, if you're doing if you have a recruiting AI and you want like give me every engineer in San Francisco who you know recently left their job and then give me like what job they left
[13:26] like how many years out of school are they any sort of data you want we'll we'll give it to you and this is very convenient for um a lot of companies. If you take away something from our current product, it's very flexible,
[13:38] right? Like some people want like uh the highest quality search that takes minutes. Some people want the fastest search. Some people want to f want to search only over like a thousand domains of the web. Some people want to search never over a thousand domains. Some people want to search only over recent
[13:50] data. Some people, you know, everyone wants something different because we're we're not serving two billion consumers, human consumers who are all kind of the same. We're serving, you know, thousands of companies who all want something different and are willing to optimize their setup. And so another thing about
[14:02] serving AIS or really serving AIs at businesses is we're building a search engine for each company in some sense. And so it's very toggable which is cool. It's also a beautiful thing to have like a very um customizable search. I think
[14:14] perfect search needs to be customizable. It's like perfect search is very personalized. It's I we don't want to be the determiners of what is truth. We want to let you determine what your truth is. Okay, cool. And yeah, like um because
[14:26] we've invested so much in this technology and we were so early and we have a really smart team, we're able to like be the top of a lot of benchmarks. Um but you can see that on our website. Okay, so that's where we are. But the future like we're certainly not perfect
[14:38] information yet. Perfect is a never- ending quest, but um uh but we're we're getting closer and closer each year. We're moving extremely fast. So yeah, again, this is the problem. It's chaotic, humongous web. We want to give
[14:50] you like full access to the world's information. you know the the like any query no matter what it feels like a database you get literally all the data you never miss out on any information it's real time all these things so this
[15:02] it should feel a little more like this it's hard to get across what it feels like to have perfect search I don't even I can't even like fully understand it it's kind of like trying to imagine what AGI is like but this is what I think it feels like just surf like the web is just like uh this is inspired actually
[15:15] by um Iron Man Age of Ultron where it's like surfing the navigating the web okay um but yeah in um we're moving really fast at Exa. Um, by 2027, I do think it will feel a lot more like you have
[15:27] perfect access to the world's information. Um, because of Exa. That's the goal. Like by 2027, I tell the team like we need uh it to feel like that. Also, in particularly important is because in 2028, the presidential election is coming up and like a really
[15:39] cool motivating factor for the team is like we want to help make sure everyone's extremely informed for that election. We also have the midterm elections coming up in November, I realize, and we should
[15:52] 2026. Um but yeah, this is the goal. Everyone should be walking around with like a full understanding of the world. And then yeah, coming back to this picture like the like what's coming is really crazy like we're only in the early days. Um you know, human AIs are
[16:05] searching roughly on the same amount as humans. They will search thousands of times. One way of think about this is like everyone's going to have an AI assistant. every software product you use is going to be an AI product and every time you interact with it, it's going to be searching under the hood, maybe multiple times. And so like humans
[16:18] on average search a few times a day on Google, but you're going to be interacting with your AI assistant like hundreds of times and each time it might be doing like a couple searches. So that's literally a thousand times. Um, and so it's kind of it's kind of got to
[16:30] feel like um you know how like you could plug your phone charger in anywhere, it's like electricity is everywhere, right? Kind of like uh high quality information will be everywhere infused into all our devices. It's a very different world, right? Um the current world we live in is like search is like
[16:43] you go to Google and you search, right? Instead it's going to be like search is in infused into like every uh software product we use. Um and if we do it right, it should feel like this. Um where uh we we basically like enable
[16:55] everyone to really understand the world. We make better decisions as individuals as a society. We actually understand what's going on. Right? There are a lot of problems in San Francisco. I think they're all political problems. What are political problems? They're like coordination problems. And I think if
[17:07] everyone has true access to what's going on, I think we could solve a lot of the stupid problems that are happening. These are not hard problems. These are coordination problems. And I I very much look forward to this kind of world. And just one last note, um maybe a nice
[17:19] story. Uh I was talking to my mom like 10 years ago on the phone and uh because like this is like 20 20 2016. Uh this like neural networks were getting really good and I was like, "Mom, like I'm I'm freaking out a little bit. Like AI is
[17:31] getting really good. Like I I think this is like the most important couple decades in history. like we all need to do our part, mom. Um like I'm trying to like be a software engineer and help people and be more productive. That was what I was interested interested in at the time. I thought that was important. And my mom's a lawyer. I was like, "Mom,
[17:43] you're a lawyer. I don't know how you're going to help, but like she's like, "Yeah, I'm a lawyer. I don't know what I'm supposed to do." And then I we got off the phone and then and then I thought about it and I called her back and I said, "Mom, like you're a lawyer. Like if we if lawyers went away, it
[17:56] would be way more chaos than if software engineers went away. like you make sure that humanity like is stable uh as we transition into this crazy sci-fi future. And she felt very good about that. I think the lesson here is that
[18:08] like literally any we all have a role to play in this crazy world like it's the most historic moment ever. Um and so like literally any way you can help society like be functional and prosper before this uh transition happens or as
[18:20] it happens is important. Okay, that's talk. Thank you everybody.

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.