Remember when scrubbing through a two-hour recording meant dragging a playhead back and forth, squinting at thumbnails, hoping you’d recognize the moment you needed? I spent an embarrassing chunk of 2019 doing exactly that, building a tool that tried to find “the part where the demo breaks” in screen recordings. My approach was crude: transcribe the audio, keyword-match, jump to the timestamp. It worked maybe 40% of the time, and only when someone said the thing out loud.
Clipto just got valued at $250M for solving a version of that problem at terabyte scale, using AI to search video. TechCrunch reported the number this week. And my first reaction wasn’t envy, it was recognition. That’s the wall I hit. That’s the wall most people building media bots hit.
Why video search breaks the usual bot playbook
If you’ve built retrieval systems for text, you know the pattern by heart. Chunk the documents, embed the chunks, stuff them in a vector store, retrieve on similarity, feed the results to a model. It’s a solved shape. You can stand up something useful in an afternoon.
Video refuses to fit that shape, and it refuses in several directions at once:
- There’s no natural chunk boundary. A paragraph ends. A scene sort of trails off. Do you cut on shot changes, on fixed intervals, on speaker turns? Each choice throws away something you’ll want later.
- The signal lives in multiple channels. Audio, spoken words, on-screen text, visual composition, motion. Query “the moment the crowd goes quiet” and you need at least three of those working together.
- Storage economics are brutal. A terabyte of text is a library. A terabyte of video is a long weekend of footage. Whatever index you build has to be dramatically smaller than the source, or you’ve just made the problem worse.
- Latency expectations are unfair. Nobody tolerates a search box that takes 90 seconds, even when it’s searching something that took 90 hours to record.
That last one is the killer. Users have been trained by text search to expect instant. Video doesn’t care what users expect.
What a $250M valuation actually signals
I’d read the Clipto number less as a bet on one product and more as a bet on a category finally becoming tractable. And the surrounding news from this week supports that read. Google’s AI Mode can now track flight prices and help book hotels, which means the assistant layer is moving from answering questions to operating on live, messy, real-world data. Micro1, an AI data startup, hit a $500M gross run rate on the training boom. Meanwhile Relay, an AI automation startup, shut down entirely, with its staff joining Google’s Chrome team.
Put those together and you get a picture of a market sorting itself out fast. Data infrastructure is printing money. Assistant surfaces are consolidating inside big platforms. And generic automation startups are getting absorbed or closed. The ones raising at real valuations are the ones sitting on a hard technical problem with an obvious buyer.
Video search is exactly that. Media companies, sports leagues, security operations, education platforms, anyone with an archive they can’t actually use. The data exists. The retrieval doesn’t. That gap is the business.
What I’d steal for my own builds
You don’t need $250M to apply the lesson. Here’s what the shape of this problem taught me, in practice:
Index at multiple granularities. Don’t pick one chunk size. Build a coarse index for “which video” and a fine index for “which second.” Query the coarse one first. It’s cheaper and it eliminates most candidates before you spend real compute.
Treat transcription as one signal, not the signal. My 2019 mistake. Transcripts are cheap and useful and will never find the shot where someone’s face falls. Combine text embeddings with frame embeddings and let retrieval score across both.
Precompute aggressively, query lazily. Every second you spend on ingest is a second you don’t spend at query time. Users forgive slow uploads. They don’t forgive slow search.
Make the timestamp the primary key. Whatever you return, return it as a jumpable moment. A search result that says “it’s in this video somewhere” is not a search result.
The uncomfortable part
There’s a tension in this week’s news that I don’t think resolves cleanly. Google also gave publishers a new way to fight AI-driven traffic losses, which is a quiet admission that better retrieval has costs for whoever produced the source material. Video search will hit the same question. When a bot can extract the one useful clip from your three-hour archive, does anyone watch the archive?
I don’t have a clean answer. But if you’re building in this space, that question is going to find you, probably right after you get the technical part working. Worth thinking about before then.
🕒 Published: