# Your AI Agent Can't Watch YouTube. Here's the Fix.

> Give your agent YouTube videos as searchable, timestamped context in about 20 lines of Python, with speech and visuals indexed and evidence clips compiled.

Category: Tutorials
Published: 2026-08-09

---

Give an agent a paper, a blog post, or an entire website and it's brilliant. Give it the URL of a 40-minute conference talk and it shrugs. The fix: an AI agent "watches" a YouTube video by ingesting the URL into video infrastructure that analyzes both what's said and what's shown. That infrastructure indexes the results with timestamps and exposes semantic search. The agent gets back not just answers, but the exact moments that prove them, playable as a clip.

## The stack everyone builds first (and regrets)

yt-dlp to fetch, Whisper to transcribe, ffmpeg to sample frames, a VLM to caption them, a vector DB to store both, plus glue code to keep timestamps aligned. That is five dependencies, two of which break when YouTube changes something. And the output is still just text *about* the video.

The failure isn't any single tool. It's that you're rebuilding video infrastructure to answer one question.

## The 20-line version

```python
import videodb

conn = videodb.connect()
coll = conn.get_collection()

# 1. Ingest straight from the URL
video = coll.upload(url="https://www.youtube.com/watch?v=LPZh9BOjkQs")

# 2. Understand both channels: speech AND visuals
understanding = video.understand(analyzers=[
    {"type": "spoken_words", "name": "transcript"},
    {"type": "vlm", "name": "scene",
     "config": {"prompt": "Describe the visual content, including any text, charts, or product shots on screen."}}
])
understanding.wait_until_complete()

# 3. Index
t_index = video.index(name="transcript", source=understanding.get_analyzer("transcript"))
s_index = video.index(name="scene", source=understanding.get_analyzer("scene"))
t_index.wait_until_complete()
s_index.wait_until_complete()

# 4. Ask
results = video.semantic_search(
    query="pricing or cost claims",
    index_ids=[t_index.index_id, s_index.index_id],
    top_k=5
)
for shot in results.get_shots():
    print(f"{shot.start:.1f}s-{shot.end:.1f}s")

clip_url = results.compile()  # playable reel of just the matching moments
```

The VLM prompt is yours, so it works as a schema for what to extract. Search runs across speech and visuals together, so "the part where they show the dashboard" works even if nobody said "dashboard." And `compile()` turns search results into a watchable clip. Your agent cites video the way a good analyst cites sources.

## Turning it into an agent tool

Wrap the flow as a `watch(url, question)` tool that returns timestamped moments plus an evidence clip. Or skip the wiring entirely: `npx skills add video-db/skills` gives Claude this capability as tools, and the same primitives exist as n8n and Zapier nodes.

Indexing is one-time per video, so every later question is just a search call. For batches, loop uploads into one collection and aggregate: a research agent that does in minutes what an analyst does in a week, with receipts.

## FAQ

**Can an AI agent summarize a YouTube video through an API?** Yes. Ingest the URL, run speech + visual analyzers, index, then search or summarize grounded in the indexed content, with timestamps and compiled clips.

**Does this only work for YouTube?** No. The same path takes direct file URLs and local files, plus live RTSP streams and screen recordings. (Process content you have the rights to work with.)

**What does it cost to index a video?** Indexing is one-time per video, and searches are cheap API calls. Free tier at console.videodb.io. Rates at https://videodb.io/pricing

**Is this just transcript search?** No. The visual channel is analyzed and indexed too. For transcript-only budget workloads, `video.index_spoken_words()` + `video.search()` is a leaner path.

---

Full guide: https://videodb.io/blogs/give-your-ai-agents-eyes · Quickstart: https://docs.videodb.io/pages/getting-started/quickstart