# Research & Blog · VideoDB

> Benchmarks, agent build notes, essays, papers and engineering notes from the VideoDB lab, published as the work happens.

Canonical: https://videodb.io/blog. Every internal post has a Markdown alternate at the same path with a `.md` suffix.

## Featured

- 2026-08-08 · [Search over the Visual World: persistent visual memory, layered indexes, and source-grounded evidence](https://videodb.io/blog/search-over-the-visual-world) · [md](https://videodb.io/blog/search-over-the-visual-world.md) (Papers) · Nagaonkar, Garg, Raj, Choithani, Trivedi: Persistent visual memory, layered indexes and source-grounded evidence for agents over cameras, screens and archives.
- 2026-09-08 · [Can a VLM annotate a robot run? Results on a revised WGO-Bench](https://videodb.io/blog/wgo-bench-action-annotations) · [md](https://videodb.io/blog/wgo-bench-action-annotations.md) (Benchmarks) · Samuel Alexander, Sankalp Nagaonkar: All 100 WGO-Bench videos re-annotated, then a VLM pipeline measured on segmenting and labeling their actions: 28.93% semantic F1 at IoU 0.5, against a 19.92% public baseline.

## Papers

- 2026-08-08 · [Search over the Visual World: persistent visual memory, layered indexes, and source-grounded evidence](https://videodb.io/blog/search-over-the-visual-world) · [md](https://videodb.io/blog/search-over-the-visual-world.md) · Nagaonkar, Garg, Raj, Choithani, Trivedi: VideoDB technical report (arXiv 2608.08075). Analyzer-defined scenes, persistent visual memory, capability-declared indexes, source-grounded evidence, and a 9,834-query retrieval evaluation against a commercial video-native engine.
- 2026-04-13 · [Do thought streams matter? Evaluating reasoning in Gemini vision-language models for video scene understanding](https://arxiv.org/abs/2604.11177) · Sharma, Nagaonkar, Choithani, Trivedi: Benchmarks how internal reasoning traces affect video scene understanding in Gemini models, including where quality gains plateau and how tight budgets increase compression-step hallucination.
- 2025-02-10 · [Benchmarking vision-language models on optical character recognition in dynamic video environments](https://arxiv.org/abs/2502.06445) · Nagaonkar, Sharma, Choithani, Trivedi: Introduces an open-source benchmark for evaluating VLMs on OCR tasks in dynamic video environments across 1,477 manually annotated frames.

## Benchmarks

- 2026-09-08 · [Can a VLM annotate a robot run? Results on a revised WGO-Bench](https://videodb.io/blog/wgo-bench-action-annotations) · [md](https://videodb.io/blog/wgo-bench-action-annotations.md) · Samuel Alexander, Sankalp Nagaonkar: We re-annotated all 100 videos of WGO-Bench, human and robot, then measured how well a VLM pipeline segments and labels the actions in them. VideoDB reaches 28.93% semantic F1 at IoU 0.5, ahead of the public baseline at 19.92%, and the remaining errors show where the work goes next.
- 2026-07-24 · [What video retrieval benchmarks taught us about ground truth](https://videodb.io/blog/video-benchmark-ground-truth-ambiguity) · [md](https://videodb.io/blog/video-benchmark-ground-truth-ambiguity.md) · Samuel Alexander: Manual review of MSRVTT, MSVD, VATEX, DiDeMo, and QVHighlights shows cases where the benchmark ground truth is too narrow, shifted, or ambiguous, so valid retrieved clips are scored as misses.
- 2026-05-19 · [A strong general-purpose VLM still fails on a chessboard](https://videodb.io/blog/claude-chessboard-spatial-reasoning) · [md](https://videodb.io/blog/claude-chessboard-spatial-reasoning.md) · Sankalp Nagaonkar: We tested GPT-5.4 and Claude Opus 4.7 on reading 72 chess positions into FEN. Claude Opus 4.7 understood the boards and still lost to square-level errors, and the reason only showed up in the reasoning traces.
- 2026-05-15 · [Evaluating video VLMs on your own task](https://videodb.io/blog/how-to-evaluate-multimodal-vlms-for-your-video-use-case) · [md](https://videodb.io/blog/how-to-evaluate-multimodal-vlms-for-your-video-use-case.md) · Sankalp Nagaonkar: A practical workflow for evaluating video VLM setups with VideoDB and Langfuse, from task definition and dataset design to tracing, scoring, and deployment decisions.
- 2025-08-25 · [NFL game analysis: 80% fewer VLM hallucinations](https://videodb.io/blog/nfl-game-analysis-vlm-hallucinations) · [md](https://videodb.io/blog/nfl-game-analysis-vlm-hallucinations.md) · Sankalp Nagaonkar: Three approaches to event-dense NFL footage, measured. Play-by-play segmentation cut VLM hallucinations from 68.1% to 11.4% at up to 70% lower cost.

## Engineering

- 2026-08-09 · [Twelve Labs alternatives, compared](https://videodb.io/blog/twelve-labs-alternatives) · [md](https://videodb.io/blog/twelve-labs-alternatives.md) · Ashutosh Trivedi: Comparing Twelve Labs alternatives for video AI in 2026: VideoDB, Mixpeek, Google, AWS, Azure, NVIDIA: organized by what you are actually building.
- 2026-08-09 · [Video RAG: how it works](https://videodb.io/blog/video-rag) · [md](https://videodb.io/blog/video-rag.md) · Ashutosh Trivedi: What video RAG is, why it breaks text-RAG assumptions, the four architectures that work, and how to build one, including RAG over live streams.
- 2026-06-04 · [Focusd: a coach for your workday](https://videodb.io/blog/focusd) · [md](https://videodb.io/blog/focusd.md) · Sankalp Nagaonkar: How we built Focusd as a local-first desktop app that records work sessions, indexes screen activity with VideoDB, and turns raw events into useful productivity coaching.
- 2026-05-08 · [Dispatch #001: playable evidence, thinking tokens, agentic streams](https://videodb.io/blog/dispatch-001) · [md](https://videodb.io/blog/dispatch-001.md) · Ashutosh Trivedi: The first issue of the VideoDB Dispatch: why agent runs need playable evidence, how much a video model should think, and builder updates from VideoDB.
- 2026-04-20 · [Reuse HTTP connections in Python services](https://videodb.io/blog/python-http-connection-pooling) · [md](https://videodb.io/blog/python-http-connection-pooling.md) · Rohit Garg: A short field note on replacing repeated bare requests calls with a shared session so Python services can reuse HTTP connections.
- 2026-04-17 · [Caching CORS preflights: hundreds of milliseconds per call](https://videodb.io/blog/cors-preflight-cache) · [md](https://videodb.io/blog/cors-preflight-cache.md) · Om Gate: A practical note on preflight caching, custom auth headers, and why OPTIONS requests quietly dominate app latency on slower connections.
- 2026-01-02 · [A Postgres backup is not real until you restore it](https://videodb.io/blog/postgres-backup-restore-drill) · [md](https://videodb.io/blog/postgres-backup-restore-drill.md) · Rohit Garg: A field note from a simple offline Postgres migration: why pg_dump was enough, where read-only mode fits, and why the restore drill matters more than the dump command.
- 2025-09-17 · [The 6 MB Lambda limit: compress before, not after](https://videodb.io/blog/lambda-compression-trap) · [md](https://videodb.io/blog/lambda-compression-trap.md) · Rohit Garg: Why API Gateway compression can make large responses look safe in testing, then still fail when Lambda enforces the raw payload limit first.
- 2025-08-25 · [Searching the slides in a conference recording](https://videodb.io/blog/conference-slide-extraction) · [md](https://videodb.io/blog/conference-slide-extraction.md) · Ashutosh Trivedi: Pull slide content out of any conference talk by combining spoken-word search with visual scene indexing. Full Python pipeline with working code.
- 2025-08-25 · [Smaller Python ML containers](https://videodb.io/blog/python-ml-container-size) · [md](https://videodb.io/blog/python-ml-container-size.md) · Lalit Gupta: Slim bases, same-layer cleanup, CPU-only PyTorch, pinned wheels, and the unglamorous work of keeping deploys fast.
- 2025-08-13 · [VideoDB × TwelveLabs: search across your video library](https://videodb.io/blog/twelvelabs) · [md](https://videodb.io/blog/twelvelabs.md) · Ashutosh Trivedi: TwelveLabs multimodal understanding plugs into VideoDB so any agent can find the exact moment it needs, not just the right file.
- 2025-06-11 · [VideoDB × LlamaIndex: video in your RAG pipeline](https://videodb.io/blog/llama-index) · [md](https://videodb.io/blog/llama-index.md) · Ashutosh Trivedi: The official VideoDB connector for LlamaIndex lets you treat video as a first-class source in any retrieval-augmented pipeline.

## Essay

- 2026-09-18 · [A query engine for robot video](https://videodb.io/blog/robot-runs) · [md](https://videodb.io/blog/robot-runs.md) · Ashutosh Trivedi: We built a query engine that sits over a lab's robot video, keeps every episode whole, and answers questions in plain English: find the failures, export them as a training manifest, and search the next model's runs the moment they land.
- 2026-07-13 · [DeepSeek’s visual primitives and the missing layer](https://videodb.io/blog/deepseek-visual-primitives-reference-gap) · [md](https://videodb.io/blog/deepseek-visual-primitives-reference-gap.md) · Sankalp Nagaonkar, Ashutosh Trivedi: The Reference Gap explains why seeing an object is insufficient when a multimodal model cannot preserve its identity, location, or path through a long reasoning trace.
- 2026-07-07 · [JEPA, from language models to world models](https://videodb.io/blog/jepa-from-language-models-to-world-models) · [md](https://videodb.io/blog/jepa-from-language-models-to-world-models.md) · Sankalp Nagaonkar, Ashutosh Trivedi: Why JEPA’s latent-prediction objective may shift AI systems from token prediction toward predictive world models for VLMs, VLAs, and embodied agents.
- 2026-06-11 · [Episodic memory for agents](https://videodb.io/blog/episodic-memory-for-agents) · [md](https://videodb.io/blog/episodic-memory-for-agents.md) · Ashutosh Trivedi: Humans remember experiences, not just facts. Your agent should too, with verifiable, playable evidence.
- 2026-06-09 · [Video was built for playback, not perception](https://videodb.io/blog/playback-vs-perception) · [md](https://videodb.io/blog/playback-vs-perception.md) · Ashutosh Trivedi: 70 years of video infrastructure for human eyes, and why AI needs perception-first architecture.
- 2026-06-07 · [MP4 is the wrong primitive](https://videodb.io/blog/mp4-is-wrong-primitive) · [md](https://videodb.io/blog/mp4-is-wrong-primitive.md) · Ashutosh Trivedi: Video files were designed for playback. AI agents need indexes, not opaque blobs.
- 2026-06-05 · [Perception is the missing layer](https://videodb.io/blog/perception-is-the-missing-layer) · [md](https://videodb.io/blog/perception-is-the-missing-layer.md) · Ashutosh Trivedi: LLMs have reasoning. RAG has retrieval. What's missing? The ability to perceive the world as it happens.
- 2026-06-03 · [Infrastructure that sees and edits](https://videodb.io/blog/infrastructure-that-sees-and-edits) · [md](https://videodb.io/blog/infrastructure-that-sees-and-edits.md) · Ashutosh Trivedi: Explore VideoDB's multimodal infrastructure that combines vision and editing for intelligent video processing.
- 2026-06-01 · [Why agents are blind today](https://videodb.io/blog/why-agents-are-blind) · [md](https://videodb.io/blog/why-agents-are-blind.md) · Ashutosh Trivedi: The gap between human perception and agent perception, and why it matters for the future of AI.

## Agents

- 2026-10-05 · [Why a search tool is not enough for an LLM working with video](https://videodb.io/blog/why-a-search-tool-is-not-enough-for-an-llm-working-with-video) · [md](https://videodb.io/blog/why-a-search-tool-is-not-enough-for-an-llm-working-with-video.md) · Aditi Priya: A search tool lets an LLM find a moment in video. But when the LLM is asked how many, or only the ones that failed, it counts whatever the search returned. We show why that happens and what the LLM needs instead.
- 2026-08-09 · [Claude edited our launch video with VideoDB skills](https://videodb.io/blog/claude-edited-our-launch-video) · [md](https://videodb.io/blog/claude-edited-our-launch-video.md) · Ashutosh Trivedi: A case study in understanding-driven editing: the agent indexed the raw takes, picked the best take per narrative beat, composed and rendered the timeline, then re-indexed its own cut to audit it.
- 2026-08-09 · [Giving an agent a YouTube video it can search](https://videodb.io/blog/ai-agent-watch-youtube) · [md](https://videodb.io/blog/ai-agent-watch-youtube.md) · Ashutosh Trivedi: Give your agent YouTube videos as searchable, timestamped context. Speech and visuals indexed, evidence clips compiled, in about 20 lines of Python.
- 2026-08-08 · [Agentic Streams: an agent researches a topic and streams you the briefing](https://videodb.io/blog/agentic-streams) · [md](https://videodb.io/blog/agentic-streams.md) · Ashutosh Trivedi: Open-source agents that research a topic on the live web, gather real clips, screenshots and charts, write a narration, assemble the video with VideoDB and hand back a stream you can play. Three agents included, with example outputs.
- 2026-08-07 · [Real-time visual perception for agents](https://videodb.io/blog/give-your-ai-agents-eyes) · [md](https://videodb.io/blog/give-your-ai-agents-eyes.md) · Ashutosh Trivedi: How AI agents get vision: ingest YouTube videos, live RTSP cameras, and screens; index continuously; search semantically; act on plain-language events.
- 2026-08-07 · [From an RTSP stream to events an agent can act on](https://videodb.io/blog/rtsp-ai-analysis) · [md](https://videodb.io/blog/rtsp-ai-analysis.md) · Ashutosh Trivedi: Turn any RTSP camera stream into AI-readable events: continuous understanding, plain-language alerts, and searchable history. No CV pipeline to build.
- 2026-06-29 · [TinyFish × VideoDB: the web your agent browses, as video it can search](https://videodb.io/blog/tinyfish) · [md](https://videodb.io/blog/tinyfish.md) · Ashutosh Trivedi: TinyFish opens the web for agents. VideoDB lets them see inside video. Together they cover both access and comprehension.
- 2026-06-04 · [Bloom: record your screen, search it later](https://videodb.io/blog/bloom) · [md](https://videodb.io/blog/bloom.md) · Lalit Gupta: How I built a local-first screen recorder on top of the VideoDB SDK. It automatically indexes your audio and screen, making everything searchable by the time you paste the share link.
- 2026-06-04 · [A camera feed for your OpenClaw agent](https://videodb.io/blog/openclaw-agent-camera) · [md](https://videodb.io/blog/openclaw-agent-camera.md) · Ashutosh Trivedi: How we built a VideoDB-powered OpenClaw skill that records, indexes, searches, summarizes, and clips an agent's remote desktop without changing the agent itself.
- 2026-05-19 · [Deep Search: finding exact moments in video](https://videodb.io/blog/deepsearch) · [md](https://videodb.io/blog/deepsearch.md) · Sankalp Nagaonkar: How we built Deep Search as a retrieval loop for finding exact moments in video using planning, indexing, validation, recovery, and follow-up state.
- 2026-05-19 · [Pair Programmer: live screen and audio context for coding agents](https://videodb.io/blog/pair-programmer-live-context) · [md](https://videodb.io/blog/pair-programmer-live-context.md) · Rohit Garg: A skill that gives Claude Code, Cursor and Codex live screen, mic and system-audio context, and how it is built on VideoDB Capture, RTStreams, local events and agent-side retrieval.
- 2026-05-08 · [How I built Call.md on VideoDB](https://videodb.io/blog/how-i-built-call-intelligence-with-videodb) · [md](https://videodb.io/blog/how-i-built-call-intelligence-with-videodb.md) · Om: A practical build story for a local-first call intelligence app that records calls, transcribes speakers, generates live nudges, and exports structured Markdown.
- 2026-05-08 · [Call.md, Meeting Copilot](https://videodb.io/apps/call-md) · [md](https://videodb.io/apps/call-md.md) (app, built with VideoDB) · Ashutosh Trivedi: Turn meetings into live agent loops.
- 2026-06-04 · [Bloom, AI-ready screen recorder](https://videodb.io/apps/bloom) · [md](https://videodb.io/apps/bloom.md) (app, built with VideoDB) · Ashutosh Trivedi: An agentic Loom alternative where recordings become inputs for AI.
- 2026-06-04 · [Focusd, Productivity Tracker](https://videodb.io/apps/focusd) · [md](https://videodb.io/apps/focusd.md) (app, built with VideoDB) · Sankalp Nagaonkar: Screen memory for understanding where your workday went.
- 2026-06-04 · [OpenClaw Monitoring](https://videodb.io/apps/openclaw-monitoring) · [md](https://videodb.io/apps/openclaw-monitoring.md) (app, built with VideoDB) · Ashutosh Trivedi: CCTV for computer-use agents.
- 2026-05-08 · [makememes.site](https://makememes.site/?utm_source=labs_videodb_io&utm_medium=website&utm_campaign=videodb_projects) (app, side project) · Ashish: A code-first video meme generator with reusable templates and VideoDB-powered source videos.
- 2026-05-08 · [Cool QR Codes Studio](https://qr-generator-delta-lake.vercel.app/?utm_source=labs_videodb_io&utm_medium=website&utm_campaign=videodb_side_projects) (app, side project) · Ashish: A no-login QR code studio with scanability checks.
- 2026-05-08 · [Dictate](https://github.com/ash-ishh/dictate) (app, side project) · Ashish: Local-first macOS dictation that pastes speech into any app.