# Research & Blog · VideoDB

> Benchmarks, agent build notes, essays, papers and engineering notes from the VideoDB lab, published as the work happens.

Canonical: https://videodb.io/blog. Every internal post has a Markdown alternate at the same path with a `.md` suffix.

## Featured

- 2026-08-08 · [Search over the Visual World: persistent visual memory, layered indexes, and source-grounded evidence](https://videodb.io/blog/search-over-the-visual-world) · [md](https://videodb.io/blog/search-over-the-visual-world.md) (Papers) · Nagaonkar, Garg, Raj, Choithani, Trivedi: Persistent visual memory, layered indexes and source-grounded evidence for agents over cameras, screens and archives.
- 2026-09-08 · [Can a VLM annotate a robot run? Results on a revised WGO-Bench](https://videodb.io/blog/wgo-bench-action-annotations) · [md](https://videodb.io/blog/wgo-bench-action-annotations.md) (Benchmarks) · Samuel Alexander, Sankalp Nagaonkar: All 100 WGO-Bench videos re-annotated, then a VLM pipeline measured on segmenting and labeling their actions: 28.93% semantic F1 at IoU 0.5, against a 19.92% public baseline.

## Papers

- 2026-08-08 · [Search over the Visual World: persistent visual memory, layered indexes, and source-grounded evidence](https://videodb.io/blog/search-over-the-visual-world) · [md](https://videodb.io/blog/search-over-the-visual-world.md) · Nagaonkar, Garg, Raj, Choithani, Trivedi: VideoDB technical report (arXiv 2608.08075). Analyzer-defined scenes, persistent visual memory, capability-declared indexes, source-grounded evidence, and a 9,834-query retrieval evaluation against a commercial video-native engine.
- 2026-04-13 · [Do thought streams matter? Evaluating reasoning in Gemini vision-language models for video scene understanding](https://arxiv.org/abs/2604.11177) · Sharma, Nagaonkar, Choithani, Trivedi: Benchmarks how internal reasoning traces affect video scene understanding in Gemini models, including where quality gains plateau and how tight budgets increase compression-step hallucination.
- 2025-02-10 · [Benchmarking vision-language models on optical character recognition in dynamic video environments](https://arxiv.org/abs/2502.06445) · Nagaonkar, Sharma, Choithani, Trivedi: Introduces an open-source benchmark for evaluating VLMs on OCR tasks in dynamic video environments across 1,477 manually annotated frames.

## Benchmarks

- 2026-09-08 · [Can a VLM annotate a robot run? Results on a revised WGO-Bench](https://videodb.io/blog/wgo-bench-action-annotations) · [md](https://videodb.io/blog/wgo-bench-action-annotations.md) · Samuel Alexander, Sankalp Nagaonkar: We re-annotated all 100 videos of WGO-Bench, human and robot, then measured how well a VLM pipeline segments and labels the actions in them. VideoDB reaches 28.93% semantic F1 at IoU 0.5, ahead of the public baseline at 19.92%, and the remaining errors show where the work goes next.
- 2026-07-24 · [What video retrieval benchmarks taught us about ground truth](https://videodb.io/blog/video-benchmark-ground-truth-ambiguity) · [md](https://videodb.io/blog/video-benchmark-ground-truth-ambiguity.md) · Samuel Alexander: Manual review of MSRVTT, MSVD, VATEX, DiDeMo, and QVHighlights shows cases where the benchmark ground truth is too narrow, shifted, or ambiguous, so valid retrieved clips are scored as misses.
- 2026-05-19 · [A strong general-purpose VLM still fails on a chessboard](https://videodb.io/blog/claude-chessboard-spatial-reasoning) · [md](https://videodb.io/blog/claude-chessboard-spatial-reasoning.md) · Sankalp Nagaonkar: We tested GPT-5.4 and Claude Opus 4.7 on reading 72 chess positions into FEN. Claude Opus 4.7 understood the boards and still lost to square-level errors, and the reason only showed up in the reasoning traces.
- 2026-05-15 · [Evaluating video VLMs on your own task](https://videodb.io/blog/how-to-evaluate-multimodal-vlms-for-your-video-use-case) · [md](https://videodb.io/blog/how-to-evaluate-multimodal-vlms-for-your-video-use-case.md) · Sankalp Nagaonkar: A practical workflow for evaluating video VLM setups with VideoDB and Langfuse, from task definition and dataset design to tracing, scoring, and deployment decisions.
- 2025-08-25 · [NFL game analysis: 80% fewer VLM hallucinations](https://videodb.io/blog/nfl-game-analysis-vlm-hallucinations) · [md](https://videodb.io/blog/nfl-game-analysis-vlm-hallucinations.md) · Sankalp Nagaonkar: Three approaches to event-dense NFL footage, measured. Play-by-play segmentation cut VLM hallucinations from 68.1% to 11.4% at up to 70% lower cost.

## Engineering

- 2026-08-09 · [Twelve Labs alternatives, compared](https://videodb.io/blog/twelve-labs-alternatives) · [md](https://videodb.io/blog/twelve-labs-alternatives.md) · Ashutosh Trivedi: Comparing Twelve Labs alternatives for video AI in 2026: VideoDB, Mixpeek, Google, AWS, Azure, NVIDIA: organized by what you are actually building.
- 2026-08-09 · [Video RAG: how it works](https://videodb.io/blog/video-rag) · [md](https://videodb.io/blog/video-rag.md) · Ashutosh Trivedi: What video RAG is, why it breaks text-RAG assumptions, the four architectures that work, and how to build one, including RAG over live streams.
- 2026-06-04 · [Focusd: a coach for your workday](https://videodb.io/blog/focusd) · [md](https://videodb.io/blog/focusd.md) · Sankalp Nagaonkar: How we built Focusd as a local-first desktop app that records work sessions, indexes screen activity with VideoDB, and turns raw events into useful productivity coaching.
- 2026-05-08 · [Dispatch #001: playable evidence, thinking tokens, agentic streams](https://videodb.io/blog/dispatch-001) · [md](https://videodb.io/blog/dispatch-001.md) · Ashutosh Trivedi: The first issue of the VideoDB Dispatch: why agent runs need playable evidence, how much a video model should think, and builder updates from VideoDB.
- 2026-04-20 · [Reuse HTTP connections in Python services](https://videodb.io/blog/python-http-connection-pooling) · [md](https://videodb.io/blog/python-http-connection-pooling.md) · Rohit Garg: A short field note on replacing repeated bare requests calls with a shared session so Python services can reuse HTTP connections.
- 2026-04-17 · [Caching CORS preflights: hundreds of milliseconds per call](https://videodb.io/blog/cors-preflight-cache) · [md](https://videodb.io/blog/cors-preflight-cache.md) · Om Gate: A practical note on preflight caching, custom auth headers, and why OPTIONS requests quietly dominate app latency on slower connections.
- 2026-01-02 · [A Postgres backup is not real until you restore it](https://videodb.io/blog/postgres-backup-restore-drill) · [md](https://videodb.io/blog/postgres-backup-restore-drill.md) · Rohit Garg: A field note from a simple offline Postgres migration: why pg_dump was enough, where read-only mode fits, and why the restore drill matters more than the dump command.
- 2025-09-17 · [The 6 MB Lambda limit: compress before, not after](https://videodb.io/blog/lambda-compression-trap) · [md](https://videodb.io/blog/lambda-compression-trap.md) · Rohit Garg: Why API Gateway compression can make large responses look safe in testing, then still fail when Lambda enforces the raw payload limit first.
- 2025-08-25 · [Searching the slides in a conference recording](https://videodb.io/blog/conference-slide-extraction) · [md](https://videodb.io/blog/conference-slide-extraction.md) · Ashutosh Trivedi: Pull slide content out of any conference talk by combining spoken-word search with visual scene indexing. Full Python pipeline with working code.
- 2025-08-25 · [Smaller Python ML containers](https://videodb.io/blog/python-ml-container-size) · [md](https://videodb.io/blog/python-ml-container-size.md) · Lalit Gupta: Slim bases, same-layer cleanup, CPU-only PyTorch, pinned wheels, and the unglamorous work of keeping deploys fast.
- 2025-08-13 · [VideoDB × TwelveLabs: search across your video library](https://videodb.io/blog/twelvelabs) · [md](https://videodb.io/blog/twelvelabs.md) · Ashutosh Trivedi: TwelveLabs multimodal understanding plugs into VideoDB so any agent can find the exact moment it needs, not just the right file.
- 2025-06-11 · [VideoDB × LlamaIndex: video in your RAG pipeline](https://videodb.io/blog/llama-index) · [md](https://videodb.io/blog/llama-index.md) · Ashutosh Trivedi: The official VideoDB connector for LlamaIndex lets you treat video as a first-class source in any retrieval-augmented pipeline.

## Essay

- 2026-09-18 · [A query engine for robot video](https://videodb.io/blog/robot-runs) · [md](https://videodb.io/blog/robot-runs.md) · Ashutosh Trivedi: We built a query engine that sits over a lab's robot video, keeps every episode whole, and answers questions in plain English: find the failures, export them as a training manifest, and search the next model's runs the moment they land.
- 2026-07-13 · [DeepSeek’s visual primitives and the missing layer](https://videodb.io/blog/deepseek-visual-primitives-reference-gap) · [md](https://videodb.io/blog/deepseek-visual-primitives-reference-gap.md) · Sankalp Nagaonkar, Ashutosh Trivedi: The Reference Gap explains why seeing an object is insufficient when a multimodal model cannot preserve its identity, location, or path through a long reasoning trace.
- 2026-07-07 · [JEPA, from language models to world models](https://videodb.io/blog/jepa-from-language-models-to-world-models) · [md](https://videodb.io/blog/jepa-from-language-models-to-world-models.md) · Sankalp Nagaonkar, Ashutosh Trivedi: Why JEPA’s latent-prediction objective may shift AI systems from token prediction toward predictive world models for VLMs, VLAs, and embodied agents.
- 2026-06-11 · [Episodic memory for agents](https://videodb.io/blog/episodic-memory-for-agents) · [md](https://videodb.io/blog/episodic-memory-for-agents.md) · Ashutosh Trivedi: Humans remember experiences, not just facts. Your agent should too, with verifiable, playable evidence.
- 2026-06-09 · [Video was built for playback, not perception](https://videodb.io/blog/playback-vs-perception) · [md](https://videodb.io/blog/playback-vs-perception.md) · Ashutosh Trivedi: 70 years of video infrastructure for human eyes, and why AI needs perception-first architecture.
- 2026-06-07 · [MP4 is the wrong primitive](https://videodb.io/blog/mp4-is-wrong-primitive) · [md](https://videodb.io/blog/mp4-is-wrong-primitive.md) · Ashutosh Trivedi: Video files were designed for playback. AI agents need indexes, not opaque blobs.
- 2026-06-05 · [Perception is the missing layer](https://videodb.io/blog/perception-is-the-missing-layer) · [md](https://videodb.io/blog/perception-is-the-missing-layer.md) · Ashutosh Trivedi: LLMs have reasoning. RAG has retrieval. What's missing? The ability to perceive the world as it happens.
- 2026-06-03 · [Infrastructure that sees and edits](https://videodb.io/blog/infrastructure-that-sees-and-edits) · [md](https://videodb.io/blog/infrastructure-that-sees-and-edits.md) · Ashutosh Trivedi: Explore VideoDB's multimodal infrastructure that combines vision and editing for intelligent video processing.
- 2026-06-01 · [Why agents are blind today](https://videodb.io/blog/why-agents-are-blind) · [md](https://videodb.io/blog/why-agents-are-blind.md) · Ashutosh Trivedi: The gap between human perception and agent perception, and why it matters for the future of AI.

## Agents

- 2026-08-09 · [Claude edited our launch video with VideoDB skills](https://videodb.io/blog/claude-edited-our-launch-video) · [md](https://videodb.io/blog/claude-edited-our-launch-video.md) · Ashutosh Trivedi: A case study in understanding-driven editing: the agent indexed the raw takes, picked the best take per narrative beat, composed and rendered the timeline, then re-indexed its own cut to audit it.
- 2026-08-09 · [Giving an agent a YouTube video it can search](https://videodb.io/blog/ai-agent-watch-youtube) · [md](https://videodb.io/blog/ai-agent-watch-youtube.md) · Ashutosh Trivedi: Give your agent YouTube videos as searchable, timestamped context. Speech and visuals indexed, evidence clips compiled, in about 20 lines of Python.
- 2026-08-08 · [Agentic Streams: an agent researches a topic and streams you the briefing](https://videodb.io/blog/agentic-streams) · [md](https://videodb.io/blog/agentic-streams.md) · Ashutosh Trivedi: Open-source agents that research a topic on the live web, gather real clips, screenshots and charts, write a narration, assemble the video with VideoDB and hand back a stream you can play. Three agents included, with example outputs.
- 2026-08-07 · [Real-time visual perception for agents](https://videodb.io/blog/give-your-ai-agents-eyes) · [md](https://videodb.io/blog/give-your-ai-agents-eyes.md) · Ashutosh Trivedi: How AI agents get vision: ingest YouTube videos, live RTSP cameras, and screens; index continuously; search semantically; act on plain-language events.
- 2026-08-07 · [From an RTSP stream to events an agent can act on](https://videodb.io/blog/rtsp-ai-analysis) · [md](https://videodb.io/blog/rtsp-ai-analysis.md) · Ashutosh Trivedi: Turn any RTSP camera stream into AI-readable events: continuous understanding, plain-language alerts, and searchable history. No CV pipeline to build.
- 2026-06-29 · [TinyFish × VideoDB: the web your agent browses, as video it can search](https://videodb.io/blog/tinyfish) · [md](https://videodb.io/blog/tinyfish.md) · Ashutosh Trivedi: TinyFish opens the web for agents. VideoDB lets them see inside video. Together they cover both access and comprehension.
- 2026-06-04 · [Bloom: record your screen, search it later](https://videodb.io/blog/bloom) · [md](https://videodb.io/blog/bloom.md) · Lalit Gupta: How I built a local-first screen recorder on top of the VideoDB SDK. It automatically indexes your audio and screen, making everything searchable by the time you paste the share link.
- 2026-06-04 · [A camera feed for your OpenClaw agent](https://videodb.io/blog/openclaw-agent-camera) · [md](https://videodb.io/blog/openclaw-agent-camera.md) · Ashutosh Trivedi: How we built a VideoDB-powered OpenClaw skill that records, indexes, searches, summarizes, and clips an agent's remote desktop without changing the agent itself.
- 2026-05-19 · [Deep Search: finding exact moments in video](https://videodb.io/blog/deepsearch) · [md](https://videodb.io/blog/deepsearch.md) · Sankalp Nagaonkar: How we built Deep Search as a retrieval loop for finding exact moments in video using planning, indexing, validation, recovery, and follow-up state.
- 2026-05-19 · [Pair Programmer: live screen and audio context for coding agents](https://videodb.io/blog/pair-programmer-live-context) · [md](https://videodb.io/blog/pair-programmer-live-context.md) · Rohit Garg: A skill that gives Claude Code, Cursor and Codex live screen, mic and system-audio context, and how it is built on VideoDB Capture, RTStreams, local events and agent-side retrieval.
- 2026-05-08 · [How I built Call.md on VideoDB](https://videodb.io/blog/how-i-built-call-intelligence-with-videodb) · [md](https://videodb.io/blog/how-i-built-call-intelligence-with-videodb.md) · Om: A practical build story for a local-first call intelligence app that records calls, transcribes speakers, generates live nudges, and exports structured Markdown.
- 2026-05-08 · [Call.md, Meeting Copilot](https://videodb.io/apps/call-md) · [md](https://videodb.io/apps/call-md.md) (app, built with VideoDB) · Ashutosh Trivedi: Turn meetings into live agent loops.
- 2026-06-04 · [Bloom, AI-ready screen recorder](https://videodb.io/apps/bloom) · [md](https://videodb.io/apps/bloom.md) (app, built with VideoDB) · Ashutosh Trivedi: An agentic Loom alternative where recordings become inputs for AI.
- 2026-06-04 · [Focusd, Productivity Tracker](https://videodb.io/apps/focusd) · [md](https://videodb.io/apps/focusd.md) (app, built with VideoDB) · Sankalp Nagaonkar: Screen memory for understanding where your workday went.
- 2026-06-04 · [OpenClaw Monitoring](https://videodb.io/apps/openclaw-monitoring) · [md](https://videodb.io/apps/openclaw-monitoring.md) (app, built with VideoDB) · Ashutosh Trivedi: CCTV for computer-use agents.
- 2026-05-08 · [makememes.site](https://makememes.site/?utm_source=labs_videodb_io&utm_medium=website&utm_campaign=videodb_projects) (app, side project) · Ashish: A code-first video meme generator with reusable templates and VideoDB-powered source videos.
- 2026-05-08 · [Cool QR Codes Studio](https://qr-generator-delta-lake.vercel.app/?utm_source=labs_videodb_io&utm_medium=website&utm_campaign=videodb_side_projects) (app, side project) · Ashish: A no-login QR code studio with scanability checks.
- 2026-05-08 · [Dictate](https://github.com/ash-ishh/dictate) (app, side project) · Ashish: Local-first macOS dictation that pastes speech into any app.