# Search over the Visual World > Persistent Visual Memory, Layered Indexes, and Source-Grounded Evidence Sankalp Nagaonkar · Rohit Garg · Ankit Raj · Ashish Choithani · Ashutosh Trivedi VideoDB · {sankalp, rohit, ankit, ashish, ashu}@videodb.io Technical Report · July 2026 Category: Research --- **The canonical document is the PDF**, published and revised at . Read it there rather than relying on any summary of it, including this page. Search over the visual world cannot be reduced to ranking video files. The report develops the infrastructure it does require — analyzer-defined scenes, persistent visual memory, capability-declared indexes, and evidence that stays playable at the original source — and evaluates it against a commercial video-native retrieval engine over 9,834 natural-language queries drawn from four public datasets. ## What the report covers 1. **Introduction** — why agents over cameras, screens, and archives face a different retrieval problem. 2. **The Shape of the Problem** — where classical retrieval assumptions fail, and the layer missing between models and media. 3. **A Conceptual Model** — sources and source time, analyzers and scenes, understanding artifacts, visual memory, indexes, and the memory / context / evidence distinction. 4. **The VideoDB Data Format (VDB)** — a logical format binding identity, source time, artifacts, provenance, and capability-declared indexes, in seven principles. 5. **Evidence as a Stream** — any temporal selection realized as playable media on demand. 6. **Search over Visual Memory** — one typed search surface: planned retrieval, stateful investigation, direct semantic / structured / aggregate contracts, and grounded synthesis; indexed versus resolved scenes. 7. **Closed-Loop Visual Search** — query-time refinement and representation refinement. 8. **Model or System** — video-native model APIs contrasted with model-agnostic visual data infrastructure, as documented interfaces. 9. **Empirical Study** — a shared-corpus retrieval comparison, plus analyzer-choice and scene-construction studies. 10. **Future and a Research Agenda** — persistent but not yet adaptive memory, native temporal representations, and governance. 11. **Conclusion** Appendices carry the exact dataset-specific prompts, the detailed reranker study, and the operational measurement setup. ## Links - Paper (PDF, canonical): - Benchmark configurations and reproduction instructions: - Open-source Deep Search implementation of the stateful retrieval loop: