# Search over the Visual World: persistent visual memory, layered indexes, and source-grounded evidence

> VideoDB technical report (arXiv 2608.08075). Analyzer-defined scenes, persistent visual memory, capability-declared indexes, source-grounded evidence, and a 9,834-query retrieval evaluation against a commercial video-native engine.

- Category: Papers (paper)
- Published: 2026-08-08
- Authors: Nagaonkar, Garg, Raj, Choithani, Trivedi
- Canonical: https://videodb.io/blog/search-over-the-visual-world
- HTML: https://videodb.io/blog/search-over-the-visual-world · Markdown: https://videodb.io/blog/search-over-the-visual-world.md
- Venue: VideoDB technical report, arXiv preprint arXiv:2608.08075
- PDF: https://videodb.io/papers/search-over-the-visual-world.pdf
- arXiv: https://arxiv.org/abs/2608.08075
- Tags: paper, technical report, retrieval, visual memory

---
**Sankalp Nagaonkar · Rohit Garg · Ankit Raj · Ashish Choithani · Ashutosh Trivedi**  
VideoDB · `{sankalp, rohit, ankit, ashish, ashu}@videodb.io`  
Technical Report · July 2026

[Read on arXiv](https://arxiv.org/abs/2608.08075) · [Download PDF](/papers/search-over-the-visual-world.pdf) · [Benchmark code](https://github.com/video-db/search-over-the-visual-world)

Search over the visual world cannot be reduced to ranking video files. This report develops the infrastructure it does require: analyzer-defined scenes, persistent visual memory, capability-declared indexes, and evidence that stays playable at the source. It then evaluates it against a commercial video-native retrieval engine over 9,834 natural-language queries drawn from four public datasets. The paper is published on [arXiv (2608.08075)](https://arxiv.org/abs/2608.08075).

<iframe src="/papers/search-over-the-visual-world.pdf#view=FitH" title="PDF" loading="lazy" style="aspect-ratio:3/4"></iframe>

Trouble viewing it here? [Open the PDF in a new tab](/papers/search-over-the-visual-world.pdf). Benchmark configurations and reproduction instructions are on [GitHub](https://github.com/video-db/search-over-the-visual-world), and the open-source [Deep Search](https://github.com/video-db/deepsearch) implementation of the stateful retrieval loop is public too.

## Cite this work

S Nagaonkar, Rohit Garg, Ankit Raj, Ashish Choithani, Ashutosh Trivedi, "Search over the Visual World: persistent visual memory, layered indexes, and source-grounded evidence", arXiv preprint arXiv:2608.08075, August 2026.

```bibtex
@article{nagaonkar2026searchvisualworld,
  author = {S Nagaonkar and Rohit Garg and Ankit Raj and Ashish Choithani and Ashutosh Trivedi},
  title = {Search over the Visual World: persistent visual memory, layered indexes, and source-grounded evidence},
  journal = {arXiv preprint arXiv:2608.08075},
  year = {2026},
  month = {aug},
  note = {https://arxiv.org/abs/2608.08075},
}
```