Efficient visual document retrieval with small distilled encoders
Project page |
Code |
Demo |
Blog |
Dataset
Small query and page encoders for visual document retrieval, trained by aligning to the
embedding space of a frozen vision-language retriever. A text encoder answers each query, so
no multi-billion model runs per request. Pages stay in the teacher's index, or move to a small
page encoder.
Model names read [Col]NanoVDR-<Q|D>-<variant>-<teacher>-<width>[-ML]. A query encoder and a
page index work together only when the teacher and the width both match. Every model card
carries its results, usage and indexing step; the
project page has the full tables.
from sentence_transformers import SentenceTransformer
query = SentenceTransformer("nanovdr/NanoVDR-Q-DistilBERT-Qwen3VL2B-2048-ML")
q_emb = query.encode(["What was the revenue growth in Q3 2024?"])
scores = q_emb @ doc_emb.T
Citation
@inproceedings{liu-etal-2026-nanovdr,
title = "{N}ano{VDR}: Distilling a 2{B} Vision-Language Retriever into a 70{M} Text-Only Encoder for Visual Document Retrieval",
author = "Liu, Zhuchenyang and
Zhang, Yao and
Xiao, Yu",
editor = "Lee, Hung-Yi and
Dev, Sunipa and
Li, Jessy and
Elliott, Desmond",
booktitle = "Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing",
month = oct,
year = "2026",
address = "Budapest, Hungary",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.emnlp-main.71/",
pages = "1631--1648",
ISBN = "979-8-89176-459-0"
}
@inproceedings{liu-etal-2026-distilvdr,
title = "{D}istil{VDR}: A Compact End-to-End Visual Document Retriever via Dual-Student Distillation",
author = "Liu, Zhuchenyang and
Wang, Ziyi and
Zhang, Yao and
Xiao, Yu",
editor = "T.Y.S.S., Santosh and
Agarwal, Amit and
Gupta, Vivek and
Kementchedjhieva, Yova and
Garimella, Aparna and
Hou, Yufang",
booktitle = "The Proceedings of Workshop on Document Intelligence and Understanding",
month = oct,
year = "2026",
address = "Budapest, Hungary",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.docinsights-1.26/",
pages = "385--399",
ISBN = "979-8-89176-477-4"
}
@misc{liu2026colnanovdrdocumentfreequerydistillation,
title={ColNanoVDR: Document-Free Query Distillation for Multi-Vector Visual Document Retrieval via Optimal Transport},
author={Zhuchenyang Liu and Ziyi Wang and Yao Zhang and Yu Xiao},
year={2026},
eprint={2609.34899},
archivePrefix={arXiv},
primaryClass={cs.IR},
url={https://arxiv.org/abs/2609.34899},
}
Acknowledgements
This project has received funding from the Business Finland co-innovation programme under grant agreement No. 69/31/2025. It is supported by the AiWo: Human-centric AI-enabled Collaborative Fieldwork Operations project (2025–2027), which aims to revolutionize fieldwork operations and enhance human-AI collaboration across the manufacturing, construction, and industrial design sectors. The calculations presented in this project were performed using computer resources within the Aalto University School of Science "Science-IT" project.
