Most document chat tools make the same bet: embed every chunk with one model, put the vectors in a store, and trust semantic similarity to find the answer. That bet often fails on the questions people actually ask, which name a specific invoice number, a person, a product or a literal phrase. Dense embeddings tend to blur those into something that merely sounds similar.
Documetro is a small open-source document intelligence workspace that hedges that bet. It ingests mixed batches of files (PDF, DOCX, XLSX, PPTX, CSV, JSON, HTML and more), converts images, audio and video to text first, and answers questions over the whole batch in a chat interface. The chat is not the interesting part. The retrieval is.
This article reads the project two ways: what the README says, and what the code in engine.py actually does. Where they differ, we say so.
What Documetro actually is
Documetro is a Python application under GPL-3.0, published on GitHub by yunusemrejr. You start it with one launcher, ./run.sh, which finds a free port, stops earlier instances, checks dependencies and starts a FastAPI app. That app serves both the API and a chat-style web UI written in plain HTML, CSS and JavaScript.
Behind the UI is a split-provider RAG pipeline. OpenRouter supplies the embeddings, the multimodal extraction and a lightweight answer model. Nous Hermes plans the query, reranks the evidence and, for several question types, writes the answer itself (more on that below). You provide both API keys in a project-local .env file.
The retrieval score: three weighted layers plus bonuses
Documetro ranks each chunk with a weighted sum of three similarity scores, plus three small additive bonuses. The README calls the weighted part the Model Harmony Configuration. The code adds the bonuses on top.
\text{score} = 0.62\,s_{\text{embed}} + 0.18\,s_{\text{tfidf}} + 0.10\,s_{\text{lsi}} + b_{\text{title}} + b_{\text{anchor}} + b_{\text{type}}Component | Weight or range | What it measures | Runs |
|---|---|---|---|
Embedding (OpenRouter) | × 0.62 | Cosine similarity between query and chunk embeddings | Remote, one call per query |
Lexical (TF-IDF) | × 0.18 | Overlap of weighted terms, hashed into a fixed-size vector | Local |
Latent (LSI/LSA) | × 0.10 | Similarity after truncated SVD of the TF-IDF matrix | Local |
Anchor bonus | up to +0.11 | Exact matches of anchors (codes, numbers, names) from the original question | Local |
Title bonus | up to +0.08 | Query terms found in the document name or section title | Local |
Type bonus | +0.03 to +0.04 | Chunks with digits, lists or comparison words, for matching question types | Local |
Reasoning (Nous Hermes) | no weight | Rewrites the query before scoring and reorders the top 8 after it | Remote |
Three things are worth knowing before you read too much into the percentages:
The weights add up to 90%, not 100%. The remaining influence comes from the unweighted bonuses, so the “62 / 18 / 10” split is not a complete description of the ranking.
The scores are not normalised against each other. Embedding cosines usually have a positive baseline even for unrelated text, while TF-IDF is often exactly zero. The real share each layer has in a ranking varies by query and corpus, and is not the nominal percentage.
The exact-match job is shared. TF-IDF here works on hashed tokens, so it matches words, not phrases, and rare hash collisions are possible. It also scores the query after Hermes has enriched it with extra keywords. The strongest literal signal is the anchor bonus, which is computed on the question exactly as the user typed it.
The idea behind the mix is sound and well established. The Haystack retriever documentation makes the same case: keyword retrieval is strong where wording matters, dense retrieval where meaning matters, and combining them makes ranking more robust. Documetro’s contribution is to wire this together in a few readable functions, with fixed hand-set weights. Fixed weights are also its main limitation: recent work such as Hybrid Retriever Evolution for Multimodal Document Reasoning Agents argues that fixed combinations cannot adapt to individual queries, and proposes learning the orchestration instead.
One vector space per corpus
Documetro picks its embedding model once per corpus, not per file, and this is the most carefully reasoned part of the design.
Text-only corpus: every chunk is embedded with the cheaper text model, by default
nvidia/llama-3.2-nv-embedqa-1b-v2:free.Corpus with at least one image: every chunk, text included, is embedded with the multimodal model, by default
nvidia/llama-nemotron-embed-vl-1b-v2:free. Image chunks are sent as text plus the image itself.Audio and video: converted to text first, then embedded in whichever space the corpus uses.
The reason is simple. Vectors from two different embedding models live in different spaces, so a similarity score between them means nothing. Re-embedding the whole corpus with one model when an image appears keeps every comparison valid. The OpenRouter embeddings documentation confirms that some models accept image input and return joint text-and-image embeddings.
The ingestion list covers pdf, doc, docx, xls, xlsx, pptx, csv, tsv, txt, json, html, xml and common text files. Images, audio and video go through OpenRouter’s multimodal processing and become text before indexing, so everything ends up in one index and one chat.
Split providers: who does what
The README describes Hermes as the planner and reranker, but in the code Hermes also writes the final answer for most open-ended questions. A single query runs through these steps:
Plan (Hermes-4-70B, Nous). Hermes rewrites the question into a search query with up to 8 keywords and related terms.
Score (OpenRouter embeddings + local TF-IDF and LSI). The enriched query is scored against every chunk, as described above.
Rerank (Hermes-4-70B). Hermes reorders the top 8 chunks.
Answer. For summary, general, comparison and how-to questions, Hermes writes the answer. For the other types, or if Hermes fails,
liquid/lfm2-8b-a1bon OpenRouter writes it. If neither is available, Documetro falls back to a templated extractive answer built from the best sentences.
So the separation is real but narrower than the README suggests. The embedding model is always separate from the answering model. The planning model and the answering model, however, are often the same.
The fallbacks do make the system degrade gracefully. Without NOUS_API_KEY, planning and reranking are skipped and the lightweight model answers. Without OPENROUTER_API_KEY, the embedding score is zero and retrieval runs on the local layers alone. Optional overrides in .env include OPENROUTER_MULTIMODAL_MODEL=openai/gpt-4o-mini for file analysis and a custom NOUS_BASE_URL.
It guesses what kind of question you asked
Before answering, Documetro classifies the question with simple trigger words and checks them in a fixed order. The README lists seven types, including Entity, Language and Numerical Analysis. The classifier in engine.py uses a different set:
Type | Triggered by | Effect |
|---|---|---|
summary | “summarize”, “summary”, “overview”, “recap” | Broader evidence (up to 8 chunks); Hermes answers |
count | starts with “how many”, or contains “number of” | +0.04 for chunks containing digits |
how | starts with “how” | Prefers sentences with action verbs; Hermes answers |
date | starts with “when”, or contains “date” | +0.04 for chunks containing digits |
who / where / what | the first word | Fact-style answer template |
list | “list”, “what are”, “which” inside the sentence | +0.03 for chunks with lists or tables |
compare | “compare”, “difference”, “versus”, “vs” inside the sentence | +0.04 for chunks with comparison words; Hermes answers |
general | anything else | Hermes answers |
Trigger-word routing is crude but better than treating every question the same. It has blind spots, though, because of the check order. “What are the risks?” matches what before it reaches list. “Compare A and B” at the start of a sentence is classified as general, because the compare markers expect a space before the word. And numerical questions do not get special treatment for CSV or tabular chunks; they only get a small bonus for chunks that contain digits.
Evidence the model can cite
The answering model receives structured evidence, not bare text fragments. Each item carries the document name, the section title, a locator (page or section) and an excerpt. The excerpt size depends on the question type:
Fact, list, count and how-to questions: 3 sentence-level excerpts (4 for lists) of about 320 characters, each with its own relevance score.
Summary, general and comparison questions: up to 8 chunk-level excerpts of about 520 characters. Here the score is fixed at 1.0, so it carries no ranking information.
Either way, the model sees where each passage came from, which lets answers point back to specific files. The interface also shows cross-document topics (k-means clusters over the LSI vectors) and document pairs with a TF-IDF similarity of at least 0.18.
Running it yourself
Setup is small. Clone the repository, put your keys in .env, and run:
./run.shto start./run.sh status,./run.sh stopand./run.sh restartfor the usual operationspython3 -m unittest discover -s tests -vto run the tests
The code follows the pipeline:
File | Role |
|---|---|
| Local extraction and multimodal preprocessing |
| OpenRouter client: embeddings, file analysis, light answers |
| Nous Hermes client: planning, reranking, answers |
| Chunking, hybrid scoring, clustering, evidence and answers |
| Workspace, upload queue, background indexing, cleanup |
| FastAPI entry point for the API and web UI |
The whole architecture is readable in an afternoon, which is rare for a RAG system.
The honest limits
It is not a local tool. Embeddings, multimodal extraction, query planning, reranking and answer generation all call hosted models on OpenRouter and Nous. Your documents’ contents leave your machine. Only extraction, chunking, TF-IDF, LSI and the workspace run locally.
Nothing persists. Uploads, indexes and runtime files are cleaned on reset, on shutdown and on the next start. That protects sensitive batches, but every restart means re-uploading and re-indexing.
Each query costs several remote calls. TF-IDF and LSI are free, but a typical question triggers a query embedding plus two or three Hermes calls (plan, rerank, often answer). Latency depends on those providers.
The documentation runs ahead of the code. The README’s question types and its description of Hermes as a non-generating planner do not match engine.py. Read the code before relying on either.
It is a small solo project. At the time of writing it has 4 commits, no releases, no hosted tier and no support. What you get is a readable codebase and a clear hybrid design under GPL-3.0.
People Also Ask
Which file formats can Documetro ingest?
PDF, DOC, DOCX, XLS, XLSX, PPTX, CSV, TSV, TXT, JSON, HTML, XML and common text files. Images, audio and video are also accepted, but OpenRouter’s multimodal models convert them to text before indexing.
Can Documetro answer questions across several documents at once?
Yes. All files in a batch share one index and one embedding space. Each piece of evidence carries its document name and a page or section locator, so answers can point back to specific files.
Which embedding model does it use for text-only corpora?
By default nvidia/llama-3.2-nv-embedqa-1b-v2:free, set through OPENROUTER_EMBEDDING_MODEL. If the corpus contains any image, the whole corpus switches to nvidia/llama-nemotron-embed-vl-1b-v2:free.
Which model writes the answers?
For summary, general, comparison and how-to questions, Nous Hermes-4-70B, when NOUS_API_KEY is set. Otherwise, or for other question types, liquid/lfm2-8b-a1b via OpenRouter. With no remote model available, it returns a templated extractive answer.
Is Documetro fully local or offline?
No. It needs API keys for OpenRouter and Nous. Extraction, chunking, TF-IDF, LSI and the workspace are local; embeddings, multimodal conversion, planning, reranking and answering are remote.
Where this leaves you
Documetro will not replace a document platform, and it does not try to. Its value is as a compact, readable example of hybrid retrieval: dense embeddings weighted at 0.62, TF-IDF at 0.18 and LSI at 0.10, plus title, anchor and question-type bonuses, a reasoning model that plans, reranks and often answers, and one vector space per corpus.
If your current design is “embed everything and hope”, clone the repository and read engine.py rather than only the README. The code is where the real design lives, and GPL-3.0 lets you reuse it.
Sources
Hybrid Retriever Evolution for Multimodal Document Reasoning Agents (arXiv 2606.29648)
Hybrid Retrieval for Hallucination Mitigation in Large Language Models
This article includes content created with AI.