Jruva
A multi-provider assistant for document questions.
Hybrid retrieval, streamed responses and provider failover. Evaluated against dense and lexical baselines, with a close look at what the numbers mean.
Read the engineering notes
The system
FastAPI serves document-question-answering workflows to a Next.js/TypeScript interface. BGE-small embeddings in Qdrant and BM25 lexical retrieval are combined with reciprocal rank fusion. Retrieval is scoped to a user’s own documents.
The reliability work
Gemini, Groq and OpenRouter routing includes retries, circuit-breaker failover, response caching and per-user quotas. The project’s CI covers backend tests, migrations, linting and a TypeScript build.
A result that changed the story
On the recorded synthetic development benchmark, dense retrieval reached MRR 0.910; BM25 reached 0.810 and hybrid fusion 0.900. Adding another retrieval method did not improve the overall result.
What this does not establish
The corpus has only 50 synthetic passages. Ninety answerable queries were scored; ten unanswerable queries were excluded from these retrieval metrics. The same dataset was used during later tuning, so it is not an independent test of generalization or answer quality.
Timing, precisely
The median of about 5 ms wraps query embedding plus in-memory dense search on CPU, with the embedding model already loaded. It excludes model startup, HTTP/network time, reranking and LLM generation. Measured on an AMD Ryzen 9 7940HX; the run was not repeated. Earlier 0.36 ms hybrid timing used cached embeddings and is not comparable.