InferLens

The problem

Anyone who has had to answer a question from a 200-page contract, manual, or filing knows the real work isn’t reading. It’s finding, and then trusting, what you found. General-purpose chat assistants will happily answer questions about a document they were never given, inventing plausible page numbers along the way. For any high-stakes domain that’s worse than useless.

InferLens is a retrieval-augmented question-answering system built around one non-negotiable rule: every claim in an answer is grounded in a retrieved passage, and every passage points back to the exact page it came from. If the document doesn’t support an answer, the system says so rather than guessing.

How it works

A PDF is extracted page by page with PyMuPDF, then split into ~500-word overlapping chunks. The detail that matters: a chunk never crosses a page boundary. Each chunk carries the page number it came from, so a citation can resolve to “page 12” rather than an opaque passage index. That costs a few extra chunks on documents with short pages, but it’s the right trade for a product where “where did this come from?” is the most common follow-up question.

Chunks are embedded with all-MiniLM-L6-v2 and indexed in FAISS. At query time the question is embedded the same way, the top five nearest passages are retrieved, and they’re handed to Claude with a system prompt that allows it to use only those passages, emitting inline [1], [2] markers after each claim. When the context is insufficient, the model returns a fixed refusal string instead of improvising.

Decisions worth surfacing

Exact search, not approximate. The index is a faiss.IndexFlatL2: exact, brute-force nearest-neighbour. For single-document corpora (hundreds to low thousands of chunks) an approximate index buys nothing but a new class of silent recall bugs. I wrote up the full trade-off in Exact over approximate.

ONNX embeddings to fit the box. Running the embedding model through ONNX Runtime (via fastembed) instead of PyTorch cut per-process memory from ~400 MB to ~100 MB. That’s the difference between fitting on a 512 MB hosting tier and OOM-ing on cold start. Same model, same vectors, a third of the footprint.

Citations are a contract, not a nicety. The generation prompt requires a citation marker after every factual claim and forbids inventing page numbers or passages. Grounding is enforced at the prompt boundary, and the UI renders the markers as clickable references to the source passages, so a reader can verify any sentence in one click.

Outcome

InferLens began as my graduate capstone (originally DocuMind) and became the reference design I reach for whenever a RAG system has to be auditable rather than merely conversational: exact retrieval, page-level provenance, and a generation step that refuses to exceed its evidence.