Studio agent — built in-houseRAG · Grounded Q&A
Document Intelligence Agent
Answers from your documents with citations — and refuses when the answer isn’t there.
$npm run rag "your question"
$ npm run rag "your question"
Document Intelligence Agent · 4-document sample handbook
In-corpus question — answered with citations. Asked about on-call escalation and Sev-1: the agent retrieves the right sections and answers, every claim cited to a chunk id.
4
Documents in the corpus
BM25
Retrieval, no embedding API
Cites
Every claim to a section
Refuses
When the answer is absent
The failure mode of document Q&A is not a wrong search — it is a confident, well-written, completely fabricated answer. A model asked something the corpus does not cover will, by default, fill the gap plausibly, and a plausible fabrication in a legal, medical, or financial document is worse than silence. A useful grounded agent therefore needs two disciplines at once: cite what it uses, and refuse what it cannot support.
Retrieval is real but dependency-free — a BM25 index over heading-anchored chunks, so a citation points at an actual section and there is no external embedding service or key involved. The agent issues its own searches, refines the terms when the first pass is thin, and only answers once it has grounding. Every factual sentence carries a chunk id. When repeated searches turn up nothing relevant, the agent is required to refuse and name what the corpus does cover instead.
A runnable agent with two recorded runs that show both halves of the discipline. Asked an in-corpus question about on-call escalation, it retrieved the right sections and answered with citations. Asked about leave policy — which the sample handbook does not contain — it searched twice with different terms, confirmed the absence, and refused rather than guessing. Both transcripts are below, on a four-document sample handbook.
Capabilities
Built with
3 tools, one decision loop.
list_documents
Surfaces the documents and their section headings, so the agent knows the shape of what it can and cannot answer from.
search
BM25 lexical search over heading-anchored chunks, returning ids and text — the agent calls it more than once, refining terms.
read_section
Pulls the full text of a chunk to confirm a citation before it is used in the answer.
The behaviour that matters most is knowing when to stop.
$ npm run rag "your question"
Document Intelligence Agent · 4-document sample handbook
Out-of-corpus question — refused honestly. Asked about leave policy, which the handbook doesn’t cover: two searches, then an honest refusal instead of a fabricated answer.
The calls that decide whether it survives contact.
A refusal is a correct answer
The hardest thing to get a language model to do is say “I don’t know.” The agent is instructed, and shown by its transcript, that declining an unsupported question is a success — because a confident wrong answer is the one failure mode that actually costs a client trust.
Search more than once before concluding absence
Absence is a strong claim. Before the agent refuses, it re-queries with different terms, so “not in the corpus” means genuinely not found, not found-under-the-wrong-words.
BM25, not a vector service
A lexical index keeps the whole thing keyless, offline-capable, and auditable — a citation resolves to a real heading you can open. For a corpus this size it is also simply faster, with nothing to embed or host.
Full transparency
This is one of our own studio agents, not client work. It runs on sample data we built ourselves, and the terminal above replays a real recorded run — the reasoning, tool calls, errors and timings are exactly what happened, only the playback is paced for watching. We show it because it is the honest version of “we build AI agents”: code you could run, not a slide.