Legal RAG
Retrieval-augmented generation over legal filings, with citations that hold up.
Solo build
- Citation-first
- answer contract
- Chunk-level
- source attribution
The problem
A legal answer without a verifiable citation is worse than no answer — it is a confident claim a lawyer cannot check. Generic RAG returns fluent summaries that quietly blend sources, which is precisely the failure mode this domain cannot tolerate.
Approach
Invert the usual contract: retrieve first, answer only from retrieved spans, and refuse when retrieval is weak. Every sentence in the output maps to a chunk the user can open.
Architecture
- Documents are chunked on structural boundaries (sections, clauses) rather than fixed token windows, so citations land on semantically complete units.
- Embeddings indexed for vector similarity search, with metadata filters for document and jurisdiction.
- Retrieved chunks are passed to the LLM under a prompt contract that forbids unsupported claims.
- The response renders inline citations that resolve to the exact source span.
Design decisions
Structural chunking over fixed-size windows
Fixed windows split clauses mid-sentence, producing citations that point at fragments. Structural boundaries cost more preprocessing but make every citation independently readable.
Refuse rather than approximate on weak retrieval
In legal context a plausible wrong answer is the worst outcome. Below a retrieval confidence threshold the system says it does not have the source, which users trust more than a hedge.
What was hard
- Long filings blow past context limits, so the ranking stage matters more than the generation stage.
- Keeping the model from smoothing over contradictions between two retrieved passages instead of surfacing them.
What I took from it
- Most RAG quality problems are retrieval problems wearing a generation costume. Improving chunking and ranking moved answer quality far more than changing models.
Stack
Other work
- Product
Atriveo
Job-search platform with a 5.0-rated Chrome extension and live customers.
- Applied Research
FOMC Intelligence
NLP pipeline turning Federal Reserve communications into market signals.
- Machine Learning
MRI Tumor Viewer
Brain tumor segmentation running entirely in the browser.