Build a working retrieval system
Implementation Guide
Build a retrieval-augmented system in plain Python, from corpus preparation and BM25 through generation, evaluation, permissioned retrieval, policy context, and connector contracts.
Core build
- 1Setup and CorpusPrepare a clean corpus and a reproducible Python environment for every chapter that follows.
- 2The Baseline: BM25Build an inspectable keyword baseline before adding semantic retrieval.
- 3Add EmbeddingsTurn the corpus into a verified vector artifact with a hosted or local embedding model.
- 4Vector Retrieval (FAISS kNN)Search the embedding space with FAISS and compare semantic results with BM25.
- 5Hybrid Retrieval with RRFFuse keyword and vector rankings while preserving evidence from both retrievers.
- 6GenerationGenerate grounded answers from retrieved evidence and reject invalid provenance.
- 7EvaluationMeasure retrieval coverage, ranking quality, and grounded-generation behavior with validated judgments.
- 8Where to Go NextChoose the next change from measured failures, route model work by difficulty, and run an observable bounded search-revision loop.
Governed Search
- 9Build Permissioned RetrievalAuthorize records before ranking, scope derived artifacts and caches, and measure filtered-vector recall against an exact eligible baseline.
- 10Carry Policy ContextSeparate trace correlation from verified authority, validate every protected receiver, and reauthorize saved state when a trajectory resumes.
- 11Build Connector ContractsClassify connectors by role, return typed outcomes, budget retries, and reconcile ambiguous effects before replay.