Phase 3 · Months 2 to 4

Build retrieval that finds the right evidence

Treat retrieval as a system

Retrieval-augmented generation (RAG) is more than a vector database. A production retrieval system handles ingestion, parsing, chunking, metadata, indexing, ranking, authorisation, freshness, deduplication, and evaluation. Ask whether the system retrieved the evidence that a correct answer needs. The fact that the model produced an answer tells you little.

Learn the retrieval basics

Learn embeddings, similarity search, hierarchical navigable small world (HNSW) indexes, keyword search with the BM25 ranking function, hybrid search, metadata filters, query rewriting, reranking, and chunking. Start with Postgres, pgvector, and Postgres full-text search, because you can follow the whole architecture in one place. Move to a dedicated search engine only when your data needs one.

Learn when agents should search with tools

Agents also find context by taking actions. Coding agents mostly search with grep, file reads, and tool calls instead of embeddings, and they often do better that way. Learn when tool-based search beats a vector index. Also learn the other ways to manage context: compacting long sessions, giving subagents their own isolated context, persistent memory files, and short tool outputs. Some platform job postings call this work a "context engine".

Keep the index fresh and permission-aware

Make ingestion idempotent. Give every source a stable ID, a content hash, a version, and access-control metadata. Re-index only what changed, and remove deleted documents. Apply access control during retrieval, not after it. If Alice cannot read a document, the document must never reach her candidate context. Learn tenant isolation, row-level security, and how to pass a user's identity through the system.

Build two retrieval systems

On your product, build a labelled set of real queries with the documents each query should retrieve. Measure recall at k, and separate retrieval failures from generation failures. For Forge, index a repository with its docs, decision records, issues, and pull requests. Answer questions such as "Why does service X use this database?" with citations. Compare vector search, hybrid search, and tool-based search on the same questions.

You are done when

When an answer is wrong, you can tell whether retrieval or generation caused it. You can show retrieval metrics, explain your chunking choices, prove that permission filtering works, and update the index without a full rebuild. You can argue when an agent should search with tools instead of a vector index.

Exit checklist