RAG System Architecture

A RAG system is two pipelines sharing an index: offline ingestion turns sources into retrievable units; online query selects the right units for a model or agent.

The conceptual spectrum lives in Retrieval-Augmented Agents. This page describes the architecture.

The evaluation target lives in Retrieval Quality: answerability, citation coverage, freshness, context sufficiency, and faithful use of retrieved evidence. Keep the split clear. Architecture says how retrieval is assembled; quality says whether it found the right thing.

Offline Ingestion Pipeline

Steps:

1. Load and parse: preserve structure, title, URL, source path, and timestamp.
2. Normalize: strip boilerplate, keep headings, preserve code blocks when relevant.
3. Chunk: split by semantic structure before fixed size. Overlap only where boundary loss matters.
4. Embed: compute vectors for semantic recall.
5. Index keywords: exact names, commands, file paths, identifiers.
6. Store metadata: citations and freshness require it.

Most RAG quality problems are ingestion problems disguised as prompt problems.

Online Query Pipeline

Steps:

1. Query transform: rewrite, expand, or split multi-hop tasks.
2. Hybrid retrieve: combine semantic and exact search.
3. Rerank: prioritize precision after cheap recall.
4. Assemble context: pack citations and enough surrounding page context.
5. Generate or act: answer, call tools, or continue agent loop.
6. Write back: update memory when useful synthesis emerges.

Kevin Wiki Implementation

The wiki uses a document-first variant:

• Markdown pages are the source corpus.
• build-index.ts produces catalog/backlink/discovery artifacts.
• qmd provides retrieval and embedding.
• _index.md gives a human-readable catalog.
• Timelines and frontmatter preserve provenance.
• qmd update && qmd embed refreshes the retrieval layer after edits.

This is intentionally not a black-box vector DB. The corpus remains readable and editable.

Design Implications

• Keep ingestion and query concerns separate.
• Do not use embeddings alone; exact identifiers matter.
• Always store enough metadata to cite and refresh.
• Reranking is often cheaper than overhauling the whole embedding model.
• Evaluate retrieval recall before blaming the generator.
• Re-index after maintenance or large doc edits.

Failure Modes

• Stale embeddings after docs change.
• Chunks too small to preserve meaning.
• Chunks too large to match precise queries.
• Missing exact keyword index.
• No source metadata.
• Agent reads a retrieved snippet but not the page context.

Architecture Position

---

Timeline

• 2026-07-01 | Architecture category refresh added this page to the Memory, context, and retrieval family, linked it to Architecture System Map, and kept it standalone because it owns this boundary: Ingestion/query split for retrieval augmented generation. Source: User request, 2026-07-01

• 2026-06-18 | Expanded with Kevin's qmd/wiki implementation, ingestion/query diagrams, and failure modes. Source: User request, 2026-06-18

• 2026-06-19 | Linked Retrieval Quality as the evaluation layer for this architecture. Source: whole-wiki concept review, 2026-06-19

• 2026-06-02 | Page created. Captured offline-ingestion / online-query split, chunking as central knob, hybrid retrieval, reranking, separate retrieval/generation evaluation, and mapping to qmd/context engine. Source: compiled from RAG system-design literature