Pattern Matrix/White Paper/M2

M2 · RAG Pipeline · Retrieval-Augmented Generation

What problem it solves

An enterprise knowledge base routinely holds hundreds of thousands of documents plus a history of tickets, and no LLM context window can fit all of it. But the agent cannot answer "I don't know" for that reason. On each question it needs to pick, out of a vast corpus, the few most relevant documents and temporarily place them into context. This is what RAG addresses: pulling in facts that live outside the input, on demand.

By 2026, the engineering difficulty of production-grade RAG is no longer "can it find anything" but "does it know how to search." Naive RAG is one query, one return, whereas RAG that actually ships to production lets the agent drive multi-step retrieval—if the first batch of results is unsatisfactory it rewrites the query and searches again, when it finds a candidate answer it actively looks for counterexamples to verify, and it cross-compares across multiple corpora.

Why the coordinate is "Memory × Chain"

• Vertical axis · Memory: RAG is an attached knowledge base that pulls in facts outside the input. At its core it is about "which facts to record and how to recall them," which belongs to the part of the memory module that manages declarative knowledge.
• Horizontal axis · Chain: RAG is a sequential pipeline—the query is embedded first, then top-K is retrieved, then reranked, then injected into the prompt, and finally handed to generation. The output of each step is the input to the next; each step passes to the one after it, the textbook form of chain passing. The name "RAG pipeline" itself points to its chained backbone. Even when hybrid retrieval (semantic plus BM25) is used to select sources, that is one link in the chain and does not change the overall sequential-pipeline form.

Core mechanism

1. Two pipelines meet at the vector store: RAG is not a single step. The offline side chunks documents, embeds them, and builds the index; the online side embeds the query, retrieves top-K, reranks, and injects into generation. The two independent pipelines share one vector store.
2. Hybrid retrieval: Embeddings are good at semantics ("ACME earnings report" ≈ "ACME performance"), but exact keywords (product names, error codes, specific IDs) are often missed. Combine semantic retrieval with BM25 keyword retrieval, then fuse with RRF (Reciprocal Rank Fusion).
3. Context-enhanced chunking: Once a chunk is split off, the original context anchors are lost. Prepending a short context summary (generated with a cheap model) to each chunk before embedding can significantly reduce the retrieval failure rate.
4. Agent-driven multi-step retrieval: Upgrade RAG from a one-shot pipeline to an agent-driven process—decompose sub-queries, iteratively rewrite, find counterexamples, triangulate across corpora, and synthesize weighted by evidence strength. This is the engineered version of a human researcher's methodology.
5. A hard iteration ceiling: Multi-step retrieval must have a hard ceiling, otherwise the agent will keep feeling "it isn't enough yet" and retrieve continuously.

Suitable production scenarios

• Large-scale natural-language knowledge bases: academic literature reviews, legal case libraries, customer-service FAQs. Large volume, containing synonyms and implicit relationships, relatively stable, public or semi-public—these are the best scenarios for RAG.
• Complex queries across many documents: scenarios that need answers synthesized from multiple sources and that need traceable citations.
• Organizing enterprise know-how: organizing the knowledge scattered across wikis, Confluence, Slack, email, and tickets into a form the agent can retrieve in five minutes. This is RAG's real proposition for enterprise deployment.

Where it tends to go wrong

• Deploying RAG before governing the knowledge source: 90% of enterprise RAG failures are rooted not in technology but in the knowledge itself never having been organized—the wiki holds many documents that contradict each other, key decisions are not in any document, all the versions are outdated, and the same concept has several names. Embedding garbage into vectors and retrieving it back is still garbage, and worse than not doing RAG at all. Do ontology and knowledge governance before building enterprise RAG.
• Forcing RAG where Agentic Search belongs: dev workflows, code search, and internal sensitive data—these three categories mostly should not use RAG. Code changes by the minute, and a just-written function cannot be found before it is indexed (data latency); the index itself is an attack surface, and a malicious insider may peek beyond their authorization (a security hazard). Such scenarios should use grep- and glob-based Agentic Search.
• Stopping at one query, one answer: when the key information is not found on the first try, naive RAG either says it doesn't know or silently fabricates. Production-grade RAG must be able to rewrite the query and search again.
• Blindly chasing retrieval precision: choosing an embedding model that is a few points better is a marginal optimization; whether the agent's multi-step retrieval resembles a person who truly knows how to search is the qualitative change. Pushing chunk recall to 80% is not necessarily useful; the final synthesized quality resembling a senior researcher's is the real value.
• Untraceable citations: in academic, legal, and medical scenarios, every conclusion must trace back to the original chunk plus page number—this is a necessary condition for compliance.

Key metrics

• Retrieval failure rate (lower is healthier): the fraction of cases where no relevant chunk appears in top-K. Stacking the three—context enhancement plus hybrid retrieval plus reranking—reduces the failure rate by about two-thirds in industrial data.
• Retrieval hit rate / citation rate (healthy zone ≥ 70%): the fraction of retrieved chunks actually cited by the agent. Too low indicates noisy recall or a query written too broadly.
• Number of multi-step retrieval rounds (healthy zone 3–5 rounds with a hard ceiling): the actual number of iterative retrieval rounds. Continuously hitting the ceiling indicates the task is genuinely hard or that single-pass recall is too poor.
• Final synthesized quality (business judgment): whether the agent's output resembles the work of a senior practitioner. This is the real criterion for judging a RAG system; reaching 70% can already change how a team works.

Minimal implementation skeleton

Offline indexing side:
                doc → chunk → prepended context summary (cheap model) → embed → vector store + BM25 index

            Online retrieval side (agent-driven, multi-step):
                decompose sub-queries
                iterative retrieval per sub-query:
                    semantic top-150 + BM25 top-150 → RRF fusion → reranker top-20
                    agent assesses whether it is enough → if not, rewrite query and search again (up to N rounds)
                form hypothesis → negate it as a new query to find counterexamples
                triangulate across multiple corpora (weight chunks that appear in several groups)
                synthesize weighted by evidence strength
            return answer + evidence + counter_evidence + full trace (with traceable citations)
            

Four points for engineering deployment: the retriever layer does context enhancement plus hybrid retrieval plus reranking; set a hard ceiling on max_iterations plus a hard ceiling on the token budget; the trace records every retrieval in full; citations must be traceable to the original document plus page number.

Enterprise deployment example

A research institution's academic literature-review agent had to answer arbitrary researcher questions over a million-paper library. The first version used keyword retrieval and returned five thousand papers already read; switching to embeddings returned only the 2018–2020 "classics." After a rewrite, all five canonical patterns were on: decompose sub-queries, iteratively refine, negate hypotheses to find counterexamples, triangulate across arXiv plus bioRxiv plus PubMed, and synthesize weighted by evidence strength. Six business decisions accompanied this—corpora kept in separate stores but not separate tables (peer-reviewed weight 1.5, preprint 1.0, unpublished 0.5), time-decay weighting (half-life 18 months), counterexample retrieval forced on, citations at the chunk level, and an author-relationship graph across corpora. After deployment the agent returned a precise twenty papers plus five counterexamples plus three implicitly related ones, each with a traceable citation; a researcher's review went from six hours down to an hour and a half, with higher quality.

Relationship to other patterns

• Layered Retention (M1): M1 addresses "how long-term, already-known memory is loaded by scope," while RAG addresses "how to look up, on demand, the vast knowledge that the scope cannot hold." Layering manages the known; RAG manages the attached.
• Procedural Memory (M5): RAG manages declarative memory (retrieving factual knowledge), while M5 manages procedural memory (reusing the ability to get work done). RAG retrieves "what is known," M5 retrieves "how a past task was done." The two play in concert in enterprise agent deployment.
• Context Triage (perception module): RAG pulls candidate information in, and triage decides which of these candidates enters first. RAG is recall; triage is the entry ordering.
• Progressive Discovery (perception module): the two are structurally similar but have different goals. Progressive discovery is the agent exploring structure in an unfamiliar space, while RAG retrieves relevant fragments from a known large library. One leans cognitive, the other leans retrieval.

What this pattern teaches

RAG is not "the more precise the better" but "the better it searches the better"—its engineering ceiling lies not in the model, the embeddings, or the vector store, but in the organizational quality of the knowledge source itself.

This page is part of the ADPS pattern white papers. Back to the pattern matrix, or see the runnable code catalog.