Pattern Matrix/White Paper/M1

M1 · Hierarchical Retention

The problem it solves

The information an agent has to "recall" at startup spans completely different scopes. A company's security policy is permanently valid for everyone; a user's preferences follow only that person; session progress matters only for this run; tool results expire after a single turn. Stuffing all of it into one prompt hits two walls at once: tokens blow up, or the critical information gets drowned in noise.

Hierarchical retention slices this information into tiers by scope, and each tier has its own storage location, expiration time, and loading strategy. The agent loads from coarse to fine at startup and drills down tier by tier when needed. The core problem it solves is not "what to store" but "how to tier"—a memory system without tiers degrades over time into a warehouse where nothing can be found.

Why the coordinate is Memory × Hierarchy

• Vertical axis · Memory: It governs what the agent stores across sessions, across users, and across projects, corresponding to the classic working / session / long-term three-tier memory division. It is the foundational pattern of the memory module.
• Horizontal axis · Hierarchy: The tiers stand in a subordinate relationship—an outer tier's settings are the defaults for inner tiers, and an inner tier can override the outer one. Data rises and falls between hot / warm / cold, which is a naturally hierarchical storage structure, not a Chain and not a Loop.

Core mechanism

1. Tier by scope: The primary basis for tiering is "who should see this piece of information," not "content type." The typical four tiers are user-level (unchanged across all sessions), project-level (shared within a project), session-level (valid for the current run), and ephemeral (expires after one turn). Scope determines which tier a piece of information belongs to.
2. Independent backend per tier: Each tier's storage backend is chosen separately according to its access pattern. The user tier is high-read, low-write, and permanent, which suits an indexed relational store; the project tier suits a file system under git management; the session tier is high read-write with a TTL, which suits Redis; the ephemeral tier is just an in-memory dictionary. Using one backend for all four tiers is the mark of a beginner implementation.
3. Load from coarse to fine: At session startup, the prompt is assembled from coarse to fine by scope, and each tier has its own token budget. One benefit of tiering is that no matter how long a session runs, it cannot squeeze out the user tier.
4. Promotion, demotion, and eviction: A mature implementation lets memory flow automatically according to access pattern—frequently referenced memory is promoted to a higher tier, memory not accessed for a long time is demoted, and expired memory is cleared per a retention policy. This turns a static schema into a dynamic working set.

Production scenarios where it fits

• Agents reused across sessions, users, and projects: programming coaches, long-term assistants, and internal enterprise agents need to remember "who this user is, what this project is about, and where the last conversation left off."
• Multi-tenant SaaS agents: Tiering by scope naturally provides isolation—user A's preferences do not pollute user B's session, and project X's rules are not carried into project Y. Scenarios that cannot tolerate cross-tenant data leakage, such as finance, healthcare, and contract review, rely on this especially.
• Enterprise developer agents: Claude Code's multi-tier CLAUDE.md is exactly this pattern. The person who writes CLAUDE.md is acting as a memory architect, placing security rules, personal preferences, and project rules in different tiers.

Where it commonly goes wrong

• Copying tier count instead of matching need: Copying CoALA's three tiers (working / episodic / semantic) outright when the product only needs two, leaving a pile of empty scaffolding in the code with nothing to put in it. The tier count should match the number of scope kinds the product actually has.
• No schema on the user tier: Letting the agent write free text like "Xiao Li knows decorators" produces, half a year later, ten phrasings for the same concept and a cliff drop in retrieval hit rate. High-frequency fields must be put into a typed schema.
• Designing everything as startup injection: The user tier and project tier are startup injection, the session tier is progressive reading, and the ephemeral tier is real-time assembly. Mashing the three access patterns into one will make the session tier's tokens explode after it accumulates dozens of turns.
• Forcing tiering onto fully stateless scenarios: Single-shot Q&A and one-off ETL transformations do not need cross-session memory.
• Eviction without a reason log: Compliance (GDPR right-to-be-forgotten) and debugging both require proving "what was deleted, why, and when." Without a log, this cannot be proven.

Key metrics

• Working set hit rate (healthy range ≥ 80%): whether all the memory the agent truly needs during reasoning is in the prompt. Below 30% means the tiering is not working, and no matter how elegant the schema is, it is wasted. This is the primary metric for judging whether the tiering is done right.
• Per-tier token share (allocated by budget): whether the tokens each tier loads into the prompt stay within budget. A tier that consistently exceeds its budget needs truncation or to be pushed down to on-demand loading.
• Cross-tier pollution rate (healthy range near 0): whether the user tier gets polluted by incidental information from a single session, or whether data leaks between tenants. In multi-tenant scenarios this is a make-or-break metric.
• Total startup tokens (measured against a baseline): how much is saved compared with a "full history dump." Done right, tiering typically saves 70-80% of startup tokens.

Minimal implementation skeleton

Define tiers (scope from coarse to fine):
                USER    → permanent / high-read / strong schema / relational store
                PROJECT → permanent / team-shared / file system + git
                SESSION → TTL 24h / high read-write / Redis
                TURN    → TTL 5min / very high frequency / in-memory dict

            Write:    select backend by tier, record last_modified
            Read:     query from fine to coarse, return on inner-tier hit (inner overrides outer)
            Assemble: at startup build prompt from coarse to fine, truncate each tier by token_budget
            Evict:    clear expired content and record a reason log (compliance + debug)
            

Four engineering points for landing this: each tier's storage backend must genuinely use a different choice; assembly must truncate by token budget rather than stuffing in everything; writes must do "write routing" (a small model or a rule decides which tier to write to); and eviction must keep a reason log.

Enterprise landing example

An execution-oriented agent at a payroll SaaS company builds its memory into three tiers. L1 is the current stream of consciousness—the minimal necessary set for this moment, holding only the information genuinely useful for advancing the immediate next step, kept extremely lean. L2 is the in-progress milestone log—recording where the task stands and what was just completed, maintaining task context across turns. L3 is cross-turn reusable experience—keeping the judgments and practices distilled from this kind of task as a long-term asset. The three tiers each manage one timescale: L1 manages "now," L2 manages "this task," and L3 manages "across tasks." This tiering lets the agent, over a long process, neither be drowned in irrelevant history (L1 stays lean) nor lose memory between turns (L2 is continuously maintained), and grow more proficient with use (L3 keeps accumulating). The accompanying engineering decisions include an independent token budget per tier, real-time assembly of L1 rather than preloading, and a trust check before writing to L3 to keep incidental single-run information from polluting long-term experience.

Relationship to other patterns

• Progress Tracking (M3): M3 is the hottest tier within the hierarchy. The todo list is the agent's register, frequently updated and loaded on every reasoning step; it is essentially the part of the session tier dedicated to managing "where we are."
• RAG (M2): Tiering solves "how long-term known memory is loaded by scope," while RAG solves "how the massive knowledge that no scope can hold is queried on demand." Tiering manages the known; RAG manages the external.
• Failure Journal (M4) / Procedural Memory (M5): These two are dedicated partitions within the long-term memory tier—M4 stores failure cases, M5 stores successful moves. Tiering is the container; they are the contents inside it.
• Semantic Compression (perception module): When the session tier fills up with tokens, semantic compression is triggered to slim it down, which is a supporting mechanism for tiering within a single tier.

What this pattern teaches

The essence of hierarchical retention is not database schema design but giving the agent a working set manager—it decides which memory stays resident in RAM, which is swapped in on demand, and which is swapped out for the long term.

This page is part of the ADPS pattern white papers. Back to the pattern matrix, or see the runnable code catalog.