Pattern Matrix/White Paper/R5

R5 · Talker-Reasoner · Dual-Process Architecture

What problem it solves

Many reasoning patterns assume something they should not: that the user can wait. Giving an agent 24 seconds to think things through is fine in an asynchronous task, but it is a disaster in real-time interaction. In a conversation, a 200-millisecond pause and the user disconnects; in a voice agent, one second of silence is awkward; in real-time collaboration, three seconds of no response breaks the rhythm. The core issue is not that the agent thinks slowly, but that it says nothing while it thinks. Users do not mind an agent taking 24 seconds to think; they mind 24 seconds of silence.

Talker-Reasoner splits the two conflicting goals — real-time response and deep reasoning — across two agents. One Talker always responds within 200 milliseconds and keeps the conversation flowing; one Reasoner runs deep analysis slowly in the background; the two coordinate through a shared belief state. The goal is not higher quality but hiding latency from the user's perception. With the same amount of thinking time, the experience is worlds apart.

Why the coordinate is Reasoning × Hierarchy

• Vertical axis · Reasoning: It separates two reasoning modalities, real-time response and deep reasoning, into a dual-process reasoning architecture — corresponding to Kahneman's System 1 (fast, automatic, low-cost) and System 2 (slow, deliberate, high-cost). This is an engineering mapping of cognitive science onto agents, not a single reasoning channel.
• Horizontal axis · Hierarchy: The Talker is in front (fast, shallow) and the Reasoner behind (slow, deep). This is a layered division of labor, not parallel competition. The two differ in responsibility, model, and timing. The Reasoner's output influences the Talker's answer through the belief state, an upper-layer–lower-layer collaboration.

Core mechanism

A single dual-process round consists of three parts:

1. Talker responds immediately: Using a cheap, fast model, it replies the moment it sees user input (200–800 ms), keeping the conversation flowing, responding to emotion, and asking clarifying questions. Its system prompt carries one hard constraint — do not give concrete advice, that is the Reasoner's job. The Talker handles the vast majority of conversation turns.
2. Reasoner thinks deeply, asynchronously: Using an expensive, slow model, once triggered it runs deep analysis in the background (10–30 s), and when finished it writes structured conclusions to the shared belief state. It does not block the conversation.
3. Belief state coordination + natural bridging: The two agents communicate through one shared state; writes must take a lock to avoid race conditions. After the Reasoner finishes, the Talker weaves the analysis into its reply at a natural moment in the next turn, rather than abruptly producing a block of conclusions.

There are three options for handling user input while the Reasoner is still running: QUEUE (wait in line for the Reasoner to finish, suited to task scenarios), INTERRUPT (immediately interrupt and rerun, suited to short exploratory tasks), and PARALLEL (the two agents truly run in parallel, the Talker replying immediately while the Reasoner runs in the background — conversation scenarios must use this). Make the three a config so the product can choose per scenario.

Production scenarios it fits

• Real-time conversation: Tutoring, customer service, personal assistants — the front end needs response in the 200-millisecond range while the back end needs deep personalized analysis, and a single agent cannot satisfy both constraints.
• Voice agents: Phone and voice assistants, where one second of silence is awkward, so the agent must think while it speaks.
• Advisory scenarios that gather information while chatting: While the Talker chats with the user about preferences and asks follow-ups, the Reasoner runs matching analysis in the background on the information already collected, so the analysis is ready by the time the chat ends.

Where it tends to go wrong

• The Talker gets clever and guesses the answer: If the Talker prompt does not firmly nail down "do not give concrete advice," it rushes to offer conclusions, which conflict with the Reasoner's deep analysis and waste the Reasoner's run. This is the most common failure mode of the dual-process design.
• The Reasoner has no timeout or cancel: A background task that runs away with no one watching, or a user who changes topic midway while the Reasoner is still working on the old question, wastes compute. Add a 30-second forced termination plus an active cancel when the user changes topic.
• The belief state is not persisted across sessions: When the user returns next time, the Talker starts from scratch and all of last session's Reasoner analysis is lost. Persist it to Redis / PostgreSQL.
• Stiff bridging: The Reasoner finishes and "abruptly drops a block of analysis," which feels jarring to the user. Use the prompt to make the Talker find a natural moment in the conversation to insert it.
• Forcing the dual-process design onto asynchronous tasks: In scenarios where the user submits a task and leaves (writing documents, batch processing), there is no need for a Talker; just let the Reasoner run slowly. Adding a Talker is over-engineering.

Key metrics

• Talker response latency p99 (healthy zone <800 ms): The time from the Talker receiving input to replying. Exceeding it loses the point of the dual-process design, and the user still feels the lag.
• Talker overreach rate (healthy zone <5%): The sampled proportion of Talker replies that are "giving concrete advice." A high value means the Talker prompt did not hold the constraint, and the Reasoner's deep analysis is wasted.
• Belief hit rate (healthy zone near 100%): The proportion of returning sessions where the Talker can give a personalized response based on the previous belief. Low means the belief state was not persisted or not read.
• Per-turn cost (note the structure): The dual-process design runs "two tiers at once," so per-turn cost is slightly higher than the single expensive tier alone (the Talker part is added), but in exchange latency drops sharply. This is another Pareto choice on the cost-latency-quality triangle, to be assessed against business tolerance.

Minimal implementation skeleton

User speaks →
                If first turn: create belief state, asynchronously trigger Reasoner (non-blocking)
                Talker replies immediately (cheap model, 200-800ms, no concrete advice)
                If Reasoner has finished (belief status = updated) → Talker weaves the analysis naturally into the reply
            Reasoner in background (expensive model, 10-30s):
                Deep analysis → write belief state (structured conclusions + recommendations)
                With timeout (30s forced termination) + cancel (cancel when user changes topic)
            belief state: persisted across sessions, writes take a lock
            Return Talker reply + the background-maintained belief
            

Four points for implementation: the Talker prompt firmly nails down "no concrete advice"; the Reasoner async task has a timeout and cancel; the belief state is persisted to Redis for reuse across sessions; writes to the shared state take a lock to prevent race conditions.

Enterprise implementation example

A study-abroad application advisory agent helps Chinese students choose schools and polish their essays. Two weeks after the first version launched, one class of feedback came in: "I asked which of two schools suited me better, and it paused for more than 30 seconds before replying. I thought the network had died, so I closed it and asked again." The team's review found the agent was not slow; its thinking path was too complete. After the user finished asking, the agent dutifully walked the full chain: fetch the user profile (800 ms), compare the two schools' characteristics (1.2 s), run deep analysis (18 s), and synthesize the answer (4 s), for a total of 24 seconds for one high-quality answer. But 24 seconds is infinitely long for a conversation, and the user experience collapsed. In the third week they added an "analyzing" loading indicator, and users still complained that "they watched the spinner and left."

In the fourth week they rewrote it entirely as Talker + Reasoner: after the user finished asking, the Talker replied within 200 milliseconds, first quickly pointing out the core difference between the two schools, telling the user "I'm running a detailed match analysis for you in the background," and asking back about the user's preference (work in the US or return home). During the dozen-odd seconds the user spent chatting with the Talker about preferences, the Reasoner finished a detailed analysis combining GPA, research experience, and interests in the background, and wrote the conclusion into the belief state. In the next turn the Talker bridged naturally: "Since you lean toward working in Silicon Valley, the analysis I just finished shows…" With the same 24 seconds of thinking time, the experience went from "frozen" to "receiving the analysis while chatting." A comparable online programming tutoring agent used this architecture (Talker on a cheap model for conversation, Reasoner on an expensive model for asynchronous analysis, PARALLEL mode, belief persisted across sessions, natural bridging, and a dual teacher-student view trace). In its second month after launch, student NPS rose from 6 to 8.5, and teachers' override rate on the agent's recommendations dropped from 35% to 12%.

Relationship to other patterns

• Complexity Routing (R2): The dual-process design is the architectural extreme of routing. Ordinary routing "picks one tier to run" (a single-point-in-time decision); the dual-process design "runs two tiers at once" (a two-time-dimension decision), pushing "tier selection" all the way to "splitting the agent."
• Iterative Hypothesis Validation (R4): Both sit in the Loop column. Iteration is a single agent looping with itself to validate hypotheses; the dual-process design is two agents collaborating, separately handling "speaking" and "thinking."
• Chain of Thought (R1): The deep reasoning on the Reasoner side is essentially a complete chain of thought; the Talker side deliberately does not do deep reasoning.
• Hierarchical Memory pattern: The belief state persists across sessions, which shares its lineage with the memory module's "User-layer belief persistence."

What this pattern teaches

The essence of Talker-Reasoner is not performance optimization but splitting the agent's persona into two — one handling real time, one handling depth. What the Talker gives is not a preview of the Reasoner; it is content of an entirely different nature, two mental modes collaborating.

This page is part of the ADPS pattern white papers. Back to the pattern matrix, or see the runnable code catalog.