ADPS Agent Design Pattern White Paper

F1 · Generator-Critic

A Generator produces an output and a Critic reviews it against evidence and a rubric. Scope, rounds, and cost are bounded; the run exits when release criteria or the iteration cap is reached.

Coordinate	Reflection × Chain (sequential)
Cost	Medium (multiple generation and review calls)
Pattern group	Reflection patterns
Summary	A Generator produces an output and a Critic reviews it against evidence and a rubric. Scope, rounds, and cost are bounded; the run exits when release criteria or the iteration cap is reached.

Problem

Tasks dense with subjective judgment have no single correct answer, only a quality spectrum—writing, design, code style, product copy, and academic abstracts all fall into this category. A first draft often misses details that become visible on review. Human writers revise until the remaining gain no longer justifies another pass. A single generation lacks this deliberate look back.

Generator-Critic encodes this process into an agent: the generator writes a draft, the critic looks back and finds problems, the generator revises with that feedback, and the loop continues until the critic is satisfied or the iteration cap is hit. Research systems such as Reflexion and Self-Refine showed that language feedback can improve later attempts without model retraining. The size of the gain depends on the task, critic, model, and evaluation method, so a production team should reproduce it on its own evaluation set.

Classification: Reflection × Chain

Vertical axis · Reflection: The Critic examines the current output using a model, rules, tests, retrieved evidence, or human judgment.
Horizontal axis · Chain: Generate → critique → revise is a linear pipeline with an explicit hand-off between stages. The chain may run more than once, but F1 starts with a usable output and applies bounded quality improvement. F4 starts with a verified failure and requires a repair loop.

Solution and mechanics

A Generator-Critic loop consists of three segments:

Generate: The generator produces a draft. The model used in this step determines how much the critic has to make up for.
Critique: The critic evaluates the current output and produces a structured verdict (an issues list / severity / an explicit no_changes_needed field). The key innovation of the Reflexion paper is that the critic produces natural-language reflection text rather than a bare score, telling the generator what is wrong, why it is wrong, and how to fix it next time.
Revise: The generator rewrites with the critique in hand. It then returns to step 2, until the critic passes the output or iteration bottoms out.

The verdict should cite its evidence rather than collapse the review into a single score. Common evidence sources have different strengths and limits:

Evidence source	Strength	Limitation
Tests, schemas, static checks	Repeatable and unambiguous within their coverage	Only checks rules that have been encoded
Rules and expert rubrics	Represents business and domain quality	Criteria may conflict or change
Self-Critic	Low-cost initial screening	Inherits blind spots from the Generator
Cross-Model review	Adds a different judgment path	Two models can still share the same error
Human review and business outcomes	Handles accountability, context, and delayed effects	Expensive and often slow

A production review also needs a Reflection Contract. At minimum, record subject, evidence_refs, rubric_version, verdict, proposed_change, scope, max_rounds, cost_budget, and release_action. This turns a verdict into an auditable input for later offline analysis.

Two feedback clocks

Online review serves the current task. It checks the artifact, allows a limited number of revisions, and stops at the release condition or budget. Latency and change scope stay small.
Offline review serves future tasks. It studies production traces, human edits, and later business outcomes to revise rubrics, graders, prompts, and evaluation sets. Changes to shared rules follow a versioned release process.

Both clocks use the same evidence structure. Online verdicts and hand-off reasons feed the offline dataset; revised standards return to production only after evaluation and release approval.

Applicability

Content-generation tasks: Writing, translation, summarization, copywriting, polishing academic abstracts—no objective right or wrong, but a clear quality difference, where the critic can reliably spot "where it falls short."
Code style and readability optimization: Naming, comments, structural adjustments, and similar improvements that do not affect correctness but do affect quality.
High-frequency generation with cost constraints: A lower-cost generator can be paired with a critic, but quality and total cost must be compared with a direct high-capability baseline on the same task set.

Known failure modes

Using an LLM where a deterministic check exists: For mathematics, SQL, schemas, and unit tests, run the check directly and use the model to explain or repair the failure.
An unvalidated critic: A critic that misses important defects or introduces noisy objections can degrade the output. Evaluate critic recall, false alarms, and revision outcomes independently of the generator's model tier.
A critic that forces problems into existence: If the critic prompt does not state that "no changes needed is a valid option," it may invent problems to fill the quota. Treat an unchanged result as a legitimate verdict, then measure false alarms on a reviewed set of already-qualified outputs.
No guard against same-source bias: When the critic and generator are the same LLM, a self-enhancement bias exists, and it tends to score its own output highly. High-stake tasks should switch vendors or add external grounding.
Collapsing a multidimensional rubric into one score: Accuracy, compliance, readability, and cost can move in different directions. Keep per-dimension verdicts and blocking criteria visible.
Reviewing only the final answer: A plausible answer may hide unnecessary tool calls, incorrect retrieval, or a lucky result. High-risk work needs both outcome and trajectory evaluation.
No circuit breaker on iteration: Not setting max_iterations triggers over-thinking—the critic keeps inventing on top of the previous round's fabrications, and after a few rounds the agent convinces itself that the output is severely flawed.

Verification and metrics

Critic convergence rate: Track how often the loop reaches an accepted result before the configured cap. Repeated exhaustion points to a weak generator, an unstable rubric, or an incapable critic.
Phantom-issue rate: Measure how often the critic flags an output that expert review already considers acceptable. Use this to calibrate the rubric and the no_changes_needed exit.
Critic-vs-expert agreement rate: Compare a representative sample of critic verdicts with expert judgment. Set an acceptance threshold from the risk of the task and the cost of human review.
Quality increment: Compare the final output with the single-generation baseline on a fixed evaluation set, and report the gain together with token cost and latency.
Trajectory efficiency: Track tool calls, failed branches, and revision rounds needed to meet the same acceptance criteria.
Critical-dimension regression: Report regressions in blocking dimensions separately instead of hiding them inside an average score.

Reference implementation

output = generator(task)
            for i in range(MAX_ITERATIONS):       # configure from task risk and evaluation evidence
                if external_critic:               # prefer a deterministic signal when present
                    critique = external_critic(task, output)
                elif multi_critic:                # multiple roles in parallel + arbitration
                    critique = merge(critic(task, output, role=r) for r in roles)
                else:
                    critique = critic(task, output)
                if critique.no_changes_needed:    # valid exit, guards against phantom issues
                    return output, "converged"
                output = generator(task, previous=output, feedback=critique)
            return output, "max_iterations_reached"   # hand off to HITL when bottomed out

The Critic prompt should allow no_changes_needed. Prefer tests, schemas, and citation checks when they apply. Configure max_iterations from local evaluation and task risk, and retain the full history, rubric version, and evidence references for calibration. Online review may revise the current artifact; durable changes to graders and rubrics go through offline evaluation and release.

Illustrative scenario

Consider an agent that polishes academic abstracts. A critic instructed to find a fixed number of problems may rewrite an already concise abstract into a verbose one, because it has no valid way to say that the draft is acceptable. A stronger design permits no_changes_needed, separates academic rigor, concision, and terminology into explicit rubrics, and grounds terminology or citations in an external database. The team then compares critic verdicts with editor review on a fixed sample and decides whether the quality gain justifies the added cost. This is an illustrative design scenario; any claimed production result would need an attributed case and a documented measurement method.

Related patterns

Self-Heal Loop (F4): Generator-Critic improves a usable output and may stop after one review. Self-Heal starts from a verified failure and runs a bounded repair loop with rollback and hand-off.
Adversarial Review (collaboration module): An upgraded form. Generator-Critic is "the same model with a different prompt," while Adversarial Review is "an independent agent with adversarial incentives." The former addresses "how an agent revises itself," needing one LLM process and separate prompts; the latter addresses "how a decision passes audit-grade independent review," requiring model routing, an independent trace, and cross-agent orchestration. Use the former when the cost of error is low, and upgrade to the latter for scenarios that demand regulatory independence, such as financial approval or medical diagnosis.
Skill Package (F2): A connecting relationship. Generator-Critic revises this one current output, while Skill Package distills the successful process across tasks.

Engineering judgment

Generator-Critic separates generation from evaluation. The Critic needs explicit criteria and should use tests, rules, source evidence, independent models, or human judgment according to the task.