ADPS Agent Design Pattern White Paper

F1 · Generator-Critic

A Generator produces an output and a Critic reviews it against evidence and a rubric. Scope, rounds, and cost are bounded; the run exits when release criteria or the iteration cap is reached.

Coordinate	Reflection × Chain (sequential)
Cost	Medium (multiple generation and review calls)
Pattern group	Reflection patterns
Summary	A Generator produces an output and a Critic reviews it against evidence and a rubric. Scope, rounds, and cost are bounded; the run exits when release criteria or the iteration cap is reached.

Problem

Writing, design, code readability, product copy, and academic abstracts combine several quality dimensions that deterministic checks do not fully cover. A first draft may satisfy the schema while still missing evidence, clarity, terminology, or audience requirements.

Generator-Critic separates drafting from review. The critic returns a structured verdict with evidence; the generator revises within a fixed scope and budget. Research systems such as Reflexion and Self-Refine provide prior evidence for language feedback, while production value still depends on the local task set, critic calibration, and acceptance criteria.

Classification: Reflection × Chain

Vertical axis · Reflection: The Critic examines the current output using a model, rules, tests, retrieved evidence, or human judgment.
Horizontal axis · Chain: Generate → critique → revise is a linear pipeline with an explicit hand-off between stages. The chain may run more than once, but F1 starts with a usable output and applies bounded quality improvement. F4 starts with a verified failure and requires a repair loop.

Solution and mechanics

A Generator-Critic loop consists of three segments:

Generate: The generator produces a draft under an output contract. Evaluate this stage independently so the critic is not used to compensate for avoidable generation defects.
Critique: the critic evaluates the current output and returns a structured verdict with an issue list, severity, evidence, and an explicit no_changes_needed field. Reflexion showed how natural-language feedback can carry the error, its cause, and repair guidance in addition to a score.
Revise: The generator applies accepted findings within the allowed scope. The loop exits on acceptance, no material change, budget exhaustion, or human hand-off.

The verdict should cite its evidence rather than collapse the review into a single score. Common evidence sources have different strengths and limits:

Evidence source	Strength	Limitation
Tests, schemas, static checks	Repeatable and unambiguous within their coverage	Only checks rules that have been encoded
Rules and expert rubrics	Represents business and domain quality	Criteria may conflict or change
Self-Critic	Low-cost initial screening	Inherits blind spots from the Generator
Cross-Model review	Adds a different judgment path	Two models can still share the same error
Human review and business outcomes	Handles accountability, context, and delayed effects	Expensive and often slow

A production review also needs a Reflection Contract. At minimum, record subject, evidence_refs, rubric_version, verdict, proposed_change, scope, max_rounds, cost_budget, and release_action. This turns a verdict into an auditable input for later offline analysis.

Two feedback clocks

Online review serves the current task. It checks the artifact, allows a limited number of revisions, and stops at the release condition or budget. Latency and change scope stay small.
Offline review serves future tasks. It studies production traces, human edits, and later business outcomes to revise rubrics, graders, prompts, and evaluation sets. Changes to shared rules follow a versioned release process.

Both clocks use the same evidence structure. Online verdicts and hand-off reasons feed the offline dataset; revised standards return to production only after evaluation and release approval.

Applicability

Content-generation tasks: Writing, translation, summarization, copywriting, and academic editing can use explicit rubrics for evidence, terminology, audience, style, and completeness.
Code style and readability optimization: Naming, comments, structural adjustments, and similar improvements that do not affect correctness but do affect quality.
High-frequency generation with cost constraints: A lower-cost generator can be paired with a critic, but quality and total cost must be compared with a direct high-capability baseline on the same task set.

Known failure modes

Using an LLM where a deterministic check exists: For mathematics, SQL, schemas, and unit tests, run the check directly and use the model to explain or repair the failure.
An unvalidated critic: A critic that misses important defects or introduces noisy objections can degrade the output. Evaluate critic recall, false alarms, and revision outcomes independently of the generator's model tier.
A critic that forces problems into existence: If the critic prompt does not state that "no changes needed is a valid option," it may invent problems to fill the quota. Treat an unchanged result as a legitimate verdict, then measure false alarms on a reviewed set of already-qualified outputs.
Correlated generator and critic errors: Shared models, prompts, or evidence can preserve the same blind spot. Add deterministic checks, independent evidence, a differently configured reviewer, or qualified human review according to risk.
Collapsing a multidimensional rubric into one score: Accuracy, compliance, readability, and cost can move in different directions. Keep per-dimension verdicts and blocking criteria visible.
Reviewing only the final answer: A plausible answer may hide unnecessary tool calls, incorrect retrieval, or a lucky result. High-risk work needs both outcome and trajectory evaluation.
No bounded exit: An unconstrained critic can keep producing low-value or false findings. Set round, cost, latency, and no-material-change limits, then preserve the unresolved verdict for hand-off.

Verification and metrics

Critic convergence rate: Track how often the loop reaches an accepted result before the configured cap. Repeated exhaustion points to a weak generator, an unstable rubric, or an incapable critic.
Phantom-issue rate: Measure how often the critic flags an output that expert review already considers acceptable. Use this to calibrate the rubric and the no_changes_needed exit.
Critic-vs-expert agreement rate: Compare a representative sample of critic verdicts with expert judgment. Set an acceptance threshold from the risk of the task and the cost of human review.
Quality increment: Compare the final output with the single-generation baseline on a fixed evaluation set, and report the gain together with token cost and latency.
Trajectory efficiency: Track tool calls, failed branches, and revision rounds needed to meet the same acceptance criteria.
Critical-dimension regression: Report regressions in blocking dimensions separately instead of hiding them inside an average score.

Reference implementation

output = generator(task)
            for i in range(MAX_ITERATIONS):       # configure from task risk and evaluation evidence
                if external_critic:               # prefer a deterministic signal when present
                    critique = external_critic(task, output)
                elif multi_critic:                # multiple roles in parallel + arbitration
                    critique = merge(critic(task, output, role=r) for r in roles)
                else:
                    critique = critic(task, output)
                if critique.no_changes_needed:    # valid exit, guards against phantom issues
                    return output, "converged"
                output = generator(task, previous=output, feedback=critique)
            return output, "max_iterations_reached"   # hand off to HITL when bottomed out

The Critic prompt should allow no_changes_needed. Prefer tests, schemas, and citation checks when they apply. Configure max_iterations from local evaluation and task risk, and retain the full history, rubric version, and evidence references for calibration. Online review may revise the current artifact; durable changes to graders and rubrics go through offline evaluation and release.

Illustrative scenario

Consider an agent that polishes academic abstracts. A critic instructed to find a fixed number of problems may rewrite an already concise abstract into a verbose one, because it has no valid way to say that the draft is acceptable. A stronger design permits no_changes_needed, separates academic rigor, concision, and terminology into explicit rubrics, and grounds terminology or citations in an external database. The team then compares critic verdicts with editor review on a fixed sample and decides whether the quality gain justifies the added cost. This is an illustrative design scenario; any claimed production result would need an attributed case and a documented measurement method.

Related patterns

Self-Heal Loop (F4): Generator-Critic improves a usable output and may stop after one review. Self-Heal starts from a verified failure and runs a bounded repair loop with rollback and hand-off.
Adversarial Review (C3): F1 generates and revises an artifact through a bounded feedback loop; generator and critic may share a model or runtime. C3 introduces independent review roles, evidence, and traces when the decision requires separation of duties or stronger challenge.
Skill Package (F2): F1 revises the current artifact. Repeatedly successful and reviewed procedures may later become F2 candidates.

Engineering judgment

Generator-Critic separates generation from evaluation. The Critic needs explicit criteria and should use tests, rules, source evidence, independent models, or human judgment according to the task.

Chronicle

Recorded source: ADPS pattern white paper; prior work and references are listed in the article
First published on ADPS: 2026-06-21

View in the ADPS Chronicle