Reflection and Self-Critique Agent Architecture
Add a critique step that inspects the agent's own output (or trajectory), produces feedback, and feeds it back for another attempt. Reflection trades extra inference for higher quality, and works because evaluating an answer is often easier than generating it.
The minimal loop is generate → critique → revise, repeated until the critique passes or a budget is hit. Two named variants anchor the design space: Self-Refine keeps the loop within a single task (the model critiques and rewrites its own output), while Reflexion persists verbal feedback across attempts in an episodic memory so the agent learns from failed trajectories on the next try. Source: Madaan et al., Self-Refine, arXiv:2303.17651; Shinn et al., Reflexion, arXiv:2303.11366
Components
- Actor — generates the candidate output or runs the task trajectory (often a ReAct loop).
- Evaluator — produces the critique. Can be the same model (self-critique), a different model (cross-modal review), or a programmatic check (tests, a linter, an eval score). Programmatic signals are the strongest because they are grounded.
- Reflection memory — stores critiques so later attempts see "what went wrong last time." Distinguishes Reflexion from one-shot refinement; this is the write path of Agent Memory Patterns.
- Stop controller — ends on a passing critique, a max-iterations cap, or no-improvement detection.
Control flow
Design implications
- Ground the evaluator or it rationalizes. A model critiquing itself with no external signal can confidently approve wrong answers. Prefer tests, compilers, retrieval checks, or a second model; reserve pure self-critique for subjective tasks.
- Bound the loop. Reflection has diminishing returns and unbounded cost; cap iterations and stop on no-improvement.
- It is the agentic form of Eval-Driven Development. The evaluator is a runtime eval; the difference is the score feeds back into another attempt instead of just gating a release.
- Latency cost is multiplicative. Each reflection round is another full generation; this only pays off on high-value outputs.
Kevin's "doctors" pattern is reflection at the harness layer: agents write code, doctors (The Eval Loop (Slop Is an Output Problem), review skills, Security and Review Skills) critique it, agents revise. The actor is the coding agent, the evaluator is the doctor, and the human reviews the converged result. Source: compiled from Agent Operating System
Architecture Position
| Axis | Value |
|---|---|
| Family | Agent runtime and control plane |
| Boundary owned | Self-critique loop that evaluates output or trajectory before continuation. |
| Read with | ReAct Agent Architecture, Agent Trajectory Evaluation, Agent Runtime Architecture |
| Use this page when | adding critique/eval steps to agent loops |
Timeline
-
2026-07-01 | Architecture category refresh added this page to the Agent runtime and control plane family, linked it to Architecture System Map, and kept it standalone because it owns this boundary: Self-critique loop that evaluates output or trajectory before continuation. Source: User request, 2026-07-01
-
2026-06-02 | Page created. Captured the generate/critique/revise loop, the Self-Refine vs Reflexion split, the component breakdown, the "ground the evaluator" rule, and the mapping to Kevin's doctors pattern. Source: Madaan et al., Self-Refine, arXiv:2303.17651; Shinn et al., Reflexion, arXiv:2303.11366