444060c3eb
- Introduced `critic`, an independent adversarial reviewer for security and correctness. - Added `fable-orchestrator` to manage task routing and verification. - Implemented `gauntlet-critic` for fresh-context evaluation of gauntlet rounds. - Created `planner` for generating executable implementation plans with dependencies. - Developed `security-auditor` for application security reviews and audits. - Established `system-steward` to improve agent prompts and skills based on verified failures. - Added `dev-loop` skill for autonomous development loops over repositories. - Implemented `gauntlet-loop` skill for iterative quality benchmarking against reference standards. - Updated project settings to utilize the new orchestrator agent. - Created documentation for `GAUNTLET.md`, `PROGRESS.md`, and `REFERENCE_BAR.md` to track project status and quality benchmarks. - Added detailed prompting style guide to enhance understanding of prompt patterns and agentic loops.
41 lines
5.0 KiB
Markdown
41 lines
5.0 KiB
Markdown
---
|
|
name: gauntlet-loop
|
|
description: Run reference-benchmarked improvement rounds on an outcome that must match or beat a concrete quality bar — decompose into independently judgeable parts, then loop builder → fresh-context gauntlet-critic on the single biggest gap until parity, diminishing returns, or budget. Use for quality-benchmarked deliverables, not routine maintenance (that is dev-loop).
|
|
allowed-tools: Read Grep Glob Bash Write Edit Skill Agent
|
|
---
|
|
|
|
Iterate work against a concrete reference until a fresh-eyes referee calls parity — the Gauntlet Loop (Matt Shumer's method behind "Claude of Duty"). Opus owns routing and acceptance; this skill is the loop discipline.
|
|
|
|
## Preconditions — refuse to start until all three hold
|
|
|
|
1. **The bar is concrete.** `docs/REFERENCE_BAR.md` names at least one inspectable reference artifact per part in scope (file, screenshot, URL, sample output, recording) and how to compare against it. An adjective is not a bar; "make it amazing" starts nothing. If the bar is missing, request it from the owner as a decision-ready item — that request never stalls other lanes.
|
|
2. **A budget exists.** Each part gets a round ceiling (an integer; add wall-clock only if the work is time-bound), written into the orchestration record and the `docs/GAUNTLET.md` row before round 1. A ceiling is a backstop so a stuck part cannot loop forever — never a plan to schedule rounds toward.
|
|
3. **The bar is not gameable.** The referee judges the artifact as a user would experience it; any single metric is supporting evidence, never the target.
|
|
|
|
## Round protocol (per part)
|
|
|
|
1. **Decompose once.** Opus splits the outcome into the smallest parts that can be improved and judged separately — coupled work stays one part. Each part gets a row in `docs/GAUNTLET.md`: part, bar row, rounds-left ceiling, status.
|
|
2. **Build.** One builder owns the part and returns the artifact plus exact instructions to render/run/see it. The builder never assesses its own round against the bar.
|
|
3. **Referee.** Spawn `gauntlet-critic` fresh. Its packet is the part contract, the bar, and artifact access including the builder's render/run steps — mechanics pass through; the builder's reasoning, summary, or self-assessment never does, and neither do prior round reports (round history lives on the board, not in the referee's context). It returns verdict, single biggest gap weighted material/cosmetic, evidence, also-observed list. Referee effort is `high`; raise to `xhigh` only for a final parity verdict at the high-risk gate.
|
|
4. **Log.** Append one line to Round history in `docs/GAUNTLET.md` — part, round, verdict, gap (weight) — and decrement the part's rounds-left. If Opus's session lacks write tools, the append rides in the next worker packet.
|
|
5. **Apply stops, then loop.** Opus checks the stop conditions below against the board's round history — the stateless referee cannot make these calls; its verdict (`parity` or `output wins`) is the only stop it can trigger. If none fires, the builder's next packet targets exactly the named gap (plus any P0 from the also-observed list). Never pre-commit to a round count — "do three rounds and stop" defeats the method; the ceiling is a backstop, not a target.
|
|
6. **Parallelize across parts** freely: different parts may sit in different rounds, with one builder and one referee per part per round.
|
|
|
|
## Stop conditions (per part — Opus applies these at each Log step, from the board's round history; Boundary fires the moment it appears)
|
|
|
|
- **Parity or better** — the round's verdict is `parity` or `output wins`.
|
|
- **Diminishing returns** — two consecutive rounds with an unchanged verdict and a gap weighted `cosmetic`.
|
|
- **Budget exhausted** — rounds-left hits zero: record the last verdict and open gap on the board; surface to the owner.
|
|
- **Recurring gap** — the board names the same gap two rounds running and Opus has no new strategy for the next packet: park it decision-ready (short options, recommended default) and move to the next part.
|
|
- **Boundary** — a round would need a destructive, external, or permission-crossing action: stop and escalate; never proceed on referee authority.
|
|
|
|
## Endgame
|
|
|
|
When every part has stopped: run one integration pass (integrator merges, verifier re-runs the full checks) so independently polished parts still work as a whole; apply the normal quality gates for the risk level; and if the per-part bars were partial views, run one final whole-artifact referee round against the bar. Record final verdicts on the board, then compress the outcome into `HANDOFF.md` and `PROGRESS.md` in owner language: what reached the bar, what stopped short and why.
|
|
|
|
## Guardrails
|
|
|
|
- Builders never self-grade; referees never see builder narrative; Opus never overrides a verdict without observable evidence.
|
|
- Evidence is observable — rendered pixels, command output, test results, a cold read of the finished writing — never a summary of them.
|
|
- Consequential actions (deploy, spend, delete, credentials) stay behind explicit owner authorization regardless of loop momentum.
|