444060c3eb
- Introduced `critic`, an independent adversarial reviewer for security and correctness. - Added `fable-orchestrator` to manage task routing and verification. - Implemented `gauntlet-critic` for fresh-context evaluation of gauntlet rounds. - Created `planner` for generating executable implementation plans with dependencies. - Developed `security-auditor` for application security reviews and audits. - Established `system-steward` to improve agent prompts and skills based on verified failures. - Added `dev-loop` skill for autonomous development loops over repositories. - Implemented `gauntlet-loop` skill for iterative quality benchmarking against reference standards. - Updated project settings to utilize the new orchestrator agent. - Created documentation for `GAUNTLET.md`, `PROGRESS.md`, and `REFERENCE_BAR.md` to track project status and quality benchmarks. - Added detailed prompting style guide to enhance understanding of prompt patterns and agentic loops.
2.7 KiB
2.7 KiB
Reference bar
The concrete quality bar for gauntlet work. Every entry must point at something a referee can open, run, or look at — an adjective is not a bar. Changing a bar mid-gauntlet is an owner decision recorded in
DECISIONS.md.Status: NOT YET CONCRETE — decision-ready. LexAI's repo contains no reference artifacts, so the rows below are proposals: the parts are real, but each needs owner-supplied artifacts (screenshots/recordings into
docs/reference/, or a named competitor install to compare live) before a gauntlet can start. Behavioral rows can start sooner — their bar is a checkable matrix, not an artifact.
Bars by part (proposed)
| Part | Reference artifact(s) — TO SUPPLY | How to compare | Minimum parity |
|---|---|---|---|
| Floating toolbar + result modal (in-page UI) | screenshots/screen-recording of Grammarly's selection toolbar + suggestion card (or another benchmark extension the owner picks) → docs/reference/ |
load unpacked, select text on a real page at the same spots, screenshot side-by-side | placement, legibility, non-intrusiveness, and interaction states read as polished as the reference |
| Options page | reference settings page screenshots (Grammarly / a best-in-class extension options UI) | side-by-side render | clarity of provider→key→model flow; error/rejected-key states as discoverable as the reference |
| Popup + Prompt Builder | reference popup/composer screenshots | side-by-side render + walk the compose flow | task flow completable as directly as the reference |
| Replace reliability (behavioral) | site matrix the owner approves (e.g. Gmail compose, GitHub textarea/PR comment, X/Twitter composer, LinkedIn, Google Docs*, Reddit, Notion) | run fix→Replace on each; record works / partial / fails | Replace works on every approved matrix site; no self-triggering; no host-page breakage (*Google Docs may be declared out of scope — record it) |
| Store listing | top-ranked writing-assistant CWS listings (live pages) | side-by-side read of store-assets/ vs the live listings |
screenshots, copy, and permission justification at parity before any CWS push (blocked by T-01 <all_urls> anyway) |
Reference sources
docs/reference/— empty until the owner supplies artifacts (screenshots, recordings)- A named competitor extension installed locally for live blind A/B, if preferred over screenshots
Out of scope for the bar
- Grammarly's backend features (tone rewriting service, plagiarism, team features) — LexAI is BYO-key by design; the bar is UI/UX and reliability parity, not feature parity.
- Anything postponed in
docs/TASKS.mdor blocked by open security items (T-01<all_urls>, T-02 key encryption) — those gate release, not gauntlet rounds.