444060c3eb
- Introduced `critic`, an independent adversarial reviewer for security and correctness. - Added `fable-orchestrator` to manage task routing and verification. - Implemented `gauntlet-critic` for fresh-context evaluation of gauntlet rounds. - Created `planner` for generating executable implementation plans with dependencies. - Developed `security-auditor` for application security reviews and audits. - Established `system-steward` to improve agent prompts and skills based on verified failures. - Added `dev-loop` skill for autonomous development loops over repositories. - Implemented `gauntlet-loop` skill for iterative quality benchmarking against reference standards. - Updated project settings to utilize the new orchestrator agent. - Created documentation for `GAUNTLET.md`, `PROGRESS.md`, and `REFERENCE_BAR.md` to track project status and quality benchmarks. - Added detailed prompting style guide to enhance understanding of prompt patterns and agentic loops.
26 lines
2.7 KiB
Markdown
26 lines
2.7 KiB
Markdown
# Reference bar
|
|
|
|
> The concrete quality bar for gauntlet work. Every entry must point at something a referee can open, run, or look at — an adjective is not a bar. Changing a bar mid-gauntlet is an owner decision recorded in `DECISIONS.md`.
|
|
>
|
|
> **Status: NOT YET CONCRETE — decision-ready.** LexAI's repo contains no reference artifacts, so the rows below are *proposals*: the parts are real, but each needs owner-supplied artifacts (screenshots/recordings into `docs/reference/`, or a named competitor install to compare live) before a gauntlet can start. Behavioral rows can start sooner — their bar is a checkable matrix, not an artifact.
|
|
|
|
## Bars by part (proposed)
|
|
|
|
| Part | Reference artifact(s) — TO SUPPLY | How to compare | Minimum parity |
|
|
| --- | --- | --- | --- |
|
|
| Floating toolbar + result modal (in-page UI) | screenshots/screen-recording of Grammarly's selection toolbar + suggestion card (or another benchmark extension the owner picks) → `docs/reference/` | load unpacked, select text on a real page at the same spots, screenshot side-by-side | placement, legibility, non-intrusiveness, and interaction states read as polished as the reference |
|
|
| Options page | reference settings page screenshots (Grammarly / a best-in-class extension options UI) | side-by-side render | clarity of provider→key→model flow; error/rejected-key states as discoverable as the reference |
|
|
| Popup + Prompt Builder | reference popup/composer screenshots | side-by-side render + walk the compose flow | task flow completable as directly as the reference |
|
|
| Replace reliability (behavioral) | site matrix the owner approves (e.g. Gmail compose, GitHub textarea/PR comment, X/Twitter composer, LinkedIn, Google Docs*, Reddit, Notion) | run fix→Replace on each; record works / partial / fails | Replace works on every approved matrix site; no self-triggering; no host-page breakage (*Google Docs may be declared out of scope — record it) |
|
|
| Store listing | top-ranked writing-assistant CWS listings (live pages) | side-by-side read of `store-assets/` vs the live listings | screenshots, copy, and permission justification at parity before any CWS push (blocked by T-01 `<all_urls>` anyway) |
|
|
|
|
## Reference sources
|
|
|
|
- `docs/reference/` — **empty until the owner supplies artifacts** (screenshots, recordings)
|
|
- A named competitor extension installed locally for live blind A/B, if preferred over screenshots
|
|
|
|
## Out of scope for the bar
|
|
|
|
- Grammarly's backend features (tone rewriting service, plagiarism, team features) — LexAI is BYO-key by design; the bar is UI/UX and reliability parity, not feature parity.
|
|
- Anything postponed in `docs/TASKS.md` or blocked by open security items (T-01 `<all_urls>`, T-02 key encryption) — those gate release, not gauntlet rounds.
|