LexAI/docs/EVALS.md

Project evaluations — LexAI

Small, repeatable checks. Prefer a deterministic command or test over a prose reminder. The standing checks below are the baseline gates for any change.

Standing gates (run on every change)

E-BASE-01 — Typecheck

• How to run: npm run typecheck
• Pass condition: tsc --noEmit exits 0.
• Cost: fast.

E-BASE-02 — Unit tests

• How to run: npm test -- --run
• Pass condition: vitest exits 0.
• Note: current unit tests exercise the chrome.storage mock, not the real handlers — passing does not prove provider routing or key decrypt. See TASKS #8.

E-BASE-03 — Production build

• How to run: npm run build
• Pass condition: builds to .output/chrome-mv3/; bundle roughly ~166 KB baseline.
• Cost: fast (~3s).

E-BASE-04 — Manual real-page check (behavior changes)

• How to run: npm run build → load unpacked .output/chrome-mv3 → select text on a textarea and a contenteditable site → run an action → Replace.
• Pass condition: toolbar shows, result modal returns, Replace edits both target types.
• Why manual: selection/replace is DOM-timing-sensitive and has no automated coverage.

Active failure-derived checks

E-RELEASE-01 — CHANGELOG version match and spot-check

• Prevents: L-RELEASE-01 — false release notes shipped to CWS
• How to run: (1) Extract version from package.json (e.g., jq -r .version package.json). (2) Grep for ## [version] in CHANGELOG.md. (3) Pick 2–3 user-visible claims (feature name, behavior, action added) and verify against git log --oneline or the code (src/lib/actions.ts, entrypoints/*/).
• Pass condition: (1) CHANGELOG has a section header matching the version; (2) each spot-checked claim is present in code or the latest commit subject(s) describe that feature being added.
• Cost: fast (~2 min).
• When to run: before git tag v*.*.*.
• Last verified: 2026-08-12 (caught two false claims in v1.1.0 prep).

Eval template

### E-YYYY-MM-DD-NN — [short check name]
- **Prevents:** [lesson ID and failure mode]
- **How to run:** `[exact command or steps]`
- **Pass condition:** [observable]
- **Cost:** fast | moderate | expensive
- **Last verified:** [date + result]

Retired checks

Move obsolete checks here with the reason and the lesson they covered.