From 2a2fecbfdde37c629bfd3333e9ca76ff54b31f98 Mon Sep 17 00:00:00 2001 From: john kevin asprec Date: Wed, 12 Aug 2026 07:54:03 +0800 Subject: [PATCH] docs: add changelog-accuracy lesson and pre-tag eval --- CLAUDE.md | 1 + docs/EVALS.md | 9 ++++++++- docs/LESSONS_LEARNED.md | 6 ++++++ 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index e3dc298..c623718 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -356,6 +356,7 @@ Keep rules general, evidence-backed, and under 20 words. Never add secrets, pers - Never `fetch` a provider from content/popup — route through the background worker. - Bump `version` in `package.json` only (+ lockfile); the manifest derives it via `pkg.version`. - Style inline; Tailwind classes do nothing until PostCSS is wired. +- Verify CHANGELOG entries against code, not commit subjects or handoff summaries. ## Memory protocol diff --git a/docs/EVALS.md b/docs/EVALS.md index 45f2d85..1e26daf 100644 --- a/docs/EVALS.md +++ b/docs/EVALS.md @@ -30,7 +30,14 @@ ## Active failure-derived checks -_No failure-derived checks yet. Add one here when a verified regression gives a deterministic trigger — e.g. a guard that fails if the built `content.js` still contains `[LexAI` logs (T-03), or a test asserting the `onMessage` listener returns `true`._ +### E-RELEASE-01 — CHANGELOG version match and spot-check + +- **Prevents:** L-RELEASE-01 — false release notes shipped to CWS +- **How to run:** (1) Extract version from `package.json` (e.g., `jq -r .version package.json`). (2) Grep for `## [version]` in `CHANGELOG.md`. (3) Pick 2–3 user-visible claims (feature name, behavior, action added) and verify against `git log --oneline` or the code (`src/lib/actions.ts`, `entrypoints/*/`). +- **Pass condition:** (1) CHANGELOG has a section header matching the version; (2) each spot-checked claim is present in code or the latest commit subject(s) describe that feature being added. +- **Cost:** fast (~2 min). +- **When to run:** before `git tag v*.*.*`. +- **Last verified:** 2026-08-12 (caught two false claims in v1.1.0 prep). ## Eval template diff --git a/docs/LESSONS_LEARNED.md b/docs/LESSONS_LEARNED.md index c9d903c..ae56616 100644 --- a/docs/LESSONS_LEARNED.md +++ b/docs/LESSONS_LEARNED.md @@ -53,6 +53,12 @@ - **Root cause / failure boundary:** Tailwind is installed but WXT PostCSS was never wired; Tailwind classes silently do nothing. - **Prevention:** style with inline objects and the existing dark palette; don't add Tailwind classes unless the task is explicitly to wire PostCSS. +### L-RELEASE-01 — Verify CHANGELOG against code before tagging + +- **Root cause / failure boundary:** release notes authored from commit subjects and handoff summaries are not facts; two false claims in v1.1.0 CHANGELOG were caught pre-tag: (1) off-by-one count of actions (`12 … plus Auto` when auto is one of 12), (2) feature listed for 1.0.1 that doesn't exist in code (commit subject claimed it but Options.tsx has no such tab). +- **Prevention:** before `git tag v*.*.*`, verify at least 2–3 user-visible changes claimed in CHANGELOG against the actual code diff or feature. Check version in CHANGELOG matches `package.json`. +- **Eval:** E-RELEASE-01. + ## Recording policy Add a lesson only after a material, evidenced learning signal (correction, unexpected failure, regression, rejected review, proven wrong assumption). Each needs a durable prevention; link a deterministic eval when possible. No secrets, credentials, personal data, or raw transcripts.