kibin/LexAI
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Add new agents and skills for enhanced project orchestration and review processes

Browse Source 
- Introduced `critic`, an independent adversarial reviewer for security and correctness.
- Added `fable-orchestrator` to manage task routing and verification.
- Implemented `gauntlet-critic` for fresh-context evaluation of gauntlet rounds.
- Created `planner` for generating executable implementation plans with dependencies.
- Developed `security-auditor` for application security reviews and audits.
- Established `system-steward` to improve agent prompts and skills based on verified failures.
- Added `dev-loop` skill for autonomous development loops over repositories.
- Implemented `gauntlet-loop` skill for iterative quality benchmarking against reference standards.
- Updated project settings to utilize the new orchestrator agent.
- Created documentation for `GAUNTLET.md`, `PROGRESS.md`, and `REFERENCE_BAR.md` to track project status and quality benchmarks.
- Added detailed prompting style guide to enhance understanding of prompt patterns and agentic loops.
This commit is contained in:
john kevin asprec
2026-08-08 16:49:07 +08:00
parent 6aee260533
commit 444060c3eb
85 changed files with 2717 additions and 171 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
docs/GAUNTLET.md Normal file
View File

@@ -0,0 +1,22 @@
# Gauntlet board
> Loop state for reference-benchmarked work. One row per part; one line per round. Move finished gauntlets to `docs/archive/`. Statuses: `not started` · `looping` · `parity — stopped` · `diminishing returns — stopped` · `budget exhausted` · `parked (decision-ready)` · `integrated`.
>
> Seeded 2026-08-06 at the tier upgrade with the screens that already have design artifacts. **Budgets are unset — owner sets them before a part's first round.** Add rows as new screens reach implementation; the bar precedence guard in `REFERENCE_BAR.md` applies to every round.
## Parts
| Part | Bar (REFERENCE_BAR.md row) | Rounds | Last verdict | Biggest open gap | Budget left | Status |
| --- | --- | --- | --- | --- | --- | --- |
| Auth screens 1–2 | Auth screens 1–2 | 0 | — | — | [set] | not started |
| Screen 06 — discount capture | Screen 06 — discount capture | 0 | — | — | [set] | not started |
| Screen 11 — printer setup | Screen 11 — printer setup | 0 | — | — | [set] | not started |
| P10 — prepaid booking / QR | P10 — prepaid booking / QR | 0 | — | — | [set] | not started |
## Round history
- _None yet._
## Final verdicts
- _None yet._

26
docs/HANDOFF.md
View File

@@ -1,16 +1,16 @@
# Handoff — LexAI
## Current state
## Handoff — 2026-08-08
- **Outcome:** Phase 1 complete (7 workitems, 2026-03-06). Codebase reviewed 2026-07-13 (`RECOMMENDATIONS.md`). Operating-system docs + agent roster aligned to the template 2026-07-15.
- **Delivered:** working MV3 extension — selection detection (textarea/input/contenteditable), floating toolbar, background LLM proxy with OpenAI/Anthropic/Groq/OpenRouter, Options page, result modal with Replace/Copy.
- **Shipped 2026-07-15 (consolidated):** refactor series (crypto consolidation, provider adapter table, context-menu registry, dev-gated debug logs); OpenAI `max_completion_tokens` + no-temperature for reasoning models; live model listing in Options (Provider → API Key → Model); new `prompt` action + Prompt Builder (popup tabs, in-page dialog, persona/style/format/model params shared via storage); security/perf pass (plaintext-key migration, `sender.id` guard, content-script listener leak fix, non-JSON error guard); CI fix — `postinstall: wxt prepare` (CI never ran it, so `.wxt/types` was missing and typecheck failed on `import.meta.env`) plus workflow hardening. All gated: typecheck + tests + build.
- **Fix (2026-07-23, Anthropic CORS):** `src/lib/providers.ts:172` — the Anthropic chat spec now sends `anthropic-dangerous-direct-browser-access: 'true'` (the model-list path at `:276` already did). Verified present in `.output/chrome-mv3/background.js`, which is the only bundle that reaches `api.anthropic.com`. **Unresolved for the user:** the CORS error still appears in their browser, which means the running extension is older than this build (a stale service worker, or a second copy installed from the pre-fix `.output/lexai-1.0.1-chrome.zip` dated 7/15). Next diagnostic: service-worker inspector → Network → `messages` → check Request Headers.
- **Fix (2026-07-23, Groq key rejected):** the Groq spec was correct; the Options flow was not. (1) `handleSave` set `modelsError` but the render gated it on `modelsStatus === 'error'`, so both save-time guards were invisible and Save silently no-opped — now rendered whenever set (amber for guidance, red for load errors). (2) `handleProviderChange` auto-listed models with the *stored* key after a provider switch, so Groq rejected the previous provider's key ("Invalid API Key") before any Groq key was entered — now tracked via `savedKeyProvider` ref; it prompts for the new key instead of guessing. (3) `listModels` errors now use `spec.label` (`Groq error: …`, matching the chat path) instead of the raw id (`groq error: …`).
- **Root cause + fix (2026-07-23, Groq "Invalid API Key"):** a stored key had no record of the provider it was entered for. `Options.handleSave` writes `{provider, model}` **without** the key when the field is blank and one is stored, so switching to Groq and saving left the OpenAI key attached to Groq — every call, and every stored-key model list, sent it and got that provider's own rejection while the field still showed 🔒. Fix: new `keyProvider` storage field (`types.ts`, in `CONFIG_STORAGE_KEYS`) written on every save; `keyProviderMismatch()` in background.ts blocks the send on the chat, COPY_AS, and stored-key LIST_MODELS paths with an actionable message (absent `keyProvider` = pre-upgrade key, allowed); Options drops the 🔒 badge and demands a new key when the saved one belongs to another provider or comes back rejected (`keyRejected` flag from `listModels` on 401/403); `callProvider` appends "open LexAI Settings and re-enter your API key" to 401/403 only.
- **Verified (2026-07-23):** `npm run typecheck` clean, `npm test -- --run` 58/58 (new: Groq bearer auth, labelled errors, the 401 hint, `keyRejected`, `providerLabel`), `npm run build` clean → `.output/chrome-mv3/` 281.72 kB. Options-page behavior is **not** covered by unit tests — a load-unpacked check of the Groq re-entry flow is still pending.
- **Open risks (ranked):**
1. `<all_urls>` host permission — privacy surface + CWS review blocker (TASKS #1).
2. Key "encryption" is obfuscation (`encKey` co-located) — TASKS #2.
3. Tests don't cover real code paths (TASKS #8), DOM replace (TASKS #10), or any Options/Popup React flow.
- **Next smallest action:** reload the unpacked extension, then in Options **re-enter the Groq key** (this stamps `keyProvider` and replaces the mis-attached key) → ↻ Load → select model → Save, and confirm a real-page action. Then the quick wins: T-03, T-06/T-09/T-15/T-16. Do T-01/T-02 before any Chrome Web Store push.
Outcome: done — Prompt Builder pattern work verified; in-page live hint added.
Delivered: `src/lib/actions.ts` — 12-entry `PROMPT_PATTERNS` (Direct/Reasoning/Agentic groups), "style" now "pattern".
Delivered: `src/lib/providers.ts` `getSystemPrompt` — ROLE+TASK → pattern block (auto rubric or one pattern's skeleton+guard) → persona/format/style modifiers → invariants last.
Delivered: `entrypoints/background.ts:71` — `prompt` action gets a 2048-token `maxTokens` floor (was truncating under the 1024 input-length floor).
Delivered: `src/lib/actions.ts` `resolvePromptPattern` — migrates legacy `promptStyle` storage values.
Delivered: `entrypoints/content.ts:722-733` — in-page dialog now shows the live per-pattern hint (popup already had it); updates on select `change` and after storage prefill.
Verified: `npm run typecheck` — clean.
Verified: `npm test -- --run` — 64 tests, 4 files, all passing.
Verified: `npm run build` — `.output/chrome-mv3/` built, no errors.
Decisions: none new this session.
Known risks: real-page load-unpacked check still outstanding — this is the owner's gate (see `docs/PROGRESS.md`).
Next smallest action: load `.output/chrome-mv3` unpacked and confirm the Pattern dropdown + hint render correctly in both popup and in-page dialog, and a saved pattern preselects.

33
docs/PROGRESS.md Normal file
View File

@@ -0,0 +1,33 @@
# Progress board
> For the owner. What works, how to see it, and what's waiting on you — plain language, no agent jargon. Refreshed at every phase seal and session end. `HANDOFF.md` speaks to the next agent; this page speaks to you.
**Updated:** 2026-08-08 · **Overall:** working MV3 extension (Phase 1 + the 2026-07 fix wave + the Prompt Builder pattern upgrade); operating system on the gauntlet-loop/opus kit (2026-08-07 audit revision).
## What works now
- The extension itself: selection → floating toolbar → fix/rephrase/shorten/expand/explain/prompt → Replace or Copy; four providers (OpenAI/Anthropic/Groq/OpenRouter); encrypted BYO key; Options with live model listing; 64/64 unit tests, typecheck and build green (2026-08-08).
- Prompt Builder now offers 12 named prompting patterns (grouped Direct / Reasoning / Agentic, plus "Auto"), each with a plain-English hint shown under the dropdown — in both the popup's Prompt tab and the in-page "Make Prompt" dialog you get from selecting text.
- Patterns like Few-shot Examples and ReAct now produce properly structured output (example blocks, step budgets) without getting cut off — a token-limit bug that truncated longer prompt patterns is fixed.
- Any pattern you'd saved before this update carries over automatically — nothing to redo.
- The agent operating system: upgraded from the older fable kit — 13 specialists (incl. your custom `lexai-extension-dev`, kept and modernized) + 4 new ones (ux-ui-designer, ux-psychologist, and the fresh-eyes `gauntlet-critic` referee), 12 skills, all your lessons and security-auditor memory preserved. Lead is now `claude --agent opus-orchestrator`.
## See it yourself
- `npm run build` → `chrome://extensions` → Load unpacked → `.output/chrome-mv3` → select text on any page → "Make Prompt" (or open the extension popup's Prompt tab).
- Open `CLAUDE.md` — your repo rules and 9 codebase invariants are carried over intact; the gauntlet protocol is new in §3.
## Waiting on you — each item blocks ONLY its own lane
| # | Decision | Options (recommended bold) | What it unblocks |
| --- | --- | --- | --- |
| 1 | Load-unpacked check of the Prompt Builder pattern upgrade: (a) grouped Pattern dropdown + hint shows in both the popup Prompt tab and the in-page dialog, (b) picking "Few-shot Examples" gives a prompt with example blocks and "ReAct (tools)" gives one with a step budget and a final-answer marker, neither cut off, (c) a pattern you'd saved before still shows selected | **do the 5-min check** / report it already done | closes this update's verification loop — nothing else is blocked meanwhile |
| 2 | Supply reference-bar artifacts (screenshots/recording of Grammarly or your chosen benchmark → `docs/reference/`) | **Grammarly toolbar + card screenshots** / pick another benchmark / defer gauntlets | UI gauntlet rounds |
| 3 | Approve the Replace-reliability site matrix in `docs/REFERENCE_BAR.md` (Gmail, GitHub, X, LinkedIn, Google Docs?, Reddit, Notion) | **approve as listed (Docs out of scope)** / edit the list | the behavioral gauntlet — can start without screenshots |
| 4 | Set gauntlet budgets on `docs/GAUNTLET.md` | **modest budget on one part first** / several at once | looping |
| 5 | Delete `_to_delete\` in the repo (replaced kit files + transfer archive parked there) | delete now / leave for later | nothing — housekeeping |
## Next up — proceeds without you
- T-01 (`<all_urls>` narrowing) and T-02 (real key encryption) remain the ranked pre-release risks from `HANDOFF.md` — routable to security-auditor + lexai-extension-dev any time.
- Nothing about the Prompt Builder update is blocked — it's complete pending item 1's owner check above.

33
docs/REFERENCE_BAR.md Normal file
View File

@@ -0,0 +1,33 @@
# Reference bar
> The concrete quality bar for gauntlet work. Every entry must point at something a referee can open, run, or look at — an adjective is not a bar. Changing a bar mid-gauntlet is an owner decision recorded in `DECISIONS.md`.
>
> **Seeded 2026-08-06 at the gauntlet-loop/fable upgrade.** This project already has a real bar: the interactive prototype + the Nocturne token authority + per-screen contracts. **Precedence guard (D-2026-07-31-01 lineage):** the prototype is *evidence, never authority* — where the prototype and the recorded spec disagree, `08-development-spec > 04-rules > PRD` wins and the difference is **not** a gap. The referee grades against the spec-corrected prototype.
Base references: `PROTO = PS Bus Ticketing App - Conductor App.html` (repo root — open in a browser, navigate to the screen) · `TOKENS = docs/06-ui-patterns.md` (Nocturne) · `SPEC = docs/08-development-spec.md` (per-screen contract) · `DESIGN = docs/design/**` (screen specs, where written).
## Bars by part
One row per screen/flow as it enters a gauntlet — seeded with the screens that already have design artifacts; add rows using the template as work reaches each screen. Budgets live on the `GAUNTLET.md` board.
| Part | Reference artifact(s) | How to compare | Minimum parity |
| --- | --- | --- | --- |
| Auth screens 1–2 | PROTO auth screens · `docs/design/` auth spec · SPEC §screen criteria | run the app on the 2 GB reference device (or emulator at its profile), screenshot vs PROTO side by side; check tokens vs TOKENS | layout/hierarchy/tokens match the spec-corrected prototype; per-screen SPEC criteria pass |
| Screen 06 — discount capture (dual-photo) | PROTO screen 06 · `docs/design/` screen-06 spec · SPEC criteria | walk the capture flow on-device; screenshot each state | every state (capture, retake, proof review) present and one-handed operable; ≥ 48 dp targets |
| Screen 11 — printer setup | PROTO screen 11 · `docs/design/` screen-11 spec | walk pairing/test-print flow (or its no-hardware stub — see orchestrator memory: no printer hardware) | states + error paths match; no-hardware path explicit, never silent |
| P10 — prepaid booking / QR | PROTO P10 · `docs/design/` P10 spec · SPEC criteria | walk the flow offline; screenshot | offline-first behavior + states match the spec-corrected prototype |
| [next screen] | PROTO screen NN · `docs/design/` spec if present · SPEC criteria | on-device screenshot side-by-side + flow walk | [what must match] |
Behavioral bars (not screenshots): the ≤ 20 s record-a-passenger contract (stopwatch on the reference device), 7-day-offline invariants (A-1…A-6), and the `TC-*` tables in `docs/09-test-plan.md` — these are already acceptance tests; the gauntlet adds the visual/UX parity layer on top, it does not replace them.
## Reference sources
- `PS Bus Ticketing App - Conductor App.html` — interactive prototype (root)
- `docs/06-ui-patterns.md` — Nocturne tokens/components (authority for visual language)
- `docs/design/**` — written screen specs (authority over the prototype)
- `docs/08-development-spec.md` — per-screen acceptance criteria
## Out of scope for the bar
- Anything the recorded spec has changed from the prototype (spec wins; log the delta as evidence, not a gap).
- Server/back-office UI (contract-only, `docs/07-api-contract.md`), iOS, passenger-facing surfaces.

219
docs/prompting_style.md Normal file
View File

@@ -0,0 +1,219 @@
```markdown
From a systems and software engineering perspective, prompt patterns and agentic loops are structured control flow mechanisms built on top of autoregressive transformer models.
Below is a detailed technical breakdown of these patterns, covering their state transitions, context memory management, prompt schemas, and failure modes.
---
## 1. Deterministic & Context-Shaping Patterns
These patterns operate at the inference step level to constrain token generation probabilities and enforce structural invariants.
### Role & System Conditioning (Logit Shaping)
* **Mechanism:** Injects instructions directly into the system message block, modifying the baseline attention weights across all subsequent user/assistant turns. It acts as an inductive bias, shifting the probability distribution of generated tokens toward domain-specific terminologies and structured logic.
* **Prompt Schema:**
```text
<system_instruction>
ROLE: Senior Distributed Systems Architect.
DOMAIN: Real-time event-driven infrastructure, gRPC, distributed consensus (Raft/Paxos).
INVARIANT: Prioritize zero-data-loss guarantees over minimal latency. Reject eventual consistency unless explicitly requested.
OUTPUT_FORMAT: Technical specification markdown with formal system invariants.
</system_instruction>
```
* **Failure Modes & Mitigations:** *Context Decay* (the model forgets constraints in long turns). Mitigate by placing critical invariant rules at the very end of the system block or repeating constraints in system system-reinforcement flags.
### Few-Shot Delimiter Scaffolding
* **Mechanism:** Imprints input-output mapping patterns directly into the model’s Key-Value (KV) cache. Utilizing explicit XML or structural delimiters prevents token boundary confusion during multi-turn parsing.
* **Prompt Schema:**
```xml
<system>Extract operational state from syslog streams.</system>
<example>
<input>2026-08-07T08:12:01Z node-04 dockerd[1042]: Error: OOMKilled process 8841</input>
<output>{"node": "node-04", "event": "OOMKilled", "pid": 8841, "severity": "CRITICAL"}</output>
</example>
<target>
<input>2026-08-07T08:14:22Z node-01 kernel: [44211.2] Out of memory: Kill process 1204 (postgres)</input>
<output>
```
* **Failure Modes:** Recency/label bias (overweighting the last example's exact values). Keep examples structurally diverse and balanced across edge cases.
---
## 2. Multi-Step Inference & Search Graph Patterns
These frameworks alter the model’s internal computation path by generating intermediate reasoning tokens before emitting the target response.
### Chain-of-Thought (CoT) & Plan-and-Solve
* **Mechanism:** Forces auto-regressive decoding to populate the context buffer with intermediate rationale steps ($z_1, z_2, \dots, z_n$) prior to predicting the target output ($y$). Mathematically:
$$P(y \mid x) = \sum_z P(y \mid x, z) P(z \mid x)$$
* **Execution Protocol:**
```text
Perform the following analysis in two explicit, separated phases:
PHASE 1 (REASONING_BUFFER):
- Identify state invariants and potential race conditions.
- Draft intermediate computational dependencies.
- Evaluate step-by-step edge cases.
PHASE 2 (EXECUTION_OUTPUT):
- Provide the final production-ready implementation wrapped in ```json tags.
```
* **When to Use:** Algorithmic execution, mathematical logic, complex SQL/query optimization.
### Tree-of-Thoughts (ToT) / Graph-of-Thoughts (GoT)
* **Mechanism:** Combines LLM generation with classical state-space search algorithms (Breadth-First Search, Depth-First Search, or $A^*$). The LLM acts both as a *Thought Generator* ($S_{t+1} \sim G(S_t)$) and a *State Evaluator* ($V(S_t) \in [0, 1]$).
```text
[Root State: Initial Prompt]
/ \
[Thought A] [Thought B]
v = 0.8 v = 0.2 (Pruned)
/ \
[Thought A1] [Thought A2]
v = 0.95 v = 0.4
```
* **Execution Pseudocode:**
```python
def tree_of_thoughts_search(root_prompt, beam_width=3, max_depth=4):
current_states = [root_prompt]
for depth in range(max_depth):
candidates = []
for state in current_states:
# 1. Expand candidate branches via LLM
branches = llm_generate_branches(state, num_samples=3)
# 2. Evaluate state heuristic score V(s) via LLM
scores = [llm_evaluate_state(branch) for branch in branches]
candidates.extend(zip(branches, scores))
# 3. Prune low-scoring branches (Beam Search)
candidates.sort(key=lambda x: x[1], reverse=True)
current_states = [branch for branch, score in candidates[:beam_width]]
return current_states[0] # Best evaluated path
```
* **When to Use:** Strategic planning, complex refactoring across multiple files, architecture synthesis.
---
## 3. Agentic Loops & State-Machine Architectures
Agentic frameworks wrap the LLM inside an external, deterministic control loop (e.g., Python/Go runtime, orchestration engines like OpenClaw, or custom middleware).
### ReAct (Reasoning + Action Protocol)
* **State Machine:**
$$\text{State}_t \rightarrow \text{Thought}_t \rightarrow \text{Action}_t(\text{Tool Call}) \rightarrow \text{Observation}_t \rightarrow \text{State}_{t+1}$$
```text
+--------------+ +-------------------+ +-----------------+
| LLM Engine | ----> | Action (Tool Call)| ----> | Execution Runtime|
+--------------+ +-------------------+ +-----------------+
^ |
|-------------- Observation (Payload) <--------------+
```
* **Prompt Engine Specification:**
```text
You operate in a strict execution loop. Available Tools: [exec_bash, query_sql, HTTP_GET].
Use the following format strictly:
Thought: <Logical about current reasoning state>
Action: <Tool_Name>(<JSON_Arguments>)
Observation: <Result by environment injected>
Loop terminates ONLY when you emit:
Final Answer: <Summary of outcome>
```
* **Failure Modes:** Infinite loops caused by unhandled tool errors.
* **Mitigation:** Enforce hard step budgets (`max_iterations = 10`) and circuit breakers on duplicate tool signatures.
### Plan-Execute-Verify (PEV) with Re-Planning
* **Mechanism:** Decouples task breakdown from task execution. The planner generates a Directed Acyclic Graph (DAG) of sub-tasks. An execution loop steps through nodes sequentially, running validation assertions after each step. If a step fails, control yields back to a Re-Planner node to mutate the remaining DAG.
```text
+--------------+
| Generate DAG |
+--------------+
|
v
+-----------------+
+->| Execute Node N |
| +-----------------+
| |
| v
| +-----------------+ FAIL +---------------+
| | Assert / Verify | -------------> | Re-Plan DAG | --+
| +-----------------+ +---------------+ |
| | PASS |
| v |
| [More Nodes Remaining?] --YES--------------------------+
| | NO
| v
| +-----------------+
+--| Final Outcome |
+-----------------+
```
### The Gauntlet Loop (Adversarial Multi-Agent Architecture)
* **Mechanism:** Implements a strict **Maker-Checker Isolation Model**. The Builder Agent generates code/artifacts. A *blind* Critic Agent—instantiated in a zero-history, isolated context window—evaluates the output against a hard reference standard or test harness.
```text
+------------------+ +--------------------+
| Builder Agent | --- Generates ---> | Artifact Payload |
| (Context Window) | +--------------------+
+------------------+ |
^ v
| +--------------------+
|-- Injects Actionable Feedback| Judge Agent |
| (No Excuses Allowed) | (Isolated Context) |
| +--------------------+
| |
+<-- [Fails Reference Standard] ---------+
```
* **System Architecture Protocol:**
```python
def gauntlet_loop(task_spec, reference_standard, max_gauntlet_runs=5):
builder_context = init_builder_context(task_spec)
for iteration in range(max_gauntlet_runs):
# Step 1: Builder generates artifact
artifact = builder_agent.run(builder_context)
# Step 2: Instantiate Judge in FRESH context window (Zero memory leak)
judge_prompt = f"""
TASK: Compare Artifact against Reference Standard.
REFERENCE: {reference_standard}
ARTIFACT TO EVALUATE: {artifact}
OUTPUT RULES:
1. Determine if Artifact >= Reference Standard in quality/correctness.
2. If FAIL, list the single most critical structural deficiency. Do not offer encouragement.
FORMAT: STATUS: [PASS|FAIL] | FEEDBACK: <concise directive>
"""
verdict = judge_agent.run_fresh_context(judge_prompt)
if verdict.status == "PASS":
return artifact
# Step 3: Append harsh feedback to builder context
builder_context.append_user_message(f"GAUNTLET REJECTION: {verdict.feedback}")
raise MaximumGauntletDepthExceeded("Quality threshold not met within limit.")
```
---
## Technical Summary Matrix
| Pattern / Loop Style | Latency Cost | Context Consumption | Determinism | Best Architectural Use Case |
| :--- | :--- | :--- | :--- | :--- |
| **Few-Shot / Schema** | Low ($O(1)$) | Low | High | API Payload Generation, Format Standardization |
| **Chain-of-Thought** | Medium ($O(k)$) | Medium | Medium | Intermediate Math, Single-Query Logic Tracing |
| **Tree-of-Thoughts** | High ($O(b^d)$) | High | High | Complex Codebase Refactoring, Architecture Search |
| **ReAct Agent** | Dynamic | Medium-High | Medium | Runtime API Orchestration, Infrastructure Ops |
| **Plan-Execute-Verify** | High | High | High | Multi-Step Migration Pipelines, CI/CD Automation |
| **Gauntlet Loop** | Very High | Extreme | Maximum | Autonomous End-to-End System/Software Synthesis |
```