LexAI/docs/prompting_style.md

From a systems and software engineering perspective, prompt patterns and agentic loops are structured control flow mechanisms built on top of autoregressive transformer models. 

Below is a detailed technical breakdown of these patterns, covering their state transitions, context memory management, prompt schemas, and failure modes.

---

## 1. Deterministic & Context-Shaping Patterns

These patterns operate at the inference step level to constrain token generation probabilities and enforce structural invariants.

### Role & System Conditioning (Logit Shaping)
*   **Mechanism:** Injects instructions directly into the system message block, modifying the baseline attention weights across all subsequent user/assistant turns. It acts as an inductive bias, shifting the probability distribution of generated tokens toward domain-specific terminologies and structured logic.
*   **Prompt Schema:**
    ```text
    <system_instruction>
    ROLE: Senior Distributed Systems Architect.
    DOMAIN: Real-time event-driven infrastructure, gRPC, distributed consensus (Raft/Paxos).
    INVARIANT: Prioritize zero-data-loss guarantees over minimal latency. Reject eventual consistency unless explicitly requested.
    OUTPUT_FORMAT: Technical specification markdown with formal system invariants.
    </system_instruction>
    ```
*   **Failure Modes & Mitigations:** *Context Decay* (the model forgets constraints in long turns). Mitigate by placing critical invariant rules at the very end of the system block or repeating constraints in system system-reinforcement flags.

### Few-Shot Delimiter Scaffolding
*   **Mechanism:** Imprints input-output mapping patterns directly into the model’s Key-Value (KV) cache. Utilizing explicit XML or structural delimiters prevents token boundary confusion during multi-turn parsing.
*   **Prompt Schema:**
    ```xml
    <system>Extract operational state from syslog streams.</system>

    <example>
    <input>2026-08-07T08:12:01Z node-04 dockerd[1042]: Error: OOMKilled process 8841</input>
    <output>{"node": "node-04", "event": "OOMKilled", "pid": 8841, "severity": "CRITICAL"}</output>
    </example>

    <target>
    <input>2026-08-07T08:14:22Z node-01 kernel: [44211.2] Out of memory: Kill process 1204 (postgres)</input>
    <output>
    ```
*   **Failure Modes:** Recency/label bias (overweighting the last example's exact values). Keep examples structurally diverse and balanced across edge cases.

---

## 2. Multi-Step Inference & Search Graph Patterns

These frameworks alter the model’s internal computation path by generating intermediate reasoning tokens before emitting the target response.

### Chain-of-Thought (CoT) & Plan-and-Solve
*   **Mechanism:** Forces auto-regressive decoding to populate the context buffer with intermediate rationale steps ($z_1, z_2, \dots, z_n$) prior to predicting the target output ($y$). Mathematically:
    $$P(y \mid x) = \sum_z P(y \mid x, z) P(z \mid x)$$
*   **Execution Protocol:**
    ```text
    Perform the following analysis in two explicit, separated phases:
    PHASE 1 (REASONING_BUFFER):
    - Identify state invariants and potential race conditions.
    - Draft intermediate computational dependencies.
    - Evaluate step-by-step edge cases.

    PHASE 2 (EXECUTION_OUTPUT):
    - Provide the final production-ready implementation wrapped in ```json tags.
    ```
*   **When to Use:** Algorithmic execution, mathematical logic, complex SQL/query optimization.

### Tree-of-Thoughts (ToT) / Graph-of-Thoughts (GoT)
*   **Mechanism:** Combines LLM generation with classical state-space search algorithms (Breadth-First Search, Depth-First Search, or $A^*$). The LLM acts both as a *Thought Generator* ($S_{t+1} \sim G(S_t)$) and a *State Evaluator* ($V(S_t) \in [0, 1]$).

```text
          [Root State: Initial Prompt]
                  /        \
          [Thought A]    [Thought B]
            v = 0.8        v = 0.2 (Pruned)
           /       \
     [Thought A1] [Thought A2]
       v = 0.95     v = 0.4

• Execution Pseudocode:

  def tree_of_thoughts_search(root_prompt, beam_width=3, max_depth=4):
      current_states = [root_prompt]
      for depth in range(max_depth):
          candidates = []
          for state in current_states:
              # 1. Expand candidate branches via LLM
              branches = llm_generate_branches(state, num_samples=3)
              # 2. Evaluate state heuristic score V(s) via LLM
              scores = [llm_evaluate_state(branch) for branch in branches]
              candidates.extend(zip(branches, scores))
  
          # 3. Prune low-scoring branches (Beam Search)
          candidates.sort(key=lambda x: x[1], reverse=True)
          current_states = [branch for branch, score in candidates[:beam_width]]
      return current_states[0] # Best evaluated path
  

• When to Use: Strategic planning, complex refactoring across multiple files, architecture synthesis.

---

3. Agentic Loops & State-Machine Architectures

Agentic frameworks wrap the LLM inside an external, deterministic control loop (e.g., Python/Go runtime, orchestration engines like OpenClaw, or custom middleware).

ReAct (Reasoning + Action Protocol)

• State Machine: \text{State}_t \rightarrow \text{Thought}_t \rightarrow \text{Action}_t(\text{Tool Call}) \rightarrow \text{Observation}_t \rightarrow \text{State}_{t+1}

  +--------------+       +-------------------+       +-----------------+
  | LLM Engine   | ----> | Action (Tool Call)| ----> | Execution Runtime|
  +--------------+       +-------------------+       +-----------------+
         ^                                                    |
         |-------------- Observation (Payload) <--------------+

• Prompt Engine Specification:

  You operate in a strict execution loop. Available Tools: [exec_bash, query_sql, HTTP_GET].
  
  Use the following format strictly:
  Thought: <Logical about current reasoning state>
  Action: <Tool_Name>(<JSON_Arguments>)
  Observation: <Result by environment injected>
  
  Loop terminates ONLY when you emit:
  Final Answer: <Summary of outcome>
  

• Failure Modes: Infinite loops caused by unhandled tool errors.
• Mitigation: Enforce hard step budgets (max_iterations = 10) and circuit breakers on duplicate tool signatures.

Plan-Execute-Verify (PEV) with Re-Planning

• Mechanism: Decouples task breakdown from task execution. The planner generates a Directed Acyclic Graph (DAG) of sub-tasks. An execution loop steps through nodes sequentially, running validation assertions after each step. If a step fails, control yields back to a Re-Planner node to mutate the remaining DAG.

       +--------------+
       | Generate DAG |
       +--------------+
              |
              v
     +-----------------+
  +->| Execute Node N  |
  |  +-----------------+
  |           |
  |           v
  |  +-----------------+      FAIL      +---------------+
  |  | Assert / Verify | -------------> | Re-Plan DAG   | --+
  |  +-----------------+                +---------------+   |
  |           | PASS                                        |
  |           v                                             |
  |  [More Nodes Remaining?] --YES--------------------------+
  |           | NO
  |           v
  |  +-----------------+
  +--| Final Outcome   |
     +-----------------+

The Gauntlet Loop (Adversarial Multi-Agent Architecture)

• Mechanism: Implements a strict Maker-Checker Isolation Model. The Builder Agent generates code/artifacts. A blind Critic Agent—instantiated in a zero-history, isolated context window—evaluates the output against a hard reference standard or test harness.

+------------------+                    +--------------------+
|  Builder Agent   | --- Generates ---> | Artifact Payload   |
| (Context Window) |                    +--------------------+
+------------------+                              |
         ^                                        v
         |                              +--------------------+
         |-- Injects Actionable Feedback|    Judge Agent     |
         |   (No Excuses Allowed)       | (Isolated Context) |
         |                              +--------------------+
         |                                        |
         +<-- [Fails Reference Standard] ---------+

• System Architecture Protocol:

  def gauntlet_loop(task_spec, reference_standard, max_gauntlet_runs=5):
      builder_context = init_builder_context(task_spec)
  
      for iteration in range(max_gauntlet_runs):
          # Step 1: Builder generates artifact
          artifact = builder_agent.run(builder_context)
  
          # Step 2: Instantiate Judge in FRESH context window (Zero memory leak)
          judge_prompt = f"""
          TASK: Compare Artifact against Reference Standard.
          REFERENCE: {reference_standard}
          ARTIFACT TO EVALUATE: {artifact}
  
          OUTPUT RULES:
          1. Determine if Artifact >= Reference Standard in quality/correctness.
          2. If FAIL, list the single most critical structural deficiency. Do not offer encouragement.
          FORMAT: STATUS: [PASS|FAIL] | FEEDBACK: <concise directive>
          """
  
          verdict = judge_agent.run_fresh_context(judge_prompt)
  
          if verdict.status == "PASS":
              return artifact
  
          # Step 3: Append harsh feedback to builder context
          builder_context.append_user_message(f"GAUNTLET REJECTION: {verdict.feedback}")
  
      raise MaximumGauntletDepthExceeded("Quality threshold not met within limit.")
  

---

Technical Summary Matrix

Pattern / Loop Style	Latency Cost	Context Consumption	Determinism	Best Architectural Use Case
Few-Shot / Schema	Low (O(1))	Low	High	API Payload Generation, Format Standardization
Chain-of-Thought	Medium (O(k))	Medium	Medium	Intermediate Math, Single-Query Logic Tracing
Tree-of-Thoughts	High (O(b^d))	High	High	Complex Codebase Refactoring, Architecture Search
ReAct Agent	Dynamic	Medium-High	Medium	Runtime API Orchestration, Infrastructure Ops
Plan-Execute-Verify	High	High	High	Multi-Step Migration Pipelines, CI/CD Automation
Gauntlet Loop	Very High	Extreme	Maximum	Autonomous End-to-End System/Software Synthesis