Failure-Diagnosis Prompts

Region-by-region diagnostic probes (Chapter 12). When the partnership misbehaves, walk the regions in order rather than guessing at a fix. These probes are sized to identify which region failed; the repair depends on the region.

Region 1 - Dialog interface (intent / framing)

Probes for whether the literal request matched what the user actually wanted.

Symptom check: the answer is technically correct but does not solve the user's problem. Probes: 1. Re-read the original prompt verbatim. What did it literally ask for? 2. What was the user actually trying to accomplish? (Different from what they asked.) 3. Did the user's framing carry an assumption that, if wrong, changes the work? Did the partner challenge it, or accept it? Verdict: dialog interface failed if (1) and (2) diverge AND the partner did not surface the divergence before acting. Repair: re-open the dialog with explicit intent-clarification (see /prompts/dialog/). Do not just re-run the task with a different prompt; the framing miss will recur.

Region 2 - Working memory (context / loop)

Probes for whether the relevant content was in context when the failing action happened.

Symptom check: the partner produces output as if missing information it should have had access to. Probes: 1. Was the relevant content within the context window at the moment of action? (Check size, model limit, position in the conversation.) 2. Did a /compact or summarization event occur between when the content was provided and when it was needed? 3. Is the content in long-term memory (file, repo, notes) that the partner could have re-read? Verdict: working memory failed if (1) was true at provision time but false at action time, AND (3) was not used. Repair: introduce externalized memory (NOTES.md or equivalent) that the partner reads at action time. See /prompts/working-memory/.

Region 3 - Long-term memory (priors)

Probes for whether the partner leaned on a generic prior instead of the specific evidence in context.

Symptom check: the answer is consistent with a generic prior pattern rather than the specific evidence provided. Probes: 1. Ask the same model the same question with NO source material attached. Does it produce a similar answer? (If yes: long-term-memory leak.) 2. Does the visible reasoning (if available) cite specific source content, or general claims? 3. Did the answer ignore specific exemptions, caveats, or counter-evidence that were present in the source? Verdict: long-term memory leaked into the answer if (1) is yes AND (3) is yes. Repair: prompt-level fix is "for each claim X, quote the specific source line." Process-level fix is a maker/checker pass with a model instructed to find exemptions/caveats.

Region 4 - Prefrontal (reasoning / metacognition)

Probes for whether the reasoning loop disconfirmed alternatives or just confirmed the first plausible answer.

Symptom check: the partner committed to a wrong answer without considering plausible alternatives. Probes: 1. If a visible chain is available, did the chain enumerate alternatives, or did it commit to one path? 2. Did the chain ever attempt to disconfirm its own answer (look for contradicting evidence)? 3. Was there a frame-before-solve step (problem restated in the partner's own words) before solving? Verdict: prefrontal failed if the chain shows reasoning but no alternatives generation OR no disconfirmation attempt. Repair: add "before answering, name 2 plausible alternatives and the specific evidence that would distinguish them" to the prompt. For high-stakes work, add a separate metacognitive pass (ACT-style adversarial review).

Region 5 - Motor / sensory (tools / actions)

Probes for whether a tool call did the wrong thing or had unexpected side effects.

Symptom check: the partner took an action with unintended or unauthorized side effects. Probes: 1. Was the action gated (did the partner ask before taking it)? If no: gating failure. 2. Was the action scope what the partner thought it was? (E.g. "update production" vs "update all environments.") If scope drift: disambiguation failure. 3. Did the action's side effects match what the partner announced before taking it? If no: tool-call failure or prompt-injection. Verdict: motor / sensory failed if any of the three diverged. Repair depends on root: gating failure -> harness policy update; scope drift -> add explicit-scope confirmation gates (see /examples/sensorimotor/); tool-call divergence -> inspect the specific tool's behavior and add a pre-call validator.

Region 6 - Nervous system (protocols / interop)

Probes for whether the protocol layer (MCP, function calling) carried unexpected payloads or context.

Symptom check: rare. The partner behaved unexpectedly and no other region's diagnostic points at a cause. Probes: 1. Inspect MCP server logs (if applicable). What did the server return in response to each call? 2. Did any tool result contain content that could be read as instructions (prompt injection via tool output)? 3. Did a server return malformed data that the partner then used as truth? Verdict: nervous system failed if a tool result was either malformed-but-trusted or injection-bearing-and-followed. Repair: tool-output sanitization at the harness level; explicit instruction to treat tool results as data, not instructions; if injection is confirmed, audit the upstream surface (the website, database, file) that delivered the injected content.

The diagnostic order matters

Walk the regions in order. Dialog interface and working memory together account for the majority of partnership failures; the deeper regions are less common (but more expensive when missed). Resist the temptation to skip ahead to the "interesting" region without ruling out the cheap ones first.

Once a region is named as the root cause, the repair lives in the prompts for that region's layer. Do not patch a working-memory failure with a dialog-layer prompt or vice versa; the fix will be cosmetic and the failure will recur.

Worked diagnosis

See the failure-diagnosis worked example for these probes applied to a real incident.