Worked Example - Failure Diagnosis by Region

Companion to Chapter 12. A real incident walked through the per-region diagnostic process. The symptom was clear (the agent produced a confidently wrong answer); the question was which region failed. Walking the regions in order - rather than guessing at a fix - is what gets the root cause named.

The symptom

An AI partner was asked to summarize a 30-page security policy and produce a list of which sections applied to the company's data-handling pipeline. It produced a clean five-page summary and a list of seven applicable sections. The user, reviewing later, noticed Section 14 (data retention) was named as applicable - but the policy explicitly exempted the pipeline category in Section 14.2. The summary did not mention 14.2 at all.

Region-by-region diagnostic

Step 1 - Dialog interface (intent / framing)

Did the user actually ask for "what applies?" or for something else? Re-read the original prompt.

Prompt was: "Summarize this security policy and list which sections apply to our data pipeline."

Dialog interface: ruled out. The literal request was answered.

Step 2 - Working memory (context / loop)

Was the full policy in the model's context when the summary was produced, or was part of it lost? Check the upload size, model context limit, and whether any compaction occurred.

Policy was 30 pages, ~24k tokens. Model context window: 200k tokens. No /compact events. The Section 14.2 text was definitely in context.

Working memory: ruled out. The content the model needed was present.

Step 3 - Long-term memory (priors)

Did the model lean on a prior pattern ("retention policies apply to all data pipelines") rather than on what this specific policy said? Check whether the model would produce the same wrong answer with the policy text removed.

Test: ask the same model the same question with no policy attached. Result: it produces a similar but generic list including "data retention" as a likely applicable section. This is a prior-pattern answer, not a policy-grounded answer.

Long-term memory: probable root cause. The model was answering from a generic prior, not from the specific exemption in 14.2.

Step 4 - Prefrontal (reasoning / metacognition)

Did the model's reasoning trace name 14.2 and then dismiss it, or did it never consider 14.2? Check the visible thinking if available.

Model: reasoning model with visible thinking enabled. Thinking trace: enumerated sections 1 through 25, summarized each in 1-2 sentences, and grouped by "applies / doesn't apply / unclear." Section 14 was in "applies" without any mention of 14.2's exemption.

Prefrontal: contributing cause. The reasoning loop never disconfirmed the prior-pattern answer because it did not specifically look for exemptions.

Step 5 - Motor / sensory (tools / actions)

No tool calls in this task; pure summarization. Ruled out by task shape.

Motor / sensory: not in play.

Step 6 - Nervous system (protocols / interop)

Not applicable to a single-model summarization task.

Nervous system: not in play.

The root cause, named

The failure was a long-term memory leak into a prefrontal-layer decision: the model's general prior about retention policies applying to pipelines overrode the specific exemption in the policy text, and the reasoning loop did not specifically check for exemptions per section, so the prior went unchallenged.

Notice what the diagnostic ruled out: the dialog was fine, the context was fine, the tools were fine. The fix is not "improve the prompt" or "increase the context window" or "switch models." The fix has to address the specific layer where the failure happened.

The fix

Two changes, sized to materiality:

Cheap and immediate (working-memory layer): add an instruction to the summarization prompt: "For each section you mark as applicable, also scan the same section for explicit exemptions and quote any you find." This forces the prefrontal layer to do the specific check it skipped.
Process-level (memory / verification layer): establish a maker / checker pattern for compliance summarizations - a second pass (different prompt, ideally different model) that specifically searches for exemption language. This protects the next compliance summarization regardless of which model is used.

What this trace shows

Walking the regions in order took five minutes and ruled out three layers cleanly. Without the diagnostic framework, the fix-it instinct would have been "improve the prompt" - which would have helped only by accident.
The visible thinking trace (D21 caveat notwithstanding) was useful as evidence. It showed the model's reasoning never considered 14.2 - that was the disconfirming evidence that pointed at long-term memory rather than working memory.
The fix is layered: a tactical patch at the prompt level and a process patch at the verification level. Different layers; same root cause.

Practitioner templates

The failure-diagnosis prompts page hosts the per-region diagnostic probes used in this trace.