Chapter 9 - Layer 4: Motor + Sensory (Agents and Computer Use)
Last updated 2026-07-13. Review the vendor-neutral primary sources behind Chapter 9 and compare current capabilities in the agents and computer-use matrix.
Companion activity
Use the sensorimotor worked example or agents matrix to mark which actions need confirmation gates, sandboxing, or monitor-model review.
What this chapter argues
When the AI partner stops being asked and starts acting, the motor / sensory layer takes over. The sensorimotor loop is the canonical pattern: perceive → plan → act → observe → update. This is the autonomous-agency pattern - the moment the human steps partly out of the loop. The chapter argues for three guardrails that its examples and primary sources both recommend: confirmation gates at materiality thresholds, monitor models that watch the agent independently, and sandboxing that bounds the side-effect surface. Agents do not solve side effects, prompt injection, or training-distribution mismatch. They are a posture choice with concrete failure modes.
Key sources
Chapter 9 cites in the print book: D5, D38, D50, D80, D81. Entries below are the sources this page draws on; some are related literature cited elsewhere in the book rather than in this chapter.
- D5 - Anthropic. (2024). New Claude 3.5 Sonnet and Claude 3.5 Haiku, and computer use in public beta. anthropic.com/news/3-5-models-and-computer-use
- D10 - Anthropic. (2024). Building effective agents. anthropic.com/engineering/building-effective-agents
- D11 - Willison, S. (2025). Designing agentic loops. simonwillison.net/2025/Sep/30/designing-agentic-loops
Primary-source links
- Anthropic, Computer Use launch (October 22, 2024) - the production-ready autonomous-agency capability that anchored the chapter's vendor-side timeline.
- Anthropic, "Building effective agents" (December 2024) - first-party patterns for safe agent construction, including monitor-model and sandboxing recommendations.
- Willison, "Designing agentic loops" - practitioner-side framing of the same patterns from a non-vendor perspective.
Chapter contents
- The sensorimotor loop in five stages, with the failure mode at each handoff.
- The autonomous-agency posture: when to grant it, when to refuse, what compensating controls are required.
- What agents do NOT solve: side effects, prompt injection, training-distribution mismatch.
- The relationship between the materiality dial (Chapter 13) and the autonomy posture.
Agent matrix
Current vendor-specific capability deltas across Claude Computer Use, OpenAI Operator / CUA, Project Mariner, Devin, and the open-source agent frameworks live in the agents and computer-use matrix. Each row carries a verified date so freshness is visible. Refreshed quarterly.
For a worked agent action log with side-effect annotations, see the sensorimotor worked example.