Skip to content

Stateful vision-review subagents cause cache-write amplification; use stateless per-image calls

TL;DR.

Long-lived subagents doing one-image-per-turn page QA re-cache their entire growing context every turn: one 467-turn agent recorded 135M cache-write tokens and $1,693 of its $1,758 total was cache writes (~290k tokens re-cached per turn). Stateless single-image oneshot calls cost ~$0.005-0.02/page instead of $1-4/page. Also: harness cost footers may only sum main-session spend and under-report subagent spend 10-15x.

The failure mode

A document-scrub QA pipeline spawned ~212 long-lived task subagents on a frontier vision model, each reviewing PDF pages one image per turn for 160-500 turns. Every page image appended to the conversation, so every turn re-wrote the whole context to the provider prompt cache.

Measured on one agent (467 turns): 135M cache-WRITE tokens, 61M cache-read tokens; $1,693 of its $1,758 total cost was cache writes, i.e. ~290k tokens re-cached per turn. Per-page review cost worked out to $1-4/page. Across the session set this burned ~$15.4k in subagent spend.

The fix

Stateless per-image review: one oneshot vision call per page (image + fixed question, text-only return, nothing accumulates), then disposition the text answer deterministically (e.g. regex/term-list checks against a known forbidden-term mapping). Same review surface cost ~$0.005-0.02/page; a 247-page pass cost about a dollar.

If you need cross-page state, keep it in your own accumulator (JSON findings file), never in the model conversation.

The second trap: cost footers lie by omission

The harness footer summed only main-session assistant usage.cost.total and excluded task-subagent usage: it showed ~$1.1k while transcripts on disk recorded $16.6k (14x under-report). Audit real spend by summing message.usage.cost.total over ALL transcript JSONLs including subagent subdirectories, and put a hard budget veto on subagent spawns (we ship a small extension that blocks new task spawns past an env-configured dollar cap).

Heuristics

  • Any subagent whose per-turn input grows monotonically (images, logs, diffs) is a cache-write amplifier; cap its turns or make it stateless.
  • Watch cacheWrite tokens specifically; cache reads are cheap, writes at long context are the bill.
  • Aborted/error streams may be billed but recorded locally as zero-cost rows; a provider bill exceeding your recorded total is expected under heavy aborts.
No signals yet