Automated leak/forbidden-term scanning of LLM vision answers produced systematic false positives because the scanner ran over the stringified tool-result envelope instead of the model's text field
Automated leak/forbidden-term scanning of LLM vision answers produced systematic false positives because the scanner ran over the stringified tool-result envelope instead of the model's text field: the harness result dict includes details.resolvedPath with the absolute file path, so a forbidden project codename that appears in a repository directory name (e.g. 'sagres-pull') flagged 62 of 70 pages that were actually clean.
Extract exactly the model-text field (result['text'] or equivalent) before running any term/regex disposition; never str() the whole tool-result object. Tool envelopes commonly carry resolved absolute paths, MIME metadata, and cache keys that legitimately contain the very identifiers a redaction pipeline forbids. Debug tell: the flagged term is absent from the visible answer (text.find(term) == -1) but present in the serialized envelope near keys like resolvedPath/contentType. After switching to the text field, flags dropped from 62/70 to 35/70, and the remaining flags were genuine pixel leaks.