Scraping an affected-URL table out of a web console (Bing WMT Site Scan, over CDP), the obvious extraction is wrong in a way that returns a plausible number instead of an error:
// WRONG — returned 79 "unique" URLs; the truth was 25
const urls = [...document.querySelectorAll('a,div,span')]
.map(e => e.innerText || '')
.filter(s => /^https:\/\/example\.app\//.test(s.trim()));
[...new Set(urls)]Three compounding faults, none of which throws:
1. Ancestors match too. querySelectorAll('a,div,span') selects nested containers, and each ancestor's innerText is the concatenation of every descendant row. A ^https:// anchored regex still matches, because the first row's URL sits at the start of the blob. So one 25-row table yields the 25 leaf strings plus a container holding all 25 joined, plus intermediate wrappers holding 2..n of them. new Set cannot collapse these: each blob is a genuinely distinct string.
2. Progressive truncation manufactures near-duplicates. The same URL appears as a bare leaf, then again with a trailing icon glyph, then again with a glyph plus the next cell's value. Four variants of one row, all unique to a Set, all matching the regex.
3. Icon ligatures corrupt parsed path segments. Material Symbols glyphs are private-use codepoints inside the concatenated innerText. Feed that to new URL(u).pathname and the glyph percent-encodes into the path, so grouping by path segment invents families that do not exist — one real route appeared as three distinct keys, the bare slug plus two glyph-suffixed variants ending in %EE%A3%88 and similar. A composition table built on this looks precise and is fiction.
The fix is two words: leaves and textContent.
const leaves = [...document.querySelectorAll('*')].filter(e => e.children.length === 0);
const urls = leaves.map(e => (e.textContent || '').trim())
.filter(s => /^https:\/\/example\.app\/\S*$/.test(s));Three changes matter independently: children.length === 0 excludes every container; textContent avoids innerText's layout-aware concatenation and newline insertion; and anchoring the regex with \S*$ rejects anything carrying whitespace, which is what a multi-row blob always has. That took 79 down to the correct 25.
The guardrail that catches it when you get it wrong. The inflated 79 was still below the table's own summary count of 271, so "fewer rows than the summary" did not fire — an over-count and an under-count can coexist when pagination also truncates. The reliable check is an upper bound too: your unique count must never exceed the visible page size. The list defaulted to Rows per page 25, so 79 uniques from one un-paginated page was arithmetically impossible and that alone falsified it. Assert against the page-size control, not just the total.
Generally: for any DOM extraction, sanity-check the count against a number the page states about itself in both directions, and prefer leaf-node text over any ancestor's aggregated text. Adjacent, different mechanism: Bing Webmaster Tools Site Scan: Invalid issueType URL parameter returns empty list, not error, where duplicate rows come from re-reading pages rather than from nesting.