Automated browser checks boot a dev server, wait for it to be "ready", then drive a browser at a URL you assembled yourself. Both halves of that can succeed while pointing at something that is not your server. Neither failure produces an error — you get a clean, confident, wrong measurement.
Two concrete instances from one session of cross-browser scroll testing, both costing real debugging time.
1. A port readiness check can be satisfied by a stranger
Dev servers commonly auto-increment when the requested port is taken. Vite/Astro do this by default:
> astro preview
Port 4321 is in use, trying another one...
astro v7.1.2 ready in 21 ms
┃ Local http://localhost:4322/The supervisor was told "ready when TCP 4321 accepts connections". It reported ready in 106 ms — because an unrelated long-running server from another project already held 4321. The harness then drove a browser at http://localhost:4321 and got someone else's app (page title: CSSError), and the probe failed waiting for a selector that was never going to exist.
The tell is the timing: a static-site preview does not become ready in 106 ms. The real one took ~1.3 s.
Fixes, in order of preference:
- Parse the bound URL out of the server's own output instead of assuming it. That is the only source of truth.
- Or pin the port and make a conflict fatal — Vite/Astro accept
--strictPort, which exits rather than incrementing. A hard failure is strictly better than a silent redirect to a stranger's server. - Or, at minimum, assert identity rather than liveness: fetch
/and check for a marker you know is yours (a title, a meta tag, a known element) before you trust the target.
"A socket is open on the port I wanted" is not evidence that your server is behind it.
2. localhost and 127.0.0.1 are not the same listener
Same session, opposite failure. The server was started with an explicit --port 4400 and printed:
┃ Local http://localhost:4400/A readiness check against 127.0.0.1:4400 timed out after 90 s. The process was up and serving the whole time.
On a dual-stack machine localhost resolves to ::1 first, and a server that binds the hostname may end up listening on IPv6 loopback only — so an IPv4-literal probe never connects. Passing --host 127.0.0.1 made it bind the v4 loopback and readiness passed in 1.3 s.
This cuts both ways, and the direction is tool-specific: a related trap is Chrome 150's DevTools endpoint 404ing when the Host header is 127.0.0.1 while localhost works ([[Chrome 150 DevTools endpoint rejects Host 127.0.0.1 - use localhost for browserUrl and lighthouse port attach]]). So the rule is not "prefer one form" — it is:
- Probe the same address family and hostname form your test client will use. A readiness check that connects differently from the browser is not checking readiness.
- When in doubt, try both forms; if only one answers, bind explicitly (
--host 127.0.0.1) rather than relying on resolution order.
Why this class is worth a guard
Both bugs share a shape that makes them expensive: the harness stays green and lies about what it measured, so the wrong readings get attributed to the code under test. In my case a probe reading "the wheel handler is still active" looked like a shipped fix that had not taken effect, and I went off reading deployed bundles before realising the target was wrong.
Cheap defences, roughly in order of value per line:
- Never hardcode the URL you asked for. Capture the one the server printed.
- Assert identity on first contact. One
fetch+ marker check turns "wrong server" from a mysterious selector timeout into an immediate, obvious failure. - Sanity-check readiness latency. Ready far faster than a cold start can plausibly be is a strong signal you attached to something already running.
- Use
--strictPort(or equivalent) in automation. Auto-increment is a convenience for humans watching a terminal and a hazard for programs that are not.
Related but distinct: the same session also hit a deployed-version version of this — measuring a CDN edge that was still serving the previous build seconds after the deploy tool reported success ([[Cloudflare Workers static-assets site serves stale content after deploy despite max-age=0, must-revalidate]]). Same lesson one layer up: confirm which build answered, not just that something did. For hashed assets, curl the HTML, pull out the bundle path, and grep it for a token unique to your change.