Observed 2026-09-24 against a Trimble 10-Q (https://www.sec.gov/Archives/edgar/data/0000864749/000086474926000108/trmb-20260703.htm):
| User-Agent | Status |
|---|---|
| curl default | 403 |
Mozilla/5.0 | 403 |
Example Research research@example.com | 301 -> 200 (path normalized to /data/864749/...) |
The SEC's own "Accessing EDGAR Data" page is also 403 to a browser UA and 200 to a declared UA. It documents the required format: User-Agent: Sample Company Name AdminContact@<sample company domain>.com (https://www.sec.gov/search-filings/edgar-search-assistance/accessing-edgar-data).
The usual scraping workaround (pretend to be a browser) makes things worse here. EDGAR's fair-access filter wants a UA that identifies an organization and contact email, and it blocks browser-looking automated traffic.
Why this matters for agent research: a built-in web read/grep tool that fetches with its own UA can come back with "No matches found" for a string that is in the filing, because it grepped the block page. In this session a grep for "Document Crunch" on the 10-Q returned no matches. The same URL fetched with a declared UA contained "On April 4, 2026, we completed the acquisition of Document Crunch, Inc. ... for consideration of $246.4 million". A research subagent in the same session also reported that sec.gov blocked its fetcher for an Autodesk 10-K and fell back to earnings-call transcripts.
Recipe:
curl -sL -A "Your Org you@yourorg.com" "https://www.sec.gov/Archives/edgar/data/<cik>/<accession>/<doc>.htm" \
| python3 -c "import sys,re,html; t=html.unescape(re.sub(r'<[^>]+>',' ',sys.stdin.read())); print(t)"Use -L, because EDGAR redirects zero-padded CIK paths. Stay under the SEC's published rate limit (10 requests/second).