url-verification
URL Verification — Always Check Before Sending
Section titled “URL Verification — Always Check Before Sending”Before presenting ANY URL to the user, verify it returns HTTP 200 with a GET or WebFetch. Do NOT include URLs from training data without verification — vendor docs change paths frequently and 404s erode trust.
curl -s -o /dev/null -w "%{http_code}" -L --max-time 25 \ -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "$url"-s -o /dev/null discards the body, -L follows redirects, the browser UA avoids bot-gating.
Never verify with HEAD (curl -I / -sI). Many vendor doc hosts 405/404 a HEAD while serving the same URL 200 over GET, manufacturing false-dead links on exactly the sources a citation needs. Measured evidence: docs/url-verification.md.
How to apply
Section titled “How to apply”- In-chat: verify before pasting. If not 200, say “search [topic] on [vendor] site” instead.
- Citations in reports / wiki: batch-verify after drafting; add
[retrieved YYYY-MM-DD]. - When a URL 404s: find the new location or mark
[vendor restructured — find via search for "<topic>"]. - Subagent prompts: include “VERIFY each URL with a curl GET (never HEAD); list unverifiable ones explicitly.”
- Bulk check one-liner for a whole file:
docs/url-verification.md.
When to skip
Section titled “When to skip”URLs the user provided verbatim · repo URLs of the active project (already grounded) · URLs inside files you are only reading, not adding · internal/intranet URLs that 404 by design (note as “internal, cannot verify”).