Skip to content

url-verification

URL Verification — Always Check Before Sending

Section titled “URL Verification — Always Check Before Sending”

Before presenting ANY URL to the user, verify it returns HTTP 200 via curl -sI or WebFetch. Do NOT include URLs from training data without verification — vendor docs change paths frequently and 404s erode trust.

  • In-chat: verify before pasting. If not 200, say “search [topic] on [vendor] site” instead.
  • Citations in reports / wiki: batch-verify after drafting; add [retrieved YYYY-MM-DD].
  • When a URL 404s: find the new location or mark [vendor restructured — find via search for "<topic>"].
  • Bulk check: for url in $(grep -oE 'https?://[^ )]+' file.md); do code=$(curl -sI -o /dev/null -w "%{http_code}" "$url" --max-time 8); [ "$code" != "200" ] && echo "$code $url"; done
  • Subagent prompts: include “VERIFY each URL with curl HEAD; list unverifiable ones explicitly.”
  • URLs the user provided — verbatim, not generated by you
  • Repo URLs of the active project — already grounded
  • URLs inside files you are only reading (not adding new ones)
  • Internal/intranet URLs that 404 by design — note as “internal, cannot verify”