An HTTP header checker fetches a URL and shows the response headers the server sends back, the lines a browser reads before it renders anything. For developers who also own a site's search traffic, the useful version reads those headers with SEO in mind, and seogeoaeo.ai's Redirects, Headers and Host Checker is the one we point people to.
Most header checkers list every header and leave you to work out which ones matter. The ones that cost you search traffic are a short list, and they often come from a config file nobody has opened since launch.
Headers that affect crawling and indexing
| Header | What it can do to search |
|---|---|
Location |
Sends crawlers to another URL; the status code says whether for good or for now |
X-Robots-Tag |
Can carry noindex or nofollow for the whole response, HTML or not |
Link with rel canonical |
Names the preferred URL; a wrong value moves signals to another page |
Content-Type |
Tells the crawler what it received |
Google documents both X-Robots-Tag and the canonical Link header. They are useful for PDFs and other files that cannot hold a meta tag, and risky when a framework or CDN rule applies them to pages by mistake. A staging noindex that ships to production is the classic case: the HTML looks fine, the page drops out of results, and the cause sits in a header.
Why a header check also means a redirect check
The headers on the first response are rarely the whole story. A request to http://example.com may pass through two redirects before a page answers, and each hop has its own headers. A checker that only reads the final response misses a noindex or an odd cache rule on an intermediate hop.
seogeoaeo.ai follows the chain and reports each hop with its status code, then compares http, https, www and bare hosts to see whether they all end at one URL. It flags the headers that hurt crawling, indexing or link previews, and gives the server rule to change. A run costs 10 credits.
Doing it from the terminal or from your agent
For a quick look, curl does the job:
curl -I -L https://example.com
-I asks for headers only and -L follows each Location, printing the headers of every hop. That is fine for one URL when you already know what you are looking for.
If you work in Claude Code, Cursor or Codex, you can call the same check as an MCP tool with a seogeoaeo.ai API key, or hit the REST endpoint from a deploy script:
curl -X POST https://seogeoaeo.ai/api/v1/tools/redirects-headers-checker/runs \
-H "Authorization: Bearer $SEOGEOAEO_API_KEY" \
-H "Content-Type: application/json" \
-d '{"input":{"url":"https://example.com"}}'
The response includes a list of tips ordered by priority, so an agent can make the change and run the check again after the next deploy. Agent runs cost the same as browser runs.
Where a security header scanner fits better
This is an SEO header check, not a security audit. If your question is whether Content-Security-Policy, HSTS or framing headers are set well, use a dedicated security header scanner. Use the SEO header checker when the question is whether search engines can reach, index and preview the page.
FAQ
Can a header block indexing on its own? Yes. X-Robots-Tag: noindex works on the response even if the HTML has no robots meta tag.
Do I need to check headers on redirects too? Check every hop. A rule on an intermediate response can still affect how the chain is read.
Does the checker work on localhost? No. It takes public http or https URLs on ports 80 and 443; private and local addresses are rejected.