How to Bulk Check URL Status Codes & Redirects in 2026

Checking one URL’s status code is trivial — curl -I and you’re done. Checking fifty thousand URLs, following every redirect chain to its final destination, recording response times, and not getting rate-limited halfway through is a different problem entirely. This is the bread-and-butter task behind SEO audits, site migrations and link monitoring, and it’s deceptively easy to get wrong. This guide covers what a real bulk status check needs to capture, how redirect chains trip people up, and how to run a large audit cleanly.

What’s worth capturing per URL

A status check that only records the final HTTP code throws away most of the useful signal. For each URL you actually want:

• HTTP status code — 200, 301, 302, 304, 404, 410, 500, 503 and the rest.
• Human-readable status message — including network-level outcomes (DNS failure, connection refused, timeout) that have no HTTP code at all.
• Final resolved URL — where the URL actually lands after following redirects.
• Full redirect chain — the ordered list of hops, not just the start and end. A 301 to a 302 to a 200 is a three-hop chain and that matters for crawl budget and link equity.
• Response time in milliseconds — total time to resolve, useful for spotting slow endpoints and timeout-prone pages.
• Redirect count — how many hops it took, so you can flag chains worth collapsing.

The redirect chain is the field most homegrown scripts drop, and it’s exactly the one SEO and migration work depends on.

Why redirect chains are the hard part

The naive approach — “follow redirects, give me the final code” — hides the most actionable information. A few realities:

• Chains accumulate. After a couple of migrations a single old URL can hop 301 → 301 → 302 → 200. Google discounts link equity across long chains and burns crawl budget walking them. You can’t fix what you can’t see, so you need the ordered hops.
• Redirect loops exist. A → B → A is a real failure mode after a botched migration. A checker needs a hop cap (configurable, up to 10 here) so a loop fails cleanly instead of hanging.
• Status changes mid-chain. A 200 final code can hide a 302 (temporary) hop that should have been a 301 (permanent). For SEO that distinction is the whole game.
• Network failures aren’t HTTP failures. DNS resolution failures, connection refusals and timeouts have no status code. A useful checker records them as explicit, human-readable outcomes rather than silently dropping the row.

How to run it at scale without getting blocked

The bottleneck on a large list isn’t CPU — it’s politeness and rate limits. The actor issues parallel HTTP GET requests with redirect following, and the knobs that matter are:

1. Concurrency — configurable up to 100 parallel checks. High concurrency is great against a diverse list of hosts; it’s a fast way to get rate-limited if every URL points at the same host.
2. Retries — transient network failures (a flaky timeout, a momentary 503) are retried up to twice before being recorded as failures, so you don’t get false negatives from a single hiccup.
3. Proxy support — routing through a proxy pool spreads requests across IPs and avoids per-IP rate limiting when you’re auditing a single large domain.

The trade-off: GET (not HEAD) is used because many servers handle HEAD inconsistently — they’ll 405 a HEAD request or return a different status than the GET would. GET costs more bandwidth but gives you the status the real crawler and real users would see.

▶ Run the Bulk URL Status Checker — feed it a list (or a whole sitemap’s worth of URLs), get back status codes, full redirect chains, final URLs and response times. Up to 100 parallel checks with retries and proxy support. Priced per URL checked.

Build it yourself vs. use a managed checker

A weekend script with aiohttp and a semaphore gets you 80% of the way for a one-off check. The gap shows up at scale:

• Building from scratch — you’ll reimplement redirect-chain capture, hop limits, loop detection, retry logic, network-error classification, and proxy rotation. Each is a small thing; together they’re a day or two, and then you maintain it.
• Using a managed actor — paste or upload your URL list, set concurrency, and export clean CSV/JSON. The redirect-chain and error-classification logic is already solved.

For a recurring audit — say a nightly check of every internal link, or a post-migration verification across tens of thousands of old URLs — the managed path saves the reimplementation tax and gives you a stable output schema you can diff run-over-run.

Schema design for downstream use

A clean per-URL record:

{
  "url": "http://example.com/old-page",
  "status_code": 200,
  "status_message": "OK",
  "ok": true,
  "final_url": "https://example.com/new-page/",
  "redirect_count": 2,
  "redirect_chain": [
    { "url": "http://example.com/old-page", "status": 301 },
    { "url": "https://example.com/old-page", "status": 301 },
    { "url": "https://example.com/new-page/", "status": 200 }
  ],
  "response_time_ms": 412,
  "checked_at": "2026-05-21T10:00:00Z"
}

Schema choices worth making:

• Keep redirect_chain as an ordered array. It’s the field that lets you find and collapse multi-hop chains. Flatten it and you lose the audit.
• Store both status_code and status_message. Network failures need the message; HTTP responses need the code.
• Record response_time_ms even on success — slow 200s are a real finding in a performance audit.
• Diff final_url against the input url to instantly flag which URLs moved.

Typical use cases

• SEO audits — find 4xx/5xx errors and long redirect chains that hurt crawlability and waste crawl budget.
• Post-migration verification — confirm every old URL 301s correctly to its new home, with no loops or 404s slipping through.
• CI/CD link health — run the checker as a pipeline step so a deploy that breaks internal links fails the build.
• Content link monitoring — scan internal and external links across a large article archive and catch link rot before users do.
• Backlink preservation — confirm inbound links still resolve and preserve link equity rather than landing on a 404.
• Redirect optimization — surface multi-hop chains so you can collapse them to a single 301.
• Endpoint and webhook validation — confirm APIs and webhooks return the expected 2xx.
• Dataset enrichment — annotate any URL dataset with live status, final URL and timing.

Cost math

Pricing is per dataset item — one row per URL checked. A 50,000-URL audit is 50,000 items; a nightly 5,000-URL internal-link check is 5,000 items per run. There’s no proxy bill to manage and no per-host infrastructure to babysit. For most SEO and migration work, a full audit costs a few dollars, and a scheduled monitoring job lands in the low single digits per month.

Self-hosting the same thing means a VPS, a proxy pool if you’re auditing single large domains, and the engineering time to build and maintain the redirect-chain and retry logic. For a recurring job, the managed checker is cheaper once your time is in the equation.

Common pitfalls

• Using HEAD instead of GET. Many servers lie on HEAD. Use GET to see what crawlers and users actually get.
• Cranking concurrency against one host. 100 parallel checks against a single domain is a self-inflicted rate-limit. High concurrency is for diverse host lists; throttle (and proxy) for single-domain audits.
• Ignoring the chain, keeping only the final code. A clean 200 can hide a temporary-vs-permanent redirect bug. Always keep the hops.
• Treating timeouts as 404s. A timeout is a network condition, not a missing page. Classify them separately or you’ll chase phantom broken links.
• No hop cap. Without a redirect limit, a loop hangs the check. Cap the hops (up to 10) so loops fail cleanly and get flagged.

Wrapping up

Bulk status checking looks trivial until you need redirect chains, network-error classification, retries and rate-limit-safe concurrency across tens of thousands of URLs. For a one-off, a script is fine. For repeatable SEO audits, migration verification and link monitoring, a managed checker gives you the full chain, the timings and a stable schema without the reimplementation tax.

▶ Open the Bulk URL Status Checker on Apify — status codes, redirect chains, final URLs and response times for thousands of links. Export to CSV/JSON. Priced per URL. Start on Apify’s free monthly credit.