# Bulk URL Checker — Status, Redirects & SEO Tags (`haketa/bulk-url-checker`) Actor

Check thousands of URLs at once: HTTP status code, redirects, final URL, response time, content type and on-page SEO tags (title, meta description, H1, canonical, Open Graph, noindex). Fast, cheap bulk link auditing for SEO, QA and migrations.

- **URL**: https://apify.com/haketa/bulk-url-checker.md
- **Developed by:** [Haketa](https://apify.com/haketa) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Features:** Limited permissions
- **Stats:** 10 total users, 3 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.70 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## 🔗 Bulk URL Checker — HTTP Status, Redirects, Response Time & SEO Tags at Scale

**Check thousands — or millions — of URLs at once.** This **bulk URL checker** takes a pasted list, an uploaded file, seed domains, or a remote text/CSV/XML sitemap of links and returns, for every URL: **HTTP status code, redirect status, final destination URL, response time (ms), HTTPS check, Content-Type, response size (`Content-Length`), server header**, plus on-page **SEO tags** — `<title>`, meta description, `<h1>`, canonical URL, Open Graph (`og:title`, `og:image`), meta robots, `isNoindex` flag and body word count.

Export every checked URL to **JSON, CSV, Excel, or API** — built for **1,000+ URL bulk audits** with high concurrency and zero duplicates.

[![Apify Actor](https://img.shields.io/badge/Apify-Actor-00b04f?style=flat-square\&logo=apify)](https://apify.com/haketa/bulk-url-checker)
[![Scale](https://img.shields.io/badge/Scale-1%2C000%2B%20to%20Millions%20of%20URLs-blue?style=flat-square)](https://apify.com/haketa/bulk-url-checker)
[![Concurrency](https://img.shields.io/badge/Concurrency-Up%20to%20500%20Parallel-orange?style=flat-square)](https://apify.com/haketa/bulk-url-checker)
[![SEO Tags](https://img.shields.io/badge/SEO%20Audit-Title%20%7C%20Meta%20%7C%20H1%20%7C%20Canonical%20%7C%20Noindex-purple?style=flat-square)](https://apify.com/haketa/bulk-url-checker)
[![Export](https://img.shields.io/badge/Export-JSON%20%7C%20CSV%20%7C%20Excel%20%7C%20API-2b7489?style=flat-square)](https://apify.com/haketa/bulk-url-checker)
[![No Auth](https://img.shields.io/badge/Login-None%20Required-success?style=flat-square)](https://apify.com/haketa/bulk-url-checker)

> Built for **technical SEO specialists, link builders, QA & site-migration engineers, webmasters, B2B lead-list cleaners and data engineering teams** who need fast, reliable bulk URL validation and on-page metadata extraction without headless-browser overhead.

***

### ⚡ At a glance

| Feature | Details |
|---|---|
| **Input sources** | Pasted URLs, bare domains (`example.com`), uploaded text/CSV files, or remote XML sitemaps (`requestsFromUrl`) |
| **HTTP & transport metrics** | `statusCode` (`200`, `301`, `404`, `500`…), `ok` (`true`/`false`), `finalUrl`, `redirected`, `responseTimeMs`, `isHttps`, `contentType`, `contentLength`, `server`, `error` |
| **On-page SEO signals** | `title`, `metaDescription`, `h1`, `canonical`, `ogTitle`, `ogImage`, `metaRobots`, `isNoindex`, `wordCount` (toggleable via `checkSeo`) |
| **Content types supported** | HTML pages, PDFs, images, JSON/XML feeds and binary CDN assets (`*/*`) |
| **Speed & concurrency** | Pure HTTP with configurable concurrency (`1`–`500`, default `50`) — checks 100 URLs in seconds and 1,000+ URLs in minutes |
| **Scale & link discovery** | Checks your exact list (unlimited when `maxItems: 0`), or automatically discovers page links when `maxItems` exceeds your seed URLs |
| **Deduplication & storage** | Strictly deduplicated by normalized URL; writes dataset items plus Key-Value Store `OUTPUT` and `SUMMARY` records |
| **Export formats** | Bulk export to JSON, CSV, Excel, XML, JSONL, Webhook & Apify REST API |

***

### 🔎 Why use a bulk URL checker?

Checking a single URL in a browser tab takes two seconds. Checking **an entire XML sitemap, a 10,000-row backlink export, a product catalog of 50,000 landing pages, or a B2B lead list of company domains** requires parallel HTTP execution, redirect resolution, timeout handling, SSL tolerance, and structured output. Spreadsheet formulas (`IMPORTXML`) and single-threaded scripts choke on slow servers, drop failed requests, or get stuck on non-HTML files.

This **bulk URL checker and SEO tag extractor** solves that at cloud scale: point it at your URLs (or a remote sitemap / seed domain) and get **one clean, deduplicated row per URL** showing its exact HTTP status, where it redirected, how fast it responded, its MIME type and byte size, and its core on-page SEO tags. Because it runs on high-concurrency plain HTTP rather than a heavy browser, large runs finish rapidly at a fraction of a cent per thousand URLs.

**Common reasons teams use this bulk URL checker:**

- 🔍 **Technical SEO & sitemap audits** — extract `title`, `metaDescription`, `h1`, `canonical` and `isNoindex` across thousands of URLs to catch missing, duplicate or de-indexed pages.
- 🚨 **Broken link & 404 detection** — filter by `ok: "false"` to isolate every `404`, `410`, `500`, `502`, DNS failure and timeout across internal links or external backlinks.
- 🔀 **Site migration & redirect verification** — audit `redirected` and `finalUrl` before and after a domain or CMS migration to ensure old URLs resolve to the right 200 destinations.
- 🧹 **B2B lead & domain list cleaning** — validate thousands of company websites before sales outreach or deeper scraping, stripping out dead domains and resolving parked redirects.
- 🖼️ **CDN, image & PDF asset verification** — confirm non-HTML assets (`image/webp`, `application/pdf`, `application/xml`) return `200 OK` and expected `contentLength`.
- ⏱️ **Uptime & latency monitoring** — schedule recurring runs to track `statusCode` and `responseTimeMs` across critical landing pages and API endpoints.

***

### ✨ What this bulk URL checker does

- ✅ **Accepts any URL list size** — paste URLs directly, upload a file, or link a remote **text, CSV or XML sitemap** (`requestsFromUrl`) containing thousands to millions of URLs.
- ✅ **Smart seed-to-1,000+ link expansion** — if you pass a few seed URLs (e.g. a homepage) and set `maxItems: 50` or `maxItems: 1000`, the Actor checks your seed URLs and automatically follows discovered links up to your exact `maxItems` target. When `maxItems: 0` (the default), it checks only the exact URLs you supplied.
- ✅ **100-URL built-in demo on empty runs** — running with empty input (`{}`) checks a curated 100-URL sample of major documentation, news and reference sites in under 30 seconds.
- ✅ **High-speed parallel execution** — configurable `maxConcurrency` up to `500` parallel connections.
- ✅ **Complete HTTP & redirect diagnostics** — reports `statusCode`, `ok`, `finalUrl`, `redirected`, `responseTimeMs`, `isHttps`, `contentType`, `contentLength`, `server` and `error`.
- ✅ **Works on HTML and non-HTML URLs alike** — PDFs, images, JSON endpoints and feeds are verified with their real HTTP status and headers instead of being rejected.
- ✅ **On-page SEO extraction (`checkSeo: true`)** — parses `<title>`, `<meta name="description">`, first `<h1>`, `<link rel="canonical">`, `og:title`, `og:image`, `<meta name="robots">`, `isNoindex` and body `wordCount`.
- ✅ **Zero silent drops & zero duplicates** — unreachable URLs are logged with `ok: "false"` and a clear `error` message, and every URL is deduplicated by normalized href.
- ✅ **Key-Value Store `OUTPUT` & `SUMMARY` mirror** — in addition to streaming items to the default Dataset, the Actor writes `OUTPUT` and `SUMMARY` (`checked`, `okCount`, `redirectedCount`, `errorCount`) to the Key-Value Store for instant API retrieval.

***

### 🎯 Use cases

| Persona | How they use this Actor |
|---|---|
| **📈 Technical SEO Auditors** | Point `requestsFromUrl` at a client's `sitemap.xml` to audit 1,000+ pages for missing titles/H1s, mismatched canonicals, `noindex` leaks and thin `wordCount`. |
| **🚀 Site Migration & QA Teams** | Upload legacy URL lists during a redesign or domain move to verify every URL returns `200` at the expected `finalUrl`. |
| **🧲 B2B Lead Gen & RevOps Teams** | Clean 10,000+ prospect domains from CRM exports (`checkSeo: false`, `maxConcurrency: 200`) to filter out dead websites and capture live `finalUrl` domains. |
| **🔗 Link Builders & Digital PR** | Verify backlink portfolios and guest-post placements in bulk to confirm pages are live (`statusCode: "200"`) and indexable (`isNoindex: "false"`). |
| **🛒 E-Commerce & Catalog Managers** | Validate thousands of product page URLs and CDN image links (`contentType`, `contentLength`) to catch broken product media before ad campaigns run. |

***

### 📥 Input configuration

All fields are optional. Leave `urls` empty for a 100-URL demo run, or use the prefill (`maxItems: 50`) for a quick 50-URL test run.

| Field | Type | Default | Description |
|---|---|---|---|
| `urls` | array | Prefill 3 seed URLs | URLs to check. Paste URLs/domains, upload a file, or link a remote text/CSV/XML sitemap via `requestsFromUrl`. Bare domains (`example.com`) are auto-prefixed with `https://`. |
| `checkSeo` | boolean | `true` | Extract on-page SEO tags (`title`, `metaDescription`, `h1`, `canonical`, `ogTitle`, `ogImage`, `metaRobots`, `isNoindex`, `wordCount`) on HTML responses. Turn `false` for pure status/redirect speed. |
| `maxItems` | integer | `0` (prefill `50`) | Maximum URLs to check. `0` = no limit (checks every URL provided, or 100 demo URLs on an empty run). When set higher than your seed URLs, discovered page links are checked up to `maxItems`. |
| `maxConcurrency` | integer | `50` | Parallel HTTP requests (`1`–`500`). Higher is faster; lower is gentler on a single origin server. |
| `timeoutSecs` | integer | `30` | Per-URL request timeout in seconds (`3`–`120`). |
| `proxyConfiguration` | object | `{ "useApifyProxy": false }` | Optional Apify Proxy. Off by default for lowest cost; enable for geo-specific checks or single-host rate-limit protection. |

#### Input example — check an exact list of URLs & bare domains

```json
{
  "urls": [
    { "url": "https://apify.com" },
    { "url": "https://docs.apify.com/platform" },
    { "url": "github.com/apify/crawlee" },
    { "url": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Status" }
  ],
  "checkSeo": true,
  "maxItems": 0,
  "maxConcurrency": 50,
  "timeoutSecs": 30
}
```

#### Input example — audit 1,000+ URLs from a remote XML sitemap

```json
{
  "urls": [
    { "requestsFromUrl": "https://blog.apify.com/sitemap-posts.xml" }
  ],
  "checkSeo": true,
  "maxItems": 1000,
  "maxConcurrency": 50,
  "timeoutSecs": 30
}
```

#### Input example — ultra-fast status-only domain validation

```json
{
  "urls": [
    { "url": "https://example.com" },
    { "url": "https://httpbin.org/status/404" }
  ],
  "checkSeo": false,
  "maxItems": 0,
  "maxConcurrency": 200,
  "timeoutSecs": 15
}
```

***

### 📤 Output & field reference

Every checked URL produces one flat record in the Dataset (and in the Key-Value Store `OUTPUT` array). Below is a real output item:

```json
{
  "url": "https://blog.apify.com/shopify-competitor-analysis/",
  "statusCode": "200",
  "ok": "true",
  "finalUrl": "https://blog.apify.com/shopify-competitor-analysis/",
  "redirected": "false",
  "responseTimeMs": "372",
  "isHttps": "true",
  "contentType": "text/html",
  "contentLength": "36692",
  "server": "openresty",
  "title": "Shopify competitor analysis with Apify and Claude",
  "metaDescription": "Will your margins survive a market's import duties? What do local buyers complain about? Build an analysis report that answers these questions.",
  "h1": "Shopify competitor analysis: how to validate a new market with Apify and Claude",
  "canonical": "https://blog.apify.com/shopify-competitor-analysis/",
  "ogTitle": "Thinking of selling your Shopify products abroad?",
  "ogImage": "https://storage.ghost.io/c/.../Shopify-competitor-analysis.png",
  "metaRobots": "index,follow",
  "isNoindex": "false",
  "wordCount": "4577",
  "error": "",
  "scrapedAt": "2026-11-10T02:03:01.520Z"
}
```

#### 🌐 HTTP Status, Redirect & Performance Fields

| Field | Type | Coverage | Description |
|---|---|---|---|
| `url` | string | 🟢 High (100%) | The input URL checked (normalized with `https://` when bare). |
| `statusCode` | string | 🟢 High (~99%) | HTTP response status code (`"200"`, `"301"`, `"404"`, `"500"`…; empty only on network/DNS failure). |
| `ok` | string | 🟢 High (100%) | `"true"` when HTTP status is `200`–`399`, `"false"` on `4xx`/`5xx` or network error. |
| `finalUrl` | string | 🟢 High (~99%) | Final destination URL after following HTTP redirects. |
| `redirected` | string | 🟢 High (~99%) | `"true"` if `finalUrl` differs from `url` (ignoring trailing slash), otherwise `"false"`. |
| `responseTimeMs` | string | 🟢 High (~99%) | Total HTTP response time in milliseconds. |
| `isHttps` | string | 🟢 High (~99%) | `"true"` when `finalUrl` uses HTTPS, `"false"` for HTTP. |
| `contentType` | string | 🟢 High (~99%) | Primary MIME type from `Content-Type` header (`text/html`, `application/pdf`, `image/png`…). |
| `contentLength` | string | 🟡 Conditional | Response body size in bytes when the server sends a `Content-Length` header. |
| `server` | string | 🟡 Conditional | Web server / CDN identifier from the `Server` header (`cloudflare`, `nginx`, `openresty`…). |
| `error` | string | 🟢 High (100%) | Empty string `""` on reachable responses; contains the failure reason on DNS/timeout/reset errors. |
| `scrapedAt` | string | 🟢 High (100%) | ISO 8601 timestamp when the URL was checked. |

#### 🔎 On-Page SEO & Content Fields (`checkSeo: true`)

| Field | Type | Coverage | Description |
|---|---|---|---|
| `title` | string | 🟢 High (~99% on HTML) | Text of the first `<title>` element. |
| `metaDescription` | string | 🟢 High (~85% on HTML) | Content of `<meta name="description">`. |
| `h1` | string | 🟢 High (~92% on HTML) | Text of the first `<h1>` heading on the page. |
| `canonical` | string | 🟢 High (~88% on HTML) | `href` of `<link rel="canonical">`. |
| `ogTitle` / `ogImage` | string | 🟢 High (~85% on HTML) | Open Graph `og:title` and `og:image` meta tag values. |
| `metaRobots` | string | 🟡 Conditional | Content of `<meta name="robots">` when declared on the page. |
| `isNoindex` | string | 🟢 High (100% on HTML) | `"true"` if `metaRobots` contains `noindex`, otherwise `"false"`. |
| `wordCount` | string | 🟢 High (100% on HTML) | Total visible word count inside `<body>`. |

***

### 🔁 Pagination, sitemaps & 1,000+ scaling

- **Exact bulk list mode (`maxItems: 0`)**: Pass 100, 1,000 or 100,000 URLs (via pasted list, uploaded file, or `requestsFromUrl` pointing to a remote CSV/txt/XML sitemap) and the Actor checks every single URL with zero artificial caps.
- **Seed expansion mode (`maxItems > seed URLs`)**: When `maxItems` is set higher than the number of direct URLs you provided (for example, 3 prefill URLs with `maxItems: 50`, or 1 homepage URL with `maxItems: 1000`), the Actor automatically discovers and checks `<a href>` links from the visited pages until `maxItems` unique URLs are audited.
- **Empty-run demo (`100` URLs)**: Running with `{}` checks 100 built-in reference URLs at high concurrency in ~15–25 seconds.

***

### ⚙️ How to use

1. Open **Bulk URL Checker** in the Apify Console and click **Try for free**.
2. Paste your **URLs**, upload a text/CSV file, or add a remote XML sitemap URL via **`requestsFromUrl`**.
3. Keep **`checkSeo: true`** for full SEO tag extraction (or set `false` for pure HTTP status speed), and adjust **`maxItems`** and **`maxConcurrency`**.
4. Click **Start**, then export your results as **JSON, CSV, Excel**, or read the Dataset / Key-Value Store `OUTPUT` via the **Apify API**.

***

### 🔌 API usage

#### Run asynchronously via REST API

```bash
curl -X POST "https://api.apify.com/v2/acts/haketa~bulk-url-checker/runs?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [
      { "requestsFromUrl": "https://blog.apify.com/sitemap-posts.xml" }
    ],
    "checkSeo": true,
    "maxItems": 500,
    "maxConcurrency": 50
  }'
```

#### Run synchronously and get checked URLs immediately

```bash
curl -X POST "https://api.apify.com/v2/acts/haketa~bulk-url-checker/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [
      { "url": "https://apify.com" },
      { "url": "https://crawlee.dev" },
      { "url": "https://github.com/apify" }
    ],
    "checkSeo": true,
    "maxItems": 3
  }'
```

***

#### Call from Python (`apify-client`)

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run_input = {
    "urls": [
        {"url": "https://apify.com"},
        {"url": "https://docs.apify.com"},
        {"url": "https://crawlee.dev"},
    ],
    "checkSeo": True,
    "maxItems": 100,
    "maxConcurrency": 50,
    "timeoutSecs": 30,
}

run = client.actor("haketa/bulk-url-checker").call(run_input=run_input)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"[{item['statusCode']}] {item['url']} -> {item['finalUrl']} ({item['responseTimeMs']} ms) | Title: {item['title']}")
```

#### Call from Node.js (`apify-client`)

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('haketa/bulk-url-checker').call({
    urls: [
        { requestsFromUrl: 'https://blog.apify.com/sitemap-posts.xml' },
    ],
    checkSeo: true,
    maxItems: 500,
    maxConcurrency: 50,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Audited ${items.length} URLs`);
```

***

### 🔄 Integrations & automated SEO / QA workflows

Connect **Bulk URL Checker** to your SEO monitoring, DevOps or RevOps pipelines:

| Integration | Workflow example |
|---|---|
| **🚨 Slack, PagerDuty & Email Alerts** | Schedule hourly or daily runs against critical landing pages and trigger an instant alert via Make/Zapier/n8n whenever any URL returns `ok: "false"` or `isNoindex: "true"`. |
| **📊 Google Sheets & Looker Studio** | Push weekly sitemap audit results (`statusCode`, `redirected`, `responseTimeMs`, `title`, `metaDescription`, `h1`, `canonical`, `wordCount`) to a live technical SEO dashboard. |
| **🧹 Clay, HubSpot & Salesforce** | Pre-validate thousands of prospect company domains before running expensive enrichment workflows — drop dead domains (`ok: "false"`) and keep clean `finalUrl` domains. |
| **🗄️ BigQuery, Snowflake & S3** | Archive historical crawl snapshots across 10,000+ product or editorial URLs to track response latency, title/meta changes and redirect hygiene over time. |
| **🤖 AI SEO & QA Agents** | Invoke the Actor via Apify MCP or synchronous REST API so autonomous SEO and QA agents can verify live URLs and inspect on-page metadata in real time. |

***

### ⚠️ Limitations & technical notes

- **Server-rendered HTML (no JavaScript execution)**: For maximum speed and low cost at 1,000+ to million-URL scale, the Actor uses plain HTTP requests (`CheerioCrawler`) rather than a headless browser. It reads the initial server-returned HTML, which matches what lightweight crawlers and link validators see, but tags injected exclusively by client-side JavaScript after page load will not appear.
- **Origin rate-limiting on single-domain runs**: Checking 1,000+ URLs on a single small website at `maxConcurrency: 200` can trigger `429 Too Many Requests` or `503` responses from the origin firewall. Lower `maxConcurrency` to `10`–`30` (and optionally enable `proxyConfiguration`) when auditing a single domain, and save high concurrency (`100`–`500`) for multi-domain lists.
- **Header-dependent fields**: `contentLength` and `server` are populated whenever the target server sends `Content-Length` and `Server` HTTP response headers (chunked responses often omit `Content-Length`).

***

### 💡 Pro tips for bulk auditing

- **Use `requestsFromUrl` for XML sitemaps and large CSVs** — paste any `https://.../sitemap.xml` URL into the URLs source editor (`From URL`) and Crawlee will automatically extract and check every `<loc>` entry.
- **Turn off `checkSeo` (`false`) when cleaning 10,000+ sales lead domains** — skipping HTML DOM parsing maximizes throughput and reduces memory usage.
- **Lower `maxConcurrency` (e.g. `10`–`25`) when auditing a single small server** so you don't trigger origin rate-limiting; raise it to `100`–`300` when checking thousands of distinct domains.
- **Read `SUMMARY` from Key-Value Store** for instant health counts (`checked`, `okCount`, `redirectedCount`, `errorCount`).

***

### ❓ FAQ

**How many URLs can I check in one run?**
From a few URLs up to **millions**. Set `maxItems: 0` to check every URL in your input list or sitemap without limit, or set `maxItems: 1000+` on seed URLs to automatically discover and check internal/outbound links up to your target.

**What happens when a URL returns 404, 500 or times out?**
It is **always included** in the output dataset with `ok: "false"`, its HTTP `statusCode` (if received), and a clean `error` message — never silently dropped.

**Does it work on PDFs, images and XML feeds?**
Yes. All MIME types (`*/*`) are accepted and reported with their real `statusCode`, `contentType`, `contentLength` and `responseTimeMs`. SEO tag fields are parsed only on HTML responses.

**Do I need proxies?**
No for most runs — direct HTTP requests are used by default so runs stay fast and cheap. Enable Apify Proxy only if you are auditing a single rate-limited host at high volume or testing geo-specific redirects.

**How does seed link discovery work when `maxItems` is set higher than my input URLs?**
When you provide a small seed list (for example, 1–3 homepage URLs) and set `maxItems` to a larger number (such as `50` or `1,000`), the Actor checks your initial URLs first and automatically extracts, normalizes and enqueues `<a href>` links found on those pages until `maxItems` unique URLs have been checked. When `maxItems: 0` (the default), link discovery is disabled and the Actor checks **only** the exact URLs in your input list or sitemap.

**Where can I find a quick summary of healthy vs. broken URLs after a run?**
In addition to the full Dataset, every run saves a `SUMMARY` JSON object (`checked`, `okCount`, `redirectedCount`, `errorCount`, `finishedAt`) and an `OUTPUT` array to the run's default **Key-Value Store**, which you can fetch in a single API call.

**Are bare domains without `https://` supported?**
Yes. Entries like `apify.com` or `github.com/apify/crawlee` are automatically normalized to `https://apify.com` and `https://github.com/apify/crawlee` before checking, and any `http:// -> https://` or `non-www -> www` redirect is captured in `redirected` and `finalUrl`.

#### Sample bulk URL auditing playbooks

| Playbook | Recommended input configuration |
|---|---|
| **🗺️ Full XML Sitemap Technical SEO Audit** | `urls`: `[{ "requestsFromUrl": "https://yoursite.com/sitemap.xml" }]`, `checkSeo`: `true`, `maxItems`: `0`, `maxConcurrency`: `40` |
| **🧹 10,000+ B2B Prospect Domain Health Check** | `urls`: uploaded CSV/TXT of domains, `checkSeo`: `false`, `maxItems`: `0`, `maxConcurrency`: `200`, `timeoutSecs`: `15` |
| **🔀 Domain Migration & Redirect Chain Verification** | `urls`: legacy URLs list, `checkSeo`: `true`, `maxItems`: `0`, `maxConcurrency`: `50` — filter output on `redirected` and `finalUrl` |
| **🕷️ Seed-URL 1,000+ Link Discovery & Audit** | `urls`: `[{ "url": "https://yoursite.com" }]`, `checkSeo`: `true`, `maxItems`: `1000`, `maxConcurrency`: `50` |

***

### 🔗 Related scrapers

- [Bulk Metadata Extractor — Titles, Meta & Open Graph](https://apify.com/haketa/bulk-metadata-extractor)
- [OutSEO Scraper — SEO Keyword & Competitor Intelligence](https://apify.com/haketa/outseo-scraper)
- [Google Trends Scraper — Search Interest & Trending Topics](https://apify.com/haketa/google-trends-scraper)
- [Google Play Scraper — Android App Store Metadata & Reviews](https://apify.com/haketa/google-play-scraper)
- [App Store Scraper — iOS App Metadata & Ratings](https://apify.com/haketa/app-store-scraper)

***

### 📋 Changelog

- **v1.2 (2026-11)** — Expanded built-in empty-run demo to **100 diverse URLs**, added automatic on-page link discovery when `maxItems` exceeds seed URLs (supporting 50-item prefill and 1,000+ URL expansion), added strict URL normalization/deduplication (`0 duplicates`), and added Key-Value Store `OUTPUT` and `SUMMARY` records.
- **v1.1 (2026-11)** — Added `additionalMimeTypes: ['*/*']` so non-HTML assets (PDFs, images, XML feeds) report real HTTP status and `contentLength`, and verified 504-URL sitemap benchmark.
- **v1.0 (2026-10)** — Initial release with high-concurrency HTTP status checking, redirect resolution, timing metrics and on-page SEO tag extraction.

# Changelog

This Actor's version history is a separate document: https://apify.com/haketa/bulk-url-checker/changelog.md

# Actor input Schema

## `urls` (type: `array`):

The URLs you want to check. Paste a list, upload a file, or link to a remote text/CSV of URLs — handles thousands to millions. Leave empty to run a quick demo on a sample of popular sites.

## `checkSeo` (type: `boolean`):

Also extract on-page SEO tags (title, meta description, H1, canonical, Open Graph, robots). Turn off for a pure status/redirect check (faster).

## `maxItems` (type: `integer`):

Maximum number of URLs to check. Set to 0 for no limit (checks every URL you provided, or a 100-URL demo sample when run empty). When set higher than your seed URLs, discovered page links are also checked up to this limit.

## `maxConcurrency` (type: `integer`):

How many URLs to check in parallel. Higher = faster, but be considerate of target servers.

## `timeoutSecs` (type: `integer`):

Give up on a URL after this many seconds.

## `proxyConfiguration` (type: `object`):

Apify Proxy. Off by default — enable it for large runs or geo-specific checks.

## Actor input object example

```json
{
  "urls": [
    {
      "url": "https://apify.com"
    },
    {
      "url": "https://github.com"
    },
    {
      "url": "https://example.com"
    }
  ],
  "checkSeo": true,
  "maxItems": 50,
  "maxConcurrency": 50,
  "timeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `url` (type: `string`):

The URL checked

## `statusCode` (type: `string`):

HTTP status code

## `ok` (type: `string`):

Whether the URL is healthy (2xx/3xx)

## `finalUrl` (type: `string`):

URL after redirects

## `redirected` (type: `string`):

Whether it redirected

## `responseTimeMs` (type: `string`):

Response time in ms

## `isHttps` (type: `string`):

Whether final URL is HTTPS

## `contentType` (type: `string`):

Content type

## `contentLength` (type: `string`):

Response size in bytes from the Content-Length header, when present

## `server` (type: `string`):

Server header

## `title` (type: `string`):

Page title

## `metaDescription` (type: `string`):

Meta description

## `h1` (type: `string`):

First H1

## `canonical` (type: `string`):

Canonical URL

## `ogTitle` (type: `string`):

Open Graph title

## `ogImage` (type: `string`):

Open Graph image

## `metaRobots` (type: `string`):

Meta robots

## `isNoindex` (type: `string`):

Whether noindex is set

## `wordCount` (type: `string`):

Body word count

## `error` (type: `string`):

Error, if the check failed

## `scrapedAt` (type: `string`):

Timestamp

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        {
            "url": "https://apify.com"
        },
        {
            "url": "https://github.com"
        },
        {
            "url": "https://example.com"
        }
    ],
    "maxItems": 50,
    "maxConcurrency": 50
};

// Run the Actor and wait for it to finish
const run = await client.actor("haketa/bulk-url-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": [
        { "url": "https://apify.com" },
        { "url": "https://github.com" },
        { "url": "https://example.com" },
    ],
    "maxItems": 50,
    "maxConcurrency": 50,
}

# Run the Actor and wait for it to finish
run = client.actor("haketa/bulk-url-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    {
      "url": "https://apify.com"
    },
    {
      "url": "https://github.com"
    },
    {
      "url": "https://example.com"
    }
  ],
  "maxItems": 50,
  "maxConcurrency": 50
}' |
apify call haketa/bulk-url-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,haketa/bulk-url-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/WMPyPQfuky6JrmyND/builds/jQ1IcuCVJ7iV7w074/openapi.json