# Wayback Machine Scraper (`glassventures/wayback-machine-scraper`) Actor

Scrape Wayback Machine archive snapshots for any URL or domain. Get archived URLs, timestamps, status codes, MIME types. Export to JSON, CSV, Excel.

- **URL**: https://apify.com/glassventures/wayback-machine-scraper.md
- **Developed by:** [Glass Ventures](https://apify.com/glassventures) (community)
- **Categories:** Developer tools, SEO tools, Automation
- **Features:** Limited permissions
- **Stats:** 46 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Wayback Machine Scraper

Scrape archived snapshots from the Wayback Machine (Archive.org) for any URL or domain. Extract archive URLs, timestamps, HTTP status codes, MIME types, and content sizes.

### What does Wayback Machine Scraper do?

Wayback Machine Scraper uses the official Wayback Machine CDX API to retrieve historical snapshots of any website. It lets you discover every archived version of a page, filter by date range, content type, and HTTP status code.

Whether you need to track how a website changed over time, recover lost content, monitor competitor website changes, or build a historical dataset of web pages, this actor makes it easy. It handles pagination, rate limiting, and exports data in JSON, CSV, or Excel format.

The Wayback Machine (Archive.org) has archived over 800 billion web pages since 1996. This actor gives you structured access to that massive archive without writing any code.

### Use Cases

- **SEO specialists** -- Track historical changes to competitor pages, find old URLs for redirect mapping, discover deleted content
- **Researchers** -- Build datasets of how websites evolved over time, study web history trends
- **Content recovery** -- Find and recover deleted or changed web pages from the archive
- **Compliance teams** -- Document historical versions of terms of service, privacy policies, or regulatory pages
- **Developers** -- Programmatically access Wayback Machine data via API for integration into tools and pipelines

### Features

- Search by exact URL or entire domain (wildcard matching)
- Filter snapshots by date range (from/to)
- Filter by MIME type (HTML, JSON, CSS, JavaScript, images)
- Filter by HTTP status code (200, 301, 404, etc.)
- Bulk processing of multiple URLs and domains
- Proxy support with automatic rotation
- Handles rate limiting and large datasets automatically
- Exports to JSON, CSV, Excel, or connect via API

### How much will it cost?

The Wayback Machine CDX API is free and public. The only cost is Apify platform compute time.

| Results | Estimated Cost |
|---------|---------------|
| 1,000   | ~$0.01        |
| 10,000  | ~$0.05        |
| 100,000 | ~$0.25        |

| Cost Component | Per 10,000 Results |
|----------------|-------------------|
| Platform compute | ~$0.05 |
| Proxy (optional) | ~$0.00 |
| **Total** | **~$0.05** |

### How to use

1. Go to the Wayback Machine Scraper page on Apify Store
2. Click "Start" or "Try for free"
3. Enter URLs to look up in the archive, or domain names for full-domain search
4. Optionally set date range filters, MIME type, and status code filters
5. Set the maximum number of items
6. Click "Start" and wait for the results

### Input parameters

| Parameter | Type | Description | Default |
|-----------|------|-------------|---------|
| startUrls | array | Website URLs to look up in the Wayback Machine | - |
| domains | array | Domain names for full-domain archive search | - |
| dateFrom | string | Only include snapshots after this date | - |
| dateTo | string | Only include snapshots before this date | - |
| mimeTypeFilter | string | Filter by content type (text/html, application/json, all) | all |
| statusCodeFilter | string | Filter by HTTP status code (e.g., "200") | - |
| maxItems | number | Maximum snapshot records to return | 1000 |
| proxyConfig | object | Proxy settings (optional) | - |

### Output

The actor produces a dataset with the following fields:

```json
{
    "originalUrl": "https://www.example.com",
    "archiveUrl": "https://web.archive.org/web/20230115120000/https://www.example.com",
    "timestamp": "20230115120000",
    "statusCode": "200",
    "mimeType": "text/html",
    "length": "1256",
    "archivedDate": "2023-01-15T12:00:00.000Z",
    "scrapedAt": "2026-04-23T10:30:00.000Z"
}
```

| Field | Type | Description |
|-------|------|-------------|
| originalUrl | string | The original URL that was archived |
| archiveUrl | string | Full Wayback Machine URL to view the snapshot |
| timestamp | string | Raw Wayback Machine timestamp (YYYYMMDDHHmmss) |
| statusCode | string | HTTP status code of the archived response |
| mimeType | string | Content type of the archived resource |
| length | string | Size of the archived resource in bytes |
| archivedDate | string | ISO 8601 date when the snapshot was taken |
| scrapedAt | string | ISO 8601 timestamp when data was extracted |

### Integrations

Connect Wayback Machine Scraper with other tools:

- **Apify API** -- REST API for programmatic access
- **Webhooks** -- get notified when a run finishes
- **Zapier / Make** -- connect to 5,000+ apps
- **Google Sheets** -- export directly to spreadsheets

#### API Example (Node.js)

```javascript
import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('YOUR_USERNAME/wayback-machine-scraper').call({
    startUrls: [{ url: 'https://www.example.com' }],
    maxItems: 100,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
```

#### API Example (Python)

```python
from apify_client import ApifyClient
client = ApifyClient('YOUR_TOKEN')
run = client.actor('YOUR_USERNAME/wayback-machine-scraper').call(run_input={
    'startUrls': [{'url': 'https://www.example.com'}],
    'maxItems': 100,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
```

#### API Example (cURL)

```bash
curl "https://api.apify.com/v2/acts/YOUR_USERNAME~wayback-machine-scraper/runs" \
  -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -d '{"startUrls": [{"url": "https://www.example.com"}], "maxItems": 100}'
```

### Tips and tricks

- Start with a small `maxItems` (10-50) to test before running large scrapes
- Use date filters (`dateFrom`/`dateTo`) to narrow results for popular sites with thousands of snapshots
- Domain-wide searches can return very large datasets -- always set a `maxItems` limit
- Filter by `statusCode: "200"` to only get successful snapshots (skip redirects and errors)
- The Wayback Machine API can be slow for domains with millions of snapshots -- be patient

### FAQ

**Q: Does this actor require login credentials?**
A: No. The Wayback Machine CDX API is completely free and public.

**Q: How fast is the scraping?**
A: Typically 1,000-10,000 results per minute depending on the API response time. Large domain searches may take longer.

**Q: What should I do if I get rate limited?**
A: Enable proxy configuration to rotate IPs automatically. Also reduce maxConcurrency.

**Q: Can I get the actual page content from the archive?**
A: This actor returns snapshot metadata (URLs, dates, status codes). Use the `archiveUrl` field to access the actual archived page content.

**Q: Why are some snapshots missing?**
A: The Wayback Machine does not archive every page on every visit. Some pages may have been excluded by robots.txt or simply not crawled.

### Is it legal to scrape the Wayback Machine?

The Wayback Machine (Archive.org) provides a public API specifically designed for programmatic access to archive data. This actor uses only the official CDX API. Always review and respect Archive.org's Terms of Service. For more information, see [Apify's blog on web scraping legality](https://blog.apify.com/is-web-scraping-legal/).

### Related Actors

- [Website Content Crawler](https://apify.com/apify/website-content-crawler) -- Crawl and extract content from live websites
- [Google Cache Scraper](https://apify.com/apify/google-cache-scraper) -- Access Google's cached versions of web pages

### Limitations

- The CDX API may rate-limit requests for very high-volume queries
- Domain-wide searches for popular domains (e.g., google.com) can return millions of records -- use date filters and maxItems
- The actor returns snapshot metadata, not the actual archived page content
- Some timestamps may have reduced precision (date only, no time)

### Changelog

- **v0.1** (2026-04-23) -- Initial release

# Actor input Schema

## `startUrls` (type: `array`):

Website URLs to look up in the Wayback Machine archive. The actor will find all archived snapshots for each URL.

## `domains` (type: `array`):

Domain names to search across the entire domain (e.g., "example.com"). Uses wildcard matching to find all archived pages under the domain.

## `dateFrom` (type: `string`):

Only include snapshots archived on or after this date.

## `dateTo` (type: `string`):

Only include snapshots archived on or before this date.

## `mimeTypeFilter` (type: `string`):

Filter snapshots by content type.

## `statusCodeFilter` (type: `string`):

Filter by HTTP status code (e.g., "200" for successful responses, "301" for redirects). Leave empty for all status codes.

## `maxItems` (type: `integer`):

Maximum number of snapshot records to return. Use 0 for unlimited.

## `maxConcurrency` (type: `integer`):

Maximum number of URLs processed in parallel.

## `debugMode` (type: `boolean`):

Enables verbose logging for troubleshooting.

## `proxyConfig` (type: `object`):

Proxy settings. The Wayback Machine API is public, but proxies can help avoid rate limits at scale.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.example.com"
    }
  ],
  "mimeTypeFilter": "all",
  "maxItems": 1000,
  "maxConcurrency": 5,
  "debugMode": false
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.example.com"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("glassventures/wayback-machine-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://www.example.com" }] }

# Run the Actor and wait for it to finish
run = client.actor("glassventures/wayback-machine-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.example.com"
    }
  ]
}' |
apify call glassventures/wayback-machine-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,glassventures/wayback-machine-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/pakArGC4iWT2LAihJ/builds/e7ngMNc9mnqHMBhgT/openapi.json