# Similarweb Scraper - Traffic, AI Traffic & WHOIS (`vortex_data/similarweb-scraper`) Actor

🔍 Spy on any website in seconds: traffic, rankings, top keywords, AI traffic share (ChatGPT/Claude/Gemini), competitors, similar sites & WHOIS — all from Similarweb. No login or API key. Bulk parallel scrape, captcha-resilient. Export to JSON/CSV/Excel. SEO, lead gen, research.

- **URL**: https://apify.com/vortex_data/similarweb-scraper.md
- **Developed by:** [VortexData](https://apify.com/vortex_data) (community)
- **Categories:** Lead generation, SEO tools, Automation
- **Features:** Limited permissions
- **Stats:** 728 total users, 182 monthly users, 100.0% runs succeeded, 9 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $1.50 / 1,000 domains

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### What is Similarweb Scraper?

Similarweb Scraper extracts **website traffic data from Similarweb** —
ranks, visits, traffic sources, AI traffic, similar sites and WHOIS — for
any list of domains, **without a Similarweb account**. Use it to **research
competitors**, **track traffic from AI chatbots** and **find alternatives**
to any website. To try it, enter one or more **domains**, pick **one
dataset** and click **Start**.

### What can Similarweb Scraper do?

- 🗂️ **Three datasets, one per run:**
  - 📊 **Base data** — global, country and category rank, monthly visits,
    bounce rate, pages per visit, time on site, traffic-source split, top
    organic keywords with volume and CPC, visits by country, and how much
    traffic AI assistants send to the site.
  - 🪞 **Similar sites** — competitors and alternatives with their traffic,
    category, top-country rank and a similarity score, plus related mobile
    apps.
  - 🆔 **AITDK** — domain WHOIS (registrar, registration and expiration
    dates, name servers, status) and keyword density of the homepage.
- ⚡ **One domain or thousands** — one domain is enough for a test run;
  large lists are processed in parallel.
- 🌐 **Any URL format** — `example.com`, `www.example.com` or
  `https://example.com/page`; the domain is extracted automatically.
- 🧾 **Nothing silently lost** — every run lists the domains that returned
  no data, so you can re-run just those.

#### Why run it on Apify?

- ⏰ **Scheduling** — check competitors' traffic every week or month.
- 🔌 **API and integrations** — REST API, Python and JavaScript clients,
  webhooks, Make, Zapier, n8n, Google Sheets, Slack and Airtable.
- 🌍 **Proxies included** — no proxy setup needed.
- 📈 **Monitoring** — run history, logs and alerts for failed runs.
- 💾 **Storage and export** — JSON, CSV, Excel, JSONL, XML, RSS or HTML.

### What data can you extract from Similarweb?

| Data group          | Fields                                                                                           |
| ------------------- | ------------------------------------------------------------------------------------------------ |
| Rankings            | Global rank · country rank · category rank                                                       |
| Engagement          | Total visits · monthly visits · bounce rate · pages per visit · time on site                     |
| Traffic sources     | Direct · search (organic / paid) · referral · social (organic / paid) · paid referrals · display ads · GenAI · mail |
| AI traffic          | Visits from AI assistants and each assistant's share (ChatGPT, Gemini, Claude and others), month by month |
| Top keywords        | Keyword · estimated value · search volume · CPC                                                  |
| Countries           | Top countries with share and monthly visits per country                                          |
| Similar sites       | Related sites with description, category, top-country rank, visits, thumbnail and similarity score |
| Related apps        | Mobile apps linked to the site — title, platform, store ranking, link, icon                      |
| WHOIS               | Registrar · registration, expiration and last-changed dates · name servers · domain status · abuse contact |
| Keyword density     | Top 20 words on the homepage with count and density                                              |
| Assets              | Desktop and mobile screenshots · favicon                                                         |

### How to scrape Similarweb data

1. Open **Similarweb Scraper** on Apify Store and click **Try for free**.
2. In **🌐 Domains**, enter the websites to analyse, one per line. One
   domain is enough for a test.
3. In **🗂️ Dataset to fetch**, pick one dataset: **📊 Base data**,
   **🪞 Similar sites** or **🆔 AITDK**.
4. Click **Start** and wait for the run to finish. **AITDK** takes longer
   than the other two, because it also queries the domain registry and
   downloads the homepage.
5. Open the **Output** tab to preview the results, then export them as
   JSON, CSV, Excel or another format — or read them through the API.

Need another dataset for the same domains? Run the Actor again with a
different **Dataset to fetch**.

### How much will it cost to scrape Similarweb?

Similarweb Scraper is priced **per result**: you pay for **each domain
that is saved to the dataset**, in whichever dataset you chose.

- ✅ **Domains that fail are free.** A domain that returns no data produces
  no dataset item, so it is not charged.
- 📋 **The price per domain for your Apify plan** is shown on the
  **Pricing** tab; higher plans pay less per domain.
- 🧮 **Easy to estimate** — the cost of a run is the number of domains
  saved times the price per domain.
- 🛑 **Set a limit** — in the run options you can set a maximum total
  charge per run.
- 🆓 **Try it for free** — Apify's free plan includes monthly platform
  credit you can use on this Actor.

### Input

The input has only two fields:

- 🌐 **Domains** — the websites to analyse, one per line.
- 🗂️ **Dataset to fetch** — **Base data**, **Similar sites** or **AITDK**.

#### Example input

```json
{
  "domains": ["openai.com", "github.com"],
  "datasetMode": "base_data"
}
```

### Output

You get **one row per domain**. The **Output** tab shows it as tables:
**📊 Overview · 🪞 Similar sites · 🚦 Traffic sources · 💫 Engagement ·
🤖 AI traffic share · 🆔 AITDK**. Below are real results, shortened, for
the domains in the example input.

#### Base data example

```json
{
  "domain": "openai.com",
  "snapshotDate": "2026-07-01T00:00:00+00:00",
  "title": "OpenAI",
  "rankGlobal": 212,
  "country": "US",
  "countryRank": 261,
  "category": "ai_chatbots_and_tools",
  "categoryRank": 5,
  "totalVisits": 197235347,
  "bounceRate": 0.5786,
  "pagesPerVisit": 2.97,
  "timeOnSite": 151.32,
  "directTraffic": 0.4539,
  "searchTraffic": 0.1953,
  "genAiTraffic": 0.2088,
  "referralTraffic": 0.0927,
  "socialTraffic": 0.0332,
  "mailTraffic": 0.0147,
  "displayAdsTraffic": 0.0014,
  "aiTrafficShareChatgpt": 0.8478,
  "aiTrafficShareGemini": 0.0123,
  "aiTrafficShareClaude": 0.0023,
  "aiTrafficSharePerplexity": null,
  "aiTrafficVisits": 40471810.45,
  "topKeywords": [
    {"keyword": "chatgpt", "estimatedValue": 25214240, "searchVolume": 164282150, "cpc": 0.14},
    {"keyword": "codex", "estimatedValue": 4064950, "searchVolume": 6264890, "cpc": 4.48}
  ]
}
```

#### Similar sites example

```json
{
  "domain": "openai.com",
  "title": "OpenAI",
  "total_visits": 620111324,
  "category": "Computers_Electronics_and_Technology/Computers_Electronics_and_Technology",
  "categoryRank": 4,
  "similar_sites": [
    {
      "url": "chatgpt.com",
      "category": "Computers_Electronics_and_Technology/Programming_and_Developer_Software",
      "topCountryRank": 12,
      "totalVisits": 5720228864,
      "similarityRank": 1,
      "similarityScore": 0.6872
    },
    {
      "url": "deepai.org",
      "category": "Computers_Electronics_and_Technology/Computers_Electronics_and_Technology",
      "topCountryRank": 3521,
      "totalVisits": 15002636,
      "similarityRank": 2,
      "similarityScore": 0.5456
    }
  ],
  "related_apps": [
    {"title": "ChatGPT", "platform": "ANDROID", "ranking": 1, "url": "https://play.google.com/store/apps/details?id=com.openai.chatgpt"}
  ]
}
```

#### AITDK (WHOIS + keyword density) example

```json
{
  "domain": "github.com",
  "whois_registrar": "MarkMonitor Inc.",
  "whois_registrar_iana_id": "292",
  "whois_registration_date": "2007-10-09T18:20:50Z",
  "whois_expiration_date": "2026-10-09T18:20:50Z",
  "whois_last_changed_date": "2024-09-07T09:16:32Z",
  "whois_abuse_email": "abusecomplaints@markmonitor.com",
  "whois_name_servers": ["dns1.p08.nsone.net", "ns-421.awsdns-52.com"],
  "whois_status": ["client delete prohibited", "client transfer prohibited", "client update prohibited"],
  "keyword_density_total_words": 486,
  "keyword_density": [
    {"keyword": "github", "count": 34, "density": 0.069959},
    {"keyword": "copilot", "count": 17, "density": 0.034979},
    {"keyword": "code", "count": 15, "density": 0.030864}
  ]
}
```

### Other scrapers you might like

#### Ready-to-run Similarweb tasks

| Task | What it does |
| ---- | ------------ |
| [Compare AI chatbot traffic to top news websites](https://apify.com/vortex_data/similarweb-scraper/examples/ai-chatbot-traffic-to-news-websites) | How much traffic ChatGPT, Gemini and Claude send to NYT, BBC, CNN, The Guardian, Reuters and The Washington Post. |
| [Find competitors and alternatives for SaaS websites](https://apify.com/vortex_data/similarweb-scraper/examples/saas-competitors-and-alternatives) | Closest competitors of Notion, Figma, Canva, HubSpot and Slack with traffic and similarity score. |
| [Check domain expiration dates and WHOIS of e-commerce brands](https://apify.com/vortex_data/similarweb-scraper/examples/ecommerce-domain-expiration-and-whois) | Registrar, expiration date, name servers and status of Shopify, Etsy, eBay, Walmart and Target. |

#### More scrapers from the same developer

| Actor | What it does |
| ----- | ------------ |
| [Google Maps Scraper](https://apify.com/vortex_data/google-maps) | Extract places, contacts and reviews from Google Maps for lead generation and market research. |
| [Instagram Scraper](https://apify.com/vortex_data/instagram-scraper) | Extract Instagram profiles, contacts, posts, reels, comments and followers. |

### FAQ

#### Do I need a Similarweb account or API key?

No. The Actor reads the same public data Similarweb shows on its website
and in its browser extension. No login, no API key.

#### Can I use it as a Similarweb API?

Yes. You can start runs and download results through the
[Apify API](https://docs.apify.com/api/v2) or the official
[Python](https://docs.apify.com/api/client/python/) and
[JavaScript](https://docs.apify.com/api/client/js/) clients. The **API**
tab of this Actor shows ready-to-copy examples.

#### Can I track websites on a schedule and send the data to Google Sheets?

Yes. In Apify Console, create a **Schedule** for the Actor (daily, weekly,
monthly or a custom cron) and connect an integration or a
[webhook](https://docs.apify.com/platform/integrations/webhooks) to send
new results to Google Sheets, Slack, Make, Zapier, n8n or your own backend.

#### What can I use it for?

- 🔎 **Competitor research** — compare traffic, engagement and traffic
  sources of competing websites.
- 🤖 **AI visibility** — see how much traffic ChatGPT, Gemini and Claude
  send to a site and how it changes month to month.
- 🧲 **Lead qualification** — rank a list of company websites by traffic.
- 🪞 **Market mapping** — find alternatives to a website with similar sites.
- 📅 **Domain portfolio checks** — track registrars and expiration dates.
- 📝 **SEO** — top organic keywords and homepage keyword density.

#### How fresh is the data?

It is the data Similarweb currently publishes. Similarweb updates traffic
data once a month.

#### Why are some fields empty?

- Similarweb shows AI traffic share only for a site's top three AI
  assistants, so the others stay empty.
- Small sites that Similarweb does not rank have no rank and no traffic
  sources.
- Sometimes Similarweb gives only a short version of a site's data; then
  AI traffic is empty and monthly visits cover one month.
- Some websites block every visit, so their keyword density stays empty.

#### Some domains returned no data. What should I do?

1. Open the finished run and go to the **Storage → Key-value store** tab.
2. Open the **FAILED_DOMAINS** record — it lists every domain that returned
   no data, with the reason.
3. Copy the list from `domainsText` into **🌐 Domains** and run again —
   only the missing domains are processed.

#### Is it legal to scrape Similarweb?

This Actor extracts only publicly available data. Traffic and ranking
data describe websites, not people. **WHOIS records can contain personal
data** — such as a domain owner's name or email — when the registry
publishes it; most registries hide it. If your results contain personal
data, you need a legitimate reason to process it under GDPR and similar
laws. If you are unsure, consult a lawyer. Read more in
[Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/)

#### I found a bug or need a custom solution

Open an issue in the **Issues** tab of this Actor.
If you need extra data, another output format or a custom version of this
scraper, write to us in the Issues tab as well.

# Changelog

This Actor's version history is a separate document: https://apify.com/vortex_data/similarweb-scraper/changelog.md

# Actor input Schema

## `domains` (type: `array`):

Websites to analyse. One domain is enough for a test run. Accepts `example.com`, `www.example.com`, or `https://example.com`; the domain is extracted automatically.

## `datasetMode` (type: `string`):

Pick one dataset for this run. Run the Actor again with another mode when you need a different data group.

## Actor input object example

```json
{
  "domains": [
    "openai.com",
    "google.com",
    "booking.com",
    "imdb.com",
    "amazon.com",
    "linkedin.com",
    "paypal.com",
    "reddit.com",
    "youtube.com",
    "wikipedia.org",
    "microsoft.com",
    "apple.com",
    "netflix.com",
    "github.com",
    "stackoverflow.com",
    "cnn.com",
    "nytimes.com",
    "bbc.com",
    "walmart.com",
    "airbnb.com"
  ],
  "datasetMode": "base_data"
}
```

# Actor output Schema

## `dataset` (type: `string`):

JSON / CSV / XLSX / JSONL / XML / RSS export of every record pushed during the run.

## `summary` (type: `string`):

JSON summary saved to the default key-value store as OUTPUT.

## `failedDomains` (type: `string`):

Complete list of domains that produced no dataset item, saved as FAILED_DOMAINS. Send it back as input to finish the batch without repeating the whole list.

## `timeline` (type: `string`):

Every request, domain and retry of the run with the second it happened on, saved to the default key-value store as TIMELINE.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "openai.com",
        "google.com",
        "booking.com",
        "imdb.com",
        "amazon.com",
        "linkedin.com",
        "paypal.com",
        "reddit.com",
        "youtube.com",
        "wikipedia.org",
        "microsoft.com",
        "apple.com",
        "netflix.com",
        "github.com",
        "stackoverflow.com",
        "cnn.com",
        "nytimes.com",
        "bbc.com",
        "walmart.com",
        "airbnb.com"
    ],
    "datasetMode": "base_data"
};

// Run the Actor and wait for it to finish
const run = await client.actor("vortex_data/similarweb-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "domains": [
        "openai.com",
        "google.com",
        "booking.com",
        "imdb.com",
        "amazon.com",
        "linkedin.com",
        "paypal.com",
        "reddit.com",
        "youtube.com",
        "wikipedia.org",
        "microsoft.com",
        "apple.com",
        "netflix.com",
        "github.com",
        "stackoverflow.com",
        "cnn.com",
        "nytimes.com",
        "bbc.com",
        "walmart.com",
        "airbnb.com",
    ],
    "datasetMode": "base_data",
}

# Run the Actor and wait for it to finish
run = client.actor("vortex_data/similarweb-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "openai.com",
    "google.com",
    "booking.com",
    "imdb.com",
    "amazon.com",
    "linkedin.com",
    "paypal.com",
    "reddit.com",
    "youtube.com",
    "wikipedia.org",
    "microsoft.com",
    "apple.com",
    "netflix.com",
    "github.com",
    "stackoverflow.com",
    "cnn.com",
    "nytimes.com",
    "bbc.com",
    "walmart.com",
    "airbnb.com"
  ],
  "datasetMode": "base_data"
}' |
apify call vortex_data/similarweb-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,vortex_data/similarweb-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/K1uLbeyZgYAoMgvwI/builds/IRR5YuVsCpdeuyZ92/openapi.json