# SEMrush Scraper - Domain Authority & SEO Metrics (`devnaz/semrush-scraper`) Actor

Extract 20+ SEO metrics from any domain: authority score, monthly traffic, bounce rate, backlinks, organic keywords, competitors & more. No SEMrush subscription needed. Perfect for competitor analysis, SEO monitoring & market research.
Fast, reliable data extraction in 40 seconds.

- **URL**: https://apify.com/devnaz/semrush-scraper.md
- **Developed by:** [DevnaZ](https://apify.com/devnaz) (community)
- **Categories:** SEO tools, Automation, Developer tools
- **Features:** Limited permissions
- **Stats:** 276 total users, 7 monthly users, 100.0% runs succeeded, 6 bookmarks
- **User rating**: 1.00 out of 5 stars

## Pricing

$25.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SEMrush Scraper - Domain Authority, Traffic & SEO Data

Fast, reliable extraction of **domain authority, traffic data, rankings, and competitor insights** from SEMrush. **No SEMrush account required**. Perfect for SEO tracking, competitor monitoring, and market research.

### Features

Extract **20+ SEO metrics** from any domain in under 20 seconds:

#### Traffic Metrics

- **Monthly visits** - Total website traffic
- **Bounce rate** - Visitor engagement percentage
- **Pages per visit** - Average pages viewed per session
- **Visit duration** - Time spent on site (in seconds)
- **Traffic change** - Month-over-month growth/decline percentage

#### Traffic Sources

- **Organic search visits** - Traffic from search engines
- **Paid search visits** - Traffic from paid ads
- **Direct traffic %** - Percentage of direct visitors
- **Desktop traffic %** - Desktop vs mobile breakdown
- **Mobile traffic %** - Mobile device traffic share

#### Rankings & Authority

- **Domain authority** - SEMrush authority score (0-100)
- **Global rank** - Worldwide ranking
- **Country rank** - Ranking in primary country
- **Industry category** - Business category classification

#### Backlinks & SEO

- **Total backlinks** - Number of incoming links
- **Referring domains** - Unique domains linking to site

#### Competitive Intelligence

- **Top 5 competitors** - Similar websites by audience
- **Top 5 organic keywords** - Best performing search terms with position & volume
- **Top 5 countries** - Geographic traffic distribution with percentages

### Input

```json
{
  "domains": [
    "amazon.com",
    "apple.com",
    "make.com"
  ],
  "maxConcurrency": 3,
  "navigationTimeout": 60,
  "pageLoadDelay": 5000,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["DATACENTER"]
  }
}
```

#### Parameters

- **`domains`** (required) - Array of domain names to analyze (e.g., `["example.com"]`)
- **`maxConcurrency`** (optional) - Number of pages to scrape simultaneously (default: 3, max: 10)
- **`navigationTimeout`** (optional) - Maximum time to wait for page navigation in seconds (default: 60, max: 300). Increase if you get timeout errors.
- **`pageLoadDelay`** (optional) - Additional wait time for dynamic content in milliseconds (default: 5000, max: 15000)
- **`proxyConfiguration`** (optional) - Proxy settings (free Apify Datacenter proxies included by default)

### Output

```json
{
  "domain": "make.com",
  "scrapedAt": "2025-11-10T20:43:27.750Z",
  "domain_authority": 55,
  "monthly_visits": 8300000,
  "bounce_rate": 37.22,
  "pages_per_visit": 6.68,
  "avg_visit_duration_seconds": 770,
  "traffic_change_percent": -2.91,
  "organic_search_visits": 657960,
  "paid_search_visits": 123320,
  "direct_visits_percent": 79.72,
  "desktop_traffic_percent": 76.92,
  "mobile_traffic_percent": 23.08,
  "global_rank": 6757,
  "country_rank": 7264,
  "country": "United States",
  "industry_category": "Computer Software and Development",
  "total_backlinks": 2370000,
  "referring_domains": 26590,
  "competitors": [
    {"domain": "n8n.io"},
    {"domain": "zapier.com"},
    {"domain": "pipedream.com"}
  ],
  "organic_keywords": [
    {
      "keyword": "make",
      "position": 1,
      "search_volume": 74000
    }
  ],
  "top_countries": [
    {
      "country": "United States",
      "traffic_percent": 16.96,
      "visits": 1410000
    }
  ]
}
```

### Use Cases

#### 1. Competitor Analysis

Monitor your competitors' traffic, keywords, and backlink profiles to identify opportunities.

#### 2. SEO Monitoring

Track domain authority, organic traffic, and keyword rankings over time.

#### 3. Market Research

Discover industry leaders, traffic patterns, and top-performing keywords in your niche.

#### 4. Lead Generation

Identify high-traffic websites in your target industry for partnership opportunities.

#### 5. Portfolio Tracking

Monitor multiple websites' performance metrics from a single dashboard.

### Performance

- **Speed**: ~35-40 seconds per domain
- **Success rate**: 95%+ on most domains
- **Concurrency**: Process up to 10 domains simultaneously
- **Data freshness**: SEMrush updates monthly (scrapes latest available data)

### Notes

- **No SEMrush account required** - Scrapes publicly available data
- **Free Apify Datacenter proxies included** - Works great for most use cases
- **Rate limiting**: Respect SEMrush's infrastructure (max 10 concurrent requests)
- Some metrics may be unavailable for low-traffic domains
- Data is as fresh as SEMrush's latest update (typically monthly)

### Technical Details

- **Runtime**: Node.js with Playwright
- **Browser**: Chromium (headless mode)
- **Rendering**: Full JavaScript execution for dynamic content
- **Extraction**: Regex pattern matching on rendered page text
- **Proxy support**: Free Apify Datacenter proxies (default) or custom proxies

### Support

Having issues? Check these common solutions:

1. **Timeout errors**: Increase the `navigationTimeout` parameter (e.g., 120 seconds) or reduce `maxConcurrency`
2. **Slow loading sites**: Increase the `pageLoadDelay` parameter (e.g., 8000ms) to allow more time for dynamic content
3. **Missing data**: Some domains may have limited public data on SEMrush
4. **Proxy blocks**: Try increasing `navigationTimeout` or reducing `maxConcurrency`. Datacenter proxies work well for SEMrush.
5. **Rate limiting**: Reduce `maxConcurrency` (e.g., set to 1 or 2) to slow down request rate

For bugs or feature requests, please contact support.

### Example Integrations

#### Zapier/Make.com

Trigger analysis when competitors launch new pages or campaigns.

#### Google Sheets

Export data automatically to spreadsheets for reporting.

#### Slack/Discord

Get alerts when competitors' traffic changes significantly.

#### Custom Dashboards

Feed data into Tableau, Power BI, or custom analytics platforms.

***

No SEMrush subscription required. Get started analyzing domains today!

# Actor input Schema

## `domains` (type: `array`):

List of domains to scrape SEMrush data for (e.g., amazon.com, apple.com)

## `maxConcurrency` (type: `integer`):

Maximum number of pages to scrape concurrently

## `navigationTimeout` (type: `integer`):

Maximum time to wait for page navigation (default: 60s). Increase if timeout errors occur.

## `pageLoadDelay` (type: `integer`):

Additional wait time for dynamic content to load (default: 5000ms)

## `proxyConfiguration` (type: `object`):

Proxy settings for scraping. Free Apify Datacenter proxies included.

## Actor input object example

```json
{
  "domains": [
    "amazon.com",
    "apple.com",
    "microsoft.com"
  ],
  "maxConcurrency": 3,
  "navigationTimeout": 60,
  "pageLoadDelay": 5000,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "make.com"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("devnaz/semrush-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "domains": ["make.com"],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("devnaz/semrush-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "make.com"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call devnaz/semrush-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,devnaz/semrush-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/bL147gFarcajpfr5W/builds/gLdOvHITfMQ6OdyiL/openapi.json