# Email Scraper (`dominic-quaiser/email-scraper`) Actor

A lightweight Apify Actor that crawls websites and extracts email addresses using HTTP requests. It bypasses Cloudflare protection, RTL obfuscation, and text patterns to deliver structured data fast. Features include configurable crawl depth, proxy support, and anti-detection measures.

- **URL**: https://apify.com/dominic-quaiser/email-scraper.md
- **Developed by:** [Dominic M. Quaiser](https://apify.com/dominic-quaiser) (community)
- **Categories:** Lead generation, Developer tools, Other
- **Features:** Limited permissions
- **Stats:** 167 total users, 5 monthly users, 100.0% runs succeeded, 3 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.50 / 1,000 emails

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Email Scraper

A powerful Apify Actor designed to crawl websites and extract email addresses using advanced detection methods. Built on lightweight HTTP requests, it runs fast while still handling Cloudflare-protected emails, RTL Unicode obfuscation, and other common hiding techniques. Simply provide a list of starting URLs, configure crawl depth, and let it run.

### 🚀 Features

- **Intelligent Email Discovery**: Finds email addresses using multiple sophisticated detection methods, including:
  - Standard text pattern matching
  - Mailto links extraction
  - Cloudflare-protected emails
  - RTL (Right-to-Left) Unicode obfuscation
  - Common text obfuscation patterns
  - Multiple recipients in a single `mailto:` link
  - Cloudflare email-protection links (`/cdn-cgi/l/email-protection#…`)
- **Email Filtering**: Optionally drop automated/role addresses (noreply@, postmaster@, …), keep only emails on the same domain as the crawled site, or apply your own include/exclude regular expressions.
- **Sitemap-Based Discovery**: Reads the site's `sitemap.xml` (including nested and gzipped sitemaps) to reach contact/about pages directly, instead of relying on link-following alone.
- **Keyword-Based Link Prioritization**: High-value pages (contact, legal notice, privacy policy, about, team, …, in English and German) are crawled first so emails are found faster, even on large sites.
- **Configurable Crawl Depth**: Control how deep the crawler follows links from your starting URLs (0-10 levels).
- **Domain-Focused or Broad Crawling**: Choose to stay on the same domain or explore external links.
- **Lightweight HTTP Crawling**: Fast, efficient method using HTTP requests without the overhead of a browser.
- **Anti-Detection Features**: Built-in measures to avoid blocking, including user agent rotation, request delays, and robots.txt compliance.
- **Proxy Support**: Integrates seamlessly with Apify's proxy service for IP rotation and avoiding rate limits.
- **Structured JSON Output**: Delivers clean, well-structured data with full context about where and how each email was discovered.

### 📥 Input Parameters

Configure the actor's behavior using these fields in the Apify Console Input tab or via API:

| Field | Type | Description | Default | Required |
| --- | --- | --- | --- | --- |
| `start_urls` | Array | The URLs to start crawling from. The scraper will extract emails from these pages and follow links up to the specified depth. | `[{ "url": "https://beispielbusiness.de/" }]` | Yes |
| `max_depth` | Integer | Maximum depth of links to follow from start URLs. 0 = only start URLs, 1 = start URLs + one level of links, etc. Range: 0-10. | `2` | No |
| `use_sitemap` | Boolean | Discover pages from the site's `sitemap.xml` to reach contact/about pages directly, in addition to following links. Discovered pages still respect the per-domain and global page limits. | `true` | No |
| `stay_on_domain` | Boolean | Only follow links that stay on the same domain as each start URL. When enabled, the crawler won't visit external sites. | `true` | No |
| `max_concurrent_pages` | Integer | Maximum number of pages to process simultaneously. Higher values crawl faster but use more memory; lower values are gentler on target sites. The actor automatically caps this per start-URL domain so workers don't time out behind the per-domain request delay (see Performance Tips). Range: 1-100. | `50` | No|
| `max_pages_per_domain` | Integer | Maximum number of pages to crawl from each individual domain. Leave empty for unlimited. This limit applies separately to each domain. | `200` | No |
| `max_requests_per_run` | Integer | Maximum number of pages to crawl globally across all domains. Leave empty for unlimited. | Unlimited | No |
| `request_delay_min` | Number | Minimum delay in seconds between consecutive requests to the same domain. A random value between min and max is used per request; different domains are crawled in parallel. Range: 0-60. | `1` | No |
| `request_delay_max` | Number | Maximum delay in seconds between consecutive requests to the same domain. A random value between min and max is used per request. Range: 0-60. | `3` | No |
| `respect_robots_txt` | Boolean | Honor robots.txt directives including crawl delays and disallowed paths. | `false` | No |
| `rotate_user_agents` | Boolean | Use a pool of realistic user agents to appear as different users. | `true` | No |
| `max_block_retries` | Integer | How many more times a page is requested after the site refused it (HTTP 401, 403, 429 or a challenge page) or the proxy failed, each with a new session. A new session gets a new IP only from a rotating proxy; without one the same IP asks again. 0 = never. | `2` | No |
| `priority_keywords` | Array | Links whose URL path or link text contains one of these keywords as a whole word are crawled first, which helps reach contact, legal notice and about pages quickly. A keyword can be several words (`about us`); case and German umlauts don't matter (`ueber-uns` also matches "Über uns"). Leave empty to use the built-in defaults (`contact`, `kontakt`, `impressum`, `datenschutz`, `privacy`, `about`, `team`, …). | Built-in list | No |
| `exclude_role_emails` | Boolean | Skip automated/role addresses such as `noreply@`, `no-reply@`, `postmaster@`, `mailer-daemon@`, and `bounce@`. | `false` | No |
| `same_domain_emails_only` | Boolean | Keep only emails whose domain matches the website they were found on (e.g. crawling `sap.com` keeps `name@sap.com` but drops `name@gmail.com`). | `false` | No |
| `include_email_patterns` | Array | Regular expressions; if any are provided, only emails matching at least one pattern are kept. Leave empty to keep all. Example: `@sap\.com$`. | `[]` | No |
| `exclude_email_patterns` | Array | Regular expressions; emails matching any of these are dropped. Applied after the include filter. Example: `@(example\|test)\.com$`. | `[]` | No |
| `proxy_configuration` | Object | Proxy settings to avoid being blocked. Apify Proxy is recommended for large crawls. | `{}` | No |

### 📤 Output Data Structure

The actor outputs one record per unique email address found during the crawl.

#### Example Output

```json
[
  {
    "email": "info@beispielbusiness.de",
    "found_on_url": "https://beispielbusiness.de/contact",
    "start_url": "https://beispielbusiness.de/",
    "extraction_method": "mailto_link",
    "depth": 1
  },
  {
    "email": "support@beispielbusiness.de",
    "found_on_url": "https://beispielbusiness.de/about",
    "start_url": "https://beispielbusiness.de/",
    "extraction_method": "text_standard",
    "depth": 1
  },
  {
    "email": "sales@beispielbusiness.de",
    "found_on_url": "https://beispielbusiness.de/impressum",
    "start_url": "https://beispielbusiness.de/",
    "extraction_method": "cloudflare_protected",
    "depth": 2
  }
]
```

### 📧 Extraction Methods Explained

The actor uses multiple sophisticated techniques to find email addresses, even when websites try to hide them from bots:

| Method                     | Description                                                                                     |
| -------------------------- | ----------------------------------------------------------------------------------------------- |
| **`mailto_link`**          | Email addresses found in standard `mailto:` links in the HTML.                                  |
| **`text_standard`**        | Email addresses found in plain text using standard pattern matching.                            |
| **`text_obfuscated`**      | Email addresses that use common text obfuscation like "info \[at] example \[dot] com".            |
| **`cloudflare_protected`** | Email addresses protected by Cloudflare's email obfuscation that are decoded from the page.     |
| **`rtl_obfuscated`**       | Email addresses hidden using Right-to-Left (RTL) Unicode characters to confuse simple scrapers. |

#### 💡 Performance Tips

- **Find emails faster**: The crawler automatically prioritizes links matching keywords like `contact`, `impressum`, or `privacy`. Keywords match whole words, so `team` matches `/team` but not `/teamtraining`. Add your own terms via `priority_keywords` to front-load the pages most likely to contain emails, including compound forms such as `kontaktformular`.
- **For small sites**: Keep the default settings for optimal speed.
- **For large crawls**: Use proxy rotation to avoid blocking and rate limits.
- **Memory constraints**: Set `max_concurrent_pages` to a lower value (2-5) if running on limited memory.
- **Faster crawling**: Increase `max_concurrent_pages` if you have sufficient resources. The effective ceiling per domain is roughly `request_handler_timeout × 0.75 / request_delay_max` (≈15 per domain at the defaults), because requests stacked deeper than this would time out while waiting in the per-domain rate limiter. To raise the cap on a single-domain crawl, lower `request_delay_max`; for multi-domain crawls the cap scales linearly with the number of start-URL domains.

### 🎯 Use Cases

- **Lead Generation**: Build targeted contact lists for sales and marketing outreach.
- **Competitive Research**: Discover contact information for companies in your industry.
- **Data Enrichment**: Enhance existing company databases with email addresses.
- **Market Analysis**: Gather communication channels for businesses in specific sectors or regions.
- **Recruitment**: Find contact emails for potential candidates or hiring managers.
- **Partnership Development**: Identify contact points for potential business partnerships.

### 🛠️ Maintainer

- **Author**: Dominic M. Quaiser
- **Contact**: <mail@dominic-quaiser.io>
- **Website**: [dominic-quaiser.io](https://dominic-quaiser.io/)

***

### 🔧 Troubleshooting

#### No Emails Found

- Check if the website contains any publicly visible emails
- Try increasing `max_depth` to crawl more pages
- Verify that `stay_on_domain` isn't preventing you from reaching contact pages on subdomains
- Check if the website might be blocking the scraper (try enabling proxies)

#### Actor Running Out of Memory

- Decrease `max_concurrent_pages` to process fewer pages simultaneously
- Use `max_requests_per_run` to limit the total crawl size
- Upgrade to a larger memory tier on Apify

A linked file does not cost its size: the crawler leaves a response that is no page at its first bytes, and reads at most 10 MB of a page.

#### Getting Blocked by Websites

- Enable proxy rotation via `proxy_configuration`
- Increase `request_delay_min` and `request_delay_max`
- Enable `rotate_user_agents`
- Consider enabling `respect_robots_txt` to honor crawl delays

#### Log Says "Reducing concurrency from X to Y"

This warning means your requested `max_concurrent_pages` exceeds what the per-domain rate limit can sustain — the actor lowered concurrency so requests don't time out while waiting for their delay slot. To restore the requested value, either lower `request_delay_max` or add more start-URL domains (the cap scales per domain).

# Changelog

This Actor's version history is a separate document: https://apify.com/dominic-quaiser/email-scraper/changelog.md

# Actor input Schema

## `start_urls` (type: `array`):

URLs to start crawling from. The scraper will extract emails from these pages and follow links up to the specified depth.

## `max_depth` (type: `integer`):

Maximum depth of links to follow from start URLs. 0 = only start URLs, 1 = start URLs + one level of links, etc.

## `stay_on_domain` (type: `boolean`):

Only follow links that stay on the same domain as each start URL. When enabled, the crawler won't visit external sites.

## `use_sitemap` (type: `boolean`):

Discover pages from the site's sitemap.xml to reach contact/about pages directly, in addition to following links. Discovered pages still respect the per-domain and global page limits.

## `priority_keywords` (type: `array`):

Links whose URL path or link text contains one of these keywords as a whole word are crawled first, which helps reach contact, legal notice and about pages quickly. A keyword can be several words ("about us"); case and German umlauts don't matter, so "ueber-uns" also matches "Über uns". Leave empty to use the built-in defaults.

## `exclude_role_emails` (type: `boolean`):

Skip automated/role addresses such as noreply@, no-reply@, postmaster@, mailer-daemon@, and bounce@.

## `same_domain_emails_only` (type: `boolean`):

Keep only emails whose domain matches the website they were found on. For example, when crawling sap.com this keeps name@sap.com but drops name@gmail.com.

## `include_email_patterns` (type: `array`):

Regular expressions; if any are provided, only emails matching at least one pattern are kept. Leave empty to keep all. Example: "@sap\\.com$".

## `exclude_email_patterns` (type: `array`):

Regular expressions; emails matching any of these are dropped. Applied after the include filter. Example: "@(example|test)\\.com$".

## `max_concurrent_pages` (type: `integer`):

Maximum number of pages to process simultaneously. Higher values crawl faster but use more memory; lower values are gentler on target sites.

## `max_pages_per_domain` (type: `integer`):

Maximum number of pages to crawl from each individual domain. Leave empty for unlimited. This limit applies separately to each domain.

## `max_requests_per_run` (type: `integer`):

Maximum number of pages to crawl globally across all domains. Leave empty for unlimited.

## `request_delay_min` (type: `number`):

Minimum delay between consecutive requests to the same domain. A random value between min and max is used per request; different domains are crawled in parallel. Recommended: 1-2 seconds.

## `request_delay_max` (type: `number`):

Maximum delay between consecutive requests to the same domain. A random value between min and max is used per request.

## `respect_robots_txt` (type: `boolean`):

Honor robots.txt directives including crawl delays and disallowed paths.

## `rotate_user_agents` (type: `boolean`):

Use a pool of realistic user agents to appear as different users.

## `max_block_retries` (type: `integer`):

How many more times a page is requested after the site refused it (HTTP 401, 403 or 429, or a challenge page) or the proxy failed, each time with a new session. A new session gets a new IP only from a rotating proxy (Apify Proxy, or several proxy URLs); without one, the same IP asks again, which rarely helps and can get it blocked for longer. 0 = never ask again.

## `proxy_configuration` (type: `object`):

Proxy settings to avoid being blocked. Recommended for large crawls.

## Actor input object example

```json
{
  "start_urls": [
    {
      "url": "https://beispielbusiness.de/"
    }
  ],
  "max_depth": 2,
  "stay_on_domain": true,
  "use_sitemap": true,
  "priority_keywords": [
    "contact",
    "contactus",
    "kontakt",
    "kontaktformular",
    "ansprechpartner",
    "reach",
    "connect",
    "email",
    "imprint",
    "impressum",
    "legal",
    "privacy",
    "datenschutz",
    "datenschutzerklaerung",
    "datenschutzerklarung",
    "about",
    "aboutus",
    "ueber-uns",
    "uber-uns",
    "team",
    "people",
    "staff",
    "mitarbeiter",
    "job",
    "jobs",
    "career",
    "careers",
    "karriere",
    "press",
    "presse",
    "support",
    "help"
  ],
  "exclude_role_emails": false,
  "same_domain_emails_only": false,
  "include_email_patterns": [],
  "exclude_email_patterns": [],
  "max_concurrent_pages": 22,
  "max_pages_per_domain": 200,
  "request_delay_min": 1,
  "request_delay_max": 2,
  "respect_robots_txt": false,
  "rotate_user_agents": true,
  "max_block_retries": 2
}
```

# Actor output Schema

## `dataset` (type: `string`):

All scraped email addresses with full metadata (JSON format)

## `datasetCsv` (type: `string`):

Download all results as CSV spreadsheet

## `datasetExcel` (type: `string`):

Download all results as Excel spreadsheet (XLSX format)

## `viewOverview` (type: `string`):

Complete view with all fields - email addresses, source URLs, extraction methods, and crawl depth

## `viewEmailsOnly` (type: `string`):

Simple list of discovered email addresses without metadata - perfect for mailing lists

## `viewEmailsWithSources` (type: `string`):

Email addresses with the URLs where they were found - useful for verification

## `viewByMethod` (type: `string`):

Results grouped by extraction technique (mailto links, text patterns, obfuscated formats, etc.)

## `viewByDepth` (type: `string`):

Emails organized by how many links away from the start URL they were discovered

## `viewByStartUrl` (type: `string`):

Results grouped by which starting URL led to each discovery - for multi-domain crawls

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "start_urls": [
        {
            "url": "https://beispielbusiness.de/"
        }
    ],
    "priority_keywords": [
        "contact",
        "contactus",
        "kontakt",
        "kontaktformular",
        "ansprechpartner",
        "reach",
        "connect",
        "email",
        "imprint",
        "impressum",
        "legal",
        "privacy",
        "datenschutz",
        "datenschutzerklaerung",
        "datenschutzerklarung",
        "about",
        "aboutus",
        "ueber-uns",
        "uber-uns",
        "team",
        "people",
        "staff",
        "mitarbeiter",
        "job",
        "jobs",
        "career",
        "careers",
        "karriere",
        "press",
        "presse",
        "support",
        "help"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("dominic-quaiser/email-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "start_urls": [{ "url": "https://beispielbusiness.de/" }],
    "priority_keywords": [
        "contact",
        "contactus",
        "kontakt",
        "kontaktformular",
        "ansprechpartner",
        "reach",
        "connect",
        "email",
        "imprint",
        "impressum",
        "legal",
        "privacy",
        "datenschutz",
        "datenschutzerklaerung",
        "datenschutzerklarung",
        "about",
        "aboutus",
        "ueber-uns",
        "uber-uns",
        "team",
        "people",
        "staff",
        "mitarbeiter",
        "job",
        "jobs",
        "career",
        "careers",
        "karriere",
        "press",
        "presse",
        "support",
        "help",
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("dominic-quaiser/email-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "start_urls": [
    {
      "url": "https://beispielbusiness.de/"
    }
  ],
  "priority_keywords": [
    "contact",
    "contactus",
    "kontakt",
    "kontaktformular",
    "ansprechpartner",
    "reach",
    "connect",
    "email",
    "imprint",
    "impressum",
    "legal",
    "privacy",
    "datenschutz",
    "datenschutzerklaerung",
    "datenschutzerklarung",
    "about",
    "aboutus",
    "ueber-uns",
    "uber-uns",
    "team",
    "people",
    "staff",
    "mitarbeiter",
    "job",
    "jobs",
    "career",
    "careers",
    "karriere",
    "press",
    "presse",
    "support",
    "help"
  ]
}' |
apify call dominic-quaiser/email-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,dominic-quaiser/email-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/KihTWAl2K4fqXtQ94/builds/NaAKx1V9pLozVcDRZ/openapi.json