# Multi-Engine Search Scraper — Google, Bing, DuckDuckGo (`maged120/search-engines-scraper`) Actor

Search engine scraper: get organic search results from Google, Bing, Yahoo, DuckDuckGo and more in one run. Titles, URLs, snippets and source domains for any keyword, in bulk. Compare rankings across engines. No API keys needed. Export to CSV, JSON, Excel or API.

- **URL**: https://apify.com/maged120/search-engines-scraper.md
- **Developed by:** [Maged](https://apify.com/maged120) (community)
- **Categories:** Automation, SEO tools
- **Features:** Limited permissions
- **Stats:** 47 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

Scrape search results from Google, Bing, Yahoo, DuckDuckGo, and more in a single run. Get structured data including titles, URLs, snippets, and source domains — no API keys, no per-query costs.

### What does Google Search and Engines Scraper do?

This Actor extracts organic search results from multiple search engines simultaneously. Provide a search query and select which engines to scrape — the Actor returns clean, structured results from all selected engines in one dataset.

Supported search engines: **Google, Bing, Yahoo, DuckDuckGo, AOL, StartPage, Dogpile, Ask**

### Why use this Actor?

- **Multi-engine in one run** — compare results across all major engines
- **No API keys required** — no Google Search API quota or billing
- **Configurable depth** — control how many pages to scrape per engine
- **Proxy support** — optional proxy for IP rotation and unblocking
- **Structured output** — title, URL, snippet, source engine, and page number for every result

### How to use Google Search and Engines Scraper

1. Open the Actor and click **Try for free**
2. Enter your `search_query`
3. Select one or more engines in `search_engines`
4. Set `max_pages` to control how deep to go
5. Run — results appear in the Output tab
6. Export as JSON or CSV, or integrate via the Apify REST API

### Input

```json
{
  "search_query": "best python web scraping libraries 2024",
  "search_engines": ["google", "bing", "duckduckgo"],
  "max_pages": 3,
  "timeout": 15,
  "proxy": null
}
```

| Field | Type | Description | Default |
|---|---|---|---|
| `search_query` | string | The search query | required |
| `search_engines` | array | Engines: `google`, `bing`, `yahoo`, `duckduckgo`, `aol`, `startpage`, `dogpile`, `ask` | `["google"]` |
| `max_pages` | integer | Pages to scrape per engine | `1` |
| `timeout` | integer | Request timeout in seconds | `10` |
| `proxy` | string | Optional proxy URL | — |

### Output

```json
[
  {
    "engine": "google",
    "page": 1,
    "title": "Scrapy — A Fast and Powerful Scraping Framework",
    "link": "https://scrapy.org",
    "host": "scrapy.org",
    "text": "An open source and collaborative framework for extracting the data you need from websites."
  },
  {
    "engine": "bing",
    "page": 1,
    "title": "Beautiful Soup Documentation",
    "link": "https://www.crummy.com/software/BeautifulSoup/",
    "host": "crummy.com",
    "text": "Beautiful Soup is a Python library for pulling data out of HTML and XML files."
  }
]
```

### Output data fields

| Field | Type | Description |
|---|---|---|
| `engine` | string | Source search engine |
| `page` | number | Result page number |
| `title` | string | Page title |
| `link` | string | Full URL |
| `host` | string | Root domain |
| `text` | string | Search result snippet |

### Use cases

- **SEO research** — track keyword rankings across multiple engines simultaneously
- **Market research** — discover what content ranks for any target query
- **SERP monitoring** — compare brand or competitor visibility across Google, Bing, and DuckDuckGo
- **Competitor analysis** — identify which competitors dominate search results for key terms
- **Lead generation** — find businesses by scraping industry-specific queries
- **Content gap analysis** — identify what questions your audience is searching for

### How many results will I get?

You're charged per result, so the cost follows the number of rows below. Your plan's rates are on the **Pricing** tab.

| Input | Results |
|---|---|
| 1 query × 4 engines | About 40 rows |
| 100 queries × 1 engine | About 1,000 rows |

Each search result is one row. Pick only the engines you need to keep runs small.

### Google mode: advanced Google search

This Actor also covers what the former **Google Search Scraper** did: one engine with Google's advanced filters. Use `query` instead of `search_query` (or set `"mode": "google"`):

| Field | Description |
|---|---|
| `query` | Search text |
| `google_domain`, `language`, `region` | Which Google and which market (e.g. `google.co.uk`, `en`, `GB`) |
| `search_type` | `web`, `news`, `images`, `videos`… |
| `results_per_page`, `pages`, `start_page` | How many results to fetch |
| `site`, `file_type`, `exact_phrase`, `all_words`, `any_words`, `none_words` | Google's advanced search operators |
| `from_date`, `to_date`, `last_update` | Date filters |
| `unique_urls`, `unique_domains`, `safe_search` | Result cleanup |

```json
{ "query": "python scraping", "pages": 2, "language": "en", "region": "US" }
```

**Coming from Google Search Scraper?** Use the same input here; the rows are the same.

### FAQ

**Do I need API keys for any of these engines?**
No. Just run the Actor.

**Can I scrape only Google?**
Yes — set `search_engines` to `["google"]` and it will only scrape Google.

**What if results are blocked?**
Use the `proxy` field with a residential proxy or Apify proxy to rotate IPs.

**Is this Actor maintained?**
Yes. For issues or feature requests, open an issue in the Issues tab.

***

⭐ **Found this Actor useful?** A quick review on the Store helps other users find it and keeps it maintained.

### Related tools

- [Google Trends Scraper – interest over time, by region and related queries](https://apify.com/maged120/google-trends-scraper)

# Actor input Schema

## `mode` (type: `string`):

Leave empty to pick automatically: "Google query" means Google mode, otherwise multi-engine.

## `search_query` (type: `string`):

The search query to look for

## `search_engines` (type: `array`):

Select one or more search engines to use

## `max_pages` (type: `integer`):

How many pages to crawl from search results

## `timeout` (type: `integer`):

Request timeout in seconds

## `proxy` (type: `string`):

Optional proxy URL to use for requests

## `query` (type: `string`):

Search text for Google mode, with Google-specific options below (domain, type, pages, language, region, safe search). Setting this instead of "Search query" switches to Google mode.

## `google_domain` (type: `string`):

Google country domain to search. The country is selected by domain, so use google.co.uk for the UK, google.de for Germany, etc.

## `search_type` (type: `string`):

Type of search (e.g. 'web', 'images', 'news')

## `results_per_page` (type: `integer`):

Number of results per page

## `pages` (type: `integer`):

How many pages to crawl from search results

## `start_page` (type: `integer`):

Page number to start crawling from

## `unique_urls` (type: `boolean`):

Return only unique URLs

## `unique_domains` (type: `boolean`):

Return only unique domains

## `language` (type: `string`):

Restrict results to a specific language (e.g. 'en')

## `region` (type: `string`):

Restrict results to a specific region (e.g. 'US')

## `last_update` (type: `string`):

Restrict results to a certain update period (e.g. 'day', 'week', 'month', 'year')

## `site` (type: `string`):

Restrict results to a specific site

## `custom_url` (type: `string`):

overrides most of the other filters, this is for advanced users who know how to use google search params and filters from the url directly

## `all_words` (type: `string`):

All these words must appear in results

## `exact_phrase` (type: `string`):

Exact phrase to appear in results

## `any_words` (type: `string`):

Any of these words can appear in results

## `none_words` (type: `string`):

None of these words should appear in results

## `numbers_from` (type: `integer`):

Minimum number value to appear in results (normally used for price lookup)

## `numbers_to` (type: `integer`):

Maximum number value to appear in results (normally used for price lookup)

## `terms_appearing` (type: `string`):

Where terms should appear (e.g. 'any', 'title', 'url', 'body')

## `file_type` (type: `string`):

Restrict results to a specific file type (e.g. 'pdf')

## `safe_search` (type: `boolean`):

Enable safe search filtering

## `from_date` (type: `string`):

Restrict results from this date (YYYY-MM-DD)

## `to_date` (type: `string`):

Restrict results up to this date (YYYY-MM-DD)

## `custom_tbs` (type: `string`):

Custom Google TBS parameter this is also for advanced users who know what tbs param is

## Actor input object example

```json
{
  "search_query": "learn python",
  "search_engines": [
    "google"
  ],
  "max_pages": 1,
  "timeout": 10
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "search_query": "learn python",
    "search_engines": [
        "google"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("maged120/search-engines-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "search_query": "learn python",
    "search_engines": ["google"],
}

# Run the Actor and wait for it to finish
run = client.actor("maged120/search-engines-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "search_query": "learn python",
  "search_engines": [
    "google"
  ]
}' |
apify call maged120/search-engines-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,maged120/search-engines-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/88Nl8oxHYRm3msDyq/builds/QfPLnwIWgBBVt8SOr/openapi.json