# AI Agent Web Fetcher (`abotapi/ai-fetch-python`) Actor

An advanced web fetcher that can fetch almost all websites and convert them to LLM-friendly Markdown format. Perfect for AI agents, RAG systems, and integration with search actors.

- **URL**: https://apify.com/abotapi/ai-fetch-python.md
- **Developed by:** [Abot API](https://apify.com/abotapi) (community)
- **Categories:** AI, Agents, Developer tools
- **Features:** Limited permissions
- **Stats:** 25 total users, 1 monthly users, 97.3% runs succeeded, 1 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.40 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## AI Web Fetcher: Any Page to Clean, LLM-Ready Markdown

AI Web Fetcher renders any URL in a full browser (default engine) and returns clean, LLM-ready Markdown, plus the page's title, description and author when the page publishes them. Fetch one URL on demand, or point it at another actor's output dataset and it works through every link in a field you choose, up to a cap you set. Read results back from the dataset (with Overview, Markdown Content, Crawl info and Metadata & SEO views), the key-value store, the API, or call the actor's own persistent Standby HTTP endpoint for real-time fetches from your own backend or agent.

### Why This Scraper?

- **Real browser rendering.** With the default engine every page loads in a full headless browser, so JavaScript-rendered content, not just the raw HTML response, ends up in the output (`engine` set to `http` skips the browser for its first, plain request and has no JavaScript rendering).
- **Clean Markdown, not raw HTML.** Scripts, navigation, ads, forms and hidden elements are stripped before conversion, so downstream LLMs and agents get readable text instead of markup noise.
- **Automatic fallback.** A page that fails or times out on the first attempt is retried in a second browser session, up to two more times within the same per-page time budget; with a proxy on, each fallback attempt opens a new proxy session.
- **More than Markdown.** Title, description and author come back with every page; optional plain text, raw HTML, an HTML file and a screenshot in the key-value store, plus canonical URL, keywords, language, Open Graph, JSON-LD and response headers.
- **Flexible URL input.** Fetch one URL, a list of start URLs, or another actor's output dataset (link field from every item); optionally crawl same-site links, bounded by a page cap and glob patterns.
- **Built for agent pipelines.** Chain it after a search actor from the Integrations tab (it picks up that run's dataset automatically) or call the persistent Standby HTTP endpoint for real-time, low-latency fetches from your own backend or agent framework.
- **Fails loudly, not quietly.** A page that cannot be fetched within its time budget is reported as a failure in the run's status and log, never as a dataset item with empty content.

### Use Cases

- **AI agent and RAG builders:** convert any page into LLM-ready Markdown as live context for an agent, chatbot or retrieval index.
- **Search-augmented apps:** chain this actor after a web search actor so every result link is fetched and converted automatically, without writing glue code.
- **Research and content teams:** pull readable text from articles, docs and product pages without manually stripping ads and navigation.
- **Backend and automation builders:** call the Standby HTTP endpoint directly from your own service for on-demand, real-time page fetches.

### Data You Get

> Sample shape: values are illustrative placeholders, not from a live record.

| Field | Example |
| --- | --- |
| `url` | `"https://example.com/articles/global-markets-update"` |
| `markdown` (page content) | `"# Global Markets Update\n\nA summary of today's session..."` |
| `markdown` (headings) | `"## Today's Highlights"` |
| `markdown` (paragraphs) | `"Plain paragraph text carried over from the page body."` |
| `markdown` (lists) | `"- First point\n- Second point"` |
| `markdown` (links) | `"[Read the full report](#)"` |
| `markdown` (tables) | `"\| Metric \| Value \|\n\| --- \| --- \|\n\| Sample \| 42 \|"` |
| `markdown` (images) | `"![Chart of quarterly results](#)"` |
| `markdown` (emphasis) | `"**bold** and *italic* text kept as written"` |
| `markdown` (blockquotes and code) | `"> A quoted passage"` and `` `inline code` `` |
| `markdown` (noise removed) | scripts, navigation, ads, forms and hidden elements never reach this field |
| `markdown` (empty string) | possible when a page loads but nothing readable is left after cleanup |
| `metadata` | `{ "title": "...", "description": "...", "author": "..." }` |
| `metadata.title` | `"Global Markets Update"` |
| `metadata.description` | `"A daily summary of major market moves."` |
| `metadata.author` | `"Jane Analyst"` |

`metadata.title`, `metadata.description` and `metadata.author` come back as empty strings, not missing keys, when the page does not publish that piece of information.

### How to Use

1. Choose a mode: fetch a single URL, or point at a dataset from another actor's run to fetch every URL listed in it.
2. Fill in the URL (or the dataset ID and the field name that holds the URL), and turn on a connection for pages that block plain requests.
3. Set Max URLs to cap a batch run, then click Start.
4. Read the Markdown and metadata from the dataset (Overview or Markdown Content view), the key-value store (single URL mode), or the API.

**Fetch a single URL:**

```json
{
  "url": "https://example.com/articles/global-markets-update"
}
```

**A JavaScript-heavy page, waiting for the network to settle:**

```json
{
  "url": "https://example.com/dashboard",
  "waitUntil": "networkidle"
}
```

**A page that blocks plain requests (residential group; availability depends on your Apify plan):**

```json
{
  "url": "https://example.com/product/4821",
  "waitUntil": "load",
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}
```

**Batch mode from another actor's dataset:**

```json
{
  "datasetId": "YOUR_DATASET_ID",
  "urlField": "link",
  "maxUrls": 25
}
```

#### Run it from your code

Python:

```python
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("abotapi/ai-fetch-python").call(
    run_input={"url": "https://example.com/articles/global-markets-update"}
)
for page in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(page["url"], page["metadata"]["title"])
```

JavaScript:

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });
const run = await client.actor('abotapi/ai-fetch-python').call({
  url: 'https://example.com/articles/global-markets-update',
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
```

Or connect it to Make, Zapier, n8n, Google Sheets or webhooks from the Integrations tab. Chained after another actor's run from the Integrations tab, this actor automatically receives that run's dataset ID, so you usually only need to set Url Field if the URLs sit in a field other than `link`.

#### How a run handles a slow or blocked page

Each URL gets its own 90 second budget, which also covers the optional plain HTTP request of `engine: http` (that request is capped at 15 seconds). The actor first renders the page in a full headless browser; if that attempt fails or times out, it retries the page in a second headless browser session, up to two times, as long as the 90 second budget allows. With Apify Proxy on, each retry uses a newly established proxy session. A dataset item is written only for a successful fetch, so a failed URL never creates a billable dataset row, it is reported through the run's status message and log instead. Max URLs (1 to 100) caps how many links a batch run reads from the dataset; it has no effect in single URL mode.

### Send results into your apps (MCP connectors)

Optionally pipe results into the apps you already use through Model Context Protocol (MCP) connectors. This is an extra delivery step **after** the fetch: the Apify dataset is never changed and no extra charge is made.

**What gets written to the connector:** a condensed summary of each fetched page (title, URL, description and the first part of the markdown), not the full record. The complete record always stays in the Apify dataset.

1. Authorize a connector once under **Apify -> Settings -> Integrations** (Notion, Linear, Airtable, or Apify).
2. Select it in the **`mcpConnectors`** field. (If the picker is empty, you have not authorized a connector yet.)
3. For **Notion**, also set **`notionParentPageUrl`** to the page where items should be created.
4. **`maxNotifyListings`** (default 50) caps how many pages are written to each connector per run.

The connection is mediated by Apify's MCP proxy, so this actor never sees your third-party credentials. Leave the field empty to skip.

### Input Parameters

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `url` | string | `https://www.google.com` | The webpage to fetch (single URL mode). If `startUrls` is left at its two built-in examples, only this URL is fetched (1 page, 1 billed row); if you edit `startUrls`, this URL and the list are fetched together (duplicates removed). Clear this field to fetch the list alone. Exception: the default value https://www.google.com is not added to an edited list; to fetch google.com together with your own list, put it in the Start URLs list. |
| `datasetId` | string | (none) | Dataset ID to pull URLs from (batch mode); leave empty for single URL mode. |
| `urlField` | string | `link` | Field in each dataset item that holds the URL. |
| `maxUrls` | integer | `10` | Maximum URLs to fetch from the dataset, from 1 to 100. |
| `proxy` | object | (none) | Optional connection settings for the fetch; enable Apify Proxy with a residential group for pages that block plain requests. |
| `waitUntil` | string | `domcontentloaded` | Navigation wait condition: `load`, `domcontentloaded`, `networkidle`, or `commit`. |
| `debug` | boolean | `false` | Adds a `debug` object (fetch seconds, HTML length, engine, links found) to each row. |
| `startUrls` | array | two sample pages (`https://www.example.com` and a Wikipedia article) | Several URLs in one run (used when no dataset is given). Left at the two built-in examples, only `url` is fetched: an untouched Console run and an empty API call both fetch one page (the `url` value) and bill one row. If you edit the list, `url` and your list are fetched together (duplicates removed); clear `url` to fetch the list alone. Capped by `maxCrawlPages`. Exception: the default value https://www.google.com is not added to an edited list; to fetch google.com together with your own list, put it in the Start URLs list. |
| `maxCrawlDepth` | integer | `0` | `0` fetches only the given URLs. 1-10 also follows same-site links that many levels deep; links that return an HTTP error, are not HTML (for example `.ttl` or `.pdf`), or point to an already fetched address (trailing slash, redirect target) are not pushed or billed. Not used with `datasetId`: dataset mode fetches only the dataset's links. |
| `maxCrawlPages` | integer | `10` | Upper bound on pages in a multi-URL or crawl run, 1-1000. |
| `includeUrlGlobs` | array | (none) | Follow only links matching one of these glob patterns (not used with `datasetId`). |
| `excludeUrlGlobs` | array | (none) | Skip links matching any of these glob patterns (not used with `datasetId`). |
| `removeElementsCssSelector` | string | (none) | Drop matching elements before the Markdown and text conversion. |
| `keepElementsCssSelector` | string | (none) | Keep only matching elements for the Markdown and text conversion. |
| `waitForSelector` | string | (none) | Wait up to 15 seconds for this selector. Uses the standard browser path instead of the stealth first path. |
| `customHttpHeaders` | object | `{}` | Extra request headers. Uses the standard browser path instead of the stealth first path. |
| `initialCookies` | array | `[]` | Cookies set before loading. Uses the standard browser path instead of the stealth first path. |
| `engine` | string | `auto` | `auto` (browser with automatic fallback, the existing behaviour), `http` (one plain HTTP request first, then `auto`; skipped when screenshots, headers, cookies or a wait selector are set), `browser`. |
| `saveHtml` | boolean | `false` | Puts the page HTML in the `html` field. |
| `saveHtmlAsFile` | boolean | `false` | Stores the HTML in the key-value store and returns its link in `htmlUrl`. |
| `saveScreenshots` | boolean | `false` | Stores a screenshot in the key-value store and returns its link in `screenshotUrl`. Uses the standard browser path instead of the stealth first path. |
| `mcpConnectors` | array | (none) | Optional MCP connectors to send a summary of each page to. |
| `notionParentPageUrl` | string | (none) | Notion parent page (Notion connector only). |
| `maxNotifyListings` | integer | `50` | Cap on pages written to each connector per run (1 to 1000). |

Each row also carries `crawl` (`loadedUrl`, `loadedTime`, `referrerUrl`, `depth`, `httpStatusCode`, `contentLengthBytes`, `contentType`), a plain `text` field, and inside `metadata`: `canonicalUrl`, `keywords`, `languageCode`, `openGraph`, `jsonLd` and `headers`. `html`, `htmlUrl` and `screenshotUrl` appear only when you turn on the matching option. Runs with several URLs or a crawl also write a `RUN_SUMMARY` record to the key-value store (requested, succeeded and failed counts, and per-URL status).

**Dataset views:** Overview (URL, title, description), Markdown Content, Crawl info (loaded URL, status, content type, depth, load time) and Metadata & SEO (canonical URL, language, keywords, Open Graph).

**Multi-URL and crawl example:**

```json
{
  "startUrls": [{ "url": "https://example.com/docs" }],
  "maxCrawlDepth": 2,
  "maxCrawlPages": 50,
  "includeUrlGlobs": ["https://example.com/docs/**"]
}
```

A crawl only ever pushes pages that loaded as HTML: a followed link that returns an HTTP error (404 and similar) or a non-HTML file is logged, counted as failed in `RUN_SUMMARY` and never billed. Start URLs are returned as requested, with their real `crawl.httpStatusCode`. A crawl stops starting new pages after about 50 minutes, keeping what it already fetched. There is no AI-written summary field: use `markdown` or `text` with your own model.

### Output Example

> Sample shape: values are illustrative placeholders, not from a live record.

```json
{
  "url": "https://example.com/articles/global-markets-update",
  "markdown": "# Global Markets Update\n\nA summary of today's session, including sector moves and a look ahead.\n\n## Today's Highlights\n\n- First point\n- Second point\n\nRead the [full report](https://example.com/reports/2026-q3).",
  "metadata": {
    "title": "Global Markets Update",
    "description": "A daily summary of major market moves.",
    "author": "Jane Analyst"
  }
}
```

### Plan Requirement

By default the actor fetches over a direct connection, which works for publicly reachable pages. Apify Proxy (automatic datacenter group and residential) was verified to load public pages such as example.com and google.com; acceptance on a specific protected site depends on that site, so this actor cannot promise it. If you pick a residential group and your Apify plan does not include it, the actor logs: "The selected connection is unavailable to this run. Apify free plans do not include residential proxies. Continuing on a backup connection; some pages may still be unavailable." and then reads pages over a backup connection ("Served over the backup connection (the default connection was unavailable)."). If every connection is refused the run fails with "The proxy connection was refused for this account, so no page could be read. Your Apify plan does not include proxy access; a paid plan is required", and if the backup also cannot load a page on a free plan the message is "No results: residential proxies are unavailable on this run (Apify free plans do not include them), and the backup connection could not load the requested web pages." When no page loads at all you get "Could not load any results page (the requests failed), so nothing was collected. This is a connection problem, not an empty search." Standby mode needs enough concurrent capacity for the requests you expect; adjust memory under the actor's Standby settings for sustained real-time traffic.

### FAQ

#### How much does it cost?

You pay for what your run uses; the **Pricing** tab on the Store page shows the current rate. Batch runs are capped by **Max URLs** (single and batch runs bill one dataset-item charge per row pushed), and a failed fetch never creates a dataset item, so a blocked or slow page does not add a billable row. Crawl and multi-URL runs bill exactly the same way: one existing dataset-item charge per row actually pushed, bounded by **Max pages**, and nothing for pages that failed. Saved HTML files and screenshots are stored in your run's key-value store.

#### Is it legal to fetch a page with this actor?

This actor renders whatever page you point it at, the same way a browser would; it does not target one site's data specifically. You are responsible for how you use the fetched content: check the target site's terms and the laws that apply to you, especially before republishing paywalled or personal content.

#### Can I run it on a schedule?

Yes, schedule it from the **Schedules** tab like any other actor. Every run is independent: it always (re)fetches the URLs or dataset you give it, there is no built-in change tracking or incremental memory, so add your own de-duplication if you only want new pages.

#### Why is the Markdown mostly empty or missing text I can see in my browser?

The default wait condition stops as soon as the initial page loads, so content a page injects later (infinite scroll, chat widgets, some single-page apps) may not be present yet; switch **Wait Until** to `networkidle` for pages like that. Scripts, navigation, ads, forms and hidden elements are also deliberately stripped before conversion, so a navigation-heavy page will look sparse in Markdown by design.

#### Why did my run fail instead of returning an empty dataset?

If a page cannot be fetched within its time budget, through both the primary and fallback browser attempts, that URL fails on its own without writing anything to the dataset, and the run's status message reports how many URLs succeeded and failed. If every URL in a run fails, the whole run fails rather than finishing with an empty dataset, so "nothing could be read" is never confused with "no data on this page". One exception: in batch mode, if no dataset item has a value in the Url Field you set, there is nothing to fetch, so the run finishes with an empty dataset and a warning in the log; check that Url Field matches your dataset.

#### Can I use it with AI agents or MCP?

Yes. Call it like any other Apify actor from your agent framework or MCP client through the API or client libraries, or enable **Standby mode** and call its `/fetch` endpoint directly (GET or POST, with `url` and optional `waitUntil`) for real-time, low-latency fetches without starting a new run each time. Standby requests use a direct connection; for pages that need a proxy, start a regular run instead.

### 🔗 Want more AI data?

Pair this actor with these related scrapers from the same team:

<table>
<tr><td>🤖 <a href="https://apify.com/abotapi/web-scraper-for-llms"><b>Web Scraper For Llms</b></a><br>Stealth web scraping engine built for LLMs. Converts any web page to clean markdown or...</td><td>🏨 <a href="https://apify.com/abotapi/doc-to-markdown-mcp"><b>Doc To Markdown MCP Server</b></a><br>An MCP server that converts documents to clean Markdown. Convert PDFs, Word docs, Excel...</td></tr>
<tr><td>🤖 <a href="https://apify.com/abotapi/ai-web-search-tool"><b>AI Search Tool</b></a><br>Give your AI agents real-world knowledge. This Actor provides high-quality web, news...</td><td>🤖 <a href="https://apify.com/abotapi/doc-to-markdown"><b>Doc To Markdown</b></a><br>Convert documents (PDF, Word, PowerPoint, Excel, HTML, images) to clean Markdown...</td></tr>
<tr><td>✈️ <a href="https://apify.com/abotapi/fuel-prices-anz-scraper"><b>Fuel Prices AU Scraper</b></a><br>Scrapes petrol and fuel prices data across Australia and New Zealand. Search by address...</td><td>📱 <a href="https://apify.com/abotapi/universal-media-extractor"><b>Universal Media Extractor</b></a><br>Extract videos, audio, and metadata from 1000+ websites including YouTube, TikTok...</td></tr>
</table>

👉 [Browse all abotapi scrapers](https://apify.com/abotapi)

### 💬 Support

- 🐞 **Found a bug or a missing field?** Open a ticket on the [Issues tab](https://apify.com/abotapi/ai-fetch-python/issues/open). We usually reply within hours.
- 🛠️ **Need another site, extra fields or a private build?** Email <contact@abotapi.com>.
- ⭐ **Enjoying it?** A quick review on the actor page helps other users find it.

# Actor input Schema

## `url` (type: `string`):

The URL of the webpage to fetch (single URL mode). If the Start URLs list is left at its two built-in examples, only this URL is fetched (1 page, 1 result row); if you edit the list, this URL and the list are fetched together (duplicates removed). Clear this field to fetch the list alone. Exception: the default value https://www.google.com is not added to an edited list; to fetch google.com together with your own list, put it in the Start URLs list.

## `startUrls` (type: `array`):

Fetch several URLs in one run (used when no Dataset ID is given). Each page becomes one result row. Capped by Max pages. The form ships with two examples (a plain static page and a long article). If this list is left at those two examples, only the URL field above is fetched (1 page, 1 result row); if you edit the list, the URL field and your list are fetched together (duplicates removed). Clear the URL field to fetch just the two examples or your own list. Exception: the default value https://www.google.com is not added to an edited list; to fetch google.com together with your own list, put it in the Start URLs list.

## `datasetId` (type: `string`):

Dataset ID containing URLs to fetch. Used when integrating with search actors. Leave empty for single URL mode.

## `urlField` (type: `string`):

Field name in dataset items containing the URL (default: 'link')

## `maxUrls` (type: `integer`):

Maximum number of URLs to fetch from dataset (1-100)

## `maxCrawlDepth` (type: `integer`):

0 (default) fetches only the given URLs. 1-10 also follows same-site links found on each page, that many levels deep. Every fetched page is one billed result row; failed pages (HTTP errors, non-HTML files) are skipped and never billed. (Not used with Dataset ID: dataset mode fetches only the dataset's links.)

## `maxCrawlPages` (type: `integer`):

Upper bound on pages fetched in a multi-URL or crawl run (1-1000).

## `includeUrlGlobs` (type: `array`):

Crawl only links matching one of these glob patterns, e.g. https://example.com/blog/\* (Not used with Dataset ID: dataset mode fetches only the dataset's links.)

## `excludeUrlGlobs` (type: `array`):

Skip links matching any of these glob patterns. (Not used with Dataset ID: dataset mode fetches only the dataset's links.)

## `waitForSelector` (type: `string`):

CSS selector to wait for (up to 15 s) before reading the page. Uses the standard browser path instead of the stealth first path.

## `customHttpHeaders` (type: `object`):

Extra request headers as a JSON object, e.g. {"Accept-Language": "de"}. Uses the standard browser path instead of the stealth first path.

## `initialCookies` (type: `array`):

Cookies set before loading, e.g. \[{"name":"a","value":"b"}] (optional domain, path). Uses the standard browser path instead of the stealth first path.

## `engine` (type: `string`):

auto (default) is the browser-based fetch with automatic fallback. http tries one plain HTTP request first (cheaper, no JavaScript) and falls back to auto. browser skips the first browser path and uses the standard browser directly. The plain HTTP step is skipped when screenshots, custom headers, cookies or a wait selector are set.

## `removeElementsCssSelector` (type: `string`):

CSS selector of elements to drop from the page before the Markdown/text conversion (e.g. nav, .sidebar). Empty keeps the built-in cleanup only.

## `keepElementsCssSelector` (type: `string`):

If set, only the matching elements are converted to Markdown/text.

## `saveHtml` (type: `boolean`):

Adds the full page HTML to the html field of each row (larger dataset).

## `saveHtmlAsFile` (type: `boolean`):

Stores each page HTML as a file in the key-value store and puts its link in htmlUrl.

## `saveScreenshots` (type: `boolean`):

Takes a screenshot per page, stores it in the key-value store and puts its link in screenshotUrl. Uses the standard browser path instead of the stealth first path.

## `waitUntil` (type: `string`):

When to consider navigation succeeded.

## `debug` (type: `boolean`):

Adds a debug object (fetch seconds, HTML length, engine, links found) to each row.

## `proxy` (type: `object`):

Optional proxy settings. Enable Apify Proxy with RESIDENTIAL group for best results on protected sites.

## `mcpConnectors` (type: `array`):

Optionally send the scraped results into the apps you already use, via Model Context Protocol (MCP) connectors. Authorize a connector once under Apify -> Settings -> Integrations, then select it here. The connector receives a condensed, human-readable summary per item (title + key fields), not the full JSON; the complete record stays in the dataset. Leave empty to skip. Supported: Notion (https://mcp.notion.com/mcp), Linear (https://mcp.linear.app/sse), Airtable (https://mcp.airtable.com/mcp), Apify (https://mcp.apify.com).

## `notionParentPageUrl` (type: `string`):

URL (or id) of the Notion page under which item pages are created. Required to enable the Notion export; ignored by other connectors.

## `maxNotifyListings` (type: `integer`):

Cap on items written to each connector per run. Does not affect the dataset.

## Actor input object example

```json
{
  "url": "https://www.example.com",
  "startUrls": [
    {
      "url": "https://www.example.com"
    },
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    }
  ],
  "urlField": "link",
  "maxUrls": 10,
  "maxCrawlDepth": 0,
  "maxCrawlPages": 10,
  "engine": "auto",
  "saveHtml": false,
  "saveHtmlAsFile": false,
  "saveScreenshots": false,
  "waitUntil": "domcontentloaded",
  "debug": false,
  "maxNotifyListings": 50
}
```

# Actor output Schema

## `overview` (type: `string`):

No description

## `markdown` (type: `string`):

No description

## `crawl` (type: `string`):

No description

## `seo` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "url": "https://www.example.com",
    "startUrls": [
        {
            "url": "https://www.example.com"
        },
        {
            "url": "https://en.wikipedia.org/wiki/Web_scraping"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("abotapi/ai-fetch-python").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "url": "https://www.example.com",
    "startUrls": [
        { "url": "https://www.example.com" },
        { "url": "https://en.wikipedia.org/wiki/Web_scraping" },
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("abotapi/ai-fetch-python").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "url": "https://www.example.com",
  "startUrls": [
    {
      "url": "https://www.example.com"
    },
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    }
  ]
}' |
apify call abotapi/ai-fetch-python --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,abotapi/ai-fetch-python"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/nqNyER2eY7MNekzd4/builds/Ns9FPWx6lB07TlzlR/openapi.json