# Article Text Extractor | News and Blog API (`xtech/article-extractor`) Actor

Extract readable text and available metadata from public news articles and blog posts. Export titles, authors, dates, images, summaries, and keywords for content research and downstream analysis. Start with a known article URL and review a small sample before processing a larger list.

- **URL**: https://apify.com/xtech/article-extractor.md
- **Developed by:** [Xtech](https://apify.com/xtech) (community)
- **Categories:** News, SEO tools, Jobs
- **Features:** Limited permissions
- **Stats:** 93 total users, 4 monthly users, 81.1% runs succeeded, 4 bookmarks
- **User rating**: 1.00 out of 5 stars

## Pricing

from $20.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Article Text Extractor | News and Blog API

Collect readable article text and available metadata from public news and blog URLs. Export titles, authors, dates, images, summaries and keywords for research or downstream processing. Metadata and summaries can be missing, and protected pages may not be extractable. Start with the published one-article example, check the output on your target sites, and then increase the URL list.

**Best first run:** [open the Article Text Extractor](https://apify.com/xtech/article-extractor), run the content-research example with one URL and a $0.03 cap, then add more URLs only after checking the returned text and metadata.

### Start with one article

Open the public content-research example to extract one known blog article with a $0.03 run cap. Review the text and available metadata before adding more URLs. The current base rate is $0.02 per successful article ($20 per 1,000), plus $0.00005 per start event per GB of memory (minimum one). The example uses 2 GB, so one successful article plus start events is $0.0201 at this rate. Failed extraction diagnostics are stored separately and do not create a paid article result. Check the Pricing tab for current rates.

For API clients, use the same bounded first run and pass the charge cap as a run parameter:

```bash
curl -X POST "https://api.apify.com/v2/acts/xtech~article-extractor/runs?token=$APIFY_TOKEN&maxTotalChargeUsd=0.03" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":[{"url":"https://blog.apify.com/what-is-web-scraping/"}],"language":"en","requestTimeout":30,"maxConcurrency":1,"maxRetries":1,"fetchImages":false}'
```

### Features

- Extract full article text from article and blog post pages
- Return titles, authors, publication dates, source URLs, and language
- Collect summaries, keywords, meta descriptions, and meta keywords when available
- Capture the main image URL, discovered image URLs, and embedded video URLs when supported by the extractor
- Process multiple article URLs in one Actor run
- Select article language, request timeout, and image metadata options
- Save successfully extracted articles in the default dataset and failed-attempt diagnostics separately in `errorResults`

### Input

Provide one or more article URLs in `startUrls`.

```json
{
  "startUrls": [
    { "url": "https://blog.apify.com/what-is-web-scraping/" }
  ],
  "language": "en",
  "requestTimeout": 30,
  "fetchImages": false
}
```

#### Input fields

| Field | Type | Description |
| --- | --- | --- |
| `startUrls` | array | Direct article or blog post URLs to extract. |
| `language` | string | Optional two-letter language hint such as `en`, `es`, `de`, or `fr`. Leave empty for automatic detection where possible. |
| `requestTimeout` | integer | Maximum time in seconds to wait for each article page. Defaults to `30`. |
| `fetchImages` | boolean | Include available article image URLs. The Actor stores image URLs, not image files. |

The form manages concurrency, retries, User-Agent, and proxy routing automatically. Existing API clients can still supply `maxConcurrency`, `maxRetries`, `browserUserAgent`, and `proxyConfiguration`; their defaults and explicit proxy choices remain supported.

### Output

The Actor returns a JSON dataset with the following fields for each article:

| Field | Description |
| --- | --- |
| `articleURL` | Input or resolved article URL. |
| `sourceURL` | Source website URL when detected. |
| `articleLanguage` | Detected article language, for example `en` or `es`. |
| `articleTitle` | Article title. |
| `articleAuthors` | Article authors as an array. |
| `articlePublishDate` | Publication date when detected. |
| `articleText` | Clean extracted article text. |
| `articleTopImage` | Main article image URL when detected. |
| `articleAllImages` | Image URLs as an array when available. |
| `articleVideos` | Embedded video URLs as an array when available. |
| `articleKeywords` | Keywords or categories as an array when available. |
| `articleSummary` | Extracted or generated article summary. |
| `articleMetaDescription` | Page meta description when available. |
| `articleMetaKeywords` | Page meta keywords as an array when available. |
| `wordCount` | Number of words in the extracted article text. |
| `characterCount` | Number of characters in the extracted article text. |
| `scrapeSuccess` | `true` when extraction succeeded, otherwise `false`. |
| `scrapeErrorMessage` | Error details for failed extractions. |
| `scrapedAt` | UTC timestamp when the URL was processed. |

### Example Output

```json
[
  {
    "articleURL": "https://www.example.com/news/article1",
    "sourceURL": "https://www.example.com",
    "articleLanguage": "en",
    "articleTitle": "Example News Article",
    "articleAuthors": ["John Doe", "Jane Smith"],
    "articlePublishDate": "2024-07-27T10:00:00Z",
    "articleText": "This is the full text of the example news article...",
    "articleTopImage": "https://www.example.com/images/article1.jpg",
    "articleAllImages": ["https://www.example.com/images/article1.jpg", "https://www.example.com/images/article2.png"],
    "articleVideos": [],
    "articleKeywords": ["news", "example", "article"],
    "articleSummary": "A brief summary of the example news article.",
    "articleMetaDescription": "An example article for demonstration.",
    "articleMetaKeywords": ["example", "article", "news", "demo"],
    "wordCount": 825,
    "characterCount": 4920,
    "scrapeMethod": "newspaper3k",
    "scrapeSuccess": true,
    "scrapedAt": "2024-07-27T12:34:56Z"
  }
]
```

With current pay-per-event pricing, failed URLs go to the separate `errorResults` dataset with `articleURL`, `scrapeSuccess: false`, `scrapeErrorMessage`, and `scrapedAt`. If at least one URL succeeds, the run can finish **SUCCEEDED** with useful articles saved and failures reported separately. If every valid URL fails, the Actor marks the run **FAILED after writing diagnostic rows**. Each run also writes `RUN_SUMMARY` with requested, extracted, saved, failed, all-failed, billing-mode, and spending-limit fields.

Only successfully extracted article rows enter the billable default dataset. Failed URLs and spending-limit diagnostics in `errorResults` are not charged as article results. The run-start event still applies; check the Pricing tab for current rates.

On Apify cloud, the residential-proxy fallback applies only when no proxy configuration is supplied. Explicitly disabled proxies, selected Apify groups, and custom proxy URLs are respected.

### Use cases

- Monitor news articles, press mentions, and competitor content
- Build article datasets for research, analysis, or machine learning
- Enrich URLs with titles, authors, dates, summaries, and clean text
- Collect SEO metadata from article and blog pages
- Feed extracted article text into downstream AI, analytics, or database workflows

### Related Actors on Apify Store

Pair this Actor with others from the same author for full content pipelines:

- [YouTube Transcript Scraper Pro](https://apify.com/xtech/youtube-transcript-scraper-pro) — video transcripts for the same research workflow
- [News Source Crawler](https://apify.com/xtech/news-source-crawler) — discover article URLs from entire news sites, then extract with this Actor
- [RSS Feed Scraper](https://apify.com/xtech/feed-extractor) — monitor feeds and pass new article URLs here

### Tips

- Use direct article URLs instead of homepages, category pages, or search result pages.
- If all URLs fail because of access restrictions, enable Apify Proxy and retry.
- Increase `requestTimeout` for slow publishers or long-form pages.
- Some JavaScript-heavy or paywalled pages may return partial text or fail if the article content is not present in the initial HTML.

### Proxy settings

An explicit proxy choice is respected: disable proxies with `{"proxyConfiguration":{"useApifyProxy":false}}`, select the Apify groups available to your account, or supply your custom proxy URLs. If no proxy configuration is supplied on the cloud, the existing residential-proxy fallback applies. Start with a public page that works directly before using proxies for sites that need them. Proxy settings cannot guarantee complete extraction from protected or dynamic pages.

# Actor input Schema

## `startUrls` (type: `array`):

Article or blog post URLs to extract. Add one or more direct article pages.

## `language` (type: `string`):

Optional two-letter language code such as en, es, de, or fr. Leave empty to let extraction libraries infer the language when possible.

## `requestTimeout` (type: `integer`):

Maximum time in seconds to wait for each article page to respond.

## `maxConcurrency` (type: `integer`):

Number of article URLs to process in parallel. Keep the default for most publishers and lower it if a website rate-limits requests.

## `maxRetries` (type: `integer`):

Number of retries for transient request failures such as timeouts, rate limits, and temporary server errors.

## `fetchImages` (type: `boolean`):

Include article image URLs when the newspaper3k fallback is used. The Actor stores image URLs, not image files.

## `browserUserAgent` (type: `string`):

Custom browser User-Agent header for article requests.

## `proxyConfiguration` (type: `object`):

Optional. Leave empty to use the actor's included connection. Custom URLs you provide are used as given, and useApifyProxy: false connects directly. Apify proxy routing is not used.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://blog.apify.com/what-is-web-scraping/"
    }
  ],
  "language": "",
  "requestTimeout": 30,
  "maxConcurrency": 3,
  "maxRetries": 2,
  "fetchImages": true,
  "browserUserAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"
}
```

# Actor output Schema

## `results` (type: `string`):

Extracted article records stored in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://blog.apify.com/what-is-web-scraping/"
        }
    ],
    "language": "",
    "browserUserAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"
};

// Run the Actor and wait for it to finish
const run = await client.actor("xtech/article-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://blog.apify.com/what-is-web-scraping/" }],
    "language": "",
    "browserUserAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36",
}

# Run the Actor and wait for it to finish
run = client.actor("xtech/article-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://blog.apify.com/what-is-web-scraping/"
    }
  ],
  "language": "",
  "browserUserAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"
}' |
apify call xtech/article-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,xtech/article-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ENblUQ4nAlQBqj9xI/builds/Mn87eQGCbIfFjhfrx/openapi.json