Readable Article Extractor

Introduction

Readable Article Extractor provides fast extraction of clean readable article text and metadata from public web URLs. It fetches supplied pages, follows HTTP redirects safely, and extracts the primary content region into plain text and Markdown while capturing metadata such as title, byline, publication date, language, word count, and a SHA-256 digest. Crucially, the Actor does not crawl external links and does not execute JavaScript, ensuring fast, deterministic, and predictable processing.

Use cases

• LLM and RAG ingestion: Prepare clean article text and structured Markdown from public news articles and blogs for embedding and context retrieval.
• Content archiving: Maintain a plain text and Markdown archive with SHA-256 integrity digests for public documentation and articles you have rights to store; email addresses and phone numbers of 10 or more digits are redacted, and content_sha256 covers the redacted text.
• Reading analysis: Inspect article lengths and word counts to estimate reading time and assess textual density.

Tutorial

Step-by-step guide

1. Prepare a JSON input object containing the urls array with 1 to 180 public HTTP or HTTPS article URLs.
2. Run the Actor on the Apify platform or start a run using the Apify REST API or Python client.
3. The Actor checks each domain's robots.txt, fetches the HTML within safe size and redirect limits, and parses the main content.
4. Once execution completes, inspect or download the resulting dataset items in JSON, CSV, or Excel format.

Pricing

The Actor is billed under the Pay-Per-Event model using the article-extracted event.

Tier	USD per article-extracted event	Maximum charge per run (180 URLs)
FREE	$0.00299	$0.5382 USD
BRONZE	$0.002691	$0.48438 USD
SILVER	$0.002392	$0.43056 USD
GOLD	$0.001993	$0.35874 USD
PLATINUM	$0.001993	$0.35874 USD
DIAMOND	$0.001993	$0.35874 USD

For the largest allowed input of 180 URLs, the maximum bill per run is 180 events: $0.5382 USD on the FREE tier, $0.48438 USD on BRONZE, $0.43056 USD on SILVER, and $0.35874 USD on GOLD, PLATINUM, and DIAMOND.

Charge contract

Duplicate URLs inside a single input list are de-duplicated before fetching, keeping the first occurrence, and are charged only once. Each row is pushed first and charged right after a successful push; a refused charge stops work and that row stays delivered and uncharged; a reached limit stops work after that row was charged; failed items are neither pushed nor charged. A robots denial, fetch error, size limit breach, or empty extraction produces no dataset row and no charge event. Non-PPE runs are not billed per event. Rows already in the dataset of a restarted run are skipped and not charged again; a row pushed just before a crash is not charged on restart; if the dataset of a restarted run cannot be read, the run fails with DATASET_READ_FAILED before any new row or charge.

Operational limits

• Accepts 1 to 180 URLs per run; each URL must not exceed 2,048 characters. Each URL must use HTTP or HTTPS, contain a valid hostname, contain no credentials, contain no whitespace, and contain no URL fragment; one invalid URL rejects the whole input. Exact duplicate URLs in the input are de-duplicated before fetching, keeping only the first occurrence.
• Maximum response size is 1,000,000 bytes per page before and after decoding.
• Fetches timeout at 20 seconds with at most 5 redirect hops.
• Crawl-delay is supported up to 50 seconds; hosts with higher delays are skipped safely to avoid container timeouts.
• Container execution budget is bounded by 3,600 seconds (RUN_TIMEOUT_SECONDS). URLs left when the budget ends are not processed or charged, and the run still succeeds with their count in the status message.
• Up to 50,000 HTML nodes are parsed per page, and candidate main-content regions must contain at least 80 visible characters.
• Pages are decoded with the charset the server or the page declares, UTF-8 when none is declared, and a page that cannot be decoded that way is not pushed or charged.

Input and output examples

Input example

{
  "urls": [
    "https://example.com/news"
  ]
}

Output example

{
  "title": "Harbor bridge reopens",
  "main_text": "Harbor bridge reopens\n\nCity crews reopened the harbor bridge on Tuesday after finishing repairs that had closed two lanes for three weeks.\n\nOfficials said the work replaced worn joints and improved drainage. Drivers can use all lanes while crews monitor the structure during the next month.",
  "main_text_markdown": "# Harbor bridge reopens\n\nCity crews reopened the harbor bridge on Tuesday after finishing repairs that had closed two lanes for three weeks.\n\nOfficials said the work replaced worn joints and improved drainage. Drivers can use all lanes while crews monitor the structure during the next month.\n",
  "url": "https://example.com/news",
  "final_url": "https://example.com/news",
  "byline": "",
  "published_date": "2026-10-02T09:30:00Z",
  "language": "en",
  "word_count": 46,
  "content_sha256": "5f2f95ad2f8e535687fad32eb395a2d86f965a4bf2ac35956c59ba5589a366f4"
}

Output fields

Field	Type	Description
title	String	Extracted article title or page heading
main_text	String	Extracted plain text of the primary article body
main_text_markdown	String	Structured Markdown representation of the article
url	String	Original requested URL
final_url	String	Final destination URL after following HTTP redirects
byline	String	Extracted author or byline information when available
published_date	String	Publication date string as declared by the page itself (not always ISO 8601), except contact-like text in it is redacted
language	String	Detected or declared language code (e.g. en)
word_count	Integer	Total count of words in the extracted plain text
content_sha256	String	SHA-256 hash of the extracted plain text

Actor recommendations

For full content pipelines, pair this Actor with:

• Sitemap URL Extractor: Discover and feed relevant article URLs directly into this extractor.
• Inline HTML and Markdown to Clean Markdown: Normalize or reformat markdown extracts for downstream publishers.
• Web Page to Markdown Converter: For general web pages that require broader layout preservation rather than article body isolation.

FAQ and support

Does this Actor crawl links discovered within the article?

No. The Actor processes only the URLs explicitly provided in the input list.

Why might an article URL produce no dataset item?

An item is skipped if the URL is disallowed by robots.txt, the page exceeds the 1 MB limit, the destination is not public, or the page requires client-side JavaScript rendering to generate its content.

Are email addresses and phone numbers preserved in outputs?

Email addresses are replaced with [email removed]. Phone numbers with 10 or more digits are replaced with [phone removed]. Shorter local numbers, such as 555-1234 (7 digits) or 030 123456 (9 digits), can stay in the text.

Support and inquiries

For questions, bug reports, or feature suggestions, open an issue in the Apify Console.