# RSS Feed Scraper — Atom, Podcast & Multi-Feed Aggregator (`devilscrapes/rss-feed-scraper`) Actor

Parse and convert any RSS or Atom feed to a clean dataset — title, link, author, published date, summary, full HTML content, tags, GUID — export to JSON or CSV. A drop-in RSS feed parser for RSS 2.0, Atom 1.0, and the content:encoded / dc:creator extensions. Also a scheduled news feed monitor.

- **URL**: https://apify.com/devilscrapes/rss-feed-scraper.md
- **Developed by:** [DevilScrapes](https://apify.com/devilscrapes) (community)
- **Categories:** Automation, News
- **Features:** Limited permissions
- **Stats:** 16 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.88 / 1,000 result emitteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

<div align="center">
  <img src=".actor/icon.svg" width="160" alt="Devil Scrapes mark" />

## RSS Feed Scraper

**💰 $1.00 / 1 000 results**  ·  pay only for results  ·  no credit card to try

*We do the dirty work so your dataset stays clean.* 😈

Parse any RSS or Atom feed into a clean structured dataset — title, link, author, published date, summary, full content (HTML), tags, GUID. Handles RSS 2.0, Atom 1.0, and the common `content:encoded` / `dc:creator` extensions.

</div>

***

### 🎯 What this scrapes

RSS and Atom are still the most reliable way to subscribe to a publication. This Actor parses any feed URL — news site, blog, podcast, GitHub release feed, Reddit, Substack, Medium per-user — and writes one row per item. Output is normalised across RSS and Atom dialects so downstream code never needs to care which format it received.

Feed sources that work out of the box:

- News publishers (New York Times, BBC, Reuters — any site that vends an RSS endpoint)
- Podcast directories and individual show feeds (`<enclosure>` tags parsed automatically)
- GitHub release and commit feeds
- Reddit `.rss` and `.json` community feeds
- Substack and Medium per-author feeds
- Google Alerts export feeds
- Any custom-built Atom or RSS 2.0 / RSS 1.0 feed

### 🔥 Features

- 🛡️ **Browser fingerprint rotation** — `curl-cffi` impersonates real Chrome / Firefox / Safari TLS handshakes so the target sees a browser, not a Python script.
- 🌐 **Residential proxy rotation** via Apify Proxy — fresh session and exit IP on every block or 429 response.
- 🔁 **Retries with exponential backoff** on `408 / 429 / 5xx` — up to 5 attempts per feed, `Retry-After` header honoured.
- 🧱 **Rate-limit-aware pacing** — when a feed host pushes back, we slow down and surface exactly what was collected before the limit hit.
- 🧊 **Clean, typed dataset rows** — Pydantic-validated, ISO-8601 timestamps, stable IDs, exportable as JSON / CSV / Excel straight from Apify Console.
- 💰 **Pay-Per-Event pricing** — you pay only for results that land in your dataset. No data, no charge (beyond the small actor-start fee).
- 📡 **Multi-feed batching** — pass a list of URLs; the Actor fetches and normalises them all in one run, deduplicating by GUID.
- 📝 **Full HTML content** — when a feed publishes `content:encoded` or Atom `content`, we capture the full body alongside the summary, not just a truncated snippet.

### 💡 Use cases

- **News aggregation dashboard** — pull 20 publications into one stream and pipe to Slack, Discord, or a webhook.
- **Brand monitoring** — track every Google Alerts RSS feed for your company name, product, or competitors.
- **Content automation** — feed company-blog RSS into a translation pipeline, summary LLM, or newsletter tool.
- **Podcast RSS parser** — podcast RSS is standard RSS with `<enclosure>` tags; this Actor surfaces the episode link, title, author, and published date for every episode in the feed.
- **LLM-ready news digest** — pass structured rows straight to an LLM pipeline; ISO-8601 timestamps and clean HTML make chunking predictable.
- **RSS-to-Google-Sheets / Notion / Airtable** — export via Apify's native integration or the API; no glue code required.
- **Feed archival** — schedule the Actor daily to build a rolling archive of feeds that don't publish full history.

### ⚙️ How to use it

1. Click **Try for free** at the top of the page.
2. Paste one or more RSS / Atom feed URLs into the `feedUrls` field — one per line.
3. Optionally set `maxItemsPerFeed` and toggle `includeContent` for full HTML.
4. Click **Start**. Output streams into the run's dataset in real time.
5. Export from **Storage → Dataset** as JSON, CSV, or Excel — or call the Apify API from your own code.

For scheduled runs, use Apify Schedules (cron syntax) so the Actor refreshes your dataset on your preferred cadence.

### 📥 Input

| Field | Type | Required | Default | Notes |
|---|---|:--:|---|---|
| `feedUrls` | `array` | **yes** | `["https://news.ycombinator.com/rss"]` | List of RSS / Atom feed URLs. One URL per item. Each URL produces one or more dataset rows. |
| `maxItemsPerFeed` | `integer` | no | `50` | Cap on items pulled from a single feed. Set to `0` for no limit. |
| `includeContent` | `boolean` | no | `true` | When `true`, includes the full HTML body (`content:encoded` / Atom `content`). When `false`, summary only. |
| `userAgent` | `string` | no | `"DevilScrapesBot/1.0 (+https://apify.com/DevilScrapes)"` | Custom User-Agent string. Default identifies as Devil Scrapes RSS reader. |
| `proxyConfiguration` | `object` | no | `{"useApifyProxy": false}` | Some publishers rate-limit scrapers. Apify Proxy provides sticky sessions and IP rotation when needed. |

#### Example input

```json
{
  "feedUrls": [
    "https://news.ycombinator.com/rss",
    "https://feeds.arstechnica.com/arstechnica/index"
  ],
  "maxItemsPerFeed": 25,
  "includeContent": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

### 📤 Output

Every row is one feed item. All fields follow Pydantic validation — no nulls where a value existed, no phantom fields.

| Field | Type | Notes |
|---|---|---|
| `feed_url` | `string` | Source feed URL passed in `feedUrls`. |
| `feed_title` | `string \| null` | Feed channel title. |
| `feed_format` | `string` | `"rss"` or `"atom"`. |
| `item_id` | `string \| null` | Item GUID (RSS) or `id` (Atom). Used for deduplication. |
| `title` | `string` | Item headline. |
| `link` | `string` | Item permalink URL. |
| `author` | `string \| null` | Author from `dc:creator` or Atom `author/name`. |
| `summary` | `string \| null` | Short description / `atom:summary`. |
| `content_html` | `string \| null` | Full HTML body when the feed includes it. |
| `categories` | `array` | Item tags / categories (empty array if none). |
| `published` | `string \| null` | Publish timestamp in ISO-8601 format. |
| `updated` | `string \| null` | Updated timestamp in ISO-8601 format. |
| `scraped_at` | `string` | ISO-8601 timestamp for when this row was recorded. |

#### Example output

```json
{
  "feed_url": "https://news.ycombinator.com/rss",
  "feed_title": "Hacker News",
  "feed_format": "rss",
  "item_id": "https://news.ycombinator.com/item?id=48000000",
  "title": "Show HN: Building a hosted RSS parser for the post-LLM web",
  "link": "https://news.ycombinator.com/item?id=48000000",
  "author": null,
  "summary": "A discussion about ...",
  "content_html": null,
  "categories": [],
  "published": "2026-05-15T20:00:00+00:00",
  "updated": null,
  "scraped_at": "2026-06-01T09:00:00+00:00"
}
```

### 💰 Pricing

Pay-Per-Event — you pay only when these events fire:

| Event | USD | What it is |
|---|---:|---|
| `actor-start` | $0.005 | One-off warm-up charge per run |
| `result` | $0.001 | Per dataset item written |

**Example:** 1 000 items at the rates above ≈ **$1.00**.

No subscription, no monthly minimum, no card to start — Apify gives every new account $5 of free credit, which covers your first 5 000 rows.

### 🚧 Limitations

- **Paginated feeds** — we don't follow `<link rel="next">` paged feeds automatically. Pass each page URL explicitly if you need full history.
- **JavaScript-rendered feeds** — feeds that require JavaScript to load are not supported. You would need a browser-based Actor for those.
- **Malformed XML** — `feedparser` is lenient and handles most broken XML, but severely corrupted feeds may yield partial or empty results. The run surfaces a warning, not a silent empty dataset.
- **Rate-limiting by feed hosts** — heavily scraped feeds (e.g. Reddit) may enforce per-IP rate limits. Enable Apify Proxy in `proxyConfiguration` to rotate IPs.

### ❓ FAQ

**Is this the same as an rss parser api?**

Functionally, yes — you call it via the Apify API (or the Console UI), pass feed URLs, and get back structured JSON. The difference is that we handle the messy parts a bare HTTP client doesn't: malformed XML, charset detection, multi-dialect normalisation, and the network-level blocks that make your home-rolled parser fail on 1 in 20 feeds.

**Does this handle podcasts?**

Yes — podcast RSS is standard RSS with `<enclosure>` tags. This Actor is a capable podcast RSS parser: the enclosure URL (the audio file) appears in the `link` field for each episode row, alongside the episode title, author, and published date.

**What about atom feed parser support?**

Full Atom 1.0 support is built in. The `feed_format` field tells you which dialect was parsed. Both RSS and Atom rows share the same output schema, so your downstream code needs no format-specific logic.

**Why is `content_html` empty for some feeds?**

Some publishers deliberately publish summary-only feeds to drive clicks to their site. The full body lives on the publisher's page, not in the feed XML. We surface what the feed provides — no fabrication.

**What if a feed URL returns an error?**

The Actor logs the failure with the HTTP status code, marks that feed as errored in the status message, and continues processing the remaining URLs. You never get a silent empty dataset — partial success is surfaced explicitly.

**Can I run this on a schedule?**

Yes. Use Apify Schedules to trigger a run on any cron cadence. Pair it with a named dataset to accumulate a rolling archive without overwriting previous results.

**Does it deduplicate items across runs?**

Within a single run, items are deduplicated by GUID / Atom `id`. Across runs, deduplication is your responsibility — filter by `item_id` in your downstream pipeline or use a named dataset with upsert logic.

### 💬 Your feedback

Spotted a bug, hit a weird edge case, or need a new field? Open an issue on the Actor's **Issues** tab on Apify Console — we ship fixes weekly and we read every report.

***

<div align="center">

Built by **[Devil Scrapes](https://apify.com/DevilScrapes)** 😈 — a small fleet of
opinionated public-data Actors. Honest pricing, real engineering, zero fine print.

</div>

# Changelog

This Actor's version history is a separate document: https://apify.com/devilscrapes/rss-feed-scraper/changelog.md

# Actor input Schema

## `feedUrls` (type: `array`):

List of RSS / Atom feed URLs. One URL per line. Each becomes one or more dataset rows.

## `maxItemsPerFeed` (type: `integer`):

Cap on items pulled from a single feed.

## `includeContent` (type: `boolean`):

When true, include the full HTML body (RSS `content:encoded` / Atom `content`). When false, only the summary.

## `userAgent` (type: `string`):

Custom UA string. Default identifies as Devil Scrapes RSS reader.

## `proxyConfiguration` (type: `object`):

Some publishers gate scrapers. Apify Proxy helps with sticky sessions.

## Actor input object example

```json
{
  "feedUrls": [
    "https://news.ycombinator.com/rss"
  ],
  "maxItemsPerFeed": 50,
  "includeContent": true,
  "userAgent": "DevilScrapesBot/1.0 (+https://apify.com/DevilScrapes)",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `datasetItems` (type: `string`):

All dataset items as JSON.

## `datasetItemsCsv` (type: `string`):

Same data exported to CSV.

## `datasetView` (type: `string`):

Open the run dataset in the Console.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "feedUrls": [
        "https://news.ycombinator.com/rss"
    ],
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("devilscrapes/rss-feed-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "feedUrls": ["https://news.ycombinator.com/rss"],
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("devilscrapes/rss-feed-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "feedUrls": [
    "https://news.ycombinator.com/rss"
  ],
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call devilscrapes/rss-feed-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,devilscrapes/rss-feed-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/VLscPdreqHPfEqKbj/builds/NdNb9dYtgx1HKPokF/openapi.json