# RSS Feed Scraper — $1/1K · Articles, Dates & Feed Updates (`apivault_labs/universal-rss-scraper`) Actor

Collect RSS, Atom and JSON Feed entries; normalize dates and filter delivered feed items. Bulk export: JSON, CSV, Excel and API. PPE: $1/1K feed items; start fee, usage extra.

- **URL**: https://apify.com/apivault_labs/universal-rss-scraper.md
- **Developed by:** [Apivault Labs](https://apify.com/apivault_labs) (community)
- **Categories:** Developer tools, News
- **Features:** Limited permissions
- **Stats:** 15 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 feed item extracteds

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## RSS Feed Scraper & Reader — RSS, Atom, RDF, JSON Feed & Sitemap to Clean JSON

**The universal RSS feed parser for AI agents and no-code automation.** Parse available RSS 2.0, Atom, RDF, [JSON Feed](https://www.jsonfeed.org/) or XML sitemap into clean, structured JSON — with title, link, author, date, content, categories, media and full article text. Built for LLM/RAG pipelines, news aggregation, podcast monitoring and n8n / Make / Zapier workflows.

Suitable for supported public feeds from blogs, news sites, podcasts and publishing platforms. No external API key is required for basic feed parsing; source coverage varies.

### Why this RSS scraper

- **5 feed formats, one actor** — RSS 2.0, Atom, RDF (RSS 1.0), JSON Feed and XML sitemaps, auto-detected.
- **Full article text for AI** — feeds usually give only a short excerpt. Enable `fullText` to fetch clean, LLM-ready markdown of the whole article (ads and boilerplate removed).
- **Feed auto-discovery** — give a plain site URL (e.g. `https://example.com`) and it finds the RSS/Atom/JSON feed for you.
- **Bulk multi-feed** — fetch hundreds of feeds in parallel, then deduplicate and merge them into one chronological stream.
- **Change monitoring (delta)** — return only items that are new since the last run. Perfect for scheduled monitoring.
- **Keyword & date filters** — keep only items matching your keywords or published in the last N days.
- **OPML import** — migrate hundreds of feeds from Feedly / Inoreader in one field.
- **Podcast-aware** — extracts iTunes tags (duration, episode, season, explicit) and enclosures (audio/MP3, images, attachments).
- **Datacenter proxy by default** — feed requests can use the configured Apify Proxy; source restrictions still apply.

### Supported formats

| Format | Example source |
|--------|----------------|
| RSS 2.0 | `https://techcrunch.com/feed/` |
| Atom | `https://xkcd.com/atom.xml` |
| RDF / RSS 1.0 | `https://rss.slashdot.org/Slashdot/slashdotMain` |
| JSON Feed | `https://daringfireball.net/feeds/json` |
| XML Sitemap | `https://example.com/sitemap.xml` |
| Site URL (auto-discover) | `https://techcrunch.com` |

### Quick start

#### Single feed

```json
{ "feedUrls": ["https://hnrss.org/frontpage"] }
```

#### Bulk news aggregator (dedup + newest first)

```json
{
  "feedUrls": [
    "https://techcrunch.com/feed/",
    "https://www.theverge.com/rss/index.xml",
    "https://feeds.arstechnica.com/arstechnica/index"
  ],
  "maxItems": 50,
  "deduplicate": true,
  "sortByDate": true
}
```

#### Full article text for AI / LLM

```json
{
  "feedUrls": ["https://techcrunch.com/feed/"],
  "maxItems": 25,
  "fullText": true
}
```

#### Keyword monitoring on a schedule

```json
{
  "feedUrls": ["https://hnrss.org/frontpage"],
  "keywords": ["AI", "python", "startup"],
  "maxAgeDays": 7,
  "onlyNew": true
}
```

#### Import your Feedly / Inoreader OPML

```json
{ "opmlUrl": "https://example.com/feedly-export.opml" }
```

### Input parameters

| Field | Type | Description |
|-------|------|-------------|
| `feedUrls` | string\[] | RSS/Atom/RDF/JSON Feed/sitemap URLs, or plain site URLs (feed auto-discovered) |
| `maxItems` | int | Limit candidate items per feed (0 = no per-feed cap). Default: 1,000 |
| `stripHtml` | bool | Return plain text (true) or original HTML (false). Default: true |
| `extractFeedMetadata` | bool | Include feed title/description/language on each item. Default: true |
| `deduplicate` | bool | Remove duplicate items across feeds. Default: true |
| `sortByDate` | bool | Merge all feeds into one newest-first stream. Default: false |
| `onlyNew` | bool | Return only items not seen in previous runs (delta monitoring). Default: false |
| `keywords` | string\[] | Keep only items containing any of these keywords |
| `maxAgeDays` | int | Keep only items from the last N days (0 = no limit) |
| `fullText` | bool | Fetch clean full article text (ads/boilerplate removed). Default: false |
| `fullTextApiKey` | string | Optional API key for the full-text reader — improves reliability at high volume |
| `opmlUrl` | string | URL of an OPML file to import (Feedly/Inoreader export) |
| `maxConcurrency` | int | Parallel fetches. Default: 5 |
| `timeout` | int | HTTP timeout per feed in seconds. Default: 20 |
| `proxyConfiguration` | object | Apify Proxy settings (datacenter by default) |

### Output

One dataset row per item:

```json
{
  "feedUrl": "https://techcrunch.com/feed/",
  "feedTitle": "TechCrunch",
  "title": "Example article title",
  "link": "https://techcrunch.com/2026/...",
  "guid": "https://techcrunch.com/?p=123",
  "pubDate": "Mon, 13 Jul 2026 18:22:10 +0000",
  "datePublished": "2026-07-13T18:22:10+00:00",
  "author": "Jane Doe",
  "description": "Short summary...",
  "content": "Article content from the feed...",
  "categories": "AI, Startups",
  "enclosures": [{ "url": "https://.../audio.mp3", "type": "audio/mpeg", "length": "12345" }],
  "fullText": "Full clean article text (when fullText is enabled)..."
}
```

Items with parseable source dates get an ISO-8601 `datePublished` so dates from RSS (RFC-822) and Atom/JSON (ISO) are consistent — ideal for sorting and feeding LLMs.

### Use cases

- **AI agents / LLM & RAG pipelines** — clean full-text JSON to summarize, embed or monitor industry news.
- **News aggregation** — build a niche news site or a Slack/Discord digest bot from dozens of sources.
- **Content monitoring** — track keywords across feeds and get only new matches on a schedule (`onlyNew`).
- **Podcast pipelines** — pull episodes with audio enclosures and iTunes metadata.
- **Medium / Substack / YouTube tracking** — follow newsletters and channels via their RSS feeds.
- **SEO & competitive research** — analyze competitor publishing frequency and topics.

### Full-text extraction notes

`fullText` fetches each article and returns clean, LLM-ready text with ads and boilerplate removed.

- Works out of the box for a small initial sample; no fixed keyless volume is guaranteed; beyond that some items may return a `fullTextError` instead of `fullText` (the item is still returned with its feed excerpt).
- Requests are spread across rotating datacenter proxy IPs to maximize throughput.
- For reader-authenticated full-text requests, set the optional `fullTextApiKey`; service quotas and access limits still apply.

### Pricing

- **$0.001 per item** ($1 per 1,000 items).
- Broken or unreachable feeds are skipped cleanly.

### How it works

1. Use the configured connection route; public sources may still deny a request.
2. Fetch the URL and auto-detect the format (RSS / Atom / RDF / JSON Feed / sitemap), or discover a feed from an HTML page.
3. Parse items with namespace-aware stdlib XML / JSON — no heavy dependencies, fast cold starts.
4. Deduplicate, filter (keywords / age), apply delta, and optionally sort by date.
5. Optionally fetch clean full article text (ads/boilerplate removed).
6. Push structured JSON to the dataset.

### Tips

- **YouTube as RSS:** `https://www.youtube.com/feeds/videos.xml?channel_id=<id>` (free, no API quota).
- **Substack:** append `/feed` to any Substack URL.
- **Medium:** `https://medium.com/feed/@user` or `medium.com/feed/publication-name`.
- **Scheduling:** Apify Scheduler + `onlyNew: true` supports periodic new-item checks with bounded shared history; review delivery and source failures.

### Choose an RSS-to-JSON workflow

Use this RSS feed reader when you already know which publications or feeds to
follow and need structured items for a digest, editorial review or an AI
pipeline. Begin with feed excerpts. Optional full-text requests add a separate
source dependency and should be requested only when the excerpt is insufficient.

| Goal | Starting settings | Check before using the output |
| --- | --- | --- |
| Small news digest | One feed and a small maxItems | Title, canonical link and available date |
| Multi-source research | Feed list, deduplication and sorting | Per-feed limits and duplicate identity |
| Topic monitoring | Keywords and an age window | Substring relevance and undated items |
| AI article processing | Small batch with fullText | fullTextError and source access/reader coverage |

#### Read two public feed entries

```json
{
  "feedUrls": [
    "https://hnrss.org/frontpage"
  ],
  "maxItems": 2,
  "fullText": false,
  "onlyNew": false,
  "maxConcurrency": 1
}
```

A two-item sample is enough to inspect the record shape before increasing collection. maxItems is applied per feed, not as a global Dataset ceiling.

#### Merge two feed samples

```json
{
  "feedUrls": [
    "https://hnrss.org/frontpage",
    "https://xkcd.com/atom.xml"
  ],
  "maxItems": 2,
  "deduplicate": true,
  "sortByDate": true,
  "fullText": false
}
```

Two sources can produce up to four items before deduplication and other filters. Different links or GUIDs can still describe the same story; this is identity deduplication rather than semantic news clustering.

#### Review a bounded topic window

```json
{
  "feedUrls": [
    "https://hnrss.org/frontpage"
  ],
  "maxItems": 10,
  "keywords": [
    "python",
    "open source"
  ],
  "maxAgeDays": 7,
  "onlyNew": false,
  "fullText": false
}
```

Keyword matching is case-insensitive substring matching over available text fields. Undated or unparseable-date items are retained by the age filter. A small candidate slice may produce zero matching results.

#### Keep a small feed excerpt for downstream text processing

```json
{
  "feedUrls": [
    "https://xkcd.com/atom.xml"
  ],
  "maxItems": 2,
  "stripHtml": true,
  "extractFeedMetadata": true,
  "autoDiscover": false,
  "fullText": false
}
```

This parses an explicit feed without searching an HTML page for a feed link. Plain text can lose inline formatting; it is still the supplied feed content, not a guarantee of the complete linked article.

### Monitoring state, completeness and cost

`onlyNew` uses persistent named history in the running Apify account. History is
shared across this Actor's monitoring jobs rather than isolated per watchlist;
there is no input for a separate monitor name. The history retains at most
20,000 seen IDs. A first run emits currently unseen matching items, not a
historical baseline of everything the publisher has ever posted. History is
saved before final output delivery, so an interrupted or budget-stopped run
needs careful reconciliation with the Dataset before relying on a later delta.

The runtime does not write SUMMARY or ERRORS records. Inspect run logs and the
Dataset; unavailable feeds are logged and skipped. A successful run with zero
items can mean filtering, seen items, an empty feed or a source failure. These
outcomes must not all be reported as 'nothing new'. Sitemap entries may contain
only link/date context, and a sitemap index is not an unlimited recursive crawl.
Feed auto-discovery follows an available alternate feed link, not every feed
published anywhere on the site.

The current item event is $0.001, or $1 per 1,000 delivered rows. Startup is
$0.00005 per allocated GB with a minimum of one event; platform usage is
additional. Two delivered items are $0.002 in item events before startup and
resources. More feeds, OPML imports and full-text enrichment increase work;
maxItems does not cap the total bill. Use an Apify run spending limit and begin
with an explicit small per-feed limit. The runtime and schema default is1,000.

Full-text extraction is best effort through an external reader. A supplied
reader key can have its own quota or service charges; it is not a promise of
unlimited access. Do not treat an excerpt plus fullTextError as a complete
article. Preserve feedUrl, link/GUID and available dates when exporting JSON,
CSV or a digest. Podcasts expose enclosure metadata and URLs; media files are
not automatically downloaded or transcribed.

**Does sorting reconstruct a complete publication history?** No. It sorts the
available selected items by parsed date. Feed retention, per-feed caps and
unknown dates limit that stream.

**Can a schedule guarantee instant, duplicate-free monitoring?** No. Publisher
updates, collection intervals, shared bounded state and source failures still
matter. Keep your own delivery ledger for an important downstream workflow.

### n8n and developer integrations

Use this Actor in your automation or application with the existing community integrations below. An Apify account and API token are required; Actor and platform charges follow the [current pricing](https://apify.com/apivault_labs/universal-rss-scraper/pricing).

- **n8n community node:** [n8n-nodes-apivault-rss](https://www.npmjs.com/package/n8n-nodes-apivault-rss). Install package name `n8n-nodes-apivault-rss` in a self-hosted n8n instance that permits community packages.
- **Python client and examples:** [GitHub](https://github.com/apivault-labs/apivault-labs/tree/main/python-clients/apivault-universal-rss-scraper).
- **n8n source and usage:** [GitHub](https://github.com/apivault-labs/apivault-labs/tree/main/community-nodes/n8n-nodes-apivault-rss).

These are community packages. Publication on npm does not establish n8n verification or availability in n8n Cloud. Review the linked package README for supported inputs and setup.

# Actor input Schema

## `feedUrls` (type: `array`):

RSS/Atom/RDF/JSON Feed/sitemap URLs — or plain site URLs (feed is auto-discovered). Examples:
• https://hnrss.org/frontpage
• https://techcrunch.com/feed/
• https://daringfireball.net/feeds/json (JSON Feed)
• https://example.com/sitemap.xml
• https://example.com (auto-discover)

## `maxItems` (type: `integer`):

Limit items per feed (0 = all)

## `stripHtml` (type: `boolean`):

If true, content fields are returned as plain text. If false, original HTML is preserved.

## `extractFeedMetadata` (type: `boolean`):

Include feed title, description, language in each item row

## `maxConcurrency` (type: `integer`):

Parallel feed fetches (default: 5)

## `timeout` (type: `integer`):

HTTP timeout per feed request

## `autoDiscover` (type: `boolean`):

If a URL is a normal web page (not a feed), find its <link rel="alternate"> RSS/Atom/JSON feed and parse that.

## `deduplicate` (type: `boolean`):

Remove duplicate items (by guid/link) across all feeds.

## `sortByDate` (type: `boolean`):

Merge all feeds into one chronologically sorted stream.

## `onlyNew` (type: `boolean`):

On each run, output only items not seen in previous runs. Enables scheduled monitoring. State is kept in the key-value store.

## `keywords` (type: `array`):

Only keep items whose title/description/content/categories contain any of these keywords (case-insensitive).

## `maxAgeDays` (type: `integer`):

Only keep items published within the last N days (0 = no limit). Items without a parseable date are kept.

## `fullText` (type: `boolean`):

Feeds often return only an excerpt. When enabled, each article is fetched and clean, LLM-ready text (ads/boilerplate removed) is added as a 'fullText' field. Works well up to ~75 articles per run; for higher volume set the full-text API key below.

## `fullTextApiKey` (type: `string`):

Optional API key for the full-text reader. Recommended for reliable high-volume full-text extraction. Leave empty to use the free tier.

## `opmlUrl` (type: `string`):

URL of an OPML file (e.g. exported from Feedly/Inoreader). All feeds inside are imported and merged with the URLs above.

## `proxyConfiguration` (type: `object`):

Route feed requests through Apify Proxy to bypass IP blocks. Datacenter (default) is cheap and works for most feeds; use residential only if a feed blocks datacenter IPs.

## Actor input object example

```json
{
  "feedUrls": [
    "https://hnrss.org/frontpage"
  ],
  "maxItems": 1000,
  "stripHtml": true,
  "extractFeedMetadata": true,
  "maxConcurrency": 5,
  "timeout": 20,
  "autoDiscover": true,
  "deduplicate": true,
  "sortByDate": false,
  "onlyNew": false,
  "maxAgeDays": 0,
  "fullText": false,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "feedUrls": [
        "https://hnrss.org/frontpage"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("apivault_labs/universal-rss-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "feedUrls": ["https://hnrss.org/frontpage"] }

# Run the Actor and wait for it to finish
run = client.actor("apivault_labs/universal-rss-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "feedUrls": [
    "https://hnrss.org/frontpage"
  ]
}' |
apify call apivault_labs/universal-rss-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,apivault_labs/universal-rss-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/XItoeDsQ5yixVaMKI/builds/YSzY7Bi1OZbWiin7M/openapi.json