# RSS / XML Scraper (`shahidirfan/rss-xml-scraper`) Actor

Meet the RSS / XML Scraper: the most advanced actor for parsing any RSS feed or XML file. It effortlessly extracts clean, structured data from even the most complex sources. Your ultimate tool for content aggregation, data monitoring, and content analysis.

- **URL**: https://apify.com/shahidirfan/rss-xml-scraper.md
- **Developed by:** [Shahid Irfan](https://apify.com/shahidirfan) (community)
- **Categories:** Developer tools, Automation, News
- **Features:** Limited permissions
- **Stats:** 116 total users, 11 monthly users, 100.0% runs succeeded, 6 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### What does RSS XML Feed Scraper do?

RSS XML Feed Scraper collects feed entries from RSS, Atom, RDF, JSON Feed, OPML, and sitemap URLs and writes them to an Apify dataset. Add one or more feed URLs, set an entry limit, and run the Actor. Each run returns feed metadata plus article records with titles, links, dates, authors, tags, media links, and optional full article text.

The Actor also accepts ordinary website URLs. It scans the page for feed links, checks each candidate, and collects the valid feeds. If a previously working feed path starts serving an HTML page, the Actor looks for the publisher's current feed instead of stopping at the first failed request.

### Why use RSS XML Feed Scraper?

- **One dataset for many publishers.** Collect BBC, Guardian, TechCrunch, Reddit, WordPress, and personal blogs in a single run.
- **Format detection without setup.** RSS, Atom, RDF, JSON Feed, OPML, and sitemaps are recognized from the response, so you do not need to sort URLs by format.
- **Recovery when a publisher blocks a request.** The Actor retries temporary failures, switches request profiles when a desktop request is rejected, and falls back to a direct request when a proxy fails.
- **Article text when you need it.** Keep short feed summaries for fast monitoring, or fetch the linked page for full text and clean HTML.
- **Scheduled monitoring.** Run hourly, daily, or weekly and compare new entries against your stored dataset.

### What data can you extract?

| Field | Description |
|-------|-------------|
| `item_type` | `feed_meta` for the feed record, `entry` for an article record |
| `feed_url` | Feed that produced the record |
| `title` | Feed title or article headline |
| `link` | Feed homepage or article URL |
| `description` | Teaser or description from the feed |
| `content` | Entry content as plain text |
| `content_html` | Entry content as HTML when the feed provides it |
| `author` / `authors` | Primary author and author list |
| `published` | Publication time in ISO 8601 when parseable |
| `updated` | Last update time in ISO 8601 when parseable |
| `tags` | Categories and tags from the entry |
| `image_url` | Best available image from media fields |
| `enclosure_url` | Podcast, video, or attachment URL |
| `full_text` | Full article text from the linked page |
| `full_html` | Cleaned article HTML |
| `meta_description` | Article meta description |
| `top_image` | Primary image found on the article page |
| `content_source` | `feed_payload` or `article_page` |
| `error` | Failure reason when a URL cannot be read |

Feed records also include `language`, `generator`, `feed_image`, `managing_editor`, `webmaster`, `entries_count`, `feed_type`, and `parsed_at`.

### Supported feed formats

- RSS 2.0, including `content:encoded`, `dc:creator`, media, and enclosure fields.
- Atom, including alternate links, summaries, content, and author lists.
- RDF and RSS 1.0 feeds.
- JSON Feed, including `content_html`, `content_text`, `authors`, and attachments.
- WordPress REST post collections at `/wp-json/wp/v2/posts`.
- OPML subscription lists.
- XML sitemaps and sitemap indexes.
- Microformats h-feed pages that expose `h-entry` items in HTML.

### How to use RSS XML Feed Scraper

1. Open the Actor on Apify.
2. Add feed URLs or website URLs under **Feed or Website URLs**.
3. Set **Maximum Entries per Feed**. Use a small number for a first run.
4. Leave **Extract Full Article Content** off for fast metadata collection.
5. Run the Actor and download the dataset as JSON, CSV, Excel, or XML.

### Input Parameters

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `urls` | Array | No | `["https://feeds.bbci.co.uk/news/rss.xml"]` | Feed URLs or website URLs. One URL per line. |
| `extractContent` | Boolean | No | `false` | Fetch every linked article and add full text and article metadata. |
| `autoExpandSnippets` | Boolean | No | `true` | Fetch the full article when an entry contains only a short summary. |
| `discoverFeeds` | Boolean | No | `false` | Treat website URLs as discovery targets before collecting entries. |
| `maxEntries` | Integer | No | `20` | Maximum entries per feed. Set to `0` to keep every entry. |
| `proxyConfiguration` | Object | No | `{"useApifyProxy": false}` | Optional proxy settings. Apify Proxy stays off by default. |

### Output Data

| Field | Type | Description |
|-------|------|-------------|
| `item_type` | String | Record type: `feed_meta` or `entry` |
| `feed_url` | String | Source feed URL |
| `title` | String | Feed or article title |
| `link` | String | Feed homepage or article URL |
| `id` | String | Entry ID or GUID from the feed |
| `description` | String | Short description from the feed |
| `summary` | String | Summary field when the feed provides one |
| `content` | String | Entry content as plain text |
| `content_html` | String | Entry content as HTML |
| `description_html` | String | Description HTML when available |
| `summary_html` | String | Summary HTML when available |
| `author` | String | Primary author |
| `authors` | Array | Author list |
| `creator` | String | Creator field from the feed |
| `published` | String | Publication date in ISO 8601 when parseable |
| `updated` | String | Update date in ISO 8601 when parseable |
| `raw_pub_date` | String | Original date string from the feed |
| `raw_iso_date` | String | Original ISO date string from the feed |
| `tags` | Array | Categories and tags |
| `comments` | String | Comment URL when the feed provides one |
| `source_title` | String | Source channel title |
| `source_url` | String | Source channel URL |
| `enclosure_url` | String | Attachment URL |
| `enclosure_type` | String | Attachment MIME type |
| `enclosure_length` | String | Attachment size |
| `media_content_url` | String | Media content URL |
| `media_thumbnail_url` | String | Media thumbnail URL |
| `image_url` | String | Best available image URL |
| `full_text` | String | Full article text |
| `full_html` | String | Cleaned article HTML |
| `meta_description` | String | Article meta description |
| `keywords` | Array | Keywords found by the article extractor |
| `top_image` | String | Primary article image |
| `publish_date` | String | Article publish date from the linked page |
| `content_source` | String | Where full content came from |
| `content_error` | String | Full content failure reason, if any |
| `error` | String | Feed or URL failure reason, if any |
| `language` | String | Feed language |
| `generator` | String | Feed generator |
| `feed_image` | String | Feed image or logo |
| `managing_editor` | String | Managing editor field |
| `webmaster` | String | Webmaster field |
| `entries_count` | Integer | Total entries available in the feed |
| `feed_type` | String | `rss`, `atom`, `rdf`, `json`, `sitemap`, or `hfeed` |
| `parsed_at` | String | Feed parse timestamp |
| `collected_at` | String | Record collection timestamp |

### Usage Examples

#### Collect one feed

```json
{
  "urls": [
    "https://feeds.bbci.co.uk/news/rss.xml"
  ],
  "maxEntries": 20
}
```

#### Collect several publishers in one run

```json
{
  "urls": [
    "https://feeds.bbci.co.uk/news/rss.xml",
    "https://www.theguardian.com/world/rss",
    "https://techcrunch.com/feed/",
    "https://hnrss.org/frontpage"
  ],
  "maxEntries": 50
}
```

#### Fetch full article text

```json
{
  "urls": [
    "https://www.theguardian.com/world/rss"
  ],
  "extractContent": true,
  "maxEntries": 10
}
```

#### Find the feed for a website

```json
{
  "urls": [
    "https://www.nasa.gov"
  ],
  "discoverFeeds": true,
  "maxEntries": 10
}
```

#### Keep only new entries for a daily job

Run the same input every day and compare the dataset against your stored records. The Actor removes duplicate entries inside a run, so repeated feed URLs and repeated discovery candidates do not inflate the dataset.

### Sample Output

```json
{
  "item_type": "entry",
  "feed_url": "https://feeds.bbci.co.uk/news/rss.xml",
  "title": "Example Article Title",
  "link": "https://www.bbc.co.uk/news/articles/example",
  "id": "https://www.bbc.co.uk/news/articles/example",
  "description": "Short teaser from the feed.",
  "summary": "Short teaser from the feed.",
  "content": "Short teaser from the feed.",
  "author": "Reporter Name",
  "authors": [
    "Reporter Name"
  ],
  "published": "2026-10-22T12:17:44.000Z",
  "raw_pub_date": "Thu, 22 Oct 2026 12:17:44 GMT",
  "tags": [],
  "image_url": "https://example.com/image.jpg",
  "full_text": "Expanded article text from the linked page.",
  "full_html": "<p>Expanded article text from the linked page.</p>",
  "content_source": "article_page",
  "collected_at": "2026-10-22T13:41:48.910Z"
}
```

### Tips for Best Results

- Start with `maxEntries` at 10 or 20 to check titles, links, and dates.
- Large podcast and archive feeds stay fast because the Actor reads only enough entries to satisfy `maxEntries`. Set `maxEntries` to `0` when you need the entire feed.
- Keep `extractContent` off for monitoring jobs that only need headlines and summaries.
- Turn `autoExpandSnippets` off when you want feed-only collection with the fastest runtime.
- Use direct feed URLs when you know them. Use `discoverFeeds` when you only have a homepage.
- Add a proxy only when a publisher rate-limits your runs. Most feeds work without one.
- A missing field usually means the publisher does not include it in the feed. Check several entries before treating it as a failure.

### Integrations

- **Google Sheets** - Send entries to a spreadsheet for editorial or research review.
- **Webhooks** - Notify internal services when a run finishes.
- **Make or Zapier** - Route new entries into publishing, alerting, or archiving workflows.
- **Apify API** - Read the dataset from your own application or scheduler.
- **Slack** - Post new headlines to a channel through a webhook workflow.

#### Export formats

- JSON for applications and data pipelines.
- CSV for spreadsheets and quick review.
- Excel for reporting.
- XML for systems that already consume feed-style data.

### Frequently Asked Questions

#### Does this Actor work with Atom and JSON Feed?

Yes. The format is detected from the response, so RSS, Atom, RDF, JSON Feed, OPML, and sitemaps can be mixed in one input list.

#### Can I collect a feed without knowing its URL?

Yes. Add the website URL and enable `discoverFeeds`. The Actor scans the page for feed links, checks each candidate, and collects the valid feeds.

#### What happens if a feed URL returns an HTML page?

The Actor scans that page for feed links. If it finds a valid feed, it processes it. If not, the dataset receives an error record with the URL and reason.

#### Why are some entries missing full text?

Full text depends on the article page. If the page blocks the request or provides no readable article body, the Actor keeps the feed summary and records the reason in `content_error`.

#### Does the Actor remove duplicate entries?

Yes. Entries are deduplicated by feed and article ID or URL within a run. Duplicate feed URLs and discovery candidates are skipped.

#### Can I run this on a schedule?

Yes. Schedule the Actor in Apify Console and connect the dataset to your downstream workflow. Daily or hourly runs work well for news and blog monitoring.

#### How many entries can I collect?

There is no fixed cap. `maxEntries` limits entries per feed, and `0` keeps every entry in the feed. Large sitemap indexes follow the first 25 child sitemaps to keep a run bounded.

#### Is this Actor legal to use?

The Actor reads publicly available feeds and pages. You are responsible for complying with publisher terms, robots policies, copyright, and applicable law.

### Support

For issues or feature requests, use the Issues tab on the Actor page or contact the developer through Apify.

### Legal Notice

This Actor is intended for legitimate collection of publicly available feed data. Users are responsible for how the collected content is stored, republished, or redistributed.

# Actor input Schema

## `urls` (type: `array`):

Feed URLs or website URLs to scan for feeds. One URL per line.

## `extractContent` (type: `boolean`):

Fetch every linked article and add full text, clean HTML, keywords, and article metadata.

## `autoExpandSnippets` (type: `boolean`):

Fetch the full article when a feed entry contains only a short snippet or summary.

## `discoverFeeds` (type: `boolean`):

Scan website URLs for RSS, Atom, RDF, and JSON feed links before collecting entries.

## `maxEntries` (type: `integer`):

Maximum entries to keep from each feed. Set to 0 to keep every entry.

## `proxyConfiguration` (type: `object`):

Optional proxy settings. Apify Proxy stays off by default for fast direct feed requests.

## Actor input object example

```json
{
  "urls": [
    "https://feeds.bbci.co.uk/news/rss.xml"
  ],
  "extractContent": false,
  "autoExpandSnippets": true,
  "discoverFeeds": false,
  "maxEntries": 20,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `overview` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://feeds.bbci.co.uk/news/rss.xml"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("shahidirfan/rss-xml-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": ["https://feeds.bbci.co.uk/news/rss.xml"] }

# Run the Actor and wait for it to finish
run = client.actor("shahidirfan/rss-xml-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://feeds.bbci.co.uk/news/rss.xml"
  ]
}' |
apify call shahidirfan/rss-xml-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,shahidirfan/rss-xml-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/NyOwKFjMHD6dmILKQ/builds/2hveXyc1BYHrvk8A4/openapi.json