# Website To Rss (`gabrielaxy/website-to-rss`) Actor

Convert any website into RSS, Atom, or JSON feeds. Auto-detects articles, tracks changes, and sends notifications. Works with WordPress, Ghost, Medium, Substack, and any blog.

- **URL**: https://apify.com/gabrielaxy/website-to-rss.md
- **Developed by:** [Gabriel Antony Xaviour](https://apify.com/gabrielaxy) (community)
- **Categories:** Developer tools, Automation, SEO tools
- **Features:** Limited permissions, Single-tenant, Server interface
- **Stats:** 18 total users, 1 monthly users, 100.0% runs succeeded, 1 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## 📡 Website to RSS

<p align="center">
  <strong>Transform any website into a standards-compliant RSS, Atom, or JSON feed with automatic article detection and change monitoring.</strong>
</p>

<p align="center">
  <a href="#features">Features</a> •
  <a href="#quick-start">Quick Start</a> •
  <a href="#input-configuration">Configuration</a> •
  <a href="#output">Output</a> •
  <a href="#change-detection">Monitoring</a> •
  <a href="#use-cases">Use Cases</a>
</p>

***

### What is Website to RSS?

Website to RSS converts any website into a subscribable feed, even if the site doesn't offer one. It automatically detects article patterns, extracts content using multiple strategies, and outputs valid RSS 2.0, Atom 1.0, or JSON Feed formats.

#### Key Benefits

| Feature | Description |
|---------|-------------|
| 🔍 **Auto-Discovery** | Automatically detects site structure and article patterns |
| 🎯 **Platform Presets** | Optimized extraction for WordPress, Ghost, Medium, Substack, Hugo |
| 📊 **Smart Extraction** | Uses OpenGraph, JSON-LD, and semantic HTML for reliable content |
| 🔔 **Change Detection** | Track new posts and content changes between runs |
| 📋 **Multiple Formats** | Generate RSS 2.0, Atom 1.0, JSON Feed, and HTML preview |
| 🔗 **Notifications** | Send webhooks or Slack alerts when new content is detected |

***

### Features

#### Auto-Discovery Engine

The actor automatically analyzes websites to find articles:

1. **Platform Detection** — Checks for WordPress, Ghost, Medium signatures
2. **Structure Analysis** — Identifies article patterns from page structure
3. **Link Filtering** — Excludes author, tag, and category pages
4. **Article Scoring** — Uses heuristics to identify real articles
5. **Content Extraction** — Tries OpenGraph → JSON-LD → Semantic HTML → Fallback

#### Platform Presets

Pre-configured extraction rules for popular platforms:

| Platform | What's Optimized |
|----------|------------------|
| WordPress | Post selectors, date formats, category extraction |
| Ghost | Card content, member content handling |
| Medium | Story pages, clap counts, reading time |
| Substack | Newsletter posts, subscriber content |
| Hugo | Front matter, taxonomy handling |
| Generic | Universal patterns for unknown sites |

#### Output Formats

| Format | Content Type | Description |
|--------|--------------|-------------|
| RSS 2.0 | `application/rss+xml` | Most compatible feed format |
| Atom 1.0 | `application/atom+xml` | Modern feed standard |
| JSON Feed | `application/json` | Developer-friendly format |
| HTML | `text/html` | Visual preview page |

***

### Quick Start

#### Basic Usage

Just provide a URL — the actor handles the rest:

```json
{
  "websiteUrl": "https://blog.example.com",
  "maxItems": 20
}
```

#### With Platform Preset

Optimize extraction for known platforms:

```json
{
  "websiteUrl": "https://my-ghost-blog.com",
  "platformPreset": "ghost",
  "maxItems": 20
}
```

#### With Manual Selectors

Full control for custom sites:

```json
{
  "websiteUrl": "https://custom-site.com",
  "discoveryMode": "manual",
  "linkSelector": ".article-link",
  "titleSelector": ".article-title",
  "contentSelector": ".article-body",
  "maxItems": 20
}
```

#### With Change Monitoring

Get notified when new content appears:

```json
{
  "websiteUrl": "https://blog.example.com",
  "stateStoreName": "my-blog-monitor",
  "monitorMode": "new_pages",
  "slackWebhook": "https://hooks.slack.com/services/..."
}
```

***

### Input Configuration

#### Core Options

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `websiteUrl` | string | *required* | URL of the website to convert |
| `maxItems` | number | 50 | Maximum items in the feed |
| `outputFormats` | array | `["rss", "json"]` | Formats: `rss`, `atom`, `json`, `html` |

#### Discovery Options

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `discoveryMode` | string | `auto` | `auto`, `preset`, or `manual` |
| `platformPreset` | string | `auto` | `wordpress`, `ghost`, `medium`, `substack`, `hugo`, `generic` |
| `autoDetectArticles` | boolean | true | Use heuristics to filter non-articles |

#### Manual Selectors

| Parameter | Type | Description |
|-----------|------|-------------|
| `linkSelector` | string | CSS selector for article links |
| `titleSelector` | string | CSS selector for article title |
| `contentSelector` | string | CSS selector for article body |
| `dateSelector` | string | CSS selector for publication date |
| `urlExcludePatterns` | array | URL patterns to skip (e.g., `/author/`) |

#### Monitoring Options

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `stateStoreName` | string | — | Named store to persist state between runs |
| `monitorMode` | string | `both` | `new_pages`, `content_changes`, or `both` |
| `webhookUrl` | string | — | URL to POST when new content found |
| `slackWebhook` | string | — | Slack webhook for notifications |

***

### Output

#### Key-Value Store Files

| Key | Content Type | Description |
|-----|--------------|-------------|
| `feed.xml` | `application/rss+xml` | RSS 2.0 feed |
| `feed.atom` | `application/atom+xml` | Atom 1.0 feed |
| `feed.json` | `application/json` | JSON Feed 1.1 |
| `feed.html` | `text/html` | HTML preview page |
| `OUTPUT` | `application/json` | Run summary and statistics |

#### Dataset Item Schema

```json
{
  "url": "https://blog.example.com/post-1",
  "title": "My First Post",
  "description": "A short description...",
  "content": "Full article content...",
  "date": "2024-01-15T10:00:00Z",
  "author": "John Doe",
  "image": "https://blog.example.com/image.jpg",
  "categories": ["tech", "tutorial"],
  "changeStatus": "new"
}
```

#### Run Summary (OUTPUT)

```json
{
  "feedTitle": "My Blog",
  "feedUrl": "https://blog.example.com",
  "itemCount": 20,
  "newItems": 3,
  "changedItems": 1,
  "formats": ["rss", "json"],
  "crawlDurationSec": 45
}
```

***

### Change Detection

Enable persistent monitoring to track content changes:

```json
{
  "websiteUrl": "https://blog.example.com",
  "stateStoreName": "my-blog-monitor",
  "webhookUrl": "https://my-webhook.com/notify"
}
```

#### How It Works

1. **First Run** — Captures all current items as baseline
2. **Subsequent Runs** — Compares current items with stored state
3. **Detection** — Identifies new pages and content changes
4. **Notification** — Sends webhook/Slack with changes
5. **State Update** — Stores new state for next run

#### Webhook Payload

```json
{
  "event": "new_items",
  "feedTitle": "My Blog",
  "itemCount": 3,
  "items": [
    {
      "title": "New Post",
      "url": "https://blog.example.com/new-post",
      "date": "2024-01-15T10:00:00Z"
    }
  ]
}
```

***

### Use Cases

| Use Case | Description |
|----------|-------------|
| **RSS for RSS-less Sites** | Create feeds for websites that don't offer them |
| **Content Monitoring** | Track competitors, news sources, or blogs for updates |
| **Aggregation** | Combine multiple sites into monitoring workflows |
| **Archiving** | Capture content changes over time |
| **Automation Triggers** | Use webhooks to trigger downstream workflows |

***

### Integrations

#### Schedule Regular Updates

Use Apify Scheduler for periodic monitoring:

| Update Frequency | Best For |
|------------------|----------|
| Every hour | News sites, high-frequency publishers |
| Every 6 hours | Active blogs, company announcements |
| Daily | Personal blogs, slow-updating sites |

#### Feed Readers

Generated feeds work with any RSS reader:

- Feedly
- Inoreader
- NewsBlur
- Feedbin
- Any RSS-compatible app

#### Automation Platforms

Connect via webhooks to:

- Zapier
- Make (Integromat)
- n8n
- Custom backends

***

### Troubleshooting

#### No articles detected

| Problem | Solution |
|---------|----------|
| Site uses JavaScript | This actor uses HTTP (not browser). Try a Playwright-based scraper |
| Custom structure | Use `discoveryMode: "manual"` with specific selectors |
| Site blocks requests | Enable proxy configuration |

#### Wrong content extracted

| Problem | Solution |
|---------|----------|
| Grabbing navigation | Provide specific `contentSelector` for article body |
| Missing dates | Add `dateSelector` for date elements |
| Extra pages included | Add patterns to `urlExcludePatterns` |

***

### FAQ

**Q: Does this work with JavaScript-heavy sites?**

This actor uses HTTP requests (CheerioCrawler), not a browser. For sites that require JavaScript rendering, consider a Playwright-based scraper.

**Q: How often should I run monitoring?**

Depends on the site's update frequency. News sites: hourly. Blogs: every 6-24 hours. Over-polling wastes resources and may trigger rate limits.

**Q: Can I monitor multiple sites?**

Run the actor separately for each site with different `stateStoreName` values. Use Apify's scheduling to orchestrate multiple monitors.

**Q: What if the site changes its structure?**

Auto-discovery adapts to many changes. For manual selectors, you'll need to update them if the site's HTML structure changes.

***

### Support

- **Issues:** Report bugs or request features on GitHub
- **Documentation:** See full README in Actor source
- **API:** Use Apify API to run this Actor programmatically

***

### License

Apache 2.0

# Actor input Schema

## `websiteUrl` (type: `string`):

URL of the website to convert to RSS

## `feedTitle` (type: `string`):

Custom title for the RSS feed

## `feedDescription` (type: `string`):

Custom description for the RSS feed

## `discoveryMode` (type: `string`):

How to discover content on the site

## `platformPreset` (type: `string`):

Pre-configured settings for common platforms (only used when Discovery Mode is 'Use Platform Preset')

## `linkSelector` (type: `string`):

CSS selector for content links (only used when Discovery Mode is 'Manual Selectors')

## `titleSelector` (type: `string`):

CSS selector for item titles (only used when Discovery Mode is 'Manual Selectors')

## `contentSelector` (type: `string`):

CSS selector for content/description (only used when Discovery Mode is 'Manual Selectors')

## `dateSelector` (type: `string`):

CSS selector for date (only used when Discovery Mode is 'Manual Selectors')

## `maxItems` (type: `integer`):

Maximum number of items to include in the feed

## `outputFormats` (type: `array`):

Feed formats to generate (rss, atom, json)

## `proxyConfig` (type: `object`):

Proxy settings for web scraping

## `monitorMode` (type: `string`):

What to detect as new content

## `stateStoreName` (type: `string`):

Named KV store to persist state between runs

## `webhookUrl` (type: `string`):

Notify this URL when new items are detected

## `slackWebhook` (type: `string`):

Post new items to Slack channel

## `urlExcludePatterns` (type: `array`):

Regex patterns for URLs to exclude (e.g., '/author/', '/tag/'). Default patterns exclude common non-article pages.

## `urlIncludePatterns` (type: `array`):

If specified, only URLs matching these patterns are included (e.g., '/blog/', '/posts/')

## `autoDetectArticles` (type: `boolean`):

Use heuristics to automatically detect article pages and filter out listing/navigation pages

## Actor input object example

```json
{
  "websiteUrl": "https://blog.apify.com",
  "discoveryMode": "auto",
  "platformPreset": "wordpress",
  "linkSelector": "a[href]",
  "titleSelector": "h1, h2, .title",
  "contentSelector": "article, .content, p",
  "maxItems": 50,
  "outputFormats": [
    "rss",
    "atom",
    "json"
  ],
  "monitorMode": "both",
  "stateStoreName": "website-rss-state",
  "urlExcludePatterns": [],
  "autoDetectArticles": true
}
```

# Actor output Schema

## `feedItems` (type: `string`):

All extracted articles and content items

## `newItems` (type: `string`):

Newly detected articles

## `rssFeed` (type: `string`):

Generated RSS feed file

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "websiteUrl": "https://blog.apify.com",
    "outputFormats": [
        "rss",
        "atom",
        "json"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("gabrielaxy/website-to-rss").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "websiteUrl": "https://blog.apify.com",
    "outputFormats": [
        "rss",
        "atom",
        "json",
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("gabrielaxy/website-to-rss").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "websiteUrl": "https://blog.apify.com",
  "outputFormats": [
    "rss",
    "atom",
    "json"
  ]
}' |
apify call gabrielaxy/website-to-rss --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,gabrielaxy/website-to-rss"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/0RHQtjHvlSllcfDtE/builds/79UDefDSqLf8juHhQ/openapi.json