# Substack Newsletter Scraper: Posts, Authors & Reactions (`stanvanrooy6/substack-scraper`) Actor

Export post data from any Substack newsletter or custom domain: titles, subtitles, URLs, authors, publish dates, tags, word counts, reactions, comment counts and paid or free audience. No login. JSON, CSV or Excel.

- **URL**: https://apify.com/stanvanrooy6/substack-scraper.md
- **Developed by:** [Stan Van Rooy](https://apify.com/stanvanrooy6) (community)
- **Categories:** Developer tools, Automation, News
- **Features:** Limited permissions
- **Stats:** 38 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.60 / 1,000 posts

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Substack Newsletter Scraper: Posts, Authors & Reactions 📰

**Free-plan sample:** Apify Free accounts can collect up to **100 result rows total per run**, shared across all inputs. Upgrade your [Apify plan](https://console.apify.com/billing/subscription) for larger runs. Normal Actor charges apply. The sample resets each run; it is not a daily or monthly quota.

**Scrape posts from any Substack publication.** Get titles, authors, dates, tags, reactions, comments, and more from any newsletter.

### 🚀 Why use this scraper?

- **Universal Compatibility**: Works with any Substack publication, on a custom domain or a substack.com subdomain.
- **Complete Metadata**: Get all post details including titles, authors, tags, reactions, comments, and restacks.
- **Fast & Lightweight**: Uses Substack's API directly, no browser overhead.
- **Cost Effective**: pay per post, no monthly plan.

### 📥 Input

**Publication URL** (required)

- **Custom domain**: e.g., `https://www.lennysnewsletter.com`
- **Substack subdomain**: e.g., `https://astralcodexten.substack.com`

**Max Posts** (optional, default: 100)

- Maximum number of posts to collect. Set to `0` for unlimited.

### 📤 Output

Each post includes comprehensive metadata:

```json
{
  "post_id": 180650333,
  "post_title": "How to spot a top 1% startup early",
  "subtitle": "Three key lessons from people who picked multiple iconic companies",
  "slug": "how-to-spot-a-top-1-startup-early",
  "post_date": "2025-12-09T13:30:55.231Z",
  "canonical_url": "https://www.lennysnewsletter.com/p/how-to-spot-a-top-1-startup-early",
  "author_name": "Lenny Rachitsky",
  "author_handle": "lenny",
  "cover_image": "https://substack-post-media.s3.amazonaws.com/...",
  "post_description": "Three key lessons from employees who picked multiple iconic companies",
  "wordcount": 2510,
  "tags": ["Career", "Investing"],
  "audience": "only_paid",
  "reactions": {"❤": 241},
  "reaction_count": 241,
  "comment_count": 7,
  "restacks": 7,
  "publication_name": "Lenny's Newsletter",
  "publication_url": "https://www.lennysnewsletter.com"
}
```

### 🎯 Example Usage

**Scrape a newsletter with custom domain:**

```json
{
  "startUrl": "https://www.lennysnewsletter.com",
  "maxPosts": 50
}
```

**Scrape a substack.com subdomain:**

```json
{
  "startUrl": "https://astralcodexten.substack.com",
  "maxPosts": 100
}
```

**Scrape all posts (unlimited):**

```json
{
  "startUrl": "https://www.notboring.co",
  "maxPosts": 0
}
```

### 💰 Pricing

Pay per result: **$2.00 per 1,000 posts** plus a small Actor start fee ($0.005 per GB of memory, one event per run). A 500-post export costs about $1. Volume users on higher Apify plans pay less per post.

Pay per result replaces the $10/month rental on October 2, 2026. Until then the rental price shown on the pricing tab applies.

### ❓ FAQ

#### How do I scrape posts from a Substack newsletter?

Provide the publication URL (either a custom domain like `lennysnewsletter.com` or a subdomain like `newsletter.substack.com`) and set your desired post limit. The scraper handles pagination automatically.

#### What data does this Substack scraper extract?

The scraper extracts 18 fields per post: post ID, title, subtitle, slug, publication date, canonical URL, author name, author handle, cover image URL, description, word count, tags, audience type, emoji reactions, reaction count, comment count, restack count, and publication info.

#### Does this scraper work with paid Substack newsletters?

Yes. The scraper fetches metadata for all posts including paid-only content. Posts behind a paywall are marked with `audience: "only_paid"`. The scraper extracts metadata, not full article content.

#### Can I scrape the full article body and HTML content from Substack?

The current version extracts post metadata only (title, description, etc.), not the full article HTML. Contact the developer if you need full content extraction.

#### What Substack URL formats are supported?

Both formats work: custom domains (e.g., `www.lennysnewsletter.com`, `newsletter.pragmaticengineer.com`) and Substack subdomains (e.g., `astralcodexten.substack.com`).

#### How many Substack posts can I scrape?

Unlimited. Set `maxPosts` to `0` for no limit. The scraper paginates through the entire archive automatically.

#### Does this work with non-Substack sites?

No. The scraper uses Substack's internal API. Non-Substack sites will be detected and the scraper exits gracefully with a message.

#### What if a Substack publication returns no posts?

This typically means: (1) the URL is not a valid Substack publication, (2) the publication has no public posts, or (3) the publication is very new/inactive.

#### Is this scraper API-based or browser-based?

API-based. It calls Substack's archive API directly without requiring a browser, making it fast and resource-efficient.

#### How do I get Substack post engagement metrics like likes and comments?

Each post includes `reaction_count` (total likes), `reactions` (breakdown by emoji), `comment_count`, and `restacks`. These are real-time values from the archive API.

#### Can I filter Substack posts by date or tag?

The scraper returns posts sorted by newest first. Post-processing filtering by date or tags can be done on the exported dataset.

#### What's the difference between audience "everyone" and "only_paid"?

`everyone` means the post is free for all readers. `only_paid` means the full post is only available to paying subscribers.

### 📞 Feedback & Support

I actively maintain this actor. Found a bug or need a field? Open an issue on the Issues tab with the newsletter URL.

### Rate this Actor

If this Actor saved you time, a rating on its Apify Store page helps other users find it. Thank you!

***

Built with ❤️ for the Apify community

# Actor input Schema

## `startUrl` (type: `string`):

URL of any Substack publication. Works with both custom domains (e.g., 'www.lennysnewsletter.com') and Substack subdomains (e.g., 'lenny.substack.com').

## `maxPosts` (type: `integer`):

Maximum number of posts to scrape. Set to 0 for unlimited.

## Actor input object example

```json
{
  "startUrl": "https://www.lennysnewsletter.com",
  "maxPosts": 100
}
```

# Actor output Schema

## `posts` (type: `string`):

One row per newsletter post. JSON; append ?format=csv or ?format=xlsx for other formats.

## `postsCsv` (type: `string`):

The same rows as a CSV download.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrl": "https://www.lennysnewsletter.com"
};

// Run the Actor and wait for it to finish
const run = await client.actor("stanvanrooy6/substack-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrl": "https://www.lennysnewsletter.com" }

# Run the Actor and wait for it to finish
run = client.actor("stanvanrooy6/substack-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrl": "https://www.lennysnewsletter.com"
}' |
apify call stanvanrooy6/substack-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,stanvanrooy6/substack-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/1nEWGBpa4KHKrWueG/builds/CxIUHlMhmdqAqmhNN/openapi.json