# Substack Newsletter Scraper (`digispruce/substack-scraper`) Actor

Extract comprehensive Substack newsletter data including author profiles, subscriber counts, social media links, and contact information for B2B outreach and market research.

- **URL**: https://apify.com/digispruce/substack-scraper.md
- **Developed by:** [Akram](https://apify.com/digispruce) (community)
- **Categories:** Lead generation, Automation, News
- **Features:** Limited permissions
- **Stats:** 135 total users, 21 monthly users, 99.9% runs succeeded, 1 bookmarks
- **User rating**: 4.00 out of 5 stars

## Pricing

from $1.90 / 1,000 newsletter info scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Substack Newsletter Subscriber & Engagement Scraper

A comprehensive Apify Actor for extracting Substack publication details, subscriber counts (when visible), and author contact information. Perfect for B2B outreach to newsletter creators, market research, and competitive analysis.

### 🚀 Key Features

- **📊 Publication Metadata**: Extract title, description, author info, author ID, categories, and subscriber counts (when publicly visible)
- **📧 Email Discovery**: Automatically find author contact emails from About pages and other content
- **🔗 Social Media Links**: Extract Twitter, LinkedIn, Facebook, Instagram, and YouTube profiles
- **📝 Flexible Post Scraping**: Choose from 3 modes - no posts, post metadata only, or full article content
- **📈 Engagement Metrics**: Extract likes, comments, and word counts for each post
- **⚡ Batch Processing**: Process multiple newsletters in a single run
- **💰 Pay-per-Event**: Charged per newsletter processed (successful or failed)

### 📋 Input Parameters

#### Required

- **newsletterUrls** (array): List of Substack newsletter URLs to scrape
  - Example: `[{"url": "https://platformer.substack.com"}, {"url": "https://lennysnewsletter.com"}]`
  - Supports both `*.substack.com` URLs and custom domains
  - Process multiple newsletters in a single run

#### Optional

- **postScrapingMode** (string, default: `"information_and_content"`): Controls what post data to extract
  - `"none"` - Newsletter metadata only (fastest, minimal data usage)
  - `"information"` - Newsletter metadata + post info (title, date, engagement, preview)
  - `"information_and_content"` - Newsletter metadata + post info + full article content

- **maxPostsPerNewsletter** (number, default: `12`, max: `12`): Number of recent posts to scrape per newsletter (only applies when postScrapingMode is not "none")

- **delayBetweenRequests** (number, default: `3000`, range: `500-10000`): Delay in milliseconds between HTTP requests to avoid rate limiting

### 💡 Example Input

```json
{
  "newsletterUrls": [
    { "url": "https://platformer.substack.com" },
    { "url": "https://lennysnewsletter.com" },
    { "url": "https://andrewsullivan.substack.com" }
  ],
  "postScrapingMode": "information_and_content",
  "maxPostsPerNewsletter": 12,
  "delayBetweenRequests": 3000
}
```

### 📊 Output Structure

The output structure varies based on the `postScrapingMode` setting:

#### Mode 1: `"none"` - Newsletter Metadata Only

```json
{
  "author_id": 241262,
  "author_name": "Casey Newton",
  "author_handle": "platformer",
  "author_bio": "Casey Newton is the founder and editor of Platformer...",
  "author_photo_url": "https://substack-post-media.s3.amazonaws.com/...",
  "author_follower_count": 209469,
  "email": "casey@platformer.news",
  "author_twitter_url": "https://twitter.com/CaseyNewton",
  "author_linkedin_url": null,
  "author_facebook_url": "https://www.facebook.com/...",
  "author_instagram_url": "https://instagram.com/crumbler",
  "website_url": "https://www.platformer.news",
  "publication_id": 7976,
  "publication_name": "Platformer",
  "publication_subdomain": "platformer",
  "publication_url": "https://www.platformer.news",
  "publication_description": "News at the intersection of Silicon Valley and democracy...",
  "publication_logo_url": "https://bucketeer-e05bbc84-baa3-437e-9518-adb32be77984...",
  "publication_custom_domain": "www.platformer.news",
  "publication_created_at": "2019-03-29T13:28:21.009Z",
  "publication_subscriber_count": 176000,
  "publication_subscriber_count_string": "176K+",
  "publication_subscriber_count_visible": true,
  "publication_is_paid": false,
  "publication_payments_enabled": false,
  "publication_founding_plan_name": "Mystery Tier",
  "publication_has_posts": true,
  "author_is_active": true,
  "author_has_likes": true,
  "author_profile_set_up_at": "2021-04-22T18:51:48.648Z",
  "author_profile_disabled": false,
  "author_is_verified": false,
  "scraped_at": "2025-11-07T19:19:13.110957+00:00"
}
```

#### Mode 2: `"information"` - Newsletter + Post Information

Includes all fields from Mode 1, plus a `posts` array:

```json
{
  // ... all newsletter metadata from Mode 1 ...
  "posts": [
    {
      "url": "https://www.platformer.news/trump-won-heres-what-comes-next/",
      "title": "Trump won. Here's what comes next.",
      "subtitle": "What a second Trump administration could mean for the internet",
      "date": "2024-11-07",
      "author": "Casey Newton",
      "likes": 13,
      "comments_count": 43,
      "content_preview": "Good morning, and wow. Donald Trump won a second term as president...",
      "word_count": 2051
    }
    // ... up to maxPostsPerNewsletter posts ...
  ]
}
```

#### Mode 3: `"information_and_content"` - Full Article Content

Includes all fields from Mode 2, plus full `content` for each post:

```json
{
  // ... all newsletter metadata ...
  "posts": [
    {
      "url": "https://www.platformer.news/trump-won-heres-what-comes-next/",
      "title": "Trump won. Here's what comes next.",
      "subtitle": "What a second Trump administration could mean for the internet",
      "date": "2024-11-07",
      "author": "Casey Newton",
      "likes": 13,
      "comments_count": 43,
      "content_preview": "Good morning, and wow. Donald Trump won a second term as president...",
      "word_count": 2051,
      "content": "Good morning, and wow. Donald Trump won a second term as president...\n\n[Full 2000+ word article text extracted here]\n\n..."
    }
    // ... up to maxPostsPerNewsletter posts with full content ...
  ]
}
```

### 🔧 Use Cases

- **B2B Outreach**: Build targeted lists of newsletter creators with contact information for partnership opportunities
- **Market Research**: Analyze newsletter subscriber counts, engagement metrics, and publishing frequency
- **Competitive Analysis**: Track competitor newsletters, their growth, and content strategies
- **Content Analysis**: Extract full article content for competitive research, trend analysis, or content strategy insights
- **Lead Generation**: Find potential customers or partners in specific niches based on newsletter topics and engagement
- **Training Data Collection**: Gather high-quality written content for AI/ML applications (respect copyright and usage rights)
- **Publishing Patterns**: Analyze posting frequency, optimal publishing times, and content length strategies
- **Engagement Research**: Study which topics, titles, and content formats drive the most likes and comments

### 💰 Pricing

This Actor uses **pay-per-event** pricing:

- You're charged per newsletter URL processed, whether extraction succeeds or fails
- Failed attempts (invalid URLs, inaccessible newsletters, parsing errors) are charged the same rate — compute is consumed regardless of outcome
- Validate your URLs before submitting to avoid charges on dead or invalid newsletters
- Transparent usage-based billing

### 🏷️ Tags

Newsletter scraping, Substack, B2B outreach, lead generation, market research, email discovery, social media extraction

### 🤖 My Other Actors

If this Actor is useful to you, check out the rest of my scrapers:

| Actor | What it does |
|---|---|
| [**Substack Leads Scraper**](https://apify.com/digispruce/substack-leads-scraper) | Find newsletter creators to pitch. Extracts author contact email, social profiles (Twitter, LinkedIn, Facebook, Instagram), subscriber counts, and publication metadata from Substack newsletters. |
| [**Reddit Posts Scraper ($0.99/1k posts)**](https://apify.com/digispruce/reddit-posts-scraper) | Scrape Reddit posts from any subreddit or keyword search: title, body, score, upvote ratio, flair, media, author, and community — as structured JSON. No login or Reddit API key needed. |

See all of them on [my Apify profile](https://apify.com/digispruce).

# Actor input Schema

## `newsletterUrls` (type: `array`):

List of newsletter URLs to scrape. Each URL will be processed one by one.

## `postScrapingMode` (type: `string`):

Choose what post data to extract for each newsletter. 'Post information' includes titles, dates, engagement metrics, and content previews. 'Post information + text content' also includes the full article text.

## `maxPostsPerNewsletter` (type: `integer`):

Maximum number of posts to scrape per newsletter (only applies when post scraping is enabled).

## `delayBetweenRequests` (type: `integer`):

Delay between HTTP requests in milliseconds to avoid rate limiting.

## `maxRetries` (type: `integer`):

Number of times to retry scraping a newsletter that fails. Total attempts will be maxRetries + 1.

## Actor input object example

```json
{
  "newsletterUrls": [
    {
      "url": "https://platformer.com"
    },
    {
      "url": "https://lennysnewsletter.com"
    },
    {
      "url": "https://andrewsullivan.substack.com"
    }
  ],
  "postScrapingMode": "information_and_content",
  "maxPostsPerNewsletter": 12,
  "delayBetweenRequests": 3000,
  "maxRetries": 2
}
```

# Actor output Schema

## `newsletterData` (type: `string`):

Complete dataset of scraped newsletter information including author profiles, publication details, subscriber metrics, social media links, and optionally recent posts with engagement metrics (when 'includePosts' is enabled). Each item contains: author details (ID, name, handle, bio), contact information (email, social links), publication info (name, URL, description, subscriber count), activity status, and when posts are enabled: array of recent posts with titles, dates, engagement metrics, content previews, and aggregate engagement statistics.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "newsletterUrls": [
        {
            "url": "https://platformer.com"
        },
        {
            "url": "https://lennysnewsletter.com"
        },
        {
            "url": "https://andrewsullivan.substack.com"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("digispruce/substack-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "newsletterUrls": [
        { "url": "https://platformer.com" },
        { "url": "https://lennysnewsletter.com" },
        { "url": "https://andrewsullivan.substack.com" },
    ] }

# Run the Actor and wait for it to finish
run = client.actor("digispruce/substack-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "newsletterUrls": [
    {
      "url": "https://platformer.com"
    },
    {
      "url": "https://lennysnewsletter.com"
    },
    {
      "url": "https://andrewsullivan.substack.com"
    }
  ]
}' |
apify call digispruce/substack-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,digispruce/substack-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Y3nUpCqMMubwjLCVz/builds/BchIkUNamHj7wnp3r/openapi.json