Substack Scraper - Newsletters, Posts & Authors

Substack Scraper — Newsletters, Posts, Authors & Leaderboards (No API Key, No Login)

Discover Substack newsletters by category and leaderboard rank, then scrape every post, author and publication. Point it at a category (technology, finance, crypto…) or a direct newsletter URL, and get structured post archives and publication metadata — titles, audience (free/paid), reactions, restacks, word counts, authors, domains and subscription tiers. Reads Substack's public API directly. No login, no API key, no proxy.

🏆 Why this Substack scraper?

Discovery + archive in one — enumerate top newsletters across 30+ categories, or pull a single publication's full post archive · post-level engagement (reactions, restacks, comments) · publication-level lead data (author, custom domain, tiers) · export to JSON / CSV / Excel. The unofficial Substack API alternative for content research, creator lead-gen and competitive intelligence.

---

✨ What this Actor does / Key features

• 🏆 Category leaderboards — enumerate the top publications in any of 30+ categories by subscribers and engagement, with leaderboard, paid or all ranking modes.
• 🔗 Direct newsletter targeting — paste a subdomain (lennysnewsletter.substack.com), a bare name (lennysnewsletter) or a custom domain (www.lennysnewsletter.com).
• 📰 Full post archives — pull every post from a publication, or run publication-only mode (0 posts).
• 📊 Post engagement — reactions, restacks, comment counts, word count and audience (free vs paid).
• 🎙️ Podcast fields — podcast URL and duration when a post is a podcast episode.
• 🏢 Publication lead data — publication name, subdomain, custom domain, author, description, category, language, subscription tiers, logo and cover.
• 🎛️ Powerful filters — date range, keyword, audience (free/paid) and language.
• 🔓 Fully keyless — Substack's public API and archive endpoints serve metadata anonymously; no account, key or proxy needed.
• 📤 Any format — export to JSON, JSONL, CSV, Excel, HTML or XML, or pull via the Apify API.

🚀 Quick start (3 steps)

1. Configure — add Category Slugs (e.g. technology, finance) with a Ranking Mode, or paste Direct Newsletter URLs. Set how many publications and posts to pull.
2. Run — click Start. The Actor enumerates the leaderboard (or resolves each URL), fetches each publication's archive and streams records into your dataset.
3. Get your data — open the Output tab and export to JSON, CSV, Excel, HTML or XML, or pull it via the Apify API.

📥 Input

Provide either category slugs or direct newsletter URLs. Everything else is optional.

Example — top technology newsletters + recent posts

{
  "categorySlugs": ["technology", "finance"],
  "rankingMode": "leaderboard",
  "maxPublicationsPerCategory": 25,
  "maxPostsPerPublication": 20
}

Example — one publication's full archive

{
  "newsletterUrls": ["lennysnewsletter.substack.com"],
  "maxPostsPerPublication": 500
}

Example — publication-only lead list for a category

{
  "categorySlugs": ["crypto"],
  "rankingMode": "paid",
  "maxPublicationsPerCategory": 100,
  "maxPostsPerPublication": 0
}

Field	Type	Description
categorySlugs	array	Substack category slugs (technology, finance, crypto…). Each becomes its own leaderboard task.
rankingMode	string	leaderboard, paid or all — how publications are ranked within a category.
newsletterUrls	array	Direct newsletter targets: subdomain, bare name or custom domain.
maxPublicationsPerCategory	integer	Cap publications enumerated per category.
maxPostsPerPublication	integer	Posts to pull per publication. 0 = publication-only mode.
dateFrom / dateTo	string	Optional post date-range filter (YYYY-MM-DD).
keyword	string	Optional keyword filter on post titles.
audienceFilter	string	all, free-only or paid-only posts.
language	string	Optional language filter.

Additional input fields

These optional fields are also available in the input form and API:

Field	Type	Default	Description
categoryRanking	string	"leaderboard"	How to sort publications within each category. 'leaderboard' = top newsletters (most subscribers + engagement), 'paid' = top paid newsletters, 'all' = all publications sorted by Substack's default ranking.
minPostDate	string	—	Filter posts to those published on or after this date (YYYY-MM-DD). Leave empty for no lower bound.
maxPostDate	string	—	Filter posts to those published on or before this date (YYYY-MM-DD). Leave empty for no upper bound.
keywordFilter	array	[]	Only save posts whose title or subtitle contains any of these terms (case-insensitive substring match, applied client-side after fetch). Examples: ['ai', 'gpt'], ['layoffs'], ['startup']. Leave empty for no filter.
includePublicationMetadata	boolean	true	When enabled, every post record is enriched with parent-publication fields (subdomain, custom domain, author name, hero description, category, language, free / paid subscription benefits, founded date). Disable for smaller per-row records.
alsoPushPublicationRecord	boolean	false	When enabled, the actor also pushes one record per discovered publication (with recordType: 'publication') in addition to per-post records. Useful when you want a denormalized list of newsletters in the same dataset.
proxyConfiguration	proxy	{"useApifyProxy": true}	Proxy for the public Substack API requests. Defaults to Apify Proxy (automatic pool). The Actor also falls back to a direct connection on the final retry, so it works without proxy too.

📤 Output

Records come in two shapes tagged by recordType. Here is a trimmed post record:

{
  "recordType": "post",
  "publicationName": "Lenny's Newsletter",
  "subdomain": "lennysnewsletter",
  "authorName": "Lenny Rachitsky",
  "postId": 154382910,
  "postSlug": "how-the-best-pms-prioritize",
  "postTitle": "How the best PMs prioritize",
  "postSubtitle": "A field guide to ruthless focus",
  "postType": "newsletter",
  "audience": "everyone",
  "postDate": "2026-07-01T13:00:00Z",
  "canonicalUrl": "https://www.lennysnewsletter.com/p/how-the-best-pms-prioritize",
  "coverImage": "https://substackcdn.com/image/.../cover.jpg",
  "reactions": 1284,
  "restacks": 212,
  "commentCount": 96,
  "wordCount": 2410,
  "sectionName": "Product",
  "category": "technology",
  "scrapedAt": "2026-07-06T12:00:00Z"
}

💡 Use cases

• Content & trend research — track which newsletters and posts are gaining traction across 30+ categories.
• Newsletter / creator lead generation — build lists of publications in a niche with authors, domains and tiers.
• Competitive intelligence — monitor a competitor's posting cadence, audience mix (free vs paid) and engagement.
• Market research — analyze topics, word counts and reactions across a category leaderboard.
• Archive backups — export a full publication's post archive to JSON or CSV.
• Influencer & sponsorship outreach — find high-engagement writers in your topic and their domains for partnership pitches.

👥 Who uses it

Content & growth marketers · newsletter operators & media analysts · creator-economy & sponsorship teams · competitive-intelligence analysts · researchers & data journalists · lead-gen teams building creator lists.

💰 Pricing

This Actor runs on a simple pay-per-result model — you pay for the records you extract, with no separate Apify platform fees to calculate. Try it on the free tier first, then scale up. See the Pricing tab on this page for the current rate.

❓ Frequently Asked Questions

Do I need a Substack login or API key? No. The Actor uses Substack's public API and archive endpoints, which serve publication and post metadata anonymously. No account, key or proxy is needed.

Can I scrape a specific newsletter directly? Yes. Add it to Direct Newsletter URLs — it accepts a subdomain (lennysnewsletter.substack.com), a bare name (lennysnewsletter) or a custom domain (www.lennysnewsletter.com).

Can I get only free or only paid posts? Yes. Use the Audience Filter to restrict results to free-only or paid-only posts, or keep all for both.

Which export formats are supported? Each run is stored in an Apify Dataset, downloadable as JSON, JSONL, CSV, Excel, HTML or XML, or available via the Apify API.

Is this a Substack API alternative?

Yes. Substack has no official public data API for bulk export, so this Actor acts as an unofficial Substack API alternative — it reads the public archive endpoints to return structured newsletter, post and author data.

How do I export Substack data to CSV or JSON?

Run the Actor on a category leaderboard or direct newsletter URL, then download the resulting Apify Dataset as CSV, JSON, Excel or XML. This gives you a clean Substack newsletter data export with no manual copy-paste.

Can I scrape Substack without an API key or login?

Yes. The scraper pulls Substack publication and post data anonymously through public endpoints, so you can scrape Substack without login, without an API key and without a proxy.

How do I find the top newsletters in a Substack category?

Add a category slug like technology or finance with ranking mode leaderboard, and the scraper enumerates the top publications by subscribers and engagement.

Can I export a single Substack publication's full post archive?

Yes. Add its subdomain, bare name or custom domain to Newsletter URLs and raise Max Posts Per Publication to pull the complete archive to CSV or JSON.

🔗 More social & content scrapers by logiover

Building a cross-platform content & creator dataset? Pair the Substack scraper with the rest of the suite:

Actor	What it does
Dev.to Articles Scraper	Articles by tag, author & feed
Product Hunt Daily Launches Scraper	Today's launches with votes & makers
LinkedIn Top Content Scraper	Top-performing LinkedIn posts
Reddit Subreddit Scraper	Posts from any subreddit by sort & window
Hacker News Search Scraper	HN stories & comments by keyword
YouTube Channel Scraper	Channel stats & full video lists
Pinterest Scraper	Pins, boards, profiles & save counts
Threads Scraper	Posts & profiles from Meta's Threads
Apple Podcasts Episode Scraper	Podcast shows & episodes
Stack Exchange Questions Scraper	Q&A across 170+ Stack Exchange sites

👉 Browse all logiover scrapers on Apify Store — 180+ actors across real estate, jobs, crypto, social media & B2B data.

⏰ Scheduling & integration

Schedule this Actor on Apify to track a category leaderboard or a competitor's archive daily or weekly. Export results to JSON, CSV or Excel, sync to Google Sheets, or push to your database, BI tools and webhooks through the Apify API. Connect it to Make, n8n or Zapier for automated content-monitoring pipelines.

🤖 AI Agents, MCP & RAG Integration

Substack Scraper works natively as an on-demand data tool for AI assistants and autonomous workflows.

Connect this Actor directly to Apify AI, Claude Desktop, Cursor, VS Code, Windsurf, LangChain, or LlamaIndex via the official Apify MCP Server — no custom wrapper code needed:

{
  "mcpServers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=logiover/substack-newsletter-scraper"
    }
  }
}

• Schema-typed inputs & outputs: Clean JSON records with explicit .actor/input_schema.json and .actor/output_schema.json definitions so LLM agents infer parameters and parse fields accurately without HTML noise.
• Bounded & predictable runs: Built-in volume caps and pay-per-result budget guards prevent runaway agent tool loops.
• RAG & automation ready: Stream run datasets straight into vector stores, n8n, Make, Zapier, or webhook pipelines.

💬 Support, Issues & Reviews

• Bug reports & target-site changes: Hit an unexpected empty result or selector change? Open a ticket in the Issues tab with your Run ID or sample input — we actively monitor issues and ship fixes fast.
• Feature & field requests: Need an additional data attribute or filter? Request it in the Issues tab.
• Enjoying this Actor? If it saved you engineering time, please consider leaving an honest review on Apify Store or bookmarking logiover/substack-newsletter-scraper — organic developer feedback helps others discover reliable tools and keeps maintenance prioritized.

⚖️ Legal

This Actor reads only publicly available Substack data and is intended for legitimate research, analytics and lead-generation use. You are responsible for complying with Substack's terms, GDPR and any applicable local laws. Not affiliated with Substack Inc.

---

Keywords: substack scraper, substack newsletter scraper, substack posts scraper, substack author scraper, substack leaderboard, substack api alternative, scrape substack without login, export substack data to csv, newsletter lead generation, creator economy data.

---

How much Substack data can you extract in one run?

A single run returned 234 articles in 98 seconds on the example input. That example is deliberately narrow; the result limit and the input list are both yours to widen. That is roughly 143 articles a minute.

There is no hard ceiling built into this Actor. What bounds a run is the result limit you set, the breadth of the input you give it, and how much Substack itself publishes. Because billing is per result, a wider run costs proportionally more and a narrow one costs almost nothing — so it is worth starting small and scaling once the output looks right.

Substack scraping — frequently asked questions

Do I need a Substack API key, account or login?

No. This Actor reads publicly available pages and feeds, so there is nothing to register for and no credentials to manage.

How do I export Substack data to CSV, Excel, JSON or Google Sheets?

Open the Storage tab after a run and pick the format you want — CSV, Excel, JSON, XML or an HTML table. For Google Sheets, Airtable or a database, use the dataset URL or the Apify API so the sheet refreshes itself on every run.

Can I schedule this Substack scraper to run daily or hourly?

Yes. Use the Schedules tab and pick a cadence. Daily is enough for most tracking work; pick hourly only if the articles you care about really do change that fast.

How many articles can I get, and what does it cost?

See the volume section above for a measured figure. Billing is per result, so you pay for the rows you actually receive and a run that finds nothing costs nothing.

Is scraping Substack legal?

This Actor collects only information that Substack publishes publicly. It does not log in, does not defeat access controls and does not touch private data. You remain responsible for how you use the output, including under GDPR and similar rules where personal data is involved.

What happens if Substack blocks the run or changes its layout?

A refused exit IP is swapped for a fresh one within a quarter of a second and retried, and rate limiting is waited out rather than fought. If nothing gets through, the run fails with the reason instead of reporting success with zero rows — so an empty result always means "no matches", never "the source would not answer".

Can I use this with an AI agent, LangChain or an MCP client?

Yes. The Actor is callable over the Apify API and through Apify's MCP server, so an agent can run it and read the dataset back as a tool call.

📝 Changelog

2026-11-11

• Added reliable terminal run status telemetry and structured Apify AI / MCP Server (mcp.apify.com) tool configuration.
• Added direct Issues & Reviews support links and refreshed live maintenance verification.

2026-11-01

• Monthly health check (2026-11-01): a live run on the current build completed successfully and returned data. Input options, output fields and pricing are unchanged.
• README refreshed: 2 links to renamed Actors updated, 1 link to retired Actors removed; documented 7 additional input fields.

2026-10-15

• Publications are now read four at a time and a request that gets no answer is abandoned after 15 seconds instead of 45, so one unresponsive newsletter no longer holds the whole run for minutes.
• Every request now has a hard time limit, so a proxy connection that stalls can no longer hold up the run or push it past its timeout.
• A run that reaches its maximum cost per run now stops cleanly with its results saved and a status message, instead of ending as aborted.
• A proxy group your account cannot use no longer stops the run at start-up; it falls back to the default Apify Proxy.
• Runs with a short timeout (API and AI-agent calls) now finish inside it with what they collected, instead of being cut off.
• Memory is managed within the run's own limit, which removes a rare out-of-memory crash on large pages.

2026-09-23

• Fleet-wide quality audit. Verified end to end against live data and re-checked the input schema, the output columns and the run configuration.
• Output verified on a live run: 32 columns returned, 87.0% of cells populated.
• Run reliability reviewed: 100.0% of public runs succeeded in the last 30 days.
• Input schema, output schema and pricing configuration reviewed.
• Noted that 1 column(s) came back empty in this sample (videoUploadId); these are under review.

2026-09-01

• Fleet-wide health check. Verified against this Actor's real run history: 30-day success rate, output row counts, per-field fill rates, and peak memory against the configured memory limit.
• Reviewed for the failure patterns that have cost this fleet runs — unguarded proxy setup, retry loops that can outlast the run's own time budget, and full-page HTML parsing that can exhaust a small container.
• No change to input, output fields or scraping logic.

2026-08-11

• Maintenance release: refreshed the build and dependencies.
• Re-verified live execution, non-empty structured output and dataset field/type integrity.
• Reviewed reliability (retries, pagination) and output quality as part of a full-fleet QA pass.

2026-08-01

• Completed the August 2026 full health check: verified empty/programmatic default, Console UI default, and two source-informed alternative inputs on Apify.
• Confirmed successful live execution, non-empty structured output, dataset-field/type integrity, and logical sample quality within the 5-minute quality window.

2026-07-06

• ✨ README overhaul: shields badge row, green highlight callout, trimmed output sample, collapsible full field reference (post + publication), ready-to-run example scenarios, social-suite cross-links and clearer quick-start.

2026-07-01

• Maintenance pass: re-verified end-to-end on live data and confirmed successful runs within the 5-minute quality window on the default input.
• Sharpened Store metadata (SEO title & description) and expanded the FAQ with high-intent, long-tail questions for easier discovery in Google and Apify Store search.
• Added ready-to-run example tasks that cover common real-world use cases.

2026-06-15

• Reliability pass: re-verified end-to-end on live data with real-world inputs. Routine maintenance build.

2026-06-07

• Docs: added coverage for Substack API alternative, exporting Substack data to CSV/JSON, and scraping Substack without login or API key.

---

*Maintained: last verified 2026-11-11 — checked end-to-end against live run history (reliability, output fields and types, and time/memory budgets).