Reddit Scraper: Posts & Full Comment Threads, Excel Rows, MCP

Reddit Scraper: posts and full comment threads, Excel rows, MCP

This Actor returns public Reddit posts with their comment threads, either nested as JSON or flattened to one Excel row per comment with depth and path. It needs no Reddit login or API key, costs $0.003 per row, and an AI assistant such as Claude can call it directly through Apify's MCP server.

Output	Switch	What one reply looks like (illustrative)
Nested JSON (default)	includeComments: true	{"id": "xyz789", "author": "curious_dev", "score": 42, "body": "How do you handle rate limits?", "replies": [{"id": "def456", "parent_id": "t1_xyz789", "depth": 1, "body": "Backoff and a small page size.", "replies": []}]}
One row per comment (Excel, CSV)	includeComments: true plus commentRows: true	post_id: abc123, comment_id: def456, parent_id: t1_xyz789, depth: 1, path: 1.1, author: op_dev, score: 17, indented_body: " Backoff and a small page size."

Nested JSON keeps the tree for code and for a model that reads the whole thread. commentRows writes each reply directly under the comment it answers, so a spreadsheet reads the thread top to bottom; every comment and reply is then its own $0.003 row. The full field lists are in the Output and Comments in Excel or Google Sheets sections below.

🔌 Use from Claude, Cursor or any MCP client

A user's public review: "Tried over 10 Reddit scrapers, this is the only one Claude Code approved for proper threading of posts and comments together."

You can connect an AI assistant to this Reddit Scraper through Apify's MCP server. It uses your Apify account and this Actor's pricing. Ask the assistant for a post with its comments and it gets the post and every reply with depth, parent_id and permalink, so it can follow who answered whom. Add commentRows: true to the request when you want the same thread as one row per comment.

{"mcpServers": {"apify": {"url": "https://mcp.apify.com/?tools=benthepythondev/reddit-scraper",
  "headers": {"Authorization": "Bearer YOUR_APIFY_TOKEN"}}}}

No Apify account? Use the MCPize server instead. Reddit: Posts, Search, Comments & History runs the same kind of Reddit reads behind one MCPize key. The free plan has 100 calls a month. Add this to Claude Desktop or Cursor:

{"mcpServers": {"reddit": {"type": "http", "url": "https://reddit-posts-search-comments.mcpize.run/mcp",
  "headers": {"Authorization": "Bearer YOUR_MCPIZE_API_KEY"}}}}

Use with: Reddit Archive on MCPize for posts older than Reddit's 1,000-post list limit, SocialBlade Creator Stats to check the growth of a creator you find in a thread, and Jobs Data Intelligence to pair hiring threads with live job listings.

📅 Reddit RSS ends November 13, 2026

Here is what still works, and how:

• Reddit's API still answers this Actor, so a subreddit, search or user feed keeps working with no Reddit login.
• Run it hourly with onlyNew and a watchName: each run returns only the posts no earlier run returned, at $0.003 per post.
• Apify serves the run as an RSS feed you paste into your reader (below). Full 3-step setup.

Feed URL template (needs a token, see the note under step 3): https://api.apify.com/v2/actor-tasks/<TASK_ID>/runs/last/dataset/items?format=rss&status=SUCCEEDED&token=<YOUR_TOKEN>

An hourly watch is 720 runs a month. Runs with nothing new cost only the $0.00005 start event ($0.036 a month); a subreddit with 300 new posts a month adds $0.90, about $0.94 a month before account discounts.

Extract Reddit posts, comments, user history and search results as clean, structured data, with a selected Markdown, HTML or plain-text content field on each post, comment and nested reply. Original text fields and word/token counts remain available. No Reddit login or user-supplied API key: point it at a subreddit, a post URL, a username or a search query and get back the available posts and comments with scores, images and nested comment hierarchies. Source access and complete-thread availability vary. Export to JSON/CSV/Excel, run on a schedule, call via API, or connect to Make, Zapier or n8n.

Transparent pricing: $0.003 per dataset row plus a $0.00005 run-start event per GB of allocated memory (minimum one). Posts and top-level comments are separate rows; nested replies stay inside their parent comment's replies field. A 10-post run at 1 GB costs $0.03005 before account discounts.

Choose the amount of data first: start with includeComments: false for a post list. Enabling comments adds top-level dataset rows and result charges; nested replies remain inside the parent row. Use a small post/comment limit and inspect partial-thread coverage before scheduling a larger run.

Need historical data?

This Actor reads what Reddit's own lists show, which ends after about 1,000 posts per list. For a backfill or research dataset of old posts and comments by UTC date range (back to 2008 for r/Python), use the Reddit Archive Scraper.

Why do Reddit keyword searches return unrelated posts?

Reddit's own search ranks by loose relevance, so a query like home assistant also brings back posts that match only one of the words. Set mode: search and strictMatch: true. The Actor then keeps a post only when its title or text contains every word of your query and drops the rest before they are saved, so you pay only for posts that match.

How do I get more than 1,000 posts?

Reddit ends a subreddit feed after about 950 posts and a search after about 240. Set deepScrape: true and maxPosts to the number you want: the Actor also reads the other sort orders and time ranges and saves every post once (2,451 posts of r/python instead of 999 in our tests). sinceDate keeps only posts from a UTC date on, which suits scheduled runs. For history by date range, use the Reddit Archive Scraper.

Quick start

Open the tested Python-posts example or start with three posts and no comment expansion:

{
  "mode": "subreddit",
  "subreddit": "Python",
  "sort": "top",
  "timeFilter": "week",
  "maxPosts": 3,
  "includeComments": false,
  "outputFormat": "text",
  "includeImages": false
}

Use content for the selected representation and content_format to identify it. The Full Content dataset view displays both fields. The original selftext, selftext_markdown, selftext_html, body, body_markdown and body_html fields remain available to existing integrations. A link-only post can have empty content; its title and destination URL remain separate.

text removes markup from source HTML while retaining paragraph/list boundaries. When source HTML is missing, it preserves the original text, which may still contain Markdown. html decodes Reddit's outer HTML encoding; if HTML is absent, it escapes the original text inside a paragraph. This field contains source markup, not a sanitized webpage component. markdown uses the existing Markdown representation.

includeImages: false clears the images array and sets thumbnail to null. It does not rewrite a post's URL or remove links from its original content.

📰 Reddit RSS ends November 13, 2026: how to keep a feed of new subreddit posts

Checked against Reddit's own announcements on October 26, 2026.

What Reddit is changing, and when

Reddit announced these dates on September 30, 2026 in r/modnews and r/redditdev (news coverage: TechWyse, TechRepublic):

Date	What happens (Reddit's wording, shortened)
October 31, 2026	New requests for public API access are no longer accepted
November 13, 2026	Reddit stops supporting RSS feeds
November 30, 2026	Registration closes for the app migration bounty program
January 12, 2027	API access starts to close for apps and users that have not registered
March 2027	Public access closes for all remaining apps and users

For moderators Reddit points to Devvit apps such as Discord Relay. For everyone else it says: "If you use RSS for feeds outside of a community you moderate, there is no replacement."

Replace a Reddit RSS feed in 3 steps

1. Paste the subreddit. Your old feed link names it: https://www.reddit.com/r/selfhosted/.rss (or /r/selfhosted/new/.rss) is the subreddit selfhosted. You can type the name or paste the whole old link into the subreddit field; the Actor reads the name out of it. A search feed (/search.rss?q=…) maps to mode: search with searchQuery, a user feed (/user/<name>/.rss) to mode: user with username.

2. Turn on onlyNew and give the watch a name, then run it once:

   {
     "mode": "subreddit",
     "subreddit": "selfhosted",
     "sort": "new",
     "maxPosts": 100,
     "onlyNew": true,
     "watchName": "selfhosted-feed",
     "includeComments": false,
     "outputFormat": "text"
   }
   

   The first run returns the newest posts and marks where the watch begins (the log says Watch: first run, … posts). Every later run returns only posts that no earlier run of this watch returned, oldest first.

3. Schedule it hourly. Save the input as a Task (the Save as a new task button under the input), then Console → Schedules → Create new, add the Task and choose @hourly (or */15 * * * * for every 15 minutes).

Read it as RSS. Point your feed reader at the Task's last successful run: https://api.apify.com/v2/actor-tasks/<TASK_ID>/runs/last/dataset/items?format=rss&status=SUCCEEDED&token=<YOUR_TOKEN>. Each run holds only that run's new posts, so let the reader check at least as often as the schedule runs, and use a separate token limited to this Task (Console → Settings → API & Integrations → Create a new token → Limit token permissions), never your main token: the feed link carries it in plain text and Apify requires a token for private Task results. Or have new posts pushed to you: in the Task's Integrations tab, send each successful run to Slack, email or a webhook instead of polling.

Tips: set maxPosts above the number of posts the target gets between two runs (the log warns when every post read was new, which means some may have been missed). Two schedules on the same subreddit need different watchName values. A run with nothing new saves no rows, so it costs only its start event.

A keyword or brand watch, no Reddit login and no API key. Reddit's search also returns posts that only relate to your words: in New order, 15 of the 30 newest results for "home assistant" lacked one of the words (October 26, 2026). Add strictMatch: true and a post is kept only when its title or text contains every word of the query; the posts left out are not saved and not charged. At $0.003 a post, 300 real mentions a month cost $0.90:

{
  "mode": "search",
  "searchQuery": "home assistant",
  "maxPosts": 100,
  "onlyNew": true,
  "strictMatch": true,
  "watchName": "home-assistant-mentions",
  "outputFormat": "text"
}

When to use a sibling Actor instead

You want	Use
A feed of one subreddit, one search or one user	This Actor with onlyNew (above)
New posts from many subreddits in one run, or a date range	Subreddit Scraper (onlyNew, monitorId)
New posts that mention a keyword anywhere on Reddit, past Reddit's ~240 results per search	Reddit Search Scraper (onlyNew, monitorId, strictMatch)
Older posts and comments by date, for a backfill or a RAG dataset	Reddit Archive Scraper

What the API closure means for this Actor

This Actor reads posts, searches and user lists from Reddit's API, and since October 22, 2026 it reads comment threads from the public Arctic Shift archive. Reddit's closure of public API access (from January 12, 2027, fully in March 2027) will reach the route it uses for posts. We will update this README with what still works as each date passes. The Reddit Archive Scraper reads public archive projects instead of Reddit's API; how fresh and complete those archives stay after March 2027 depends on those projects, so test with a small run before relying on them.

🤖 What is the Reddit Scraper?

It turns any Reddit page into a structured dataset. Choose one of four modes — subreddit, single post + comments, user posts or a Reddit search — add optional filters for sort order, time range, date and result caps, then Run. The Actor reads Reddit's public OAuth API (no browser or user login needed), paginates automatically, and can expand Reddit's "load more comments" and "continue this thread" stubs. The returned count may be below num_comments because of your cap, removed content or source limits. Posts and top-level comments are dataset rows; nested replies retain their hierarchy inside those rows.

What data does it extract?

• Post title, body and content in markdown, html or text (whichever you pick)
• Scores & engagement — score, upvote_ratio, num_comments, total awards
• Author & subreddit (with subreddit ID) and the post permalink + url
• Nested comment threads with full hierarchy, depth, scores and timestamps
• User post history across any public account
• Images & media — image URLs with width/height, plus thumbnails
• Post type & flags — self/video, NSFW, spoiler, stickied, locked, post hint, domain
• Flair (link_flair_text) and timestamps (created_utc)
• AI-ready stats — word_count and estimated token_count for every post and comment

⬇️ Input

Run it four ways — pick a mode and fill in the matching field:

Field	Description
mode	subreddit, post, user, or search
subreddit	Subreddit to scrape, e.g. python or r/python (subreddit mode)
postUrl	Full URL of a Reddit post (post mode)
username	Reddit username to pull posts from (user mode)
searchQuery	Search term to find posts (search mode)
searchSubreddit	Optionally limit a search to one subreddit
strictMatch	Search mode: keep only posts whose title or text contains every word of the query. Off by default
sort	hot, new, top, rising, controversial, or relevance (search)
timeFilter	hour, day, week, month, year, all (for top/controversial)
maxPosts	Max posts to save; 0 = as many as Reddit's list gives. One subreddit feed ends after about 950 posts, one search after about 240
deepScrape	Read past that end: the other orders and time ranges of the same subreddit or search are read too and every post is saved once, until maxPosts. Off by default
onlyNew	Monitoring: each run returns only the posts earlier runs have not returned (subreddit, search and user mode). Off by default
watchName	Optional name that keeps two watches on the same target apart
maxComments	Max comments per post, 0 = unlimited
includeComments	Also scrape comments for each post in subreddit mode
expandMoreComments	Resolve "load more" / "continue thread" stubs (on by default)
sinceDate	Subreddit mode only: posts on or after this UTC date (YYYY-MM-DD)
outputFormat	markdown, html, or text, returned in content throughout the comment tree
includeImages	false returns images: [] and thumbnail: null; default true preserves them
delaySeconds	Delay between requests to respect rate limits

Example input

{
  "mode": "subreddit",
  "subreddit": "ArtificialInteligence",
  "sort": "top",
  "timeFilter": "week",
  "maxPosts": 100,
  "includeComments": true,
  "maxComments": 200,
  "outputFormat": "markdown",
  "includeImages": true
}

⬆️ Output

Posts and top-level comments are dataset rows; nested replies remain inside replies. Export JSON to preserve the tree, or switch on commentRows for one row per comment and reply in CSV/Excel.

A post has the following shape. This is an illustrative record, not a live result:

{
  "id": "abc123",
  "title": "How I built an AI agent that scrapes Reddit",
  "url": "https://www.reddit.com/r/ArtificialInteligence/comments/abc123/how_i_built_an_ai_agent/",
  "permalink": "/r/ArtificialInteligence/comments/abc123/how_i_built_an_ai_agent/",
  "selftext": "Here's my complete guide to building a scraping agent...",
  "selftext_markdown": "Here's my complete guide to building a **scraping agent**...",
  "content": "Here's my complete guide to building a **scraping agent**...",
  "content_format": "markdown",
  "author": "ai_developer",
  "subreddit": "ArtificialInteligence",
  "subreddit_id": "t5_2qh0y",
  "score": 1250,
  "upvote_ratio": 0.97,
  "num_comments": 89,
  "is_self": true,
  "is_video": false,
  "post_hint": "self",
  "domain": "self.ArtificialInteligence",
  "thumbnail": "https://b.thumbs.redditmedia.com/example.jpg",
  "images": [
    { "url": "https://i.redd.it/example.jpg", "width": 1200, "height": 800, "caption": null }
  ],
  "created_utc": "2026-06-20T10:30:00",
  "total_awards_received": 3,
  "link_flair_text": "Discussion",
  "over_18": false,
  "spoiler": false,
  "stickied": false,
  "locked": false,
  "word_count": 850,
  "token_count": 1200
}

An illustrative comment, returned in post mode or in subreddit mode with includeComments:

{
  "id": "xyz789",
  "post_id": "abc123",
  "parent_id": "t3_abc123",
  "permalink": "/r/ArtificialInteligence/comments/abc123/how_i_built_an_ai_agent/xyz789/",
  "body": "This is brilliant — how do you handle rate limits?",
  "body_markdown": "This is brilliant — how do you handle rate limits?",
  "content": "This is brilliant — how do you handle rate limits?",
  "content_format": "markdown",
  "author": "curious_dev",
  "score": 42,
  "ups": 42,
  "downs": 0,
  "created_utc": "2026-06-20T11:05:00",
  "edited": false,
  "is_submitter": false,
  "stickied": false,
  "depth": 0,
  "replies": [],
  "total_awards_received": 0,
  "word_count": 8,
  "token_count": 11
}

📗 Comments in Excel or Google Sheets

JSON keeps the tree. In this Actor a post's comments come back nested: each comment row carries its answers in replies, and those carry theirs. That is the right shape for code, and it is why a CSV or Excel export of this Actor shows the top-level comments only, with the replies squeezed into replies/0/... columns.

For a flat sheet, switch on commentRows (One row per comment). Every comment and reply becomes its own row, written directly after the comment it answers, so the sheet reads like the thread top to bottom. Each row has type, post_id, post_title, comment_id, parent_id, depth (0 is a comment on the post), path (for example 1.3.2: the third reply to the first comment's third reply), author, score, created_utc, body and indented_body (the body with four spaces per level). Pricing is unchanged: each comment row is one dataset row at the row price above, and the replies are rows too, so a thread with 200 comments and replies is 200 rows. Without the option the output is exactly as before.

What three rows of a thread look like (columns picked: post_id,path,depth,author,score,indented_body; illustrative values):

post_id	path	depth	author	score	indented_body
abc123	1	0	op_dev	42	How do you handle rate limits?
abc123	1.1	1	curious_dev	17	Backoff and a small page size.
abc123	1.1.1	2	op_dev	5	That fixed it, thanks.

1. Turn on Include Comments (or use Post mode) and One row per comment (Excel-ready), then run.
2. On the run's Export button pick Excel (XLSX) or CSV. To keep only the columns you read, choose them under fields, for example post_id,path,depth,author,score,created_utc,indented_body. The same file comes from the API: https://api.apify.com/v2/datasets/<datasetId>/items?format=xlsx&fields=post_id,path,depth,author,score,created_utc,indented_body.
3. Do not sort the comments by date: the rows are already in thread order (post, then each comment followed by its replies). To restore the thread after sorting, sort on path within one post_id.
4. indented_body already shows the tree; filter depth to 0 for top-level comments only.
5. Several posts in one run: sort or filter by post_id only, which keeps each thread's own order.

Without the option, a CSV or Excel export shows the top-level comments only, with replies squeezed into replies/0/... columns. The Reddit Comments Scraper also returns flat rows if you only need comments from post links.

To fill a Google Sheet without downloading a file, run Google Sheets Import & Export with the run's dataset id, or add it as an integration on a scheduled task so every run lands in the same sheet.

💡 Use cases

• 💬 Conversation research: pull public discussions and expert Q&A from any community as clean Markdown with token counts, for reading, tagging and analysis.
• 📊 Market & brand research: track mentions, pain points and product feedback across subreddits to understand what people actually think.
• 😊 Sentiment analysis: scrape thousands of comments on a launch, brand or topic and feed scores, text and threads straight into your model.
• ✍️ Content & trend research: mine top posts in your niche for viral angles, headlines and audience questions worth writing about.

❓ FAQ

Can Claude read whole Reddit threads? Yes. Through the hosted MCP server, the get_post_comments tool takes up to 10 post links per call and returns each thread's comments in thread order, with "load more comments" and "continue this thread" stubs expanded. A call returns at most 200 comment rows, replies included.

How do I scrape a subreddit? Set mode to subreddit, enter the subreddit name (e.g. python), pick a sort and maxPosts, then Run. You get every matching post with scores, flair, images and AI-ready text. Flip on includeComments to also pull each post's comments.

Can I scrape all the comments on a single post? Use mode: post, paste the postUrl, and set maxComments (0 removes the requested cap). With expandMoreComments on (the default), the Actor attempts to expand more-comment stubs. A complete thread is not guaranteed. The cap includes nested replies, while dataset rows and result charges count only posts and top-level comments. Export JSON to retain the nested replies objects.

What if Reddit blocks the Actor? Reddit refuses requests for stretches of time. When it does, the Actor reads the same subreddit feed or search from the public Arctic Shift archive instead, newest first, and sets data_source to arctic-shift-archive on every post that came from it (the field is empty for posts read from Reddit). Archive data can lag Reddit by minutes, can miss posts, and keeps posts deleted on Reddit after they were archived. Its search matches titles only, and a search across all of Reddit looks only at title matches in 30 large subreddits. Sorting by hot, rising or controversial returns newest first. Arctic Shift limits requests dynamically (HTTP 429 with an X-RateLimit-Reset header), so large runs on the archive can be slower or end early. Data credit: Arctic Shift, a community archive of Reddit. Filter on data_source if you only want posts read from Reddit.

Where do comments come from? Since October 22, 2026 Reddit refuses comment threads to visitors without a login. The Actor then reads a post's comments from the public Arctic Shift Reddit archive, which holds them minutes after they are written: same fields, top-level comments ordered by score, and a comment deleted on Reddit after it was archived can still appear. Posts, search and subreddit lists come from Reddit itself as before.

Can I search Reddit or scrape a user's posts? Yes. Use mode: search with a searchQuery (optionally limited to one subreddit via searchSubreddit), or mode: user with a username to pull that account's public post history.

Do I need an API key? No. You do not provide a Reddit login, client ID, or secret. The Actor obtains an anonymous app-only OAuth token internally, then reads public Reddit data.

Where is my chosen format? Read content and content_format on each post, comment and nested reply. Existing fields retain their original meanings. The word and token counts describe the original source text, not the selected representation.

Can I get only recent posts? Use sort: top or controversial with a timeFilter (hour/day/week/month/year/all). In subreddit mode, sinceDate (YYYY-MM-DD) keeps posts on or after midnight UTC on that date. Old pinned or ranked posts are skipped without stopping pagination. Invalid calendar dates and sinceDate in other modes fail before scraping. This filter checks the posts Reddit exposes; it does not search a complete archive.

Can I run it on a schedule or via API? Yes — schedule recurring runs in Apify, call it via the API/SDK, or connect it to Make, Zapier or n8n to push fresh Reddit data into your stack.

Can it scrape private or deleted content? No. Only public posts and comments are available — deleted/removed items and private subreddits can't be accessed without authentication. A private, quarantined, banned or missing subreddit, a deleted post and a suspended account end the run right away as a success with zero rows; the log and the status message name the reason.

How many posts can it return? Up to your maxPosts cap, as far as Reddit's lists go. Reddit ends a subreddit feed after about 950 posts and a search after about 240, and by default the Actor pages to the end of the one list you chose; the status message then says that more is available. With deepScrape: true it goes on with the other orders and time ranges of the same subreddit or search and saves every post once: 2,451 posts of r/python instead of 999, and 1,000 posts for the query "home assistant" instead of 240 in our tests. The chosen list comes first, the further posts follow in the order they were found. Every further list is read to its end, so such a run takes longer: a 600-post search that read three lists took 52 seconds. Near the run's timeout the Actor stops reading and saves what it has. Many subreddits or queries in one run, a date range and an only-new mode are in the Subreddit Scraper and the Reddit Search Scraper; for complete history by date use the Reddit Archive Scraper.

Does this replace a Reddit RSS feed? For new posts, yes. Reddit stops supporting RSS feeds on November 13, 2026. A scheduled run with onlyNew returns each new post of a subreddit, search or user once, with title, author, link, text, score, comment count and date, and Apify serves the run's rows as RSS (format=rss). It does not return posts from private, quarantined or banned subreddits, and it does not deliver new comments as a feed. Set up: see Replace a Reddit RSS feed in 3 steps. Full guide with timeline, reader setup (Inoreader, Feedly, n8n) and a cost table: Keep a Reddit feed in your RSS reader after November 13. Short version on dev.to: Reddit RSS ends Nov 13: keep a feed of new subreddit posts. Each feed entry carries link (the Reddit post) and pubDate, so readers open and date it.

Will it keep working after Reddit closes its public API? Reddit says public API access starts to close on January 12, 2027 for apps that have not registered and closes for all remaining apps and users in March 2027. This Actor reads posts, searches and user lists from that API, so that closure will reach it. We will say in this README what still works as each date passes; we cannot promise more than that today.

Is scraping Reddit legal? It extracts publicly available data through Reddit's OAuth API and paces requests to respect rate limits. Use it responsibly for research and analysis, and follow applicable laws and Reddit's terms.

Does this supply a registered app or source approval? No. If your organization requires a registered Reddit application or an approved data agreement, confirm that requirement before adopting the Actor. A successful run does not establish approval for your intended use.

🔗 You might also like

• Reddit Archive Scraper — years of historical Reddit posts & comments
• Reddit Search Scraper — posts by keyword past Reddit's 240 per search, with a date range and an only-new mode
• Subreddit Scraper — the posts of many subreddits in one run, past Reddit's 950 per feed
• Reddit Comments Scraper — whole threads as a flat table, one row per comment or reply
• Lemmy Scraper — posts & comments from Lemmy communities
• Stock Sentiment Intelligence — social sentiment signals for stocks & tickers
• Tech Stack and Email Provider Detector — after a Reddit thread names a company, see which CMS, tools and email provider its domain runs

---

Keywords: reddit scraper, reddit api, scrape reddit, reddit data extraction, reddit comments scraper, subreddit scraper, reddit search api, reddit user scraper, reddit posts, sentiment analysis, social media scraping, markdown export, reddit json api, no api key, market research, content research.