# X / Twitter Scraper - 42 Fields Per Post (`parseforge/x-com-scraper`) Actor

Scrapes tweets from X.com (formerly Twitter) using GraphQL API without authentication. Supports direct URLs, user timelines, and search filtering.

- **URL**: https://apify.com/parseforge/x-com-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Social media, Developer tools
- **Features:** Limited permissions
- **Stats:** 954 total users, 24 monthly users, 100.0% runs succeeded, 16 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $0.22 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner-v4.webp)](https://apify.com/parseforge?fpr=vmoqkp)

### X.com Posts Scraper

**Scrape X.com posts from user profiles and post URLs, up to a million per run.** Each post comes with its full text, engagement counts, view count, timestamp and author profile. No API key and no login needed. Export to CSV, JSON, Excel or XML.

X's official API needs a developer account, project approval and a paid tier for any real volume. This Actor reads the same public data a logged-out visitor sees on x.com and returns it in one fixed schema. Keyword search across all of X is possible only if you add your own X session (see below), because X allows no search at all to logged-out visitors.

| Who uses it | What they scrape X.com for |
|---|---|
| Market researchers | Track what a set of influential accounts are saying about a product category this month. |
| PR and comms teams | Monitor executive or brand account posts for sentiment and narrative shifts. |
| Data journalists | Collect a corpus of posts around a breaking event from known sources. |
| Growth marketers | Analyze competitor posting cadence and top-performing content by engagement. |

### What it does

The Actor collects posts from user profiles and direct post URLs and returns each one as a row with full text, engagement counts and the author's profile.

- **Profiles:** give usernames or profile URLs and it returns their posts, newest first.
- **Single posts:** give post URLs (https://x.com/user/status/123) and it returns those posts.
- **Keyword filter:** add a search term to keep only the posts from those accounts that contain it.
- **Keyword search across X (optional, needs your session):** with your own X session cookies, the search term also runs as a search of all of X, latest posts first.
- **Enrichment (optional):** attach Google Trends momentum, YouTube interest or Apollo.io company profiles to each post.

### What you get for very large accounts

For some very large accounts (for example elonmusk, BarackObama, nytimes, naval, BillGates), X does not give logged-out visitors their timeline. It returns the account's latest few posts on the profile page and, through its API, a fixed set of about 100 all-time top posts from past years, with no way to page further back.

The Actor handles this case as follows:

- The latest posts from the profile page come first, in date order, with `isTopPostsFallback: false`.
- The all-time top posts follow, with `isTopPostsFallback: true`. They are real posts with real counts, but they are not the account's recent timeline and they skip long stretches of time.
- The run log says when an account was served this way.

If you need only recent posts, filter out rows where `isTopPostsFallback` is true. Accounts of normal size are not affected: their timeline pages back through time as usual.

### What you can do with X.com data

**Monitor brand or executive accounts.**

A comms team lists five executive usernames, sets maxItems to 5000, and exports a CSV of their posts to audit messaging consistency.

**Research a topic across known voices.**

A market researcher enters ten fintech founder usernames and the search term 'regulation', then keeps only the matching posts.

**Collect posts around a breaking event.**

A journalist feeds the Actor a list of post URLs from a news thread and exports the full text and timestamps.

**Benchmark competitor content performance.**

A growth marketer scrapes a competitor's profile, sorts by views or likes in a spreadsheet, and looks at which post formats got the most replies.

### Why choose this scraper

| | What you get |
|---|---|
| **No API key or approval** | Reads the public, logged-out view of X. No developer account and no OAuth app. |
| **Up to 1,000,000 posts per run** | Set the volume once. Pagination, proxy rotation and retries are handled for you. |
| **Honest rows** | Top posts served instead of a timeline are flagged, and failed targets come back as unbilled error rows with a code. |
| **Fixed flat schema** | Every post arrives with the same columns, ready for pivot tables, BI tools or a database. |

### Configure the run

Give usernames, profile URLs or post URLs. The optional search term filters the collected posts. The Input tab lists every parameter.

A first run with the defaults:

```json
{
 "maxItems": 10,
 "usernames": [
 "elonmusk"
 ]
}
```

A larger pull from two accounts, keeping only posts that mention Artemis:

```json
{
 "maxItems": 200,
 "usernames": ["NASA", "NASAArtemis"],
 "searchQuery": "artemis"
}
```

#### Proxy

Runs use Apify Proxy datacenter IPs by default. That is enough for public profiles and posts. When X rate-limits a request, the Actor switches to a new IP with a new guest token and retries, a few times at most. You can pick Residential in the Proxy field if an account keeps coming back empty.

#### Optional: your own X session for keyword search

Logged out, X has no search at all: the search page redirects to login and the search API refuses guest requests. If you want the search term to search all of X, you can add the cookies of your own logged-in X session.

1. Log in to x.com in Chrome with a secondary account. Do not use your main account: X can lock accounts that are used for automated searching.
2. Open DevTools (F12), then Application, then Cookies, then https://x.com.
3. Copy the value of `auth_token` into "X session: auth_token cookie" and the value of `ct0` into "X session: ct0 cookie". You can instead paste the whole cookie string into the third field.

How the session is used:

- The values are stored encrypted by Apify, are never written to the log or the dataset, and are sent only to x.com.
- Search requests keep one IP for the whole run and wait a few seconds between pages.
- If X answers 401, 403 or 429, the Actor stops using the session at once. The search then comes back as an unbilled error row with `errorCode: "INVALID_SESSION"`, and the profiles and posts you listed are still collected.
- Search results have `collectedFrom: "search"` and are not re-filtered by the local keyword rules.

The logged-in search path has been checked against X's answers to invalid cookies only. It has not been run with a real account, so treat it as experimental.

### Output fields

Every post row has the same columns as before, plus three newer ones:

| Field | Meaning |
|---|---|
| `viewCount` | Views X shows under the post, or null when X does not report it. |
| `isTopPostsFallback` | True for all-time top posts X served in place of the timeline (see above). False otherwise. |
| `collectedFrom` | `timeline`, `profilePage`, `post` or `search`. |

A target that produced nothing (a handle that does not exist, a suspended or protected account, a deleted post, a rejected session) is written as one error row with `error`, `errorCode` and `target`. Error rows are never charged.

### Pricing

Pay per result: **$0.25 per 1,000 posts** on the Free plan, a little less on paid plans. Each run also has a start fee of about $0.08 per GB of memory (the default 256 MB counts as one). Error rows are not charged.

| Posts collected | Approximate cost (Free plan) |
|---|---|
| 100 posts | $0.03 plus the start fee |
| 1,000 posts | $0.25 plus the start fee |
| 10,000 posts | $2.50 plus the start fee |

**Optional connector enrichment:** with `enrichOutput` on, each post whose connector lookup (Apollo or Trends) comes back adds a `connector-enrichment` event at **$0.005** ($5 per 1,000). Leave it off and you pay only the prices above.

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [X.com Posts Scraper](https://apify.com/parseforge/x-com-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to X.com through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/x-com-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting no results?**

Check that the usernames exist and are not suspended, and that your search term, if any, appears in their posts. Each target that produced nothing has an error row in the dataset that says why. A run that collects no posts at all ends as FAILED with the reason in its status message, and nothing is charged for it. Some accounts return an empty timeline to logged-out visitors even though they have posts; the Actor then falls back to the posts on the profile page, and a residential proxy may also help.

**The newest posts of a big account are there, but then the dates jump back years.**

That is X's top-posts behavior for very large accounts, described above. Those rows have `isTopPostsFallback: true`.

**The Actor stops before reaching my maxItems limit.**

It stops when it runs out of matching posts. For the very large accounts above, only the latest few posts and about 100 top posts exist in the logged-out view. If you filter by keyword, try a broader term or remove it.

**I get an error about enrichment failing.**

Make sure you have selected a valid enrichment connector and that its MCP server is reachable. If you use the override fields for proxy URL or token, clear them and try again with the defaults.

**The dataset has fewer columns than I expected.**

Enrichment columns appear only when enrichOutput is on and a connector is selected.

### FAQ

| Question | Answer |
|---|---|
| Do I need an X.com API key or login? | No, not for profiles and posts. A login (your own session cookies) is needed only for keyword search across all of X. |
| Can I search all of X for a keyword without listing usernames? | Only with your own X session, as described above. Without it, the search term filters the posts of the accounts you list, and a search term with no accounts ends the run with an explanation. |
| What is the difference between startUrls and usernames? | startUrls accepts post or profile URLs. usernames accepts handles. You can use both: if they name the same account it is scraped once, and if they disagree the URLs win and the ignored usernames are listed in the run log. |
| Does the run fail if my input is wrong? | Input that can be understood is repaired (a leading @, a missing https://, a profile URL typed into the usernames field). The run ends as FAILED only when it collects no posts at all, and then the status message says why. |
| Why are some rows marked isTopPostsFallback? | X served that account's all-time top posts instead of its timeline to a logged-out visitor. The rows are real, but not in date order. |
| How many posts can I collect in one run? | Set maxItems from 1 to 1,000,000. The Actor stops at your limit or when no more posts are available. |
| What does the enrichment feature do? | It attaches cross-platform context to each post: Google Trends momentum and YouTube interest for the topic, or Apollo.io company data for the post author, depending on the connector. |
| What export formats are supported? | CSV, JSON, Excel or XML, straight from the Apify platform. |
| Can I scrape replies to a specific post? | No. The Actor does not fetch reply threads. Give it the individual post URLs you need. |
| Is this legal? | The Actor reads publicly available pages. You are responsible for complying with X's terms of service and the laws that apply to you. Using your own session for search is your decision and your account. |
| How do I scrape a private account? | You can't. Protected accounts come back as an error row with errorCode PROTECTED. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by X Corp. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `maxItems` (type: `integer`):

How many posts to collect per run.

## `startUrls` (type: `array`):

Direct URLs to scrape. Supports post URLs (https://x.com/username/status/1234567890) and profile URLs (https://x.com/username). x.com and twitter.com are both accepted, with or without https://. If you also fill Usernames below, these URLs take precedence and any extra usernames are listed in the run log.

## `searchQuery` (type: `string`):

Keyword filter applied to the posts collected from the URLs and usernames above. Quote "an exact phrase" to require it, and prefix -word to exclude it. Searching all of X is not possible logged out: without a session this only filters the accounts you list. If you add your own X session below, the same term also runs as a search across all of X (latest posts).

## `usernames` (type: `array`):

List of X.com usernames to scrape posts from. A leading @ is accepted and removed, and a full profile URL is accepted too. Combine with the search term to filter a user's posts by keyword.

## `proxyConfiguration` (type: `object`):

Apify Proxy datacenter IPs are used by default and are enough for public profiles and posts. Pick Residential if X keeps rate-limiting your runs or serves empty timelines. When X answers with a rate limit, the run switches to a new IP and retries a few times.

## `sessionAuthToken` (type: `string`):

Only needed for keyword search across all of X. Use a secondary X account, not your main one: X can lock accounts that are used for automated searching. How to get it: 1) log in to x.com in Chrome with that account; 2) open DevTools (F12), go to Application, then Cookies, then https://x.com; 3) copy the value of auth_token here and the value of ct0 in the next field. The value is stored encrypted, is never written to the log or the dataset, and is only sent to x.com.

## `sessionCt0` (type: `string`):

The ct0 cookie from the same browser session as auth_token (DevTools, Application, Cookies, https://x.com). Both are required.

## `sessionCookies` (type: `string`):

Alternatively, paste the whole Cookie header of a logged-in x.com request (for example from a cookie export extension). auth_token and ct0 are read from it.

## `enrichOutput` (type: `boolean`):

Toggle on to add Google/YouTube/News momentum + growth rates to every tweet, keyed by the topic (search query, mentioned username, or hashtag). Default off (free of charge). Requires a connector below.

## `enrichConnector` (type: `string`):

Pick Apollo.io (primary) to enrich each tweet's author with their company profile (size, funding, industry, LinkedIn). Or pick Trends MCP (secondary) for cross-platform topic momentum.

## `enrichTool` (type: `string`):

Override the auto-picked tool name. Leave empty for auto-detect.

## `mcpProxyUrlOverride` (type: `string`):

Diagnostic only. Bypasses Apify's MCP proxy. Leave empty for normal use.

## `mcpTokenOverride` (type: `string`):

Diagnostic only. Used with URL override. Leave empty for normal use.

## Actor input object example

```json
{
  "maxItems": 10,
  "usernames": [
    "elonmusk"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  },
  "enrichOutput": false
}
```

# Actor output Schema

## `tweets` (type: `string`):

Complete dataset with all scraped tweets including text, author, engagement metrics, media, and metadata

## `overview` (type: `string`):

Overview view of tweets with key fields displayed in a table format

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "maxItems": 10,
    "usernames": [
        "elonmusk"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/x-com-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "maxItems": 10,
    "usernames": ["elonmusk"],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("parseforge/x-com-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "maxItems": 10,
  "usernames": [
    "elonmusk"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call parseforge/x-com-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/x-com-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/bQRZfubEYXaXORDQj/builds/6qJbhv9FmBLODrgg5/openapi.json