# arXiv Scraper (`dami_studio/arxiv-scraper`) Actor

arXiv scraper for papers by title, author, abstract or category, in arXiv's own query syntax. Each row holds the arXiv id, title, full abstract, authors, categories, first-submitted and last-revised dates, the DOI when there is one, and abstract and PDF links. $2.00 per 1,000 papers.

- **URL**: https://apify.com/dami_studio/arxiv-scraper.md
- **Developed by:** [Dami's Studio](https://apify.com/dami_studio) (community)
- **Categories:** AI, Developer tools, Other
- **Features:** Limited permissions
- **Stats:** 6 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $1.94 / 1,000 papers

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

**arXiv scraper for arXiv queries, one row per paper**: the arXiv id, the title, the whole abstract,
every author, the categories, the first-submitted and last-revised dates, the DOI when there is one,
and direct links to the abstract page and the PDF.

Metadata and links only. The PDF link is there for you to fetch; this actor does not download or
read the paper, and arXiv publishes no citation counts, so there are none here either.

| | |
|---|---|
| **Input** | An arXiv search query, using arXiv's own field prefixes |
| **Output** | One row per paper: id, title, full abstract, authors, categories, both dates, DOI, abstract and PDF links |
| **Ceiling** | 30,000 papers per query, which is arXiv's own depth limit |
| **Account needed** | None, and no API key |
| **Price** | $2.00 per 1,000 papers on the Free plan, a little less on paid plans. The free plan's $5 a month covers about 2,500 |

### 🔍 What arXiv Scraper does

It runs your query against arXiv and pages through the results, pausing between pages because arXiv
asks callers to. A paper that turns up on two pages is delivered once.

The query uses arXiv's own syntax. It is worth two minutes, because it is what makes a search
precise:

| Prefix | Searches |
|---|---|
| `all:` | every field |
| `ti:` | the title |
| `au:` | the authors |
| `abs:` | the abstract |
| `cat:` | the category, like `cs.CL` or `math.AG` |

Combine them with `AND`, `OR` and `ANDNOT`. `ti:attention AND au:vaswani` does exactly what it looks
like. Results come back by relevance, by submission date or by last revision, and both date orders
put the newest first.

### 📋 What data you get from each arXiv paper

| What you get | Field |
|---|---|
| The arXiv id, without its version marker | `arxivId` |
| The title and the whole abstract | `title`, `abstract` |
| Every author, in the order on the paper | `authors` |
| The category it was filed under, and all the others | `primaryCategory`, `categories` |
| When it was first submitted and when it was last revised | `publishedAt`, `updatedAt` |
| The journal DOI, when the author added one | `doi` |
| The abstract page and the PDF | `absUrl`, `pdfUrl` |

### ▶️ How to scrape arXiv papers

1. Open [arXiv Scraper](https://apify.com/dami_studio/arxiv-scraper) and click **Try for free**.
2. Type a query into **Search query**, for example `cat:cs.CL AND abs:retrieval`.
3. Pick a **Sort by**. Use `submittedDate` when what you care about is new work.
4. Set **Max papers**. Start around 20 while you tune the query, then click **Start**.
5. Download the dataset as JSON, CSV or Excel, or read it from the Apify API.

### 💰 How much does it cost to scrape arXiv?

**$2.00 per 1,000 papers** on the Free plan. Paid plans pay a little less: $1.98 on Starter, $1.96 on Scale and $1.94 on Business. On the free plan, the $5
Apify gives you each month covers about 2,500 papers.

You pay per paper row delivered. Duplicates across pages are dropped before they reach you,
diagnostic rows are not charged, and a query that matches nothing costs you nothing. Set a maximum
cost on the run and it stops when it gets there.

### 📥 What you give it

```json
{
  "query": "all:large language models",
  "sortBy": "relevance",
  "maxItems": 50
}
```

| Field | Default | What it is |
|---|---|---|
| `query` | none, the form starts with `all:large language models` | The arXiv query. Use the prefixes above. A bare phrase with no prefix is not what arXiv expects. |
| `sortBy` | `relevance` | `relevance`, `submittedDate` or `lastUpdatedDate`. Both date orders are newest first. |
| `maxItems` | `50` | Papers to return, from 1 to 30,000. |
| `notionConnector` | none | Optional. Writes every paper into your own Notion as well as the dataset, a quick start on a literature review. |
| `notionParentId` | none | Optional. The Notion data source to write into. Leave it empty and the pages land privately in your workspace. |
| `proxyConfiguration` | off | Optional, and off by default because arXiv is public and a normal run does not need it. |

**There is no date-range field.** To narrow by time, sort on `submittedDate` and cut the rows on
`publishedAt` yourself.

### 📤 What you get back

A real row from a run on 2 October 2026, with the abstract cut short:

```json
{
  "ok": true,
  "charged": true,
  "arxivId": "2501.05032",
  "title": "Enhancing Human-Like Responses in Large Language Models",
  "abstract": "This paper explores the advancements in making large language models (LLMs) more human-like. ...",
  "authors": ["Ethem Yağız Çalık", "Talha Rüzgar Akkuş"],
  "primaryCategory": "cs.CL",
  "categories": ["cs.CL", "cs.AI"],
  "publishedAt": "2025-01-09T07:44:06Z",
  "updatedAt": "2026-02-01T13:32:03Z",
  "doi": null,
  "absUrl": "https://arxiv.org/abs/2501.05032v2",
  "pdfUrl": "https://arxiv.org/pdf/2501.05032v2"
}
```

| Field | How to read it |
|---|---|
| `arxivId` | The id with any trailing version marker removed, so `2501.05032v2` arrives as `2501.05032`. Stable, and the dedupe key inside a run. |
| `abstract` | The whole abstract, with line breaks collapsed into single spaces. |
| `publishedAt`, `updatedAt` | First submission and latest revision. On a paper never revised they are the same. Some rows carry the submission day only, at midnight, with `updatedAt` `null`. |
| `doi` | `null` on most papers. arXiv only has one when the author added it after journal publication. |
| `absUrl`, `pdfUrl` | Usually the exact version the search returned, like `v2` above. Nothing is downloaded for you. |
| `primaryCategory` | The one category the author filed it under. `categories` holds all of them, without repeats. |

### 🧾 Reading the output

Two kinds of row can land in your dataset.

| Row | How to spot it | Billed |
|---|---|---|
| A paper | `ok: true`, `charged: true` and an `arxivId` | yes |
| A diagnostic | `ok: false` and an `errorCode` | no |

`charged: true` is simply the marker every paper row carries. Use it, or `ok`, to tell results from
diagnostics, and do not read it as a receipt.

| Code | What it means |
|---|---|
| `BAD_INPUT` | The query was empty. Give one in arXiv syntax. |
| `NO_RESULTS` | The query ran and matched no papers. Loosen it, or check the prefix spelling. |
| `NOT_FOUND` | arXiv answered 404 for the request. |
| `RATE_LIMITED` | arXiv asked for a slower pace. Re-run with a smaller `maxItems`. |
| `SERVER_ERROR` | arXiv answered with a server error. Usually passes on its own. |
| `NOT_SERVED` | This paper was not served this run. Re-run it later. Runs before 13 October 2026 wrote `BLOCKED` here, and runs from 13 to 24 October wrote `NETWORK`. |
| `NETWORK` | arXiv could not be reached. |
| `TIME_LIMIT` | The run got close to its time limit and stopped on its own. The papers delivered before that are complete and charged. Give the run more time, or ask for fewer papers, to get the rest. |

**Rows arrive at the end.** Everything is collected before anything is written, so a large run shows
an empty dataset until it finishes. If the run gets close to its time limit first, it writes the
papers it has and adds a `TIME_LIMIT` row. Keep a first run small while you get the query right.

**The default table view has no `ok` column**, so a diagnostic row renders as one blank line. Switch
to **All fields** or export as JSON.

### 💡 What people use it for

- Watching a category like `cat:cs.CL` on a schedule, sorted by `submittedDate`, and keeping only the
  `arxivId` values you have not seen before.
- Building a reading list on a topic, with every abstract already in the row to read or summarise.
- Following one author with `au:` to see what they have put up and when it was last revised.
- Collecting PDF links for a set of papers to fetch and process in your own pipeline.

### 🚧 What it does not do

- **No full text.** Abstracts and links. The PDF is yours to fetch from `pdfUrl`.
- **No citation counts, no references, no related-paper graph.** arXiv does not publish them.
- **No peer-review status.** A paper on arXiv may be a preprint, a submitted manuscript or a
  published article, and nothing on the row tells you which.
- **`doi` is usually `null`.** Most preprints have none, and arXiv does not go looking.
- **No date-range filter, and no ascending order.** Sort by date and trim the rows yourself.
- **About 30,000 results per query, at most.** That is arXiv's depth limit, not a setting here. A
  pull that size runs for close to an hour, the default run timeout, and rows are only written at the
  end, so split a broad query into narrower ones rather than raising the number.
- **The run finishes even when every request failed**, so read `ok` rather than the run status. A run the time limit stopped before any paper came back is no exception: it ends as succeeded too, with a `TIME_LIMIT` row and an amber status message.
- It is not an official arXiv tool. Dami's Studio is independent and is not affiliated with or endorsed by arXiv or Cornell University, or by any other company named on this page.

### 🧭 Which research scraper do you need?

| If you want | Use |
|---|---|
| arXiv preprints with full abstracts and PDF links | This one |
| Anything with a registered DOI, with its journal, publisher and ISSN | [Crossref Scraper](https://apify.com/dami_studio/crossref-scraper) |
| Papers with readable abstracts, institutions and open-access links | [OpenAlex Scraper](https://apify.com/dami_studio/openalex-scraper) |
| Patents rather than papers | [Google Patents Search Scraper](https://apify.com/dami_studio/google-patents-search-scraper) |
| Book records by keyword, title or ISBN | [Books Scraper](https://apify.com/dami_studio/books-scraper) |
| Encyclopedia articles as plain text | [Wikipedia Scraper](https://apify.com/dami_studio/wikipedia-scraper) |
| arXiv, OpenAlex and Wikipedia searches as tools for an AI agent | [Research MCP Server](https://apify.com/dami_studio/research-mcp-server) |

### ❓ Questions people ask

#### Do I need an arXiv account or an API key?

No. arXiv's search is open to anyone.

#### Why did my arXiv search return nothing?

Usually the syntax. A bare phrase with no prefix is not a valid arXiv query; start it with `all:`
and try again.

#### Can I download the papers themselves?

Not from this actor. `pdfUrl` gives you a direct link to fetch.

#### Which date should I sort on?

`submittedDate` for when work first appeared, `lastUpdatedDate` for what was revised recently. They
differ a lot on papers that keep being updated.

#### Can I call it from code or connect it to an AI assistant?

The [API tab](https://apify.com/dami_studio/arxiv-scraper/api/python) has code for Python, JavaScript and the CLI. MCP
clients like Claude and ChatGPT use `https://mcp.apify.com/?tools=fetch-actor-details,dami_studio/arxiv-scraper`;
coding assistants can load [our skill
file](https://api.apify.com/v2/key-value-stores/BTBHU3DdiuoF4JOsK/records/apify-open-data-research-SKILL.md). All
three bill your Apify account at the same price.

#### Is scraping arXiv legal?

Individual papers carry their own licences, so check those before republishing
text. Apify's write-up on [the legality of web scraping](https://blog.apify.com/is-web-scraping-legal/)
is a good starting point, and we are not lawyers.

### 🆘 If something breaks

Open the **Issues** tab on the actor page. Send the run ID and the exact query string you used. The
`errorCode` and `hint` on the diagnostic row usually name the problem on their own.

# Actor input Schema

## `query` (type: `string`):

arXiv search query. Use arXiv field prefixes: all: (all fields), ti: (title), au: (author), abs: (abstract), cat: (category). Examples: "all:large language models", "cat:cs.CL", "ti:attention AND au:vaswani". Combine terms with AND / OR / ANDNOT.

## `sortBy` (type: `string`):

How to order results. Relevance ranks by match quality; Submitted date sorts by original submission; Last updated date sorts by most recent revision. Newest/most-relevant first.

## `maxItems` (type: `integer`):

Maximum number of papers to return. The actor paginates 100 per request and pauses ~3s between pages to respect arXiv's rate guidance. arXiv hard-limits total reachable results to about 30000.

## `notionConnector` (type: `string`):

Optional. Write each paper as a page into your Notion when the run finishes, handy for building a literature-review database. Authorize a Notion connector once in Settings → API & Integrations → MCP connectors, then pick it here. Leave empty to skip (default), results are always saved to the dataset regardless.

## `notionParentId` (type: `string`):

Optional. The Notion data source ID of the database to write papers into (only used if a Notion connector is set). Leave empty to create the pages privately in your workspace instead.

## `proxyConfiguration` (type: `object`):

Optional. arXiv is public and needs no login, so no proxy is used by default.

## Actor input object example

```json
{
  "query": "all:large language models",
  "sortBy": "relevance",
  "maxItems": 50,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

Scraped rows are stored in the default dataset (one row per result). Empty or failed runs return a single uncharged diagnostic row instead.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "all:large language models"
};

// Run the Actor and wait for it to finish
const run = await client.actor("dami_studio/arxiv-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "query": "all:large language models" }

# Run the Actor and wait for it to finish
run = client.actor("dami_studio/arxiv-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "all:large language models"
}' |
apify call dami_studio/arxiv-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,dami_studio/arxiv-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Uvnv5ugbNNsGaksVg/builds/U4PyPiIm5ceopHe1x/openapi.json