# arXiv Papers Extractor (`cynix_dev/arxiv-papers`) Actor

Search arXiv and extract papers as clean typed records: title, abstract, authors, categories, DOI, and direct PDF links. No API key required.

- **URL**: https://apify.com/cynix_dev/arxiv-papers.md
- **Developed by:** [Cynix Dev](https://apify.com/cynix_dev) (community)
- **Categories:** AI, Developer tools
- **Features:** Limited permissions
- **Stats:** 3 total users, 2 monthly users, 96.6% runs succeeded, 1 bookmarks
- **User rating**: No ratings yet

## Pricing

from $22.50 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## arXiv Papers Extractor

Search **arXiv** and get papers back as clean, typed records — title, full abstract, authors, categories, DOI, and a direct PDF link. Uses arXiv's official public API, so there is no scraping, no API key and no rate-limit contract to negotiate.

### What it does

arXiv hosts millions of preprints across physics, mathematics, computer science, quantitative biology, finance and statistics. Its official API returns Atom XML, which is awkward to work with. This Actor queries that API for you and emits flat JSON records you can load into a notebook, a vector database or a spreadsheet without writing a parser.

You can search with plain language (`diffusion models`), use arXiv's fielded query syntax for precision, filter by category, or skip search entirely and fetch exact papers by ID.

### Features

- **Official arXiv API** — no HTML scraping, so results don't break when a page template changes.
- **Fielded query syntax** supported: `ti:` (title), `au:` (author), `abs:` (abstract), `cat:` (category), `all:`, combined with `AND` / `OR` / `ANDNOT`.
- **Category filter** — narrow to `cs.LG`, `cs.CL`, `stat.ML`, `q-fin.TR` and any other arXiv category.
- **Fetch by ID** — pass a list of arXiv IDs in `idList` to retrieve exact papers.
- **Sort control** — by `submittedDate`, `lastUpdatedDate` or `relevance`, ascending or descending.
- **Full abstracts included**, not truncated snippets — ready for embedding or summarisation.
- **Direct PDF URLs** on every record, so downstream download steps are trivial.

### What people use it for

- Literature reviews — pull everything published on a topic in a date window.
- RAG and embeddings — abstracts are clean, well-formed text, ideal for vector search.
- Research monitoring — schedule a daily run on a category and get new preprints automatically.
- Competitive/technical intelligence — track which labs and authors publish in your field.
- Dataset building — assemble a labelled corpus of abstracts by category for ML experiments.

### Query syntax cheat-sheet

Plain text works fine, but arXiv's fielded syntax gives you much sharper results:

| Goal | Query |
| --- | --- |
| Phrase in the title | `ti:"reinforcement learning"` |
| Specific author | `au:hinton` |
| Phrase in the abstract | `abs:"chain of thought"` |
| Everything by an author in a field | `au:lecun AND cat:cs.LG` |
| Two topics together | `all:transformer AND all:distillation` |
| Exclude a topic | `all:llm ANDNOT all:survey` |

#### Useful categories

`cs.LG` machine learning · `cs.CL` computation and language (NLP) · `cs.CV` computer vision · `cs.AI` artificial intelligence · `cs.CR` cryptography and security · `stat.ML` statistics/ML · `q-fin.TR` trading and market microstructure · `math.OC` optimisation. The full list is on arXiv's category taxonomy page.

### Input

Everything is optional. `query` plus `category` covers most searches; `idList` overrides searching and fetches specific papers instead.

| Field | Type | Default | What it does |
| --- | --- | --- | --- |
| `query` | string | `transformer` | Free text (e.g. 'diffusion models') or fielded arXiv syntax (ti:, au:, abs:, all:, cat:). Example: ti:"reinforcement learning" AND au:silver |
| `category` | string | — | arXiv category filter, e.g. cs.LG, cs.CL, stat.ML, q-fin.TR. ANDed with the query. |
| `idList` | array | `[]` | Fetch specific papers by ID, e.g. 2301.00001. Overrides paging behavior. |
| `maxResults` | integer | `50` | Cap on papers returned. Range 1–2000. |
| `sortBy` | string | `submittedDate` | Result ordering. Options: `relevance`, `lastUpdatedDate`, `submittedDate`. |
| `sortOrder` | string | `descending` | Ascending or descending. Options: `ascending`, `descending`. |

#### Input example

```json
{
  "query": "machine learning",
  "category": "cs.LG",
  "maxResults": 10,
  "sortBy": "submittedDate",
  "sortOrder": "descending"
}
```

### Output

One record per paper, including the arXiv ID and version, the full abstract, the author list, category tags, and links to both the abstract page and the PDF.

Every dataset record contains: `arxivId`, `version`, `title`, `abstract`, `authors`, `primaryCategory`, `categories`, `published`, `updated`, `doi`, `journalRef`, `comment`, `absUrl`, `pdfUrl`, `fetchedAt`.

#### Output example

A real record from a run of this Actor:

```json
{
  "arxivId": "2608.19181",
  "version": "v1",
  "title": "Beyond Teacher Likelihood: Group-Calibrated On-Policy Distillation for Long-Context Reasoning",
  "abstract": "On-policy distillation (OPD) trains a student on its own responses using dense token-level guidance from a stronger teacher. In long-context tasks, however, token-level teacher support can favor locally plausible respons …",
  "authors": [
    "Zhu Zhang",
    "Jixun Wang",
    "Xiaoang Xu",
    "… +6 more"
  ],
  "primaryCategory": "cs.LG",
  "categories": [
    "cs.LG",
    "cs.AI",
    "cs.CL"
  ],
  "published": "2026-08-19T17:54:58Z",
  "updated": "2026-08-19T17:54:58Z",
  "doi": null,
  "journalRef": null,
  "comment": "20 pages, 5 figures",
  "absUrl": "https://arxiv.org/abs/2608.19181",
  "pdfUrl": "https://arxiv.org/pdf/2608.19181v1",
  "fetchedAt": "2026-08-20T02:12:58.385Z"
}
```

Export the dataset as JSON, CSV, Excel, XML or JSONL from the Console, or pull it programmatically through the Apify API and any of the official clients.

### How to use it

1. Click **Try for free** (or **Start** if you already have an Apify account).
2. Fill in the input fields described above — the defaults already produce a working run.
3. Press **Start** and watch the log; results stream into the dataset as they are found.
4. When the run finishes, open the **Output/Storage** tab and export as JSON, CSV or Excel.

Runs can be scheduled (hourly, daily, weekly) and wired into Slack, Google Sheets, Zapier, Make, webhooks or your own backend through Apify integrations. Everything the Console does is also available over the [Apify API](https://docs.apify.com/api/v2).

### FAQ

#### Do I need an arXiv API key?

No. arXiv's API is public and free. The Actor paces its requests to stay inside arXiv's published rate guidance.

#### Can I get the full paper text, not just the abstract?

Each record includes a direct `pdfUrl`. Fetch and parse that PDF in a downstream step if you need the body text — this Actor deliberately returns metadata plus abstract, which is what most pipelines index.

#### How do I fetch new papers only?

Sort by `submittedDate` descending and schedule the run. Compare `arxivId` values against what you already stored, or push the dataset into your own database with a unique index on `arxivId`.

#### Why does a paper show a version suffix?

arXiv preprints are revised. `version` (`v1`, `v2`, …) tells you which revision the record describes — worth storing if you track changes.

#### Is there a result limit?

`maxResults` caps each run. arXiv's API paginates large result sets, so for very large harvests run several narrower queries (by category or date range) rather than one enormous one.

### Other Actors by cynix_dev

| Actor | What it does |
| --- | --- |
| [CoinGecko Markets — Crypto Data API](https://apify.com/cynix_dev/coingecko-markets) | Live cryptocurrency market data from CoinGecko as clean typed JSON: price, market cap, volume, 24h change, ATH/ATL, … |
| [FX Rates & History](https://apify.com/cynix_dev/fx-rates-history) | Latest and historical foreign exchange rates (ECB reference data) as clean, typed dataset records. |
| [USGS Earthquakes — GeoJSON Extractor](https://apify.com/cynix_dev/usgs-earthquakes) | Pull live and historical earthquakes from the USGS FDSN event service as clean typed JSON: magnitude, place, time, lat/lon/depth, … |
| [Launch Library 2 — Rocket Launch Tracker](https://apify.com/cynix_dev/launch-library-launches) | Upcoming, previous, and specific rocket launches from The Space Devs' Launch Library 2 API. |
| [Open Food Facts Extractor](https://apify.com/cynix_dev/open-food-facts) | Search and extract food-product data from Open Food Facts as clean typed JSON: name, brand, ingredients, allergens, nutrition … |
| [Website to RAG Chunks](https://apify.com/cynix_dev/web-to-rag-chunks) | Crawl any website and turn its pages into clean, chunked, metadata-rich Markdown records ready for RAG pipelines, vector stores, … |

### Legal and responsible use

This Actor collects only publicly available information. You are responsible for how you use the data, including compliance with the target site's Terms of Service, robots directives, copyright, and data protection law such as GDPR and CCPA. Do not use it to gather personal data without a lawful basis.

### Support and feedback

Found a bug, hit a site change, or need an extra field? Open a ticket on the **Issues** tab of this Actor — issues are read and fixed. Feature requests and custom-scraper enquiries are welcome through the same channel.

# Actor input Schema

## `query` (type: `string`):

Free text (e.g. 'diffusion models') or fielded arXiv syntax (ti:, au:, abs:, all:, cat:). Example: ti:"reinforcement learning" AND au:silver

## `category` (type: `string`):

arXiv category filter, e.g. cs.LG, cs.CL, stat.ML, q-fin.TR. ANDed with the query.

## `idList` (type: `array`):

Fetch specific papers by ID, e.g. 2301.00001. Overrides paging behavior.

## `maxResults` (type: `integer`):

Cap on papers returned.

## `sortBy` (type: `string`):

Result ordering.

## `sortOrder` (type: `string`):

Ascending or descending.

## Actor input object example

```json
{
  "query": "transformer",
  "category": "cs.LG",
  "idList": [],
  "maxResults": 10,
  "sortBy": "submittedDate",
  "sortOrder": "descending"
}
```

# Actor output Schema

## `dataset` (type: `string`):

One record per paper: title, abstract, authors, categories, DOI, abs/PDF URLs.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "category": "cs.LG"
};

// Run the Actor and wait for it to finish
const run = await client.actor("cynix_dev/arxiv-papers").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "category": "cs.LG" }

# Run the Actor and wait for it to finish
run = client.actor("cynix_dev/arxiv-papers").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "category": "cs.LG"
}' |
apify call cynix_dev/arxiv-papers --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,cynix_dev/arxiv-papers"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/TavuY3UvOqzejMs7H/builds/qwkSt0hfWkXRp44GH/openapi.json