# Wikipedia Scraper — Markdown, Infobox Data & Pageviews (`fabrikit/wikipedia-scraper`) Actor

Wikipedia scraper for AI and RAG: fetch articles by title, URL, search or category in any language as clean Markdown, with the infobox as key/value JSON, summary, sections, categories, Wikidata id, coordinates and pageviews. Public Wikimedia APIs, no proxy. From US$0.80 per 1,000 articles.

- **URL**: https://apify.com/fabrikit/wikipedia-scraper.md
- **Developed by:** [Fabrikit](https://apify.com/fabrikit) (community)
- **Categories:** AI, Developer tools, Other
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.80 / 1,000 article scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Wikipedia Scraper

Wikipedia Scraper extracts articles from wikipedia.org in any language edition and returns each one as a single JSON record: the article body as clean Markdown (or plain text / cleaned HTML), the infobox as key/value fields, the lead summary, the table of contents, categories, Wikidata id, main image, coordinates and human pageviews for the last N days. Pick articles by title or URL, by full-text search or by category. It reads Wikipedia through the public MediaWiki and Wikimedia REST APIs with a descriptive User-Agent, so there is no browser, no CAPTCHA and no proxy to pay for.

### What you get

| Field | Example (Machu Picchu) |
|---|---|
| `title`, `pageId`, `language`, `url` | Machu Picchu, 80019, en, https://en.wikipedia.org/wiki/Machu_Picchu |
| `description`, `summary` | "15th-century Inca citadel in Peru", the lead paragraph in plain text |
| `markdown` | `# Machu Picchu` + every section as `##` / `###`, lists, GFM tables, fenced code, LaTeX math (`$a^{2}+b^{2}=c^{2}$`) |
| `text` / `html` | the same body as plain text or cleaned HTML, when you choose that format |
| `wordCount` | 7015 |
| `infoboxType`, `infoboxTitle`, `infobox` | historic site, Machu Picchu, `{ "Built": "c. 1450 – c. 1470", "Elevation": "2,430 m (7,970 ft)", … }` |
| `sections[]` | title, level, number and anchor of every heading |
| `categories[]` | visible categories only (maintenance categories are left out) |
| `wikidataId`, `imageUrl`, `thumbnailUrl`, `latitude`, `longitude` | Q676203, main image, -13.16333333, -72.54555556 |
| `lastModified`, `revisionId`, `isDisambiguation` | time of the last edit, revision read, disambiguation flag |
| `languageCount`, `languageVersions[]` | 132 editions; the list of other-language titles and URLs on request |
| `pageviews`, `pageviewsDailyAverage`, `pageviewsFrom`, `pageviewsTo`, `pageviewsDaily[]` | 96642 human views in 30 days, 3221.4 per day; day-by-day series on request |
| `redirectedFrom`, `sourceType`, `sourceValue`, `position`, `scrapedAt` | how the article was reached and its rank |

What is removed from the body: citation markers like \[1], edit links, navigation boxes, maintenance banners, "For other uses" notes, image galleries and, by default, the trailing reference sections (See also, Notes, References, Further reading, External links, and their equivalents in German, Spanish, French, Portuguese, Italian, Dutch, Polish and Russian). "Bibliography" is kept, because in biographies it lists the person's works.

### Use cases

- **RAG and AI agents**: feed an LLM or a vector database with Markdown that keeps the heading structure (easy to chunk by section) and none of the page furniture. Use `summary` for short context and `markdown` for full context.
- **Knowledge graphs and entity enrichment**: the infobox gives structured facts (founded, headquarters, population, elevation, designer, license...) and `wikidataId` links each record to Wikidata.
- **Trend and demand research**: pageviews show which topics, products, places or people the public is reading about; compare a list of articles or a whole category.
- **Multilingual datasets**: the same article in several editions (pass the URLs, or turn on `includeLanguageLinks` to get the titles).
- **SEO and content research**: word counts, section outlines and categories of the top articles for a search phrase.

### Input

```json
{
    "articles": ["https://en.wikipedia.org/wiki/Large_language_model", "Machu Picchu", "https://de.wikipedia.org/wiki/Berlin"],
    "searchQueries": ["retrieval augmented generation"],
    "categories": ["Machine learning algorithms"],
    "language": "en",
    "maxItems": 50,
    "maxArticlesPerQuery": 10,
    "contentFormat": "markdown",
    "includeLinks": false,
    "dropReferenceSections": true,
    "includePageviews": true,
    "pageviewsDays": 30,
    "includeDailyPageviews": false,
    "includeLanguageLinks": false
}
```

- `articles`: titles (looked up in the `language` edition) or URLs from any edition, including mobile links and `?curid=` links. Redirects are followed and reported in `redirectedFrom`.
- `searchQueries`: Wikipedia's own full-text search, best match first; `maxArticlesPerQuery` articles per query.
- `categories`: articles filed directly in each category, up to `maxArticlesPerQuery` each. Subcategories are not followed.
- `maxItems` is exact and counts across all sources; an article reached twice (for example by title and by search) is saved and charged once.
- `contentFormat`: `markdown`, `text`, `html`, or `none` for metadata, infobox and summary only.
- `includeLinks` keeps links as `[text](https://...)`; off by default because plain text embeds better.
- `maxConcurrency` (1-5, default 3) sets how many articles are fetched at once.

### Output example

One record from the run above, with the long fields shortened:

```json
{
    "title": "Machu Picchu",
    "pageId": 80019,
    "language": "en",
    "url": "https://en.wikipedia.org/wiki/Machu_Picchu",
    "description": "15th-century Inca citadel in Peru",
    "summary": "Machu Picchu is a 15th-century Inca citadel located in the Eastern Cordillera of southern Peru on a mountain ridge at 2,430 meters (7,970 ft). …",
    "markdown": "# Machu Picchu\n\n**Machu Picchu** is a 15th-century Inca citadel located in the Eastern Cordillera of southern Peru …\n\n## Etymology\n\n…",
    "wordCount": 7015,
    "infoboxType": "historic site",
    "infoboxTitle": "Machu Picchu",
    "infobox": {
        "Type": "Llaqta (settlement)",
        "Cultures": "Inca civilization",
        "Location": "Machupicchu District, Urubamba, Cusco, Peru",
        "Built": "c. 1450 – c. 1470",
        "Elevation": "2,430 m (7,970 ft)",
        "Discovered": "Agustín Lizárraga (1902); Hiram Bingham (1911)",
        "UNESCO World Heritage Site: Location": "Latin America and the Caribbean",
        "Inscription": "1983 (7th Session)",
        "…": "…"
    },
    "sections": [{ "title": "Etymology", "level": 2, "number": "1", "anchor": "Etymology" }, "…"],
    "categories": ["Machu Picchu", "Archaeological sites in Peru", "World Heritage Sites in Peru", "…"],
    "wikidataId": "Q676203",
    "imageUrl": "https://thumb.wikimedia.org/wikipedia/commons/thumb/b/bb/Machu_Picchu%2C_2023_%28012%29.jpg/3840px-Machu_Picchu%2C_2023_%28012%29.jpg",
    "latitude": -13.16333333,
    "longitude": -72.54555556,
    "isDisambiguation": false,
    "revisionId": 1380992747,
    "lastModified": "2026-10-19T08:52:19Z",
    "languageCount": 132,
    "pageviews": 96642,
    "pageviewsDailyAverage": 3221.4,
    "pageviewsFrom": "2026-09-24",
    "pageviewsTo": "2026-10-23",
    "redirectedFrom": null,
    "sourceType": "article",
    "sourceValue": "Machu Picchu",
    "position": 2,
    "scrapedAt": "2026-10-24T08:27:16.078Z"
}
```

A `SUMMARY` record in the key-value store lists each source (search hits on Wikipedia, articles listed and saved), titles that do not exist, duplicates skipped and the number of requests.

### Pricing

Pay per event: **US$0.80 per 1,000 articles** (`article-result`, US$0.0008 each), plus Apify platform usage, which is small: three light API calls per article, no browser and no proxy.

| Run | Articles | Price |
|---|---|---|
| Default input (2 titles + 1 search of 3) | 5 | US$0.004 |
| Top 100 articles for 10 search phrases | 1,000 | US$0.80 |
| A 5,000-article knowledge base for RAG | 5,000 | US$4.00 |

Titles that do not exist and duplicates are not charged.

### How it compares

- **Wikipedia APIs directly**: free, but you need three different APIs (Action API for HTML and metadata, REST for the summary, Wikimedia for pageviews), the HTML still has to be cleaned and converted, and infobox tables differ between editions. This Actor does all of that and gives you one flat record per article, exportable as JSON, CSV or Excel.
- **Wikipedia dumps**: best for the whole encyclopedia at once (tens of GB of wikitext); overkill when you need hundreds or thousands of current articles, and wikitext is not Markdown.
- **Other Wikipedia Actors in the Store**: most return the plain-text extract or the full text without structure; this one keeps headings, tables, code and math as Markdown and adds the infobox and pageviews, at US$0.80 per 1,000.

### Related Actors

- [OpenAlex Scraper](https://apify.com/fabrikit/openalex-papers-scraper): the Actor this one is built on (same API client), for scholarly papers with abstracts and citations.
- [Website to Markdown for AI/LLM](https://apify.com/fabrikit/website-to-markdown-scraper): turns any other website into Markdown; its converter is reused here.

### Integrations

Use it from the Apify API (`/v2/datasets/{id}/items?format=json`), schedule it, or connect it to n8n, Make, Zapier, Google Sheets or Airbyte. AI agents can call it as a tool through the Apify MCP server, and LangChain / LlamaIndex can load the dataset directly.

### FAQ

**Is this legal?** Wikipedia text is published under CC BY-SA, which allows reuse with attribution: keep the `url` of each article with the text you reuse. The Actor reads only public article pages through Wikimedia's documented APIs, identifies itself with a descriptive User-Agent as Wikimedia asks, and does not collect editor or user data.

**Do I need a proxy?** No. Wikimedia's APIs accept direct connections, and the default input runs without any proxy. The `proxyConfiguration` field is there only in case your runs ever get HTTP 403 / 429.

**What are the limits?** A search can go up to 10,000 results deep (a limit of Wikipedia's search). Categories return the pages filed directly in them. The pageviews window is 1 to 365 days and ends yesterday (UTC); very new or rarely read articles show 0. Requests are spaced to stay inside Wikimedia's API etiquette, so expect about 2 articles per second.

**Which languages work?** Any Wikipedia edition: set `language` (en, de, es, fr, pt, ja, zh, simple...) or pass URLs from that edition. Infobox parsing was checked on English, German and Spanish articles.

**How fresh is the data?** Live: each article is read at run time, and `lastModified` tells you when it was last edited.

### Changelog

- **0.1** (October 2026): first release. Titles / URLs / curid, search and category sources, any language, Markdown / text / HTML / none, infobox fields and type, summary, sections, categories, Wikidata id, image, coordinates, language versions, pageviews with optional daily series.

***

### En español

Wikipedia Scraper extrae artículos de Wikipedia en cualquier idioma (por título, URL, búsqueda o categoría) y entrega cada uno como un registro JSON con el texto en Markdown limpio (o texto plano / HTML), la ficha (infobox) como campos clave-valor, el resumen, las secciones, las categorías, el id de Wikidata, la imagen, las coordenadas y las visitas de los últimos N días. Usa las APIs públicas de Wikimedia, sin navegador ni proxy. Ideal para RAG y agentes de IA. Precio: US$0,80 por cada 1.000 artículos.

# Actor input Schema

## `articles` (type: `array`):

Article titles ("Machu Picchu", in the edition set by Language) or full Wikipedia URLs in any language (https://de.wikipedia.org/wiki/Berlin, mobile en.m.wikipedia.org links and ?curid= links work too). Redirects are followed.

## `searchQueries` (type: `array`):

Full-text searches on Wikipedia (the same engine as the site's search box). Each query returns up to "Max articles per query or category" articles, best match first.

## `categories` (type: `array`):

Wikipedia categories ("Machine learning algorithms", "Category:Inca" or a category URL). Takes the articles filed directly in the category; subcategories are not followed.

## `language` (type: `string`):

Wikipedia language edition for titles, search queries and categories given without a URL: en, de, es, fr, pt, it, ja, zh, ru, nl, pl, simple... URLs always use their own edition.

## `maxItems` (type: `integer`):

Total number of articles to save across all sources (exact). An article reached twice is saved and charged once.

## `maxArticlesPerQuery` (type: `integer`):

How many articles to take from each search query and from each category.

## `contentFormat` (type: `string`):

Markdown (headings, lists, tables, code and LaTeX math kept), plain text, cleaned HTML, or none (metadata, infobox and summary only).

## `includeLinks` (type: `boolean`):

Keep links as [text](https://...) in the Markdown / HTML. Off gives cleaner text for embeddings and LLMs.

## `dropReferenceSections` (type: `boolean`):

Remove trailing sections that only hold references and links (See also, Notes, References, Further reading, External links and their equivalents in the main languages). Citation markers like \[1] are always removed.

## `includePageviews` (type: `boolean`):

Add the article's human pageviews (bots and crawlers excluded) for the last N days from the Wikimedia Pageviews API.

## `pageviewsDays` (type: `integer`):

Number of full days, ending yesterday (UTC), to add up in pageviews.

## `includeDailyPageviews` (type: `boolean`):

Also return the day-by-day series (date, views) in pageviewsDaily.

## `includeLanguageLinks` (type: `boolean`):

List the article's versions in other languages (language code, title, URL) in languageVersions. The count is always in languageCount.

## `maxConcurrency` (type: `integer`):

How many articles are fetched at the same time (1-5). The default keeps well within Wikimedia's API etiquette.

## `proxyConfiguration` (type: `object`):

Not needed: the Wikimedia APIs accept direct connections. Leave off unless your runs get HTTP 403 / 429.

## Actor input object example

```json
{
  "articles": [
    "https://en.wikipedia.org/wiki/Large_language_model",
    "Machu Picchu"
  ],
  "searchQueries": [
    "retrieval augmented generation"
  ],
  "language": "en",
  "maxItems": 10,
  "maxArticlesPerQuery": 3,
  "contentFormat": "markdown",
  "includeLinks": false,
  "dropReferenceSections": true,
  "includePageviews": true,
  "pageviewsDays": 30,
  "includeDailyPageviews": false,
  "includeLanguageLinks": false,
  "maxConcurrency": 3,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `articles` (type: `string`):

All Wikipedia articles collected during the run (one record per article and language)

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "articles": [
        "https://en.wikipedia.org/wiki/Large_language_model",
        "Machu Picchu"
    ],
    "searchQueries": [
        "retrieval augmented generation"
    ],
    "maxItems": 10,
    "maxArticlesPerQuery": 3,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("fabrikit/wikipedia-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "articles": [
        "https://en.wikipedia.org/wiki/Large_language_model",
        "Machu Picchu",
    ],
    "searchQueries": ["retrieval augmented generation"],
    "maxItems": 10,
    "maxArticlesPerQuery": 3,
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("fabrikit/wikipedia-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "articles": [
    "https://en.wikipedia.org/wiki/Large_language_model",
    "Machu Picchu"
  ],
  "searchQueries": [
    "retrieval augmented generation"
  ],
  "maxItems": 10,
  "maxArticlesPerQuery": 3,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call fabrikit/wikipedia-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,fabrikit/wikipedia-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/CRAbcZkyIybce21Je/builds/YtBkAoyu6LgZScjtl/openapi.json