Wikipedia Scraper — Markdown, Infobox Data & Pageviews

Wikipedia Scraper

Wikipedia Scraper extracts articles from wikipedia.org in any language edition and returns each one as a single JSON record: the article body as clean Markdown (or plain text / cleaned HTML), the infobox as key/value fields, the lead summary, the table of contents, categories, Wikidata id, main image, coordinates and human pageviews for the last N days. Pick articles by title or URL, by full-text search or by category. It reads Wikipedia through the public MediaWiki and Wikimedia REST APIs with a descriptive User-Agent, so there is no browser, no CAPTCHA and no proxy to pay for.

What you get

Field	Example (Machu Picchu)
title, pageId, language, url	Machu Picchu, 80019, en, https://en.wikipedia.org/wiki/Machu_Picchu
description, summary	"15th-century Inca citadel in Peru", the lead paragraph in plain text
markdown	# Machu Picchu + every section as ## / ###, lists, GFM tables, fenced code, LaTeX math ($a^{2}+b^{2}=c^{2}$)
text / html	the same body as plain text or cleaned HTML, when you choose that format
wordCount	7015
infoboxType, infoboxTitle, infobox	historic site, Machu Picchu, { "Built": "c. 1450 – c. 1470", "Elevation": "2,430 m (7,970 ft)", … }
sections[]	title, level, number and anchor of every heading
categories[]	visible categories only (maintenance categories are left out)
wikidataId, imageUrl, thumbnailUrl, latitude, longitude	Q676203, main image, -13.16333333, -72.54555556
lastModified, revisionId, isDisambiguation	time of the last edit, revision read, disambiguation flag
languageCount, languageVersions[]	132 editions; the list of other-language titles and URLs on request
pageviews, pageviewsDailyAverage, pageviewsFrom, pageviewsTo, pageviewsDaily[]	96642 human views in 30 days, 3221.4 per day; day-by-day series on request
redirectedFrom, sourceType, sourceValue, position, scrapedAt	how the article was reached and its rank

What is removed from the body: citation markers like [1], edit links, navigation boxes, maintenance banners, "For other uses" notes, image galleries and, by default, the trailing reference sections (See also, Notes, References, Further reading, External links, and their equivalents in German, Spanish, French, Portuguese, Italian, Dutch, Polish and Russian). "Bibliography" is kept, because in biographies it lists the person's works.

Use cases

• RAG and AI agents: feed an LLM or a vector database with Markdown that keeps the heading structure (easy to chunk by section) and none of the page furniture. Use summary for short context and markdown for full context.
• Knowledge graphs and entity enrichment: the infobox gives structured facts (founded, headquarters, population, elevation, designer, license...) and wikidataId links each record to Wikidata.
• Trend and demand research: pageviews show which topics, products, places or people the public is reading about; compare a list of articles or a whole category.
• Multilingual datasets: the same article in several editions (pass the URLs, or turn on includeLanguageLinks to get the titles).
• SEO and content research: word counts, section outlines and categories of the top articles for a search phrase.

Input

{
    "articles": ["https://en.wikipedia.org/wiki/Large_language_model", "Machu Picchu", "https://de.wikipedia.org/wiki/Berlin"],
    "searchQueries": ["retrieval augmented generation"],
    "categories": ["Machine learning algorithms"],
    "language": "en",
    "maxItems": 50,
    "maxArticlesPerQuery": 10,
    "contentFormat": "markdown",
    "includeLinks": false,
    "dropReferenceSections": true,
    "includePageviews": true,
    "pageviewsDays": 30,
    "includeDailyPageviews": false,
    "includeLanguageLinks": false
}

• articles: titles (looked up in the language edition) or URLs from any edition, including mobile links and ?curid= links. Redirects are followed and reported in redirectedFrom.
• searchQueries: Wikipedia's own full-text search, best match first; maxArticlesPerQuery articles per query.
• categories: articles filed directly in each category, up to maxArticlesPerQuery each. Subcategories are not followed.
• maxItems is exact and counts across all sources; an article reached twice (for example by title and by search) is saved and charged once.
• contentFormat: markdown, text, html, or none for metadata, infobox and summary only.
• includeLinks keeps links as [text](https://...); off by default because plain text embeds better.
• maxConcurrency (1-5, default 3) sets how many articles are fetched at once.

Output example

One record from the run above, with the long fields shortened:

{
    "title": "Machu Picchu",
    "pageId": 80019,
    "language": "en",
    "url": "https://en.wikipedia.org/wiki/Machu_Picchu",
    "description": "15th-century Inca citadel in Peru",
    "summary": "Machu Picchu is a 15th-century Inca citadel located in the Eastern Cordillera of southern Peru on a mountain ridge at 2,430 meters (7,970 ft). …",
    "markdown": "# Machu Picchu\n\n**Machu Picchu** is a 15th-century Inca citadel located in the Eastern Cordillera of southern Peru …\n\n## Etymology\n\n…",
    "wordCount": 7015,
    "infoboxType": "historic site",
    "infoboxTitle": "Machu Picchu",
    "infobox": {
        "Type": "Llaqta (settlement)",
        "Cultures": "Inca civilization",
        "Location": "Machupicchu District, Urubamba, Cusco, Peru",
        "Built": "c. 1450 – c. 1470",
        "Elevation": "2,430 m (7,970 ft)",
        "Discovered": "Agustín Lizárraga (1902); Hiram Bingham (1911)",
        "UNESCO World Heritage Site: Location": "Latin America and the Caribbean",
        "Inscription": "1983 (7th Session)",
        "…": "…"
    },
    "sections": [{ "title": "Etymology", "level": 2, "number": "1", "anchor": "Etymology" }, "…"],
    "categories": ["Machu Picchu", "Archaeological sites in Peru", "World Heritage Sites in Peru", "…"],
    "wikidataId": "Q676203",
    "imageUrl": "https://thumb.wikimedia.org/wikipedia/commons/thumb/b/bb/Machu_Picchu%2C_2023_%28012%29.jpg/3840px-Machu_Picchu%2C_2023_%28012%29.jpg",
    "latitude": -13.16333333,
    "longitude": -72.54555556,
    "isDisambiguation": false,
    "revisionId": 1380992747,
    "lastModified": "2026-10-19T08:52:19Z",
    "languageCount": 132,
    "pageviews": 96642,
    "pageviewsDailyAverage": 3221.4,
    "pageviewsFrom": "2026-09-24",
    "pageviewsTo": "2026-10-23",
    "redirectedFrom": null,
    "sourceType": "article",
    "sourceValue": "Machu Picchu",
    "position": 2,
    "scrapedAt": "2026-10-24T08:27:16.078Z"
}

A SUMMARY record in the key-value store lists each source (search hits on Wikipedia, articles listed and saved), titles that do not exist, duplicates skipped and the number of requests.

Pricing

Pay per event: US$0.80 per 1,000 articles (article-result, US$0.0008 each), plus Apify platform usage, which is small: three light API calls per article, no browser and no proxy.

Run	Articles	Price
Default input (2 titles + 1 search of 3)	5	US$0.004
Top 100 articles for 10 search phrases	1,000	US$0.80
A 5,000-article knowledge base for RAG	5,000	US$4.00

Titles that do not exist and duplicates are not charged.

How it compares

• Wikipedia APIs directly: free, but you need three different APIs (Action API for HTML and metadata, REST for the summary, Wikimedia for pageviews), the HTML still has to be cleaned and converted, and infobox tables differ between editions. This Actor does all of that and gives you one flat record per article, exportable as JSON, CSV or Excel.
• Wikipedia dumps: best for the whole encyclopedia at once (tens of GB of wikitext); overkill when you need hundreds or thousands of current articles, and wikitext is not Markdown.
• Other Wikipedia Actors in the Store: most return the plain-text extract or the full text without structure; this one keeps headings, tables, code and math as Markdown and adds the infobox and pageviews, at US$0.80 per 1,000.

Related Actors

• OpenAlex Scraper: the Actor this one is built on (same API client), for scholarly papers with abstracts and citations.
• Website to Markdown for AI/LLM: turns any other website into Markdown; its converter is reused here.

Integrations

Use it from the Apify API (/v2/datasets/{id}/items?format=json), schedule it, or connect it to n8n, Make, Zapier, Google Sheets or Airbyte. AI agents can call it as a tool through the Apify MCP server, and LangChain / LlamaIndex can load the dataset directly.

FAQ

Is this legal? Wikipedia text is published under CC BY-SA, which allows reuse with attribution: keep the url of each article with the text you reuse. The Actor reads only public article pages through Wikimedia's documented APIs, identifies itself with a descriptive User-Agent as Wikimedia asks, and does not collect editor or user data.

Do I need a proxy? No. Wikimedia's APIs accept direct connections, and the default input runs without any proxy. The proxyConfiguration field is there only in case your runs ever get HTTP 403 / 429.

What are the limits? A search can go up to 10,000 results deep (a limit of Wikipedia's search). Categories return the pages filed directly in them. The pageviews window is 1 to 365 days and ends yesterday (UTC); very new or rarely read articles show 0. Requests are spaced to stay inside Wikimedia's API etiquette, so expect about 2 articles per second.

Which languages work? Any Wikipedia edition: set language (en, de, es, fr, pt, ja, zh, simple...) or pass URLs from that edition. Infobox parsing was checked on English, German and Spanish articles.

How fresh is the data? Live: each article is read at run time, and lastModified tells you when it was last edited.

Changelog

• 0.1 (October 2026): first release. Titles / URLs / curid, search and category sources, any language, Markdown / text / HTML / none, infobox fields and type, summary, sections, categories, Wikidata id, image, coordinates, language versions, pageviews with optional daily series.

---

En español

Wikipedia Scraper extrae artículos de Wikipedia en cualquier idioma (por título, URL, búsqueda o categoría) y entrega cada uno como un registro JSON con el texto en Markdown limpio (o texto plano / HTML), la ficha (infobox) como campos clave-valor, el resumen, las secciones, las categorías, el id de Wikidata, la imagen, las coordenadas y las visitas de los últimos N días. Usa las APIs públicas de Wikimedia, sin navegador ni proxy. Ideal para RAG y agentes de IA. Precio: US$0,80 por cada 1.000 artículos.