OpenAlex Scraper — Papers, Authors & Institutions

OpenAlex Scraper — Scholarly Papers, Citations, Authors, ORCID & Institutions

The ultimate OpenAlex scraper to search, discover, and extract comprehensive scholarly literature and academic intelligence at scale. Extract full metadata for research publications (title, authors, DOI, citation count, journal/venue, open-access PDF links, reconstructed full abstracts, keywords, topics, Field-Weighted Citation Impact), academic researchers (full name, ORCID, institutional affiliations, publication count, h-index, i10-index), and research institutions (ROR identifier, country, city, academic output).

Export research datasets instantly to JSON, CSV, Excel, XML, or connect via Apify REST API, webhooks, and cloud data pipelines.

---

⚡ At a glance

Feature	Specification	Description
Coverage Scope	250M+ Papers, 90M+ Authors, 100K+ Institutions	Comprehensive global index spanning all academic disciplines
Supported Entities	Works (Papers), Authors (Researchers), Institutions	Flexible entity extraction engine with unified schema formatting
Search & Filtering	Keywords, Topics, Author Names, Advanced Filters	Filter by publication year, open access, citation thresholds, country
Abstract Engine	Automated Plain-Text Abstract Reconstruction	Decodes inverted token index into continuous natural-language abstracts
Bibliometric Metrics	Citations, FWCI, h-index, i10-index, 2yr-Mean	Quantitative impact metrics for research evaluation and ranking
Author Intelligence	ORCID, Affiliations, Historical Output, Topics	Precision researcher profiling for recruiting and academic outreach
Deep Pagination	Up to 10,000 Records per Search Query	Paginate up to 50 pages (200 records/page) automatically
Performance	Pure HTTP Datacenter Execution	Under 30 seconds for 200+ enriched publications
Export Formats	JSON, CSV, Excel, XML, HTML, REST API	Ready for Python pandas, R bibliometrix, CRMs, and vector stores

---

🔎 Why scrape OpenAlex?

OpenAlex represents the modern, fully open catalog of the global scholarly ecosystem. Serving as the open successor to Microsoft Academic Graph and an accessible alternative to closed paywalled indexes like Scopus or Web of Science, OpenAlex links hundreds of millions of scientific works, authors, academic institutions, publication venues, and research topics.

However, extracting large volumes of scholarly metadata presents technical obstacles. Web portal interfaces limit browsing to one record at a time without bulk data export tools. Raw JSON endpoints store abstracts as inverted positional token maps rather than human-readable text, bury author institutional affiliations within nested relationship trees, and impose strict cursor navigation rules.

This OpenAlex scraper bridges the gap. It automatically handles pagination, decodes and reconstructs plain-text abstracts, flattens relational institutional arrays, de-duplicates records by unique OpenAlex IDs, and delivers clean, standardized tabular datasets ready for data science, talent acquisition, and literature synthesis.

High-Impact Applications

• Systematic Literature Reviews & Meta-Analyses: Harvest comprehensive corpora of peer-reviewed articles matching specific therapeutic areas, technological innovations, or environmental topics.
• Bibliometric Analysis & Research Evaluation: Benchmark institutional research impact, track citation velocity, calculate open-access percentages, and examine Field-Weighted Citation Impact (FWCI).
• Researcher Talent Discovery & Sourcing: Identify leading scientists, principal investigators, and postdoctoral candidates by h-index, ORCID, publication count, and verified institutional affiliations.
• R&D Competitor Intelligence: Monitor publications from corporate research labs (e.g., DeepMind, OpenAI, Genentech, Pfizer) to detect emerging scientific breakthroughs.
• AI/ML Dataset Construction: Assemble large-scale domain-specific corpora of scientific titles, keywords, and full abstracts for training language models or fine-tuning embeddings.
• Academic Grant & Policy Analysis: Trace grant impact, funding affiliations, and inter-institutional collaboration networks across countries.

---

✨ Key Features & Capabilities

• ✅ Three Entity Engines in One Actor: Switch effortlessly between works (scientific papers), authors (researchers), and institutions via a single dropdown configuration.
• ✅ Automated Abstract Reconstruction: Automatically rebuilds inverted token indexes into complete, coherent natural-language scientific abstracts up to 5,000 characters.
• ✅ Researcher Impact Profiling: Extract complete researcher profiles featuring verified ORCID links, h-index, i10-index, total citation counts, historical institutions, and topical expertise.
• ✅ Advanced Boolean Filtering: Combine free-text keyword search with OpenAlex filter syntax (e.g., publication_year:2024, is_oa:true, cited_by_count:>50, authorships.institutions.country_code:us).
• ✅ Deep Batch Pagination: Requests up to 200 records per page and smoothly traverses pages up to the 10,000-record query threshold without manual intervention.
• ✅ Polite Pool Optimization: Supply an optional contact email to access prioritized routing and high-throughput connection pipelines.
• ✅ Complete Venue & Open Access Metadata: Captures journal/conference names, DOI identifiers, open-access status, and direct open-access PDF repository links.
• ✅ Built-in De-Duplication: Guarantees 100% unique records across multi-term batch queries using canonical OpenAlex entity IDs.

---

💼 Industry Use Case Playbooks

1. Life Sciences & Pharma Scientific Advisory Board Sourcing

Biotech executive search teams need to identify key opinion leaders (KOLs) in specific therapeutic modalities (e.g., mRNA delivery, CAR-T cell therapy). By querying relevant publication keywords with high citation filters (cited_by_count:>100), recruiters can extract first authors, corresponding institutions, verified ORCIDs, and calculate researcher h-indexes to rapidly compile elite advisory panels.

2. Systematic Literature Synthesis for Medical Affairs

Medical science liaisons and healthcare researchers must synthesize clinical evidence across therapeutic indications. Using publication year filters and open-access constraints (is_oa:true), teams can automatically download structured datasets containing titles, journals, DOIs, and full plain-text abstracts for rapid meta-analysis and systematic literature reviews.

3. Corporate R&D Technology Scouting

Enterprise technology strategists can monitor patents and academic output from rival institutions or emerging university incubators. By filtering by institution country or specific academic lab names, analysts can track topic shift trajectories, emerging machine learning architectures, and material science developments years before market commercialization.

4. Vector Database Ingestion for Scientific RAG Agents

Generative AI engineers building retrieval-augmented generation (RAG) applications for legal, patent, or academic research need pristine scientific text. This actor exports reconstructed abstracts, titles, and topic tags formatted for instant chunking, embedding generation, and vector store ingestion.

---

📥 Input Parameters

All parameters are optional. Running the actor with empty input performs a demonstration search across key technology disciplines.

Parameter	Type	Default	Description
entityType	String (Select)	"works"	Type of entity to extract: works (papers), authors (researchers), or institutions.
searchTerms	Array of Strings	["machine learning"]	Keywords, author names, or institution names to search and paginate across.
filters	Array of Strings	[]	OpenAlex filter syntax expressions (e.g. publication_year:2024, is_oa:true, cited_by_count:>100).
email	String	""	Optional email address to access OpenAlex's prioritized high-throughput polite pool.
reconstructAbstract	Boolean	true	Rebuild plain-text natural language abstracts from inverted token positional maps.
maxItems	Integer	0	Maximum total records across all search queries. 0 = unlimited (governed by available results).
maxPagesPerSearch	Integer	0	Maximum pages per search term (200 records/page; up to 50 pages). 0 = auto/unlimited.
proxyConfiguration	Object	{ "useApifyProxy": true }	Apify Proxy configuration. Standard datacenter proxies provide optimal performance.

Sample JSON Configuration: High-Impact AI Papers

{
  "entityType": "works",
  "searchTerms": ["transformer architecture", "reinforcement learning"],
  "filters": ["publication_year:2023", "is_oa:true", "cited_by_count:>50"],
  "reconstructAbstract": true,
  "maxItems": 500,
  "maxPagesPerSearch": 0,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}

---

📊 Extracted Data Structure

📄 Works (Research Papers) Fields

Field	Description	Type	Coverage
id	Short canonical OpenAlex work identifier (e.g., W2100837269)	String	🟢 High (100%)
title	Full registered scientific publication title	String	🟢 High (100%)
doi	Canonical Digital Object Identifier URL	String	🟢 High (~85%)
publicationYear	Year of official academic publication	Number	🟢 High (100%)
publicationDate	Exact ISO publication date (YYYY-MM-DD)	String	🟢 High (100%)
type	Publication type classification (article, preprint, book-chapter)	String	🟢 High (100%)
citedByCount	Total global academic citation count	Number	🟢 High (100%)
authorNames	Complete array of contributing author display names	Array	🟢 High (~98%)
firstAuthor	Primary lead author display name	String	🟢 High (~98%)
authorInstitutions	List of distinct academic institutions associated with authors	Array	🟢 High (~90%)
venue	Publishing journal, conference proceeding, or repository name	String	🟢 High (~85%)
isOpenAccess	Boolean flag indicating public open-access availability	Boolean	🟢 High (100%)
oaUrl	Direct link to public open-access full-text PDF document	String	🟡 Conditional (~65%)
language	Primary ISO publication language code	String	🟢 High (~90%)
topics	Extracted scientific topic classification labels	Array	🟢 High (~95%)
keywords	High-relevance scientific keyword terms	Array	🟢 High (~85%)
fwci	Field-Weighted Citation Impact relative to global subject benchmark	Number	🟡 Conditional (~60%)
referencedWorksCount	Total count of cited references in bibliography	Number	🟢 High (~95%)
abstract	Complete reconstructed plain-text publication abstract	String	🟢 High (~80%)
openAlexUrl	Direct URL to public OpenAlex entity landing page	String	🟢 High (100%)
scrapedAt	ISO 8601 extraction timestamp	String	🟢 High (100%)

🧑‍🔬 Authors (Researchers) Fields

Field	Description	Type	Coverage
id	Short canonical OpenAlex author identifier (e.g., A5023880000)	String	🟢 High (100%)
name	Researcher full display name	String	🟢 High (100%)
orcid	Official verified ORCID identifier URL	String	🟡 Conditional (~45%)
worksCount	Total cumulative published scientific works	Number	🟢 High (100%)
citedByCount	Lifetime aggregate academic citation count	Number	🟢 High (100%)
hIndex	Lifetime calculated author h-index metric	Number	🟢 High (100%)
i10Index	Count of authored publications with at least 10 citations	Number	🟢 High (100%)
meanCitedness2yr	2-year mean citation metric for recent output	Number	🟢 High (100%)
lastKnownInstitution	Current or most recent affiliated research institution	String	🟢 High (~85%)
lastKnownInstitutionCountry	ISO two-letter country code of current institution	String	🟢 High (~85%)
affiliations	Historical academic and research affiliations	Array	🟢 High (~85%)
topics	Primary specialized research topic areas	Array	🟢 High (100%)
openAlexUrl	Direct URL to public author profile page	String	🟢 High (100%)

🏛️ Institutions Fields

Field	Description	Type	Coverage
id	Short canonical OpenAlex institution identifier (e.g., I136199984)	String	🟢 High (100%)
name	Registered academic institution display name	String	🟢 High (100%)
countryCode	ISO two-letter country code of institution headquarters	String	🟢 High (100%)
type	Institution classification (education, healthcare, facility, company)	String	🟢 High (100%)
worksCount	Total institutional scientific publications recorded	Number	🟢 High (100%)
citedByCount	Total institutional cumulative citation count	Number	🟢 High (100%)
homepageUrl	Official institutional website homepage URL	String	🟢 High (~80%)
ror	Official Research Organization Registry (ROR) identifier	String	🟢 High (~85%)
city	Registered municipality / city location	String	🟢 High (~90%)
region	Geographic administrative state or region	String	🟢 High (~85%)
openAlexUrl	Direct URL to public institution profile page	String	🟢 High (100%)

---

💻 Sample Output Record: Research Paper

{
  "id": "W2100837269",
  "title": "Scikit-learn: Machine Learning in Python",
  "doi": "https://doi.org/10.48550/arxiv.1201.0490",
  "publicationYear": 2012,
  "publicationDate": "2011-01-01",
  "type": "article",
  "citedByCount": 63567,
  "authorNames": [
    "Fabián Pedregosa",
    "Gaël Varoquaux",
    "Alexandre Gramfort",
    "Vincent Michel",
    "Bertrand Thirion",
    "Olivier Grisel"
  ],
  "firstAuthor": "Fabián Pedregosa",
  "authorInstitutions": [
    "French Institute for Research in Computer Science and Automation",
    "CEA Paris-Saclay"
  ],
  "venue": "Journal of Machine Learning Research",
  "isOpenAccess": true,
  "oaUrl": "https://arxiv.org/pdf/1201.0490",
  "language": "en",
  "topics": [
    "Machine Learning Algorithms",
    "Python Scientific Computing",
    "Open Source Software"
  ],
  "keywords": [
    "machine learning",
    "scikit-learn",
    "data science",
    "python"
  ],
  "fwci": 141.89,
  "referencedWorksCount": 23,
  "abstract": "Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license.",
  "openAlexUrl": "https://openalex.org/W2100837269",
  "scrapedAt": "2026-11-10T14:35:00.000Z"
}

---

🛠️ Step-by-Step Practical Integration Guide

1. Interactive Console Execution

1. Open the actor in the Apify Console.
2. Select your target entityType (works, authors, or institutions).
3. Enter keywords or researcher names in searchTerms.
4. (Optional) Enter advanced filter expressions such as publication_year:2024.
5. Set maxItems to your target volume and click Start.

2. Python Bibliometrics Pipeline

from apify_client import ApifyClient
import pandas as pd

client = ApifyClient("YOUR_APIFY_TOKEN")

# Run OpenAlex scraper for deep learning literature
run = client.actor("haketa/openalex-scraper").call(run_input={
    "entityType": "works",
    "searchTerms": ["diffusion models"],
    "filters": ["publication_year:2023-2024", "is_oa:true"],
    "maxItems": 200,
    "maxPagesPerSearch": 0
})

# Load dataset items directly into a pandas DataFrame
dataset_items = client.dataset(run["defaultDatasetId"]).list_items().items
df = pd.DataFrame(dataset_items)

# Rank top cited papers
top_cited = df[["title", "citedByCount", "venue", "firstAuthor"]].sort_values(
    by="citedByCount", ascending=False
)
print(top_cited.head(10))

3. Node.js & TypeScript Pipeline

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('haketa/openalex-scraper').call({
    entityType: 'authors',
    searchTerms: ['Yann LeCun', 'Geoffrey Hinton', 'Yoshua Bengio'],
    maxItems: 50,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const author of items) {
    console.log(`${author.name} (h-index: ${author.hIndex}, Citations: ${author.citedByCount})`);
}

4. cURL Direct Invocation

curl -X POST "https://api.apify.com/v2/acts/haketa~openalex-scraper/runs?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "entityType": "works",
    "searchTerms": ["crispr gene editing"],
    "filters": ["cited_by_count:>100"],
    "maxItems": 100,
    "maxPagesPerSearch": 0
  }'

---

⚖️ OpenAlex Scraper vs. Proprietary Scholarly Databases

Capability	OpenAlex Scraper	Web of Science / Scopus	PubMed / arXiv
Cost & Licensing	Fully Open, Pay-per-compute only	$10,000s / year enterprise license	Free (Narrow disciplinary scope)
Cross-Disciplinary Breadth	250M+ Works across all sciences	High, but heavily gated	Life sciences & physics/CS only
Author Metrics (h-index, i10)	Directly included in author schema	Restricted to paid analytics tiers	Not natively computed
Abstract Availability	Fully reconstructed plain-text abstracts	Paywalled or partial abstracts	Partial / abstracts only
Data Export Flexibility	JSON, CSV, Excel, XML, API, Webhooks	Proprietary export formats, limited caps	Manual downloads, XML dumps
Cloud Automation	Apify scheduler, Make, Zapier, Python	Manual web interface only	Requires custom local servers

---

💡 Best Practices for Research Extraction

• Author Search vs. Topic Search: When searching for researchers by topic rather than name, scrape works matching the research topic and extract unique entries from authorNames and authorInstitutions. Author entity search matches researcher names directly.
• Use the Polite Pool: Providing an email address in the email field routes requests through prioritized processing, ensuring uninterrupted high-throughput execution for large runs.
• Toggle Abstract Reconstruction: If you only require bibliographic citation metrics and authors without abstracts, set reconstructAbstract: false to reduce network payload and optimize processing speed.
• Filter Syntax Mastery: Combine filters using commas for logical AND operations (e.g., publication_year:2024,is_oa:true,type:article).
• Exporting for Citation Managers: Download the dataset as CSV or JSON and easily import into citation managers like Zotero, Mendeley, or EndNote.

---

❓ Frequently Asked Questions

Does this actor require an API key or paid subscription? No. OpenAlex is an open scholarly index. The scraper runs smoothly without private keys or access credentials.

What is the maximum number of results I can extract per search? OpenAlex allows up to 10,000 records per individual search query. Setting maxPagesPerSearch: 0 traverses up to 50 pages (200 records per page) to reach the 10,000 ceiling. To extract larger corpora, divide queries across multiple years or topics.

How does abstract reconstruction work? OpenAlex stores abstracts as inverted index dictionaries mapping words to character positions. This actor decodes the positional index and reconstructs coherent, human-readable continuous text.

Can I run scheduled monitors for newly published papers? Yes. You can configure scheduled Apify jobs to query topics weekly, combined with current-year filters (publication_year:2026), sending newly published literature directly into your database or webhook.

What proxies are used for scraping? Standard Apify Datacenter proxies are enabled by default, ensuring rapid and resilient data extraction.

Can I export data directly to Google Sheets or Airtable? Yes. You can use Apify's native cloud integrations or Make/Zapier connections to send freshly scraped papers or author profiles directly into Google Sheets, Notion, or Airtable.

How often is the underlying scholarly database refreshed? OpenAlex synchronizes continuously with Crossref, PubMed, arXiv, institutional repositories, and publisher feeds, ensuring immediate visibility for newly indexed preprints and journal articles.

Can I look up specific papers by DOI? Yes. You can pass exact DOIs into the filters parameter using the syntax doi:https://doi.org/10.xxxx/xxxx or search for the title directly.

Is Field-Weighted Citation Impact (FWCI) available for all papers? FWCI is computed for scientific publications that have accumulated standard subject-level benchmark baselines. For very newly published papers or preprints, FWCI will be populated once sufficient citation windows have elapsed.

Does this actor support multilingual research literature? Yes. OpenAlex indexes research published in over 50 languages globally. You can filter publications by language code using language:fr, language:es, language:de, or language:zh.

---

📋 Changelog

Version 1.2 (2026-11)

• Unlocked 1,000+ Deep Pagination: Removed restrictive default page limits; setting maxPagesPerSearch: 0 now enables automatic infinite pagination up to the 10,000-record query boundary.
• Enhanced Empty Input Fallbacks: Integrated multi-disciplinary academic default queries ensuring zero-configuration runs return rich multi-sector research results without early termination.
• Optimized Abstract Reconstruction: Accelerated inverted index decoding speed and increased abstract buffer ceiling to 5,000 characters.
• Enhanced Normalization: Added standard ROR identifier mapping for institutions and unified ORCID URL formatting.

Version 1.1 (2026-10)

• Added Field-Weighted Citation Impact (FWCI) and keyword extraction for works.
• Added summary statistics (hIndex, i10Index, meanCitedness2yr) for authors.
• Introduced polite pool email routing option.

Version 1.0 (2026-10)

• Initial release featuring multi-entity extraction for works, authors, and institutions with inverted index abstract reconstruction.