# Google Scholar Scraper: Articles, Citations & PDFs (`primeparse/google-scholar-scraper`) Actor

Extract academic data from Google Scholar: titles, authors, years, citations, abstracts, PDF links. Supports queries, year filters (1900-2100), pagination (up to 5 pages). Rate-limited for safety. Ideal for research, citations, datasets, AI. Clean JSON output. Run on Apify with proxies.

- **URL**: https://apify.com/primeparse/google-scholar-scraper.md
- **Developed by:** [PrimeParse](https://apify.com/primeparse) (community)
- **Categories:** Developer tools, AI, Other
- **Features:** Limited permissions
- **Stats:** 55 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## 🔬 Google Scholar Scraper: Academic Research Data Extractor

Enterprise-grade Google Scholar scraper for academic research and data analysis. Collects structured data from Google Scholar search results including titles, authors, citations, abstracts, and PDF links. Ideal for literature reviews, citation analysis, and academic dataset building. Features intelligent parsing, rate limiting, and year filtering.

**High-quality Google Scholar Data Extractor for Researchers, Academics, and Data Scientists**

Automatically searches Google Scholar, extracts article metadata, filters by publication year, and collects citation data — clean, structured, ready for analysis or academic research.

**Built for:**

- Academic researchers conducting literature reviews
- Data scientists building research datasets
- PhD students tracking citations and publications
- Librarians organizing academic resources
- Research teams monitoring publication trends
- AI/ML engineers collecting training data from academic sources

✅ **Smart search with keyword queries**
✅ **Year range filtering (1900-2100)**
✅ **Rich metadata extraction (title, authors, year, citations, abstract, PDF links)**
✅ **Automatic pagination support (up to 5 pages)**
✅ **Rate limiting & respectful crawling**
✅ **AI-ready structured output**

👉 **Runs on Apify • No code required**

### 🚀 Why This Scraper

#### ✔ Purpose-Built for Academic Research

Intelligently extracts structured data from Google Scholar search results — perfect for literature reviews, citation analysis, and academic research.

#### ✔ Comprehensive Metadata Extraction

Extracts all essential academic metadata: article titles, author lists, publication years, citation counts, abstracts, PDF links, and Google Scholar page URLs.

#### ✔ Clean & Structured Output

Produces clean, structured JSON output ready for analysis, database import, or further processing. Perfect for academic datasets and research workflows.

#### ✔ Smart Year Filtering

Filter results by publication year range to focus on recent research or historical publications. Supports years from 1900 to 2100.

#### ✔ AI & ML Ready

Structured JSON output perfect for RAG systems, LLM fine-tuning, academic knowledge bases, or training datasets for research applications.

#### ✔ Fast & Efficient

Powered by Puppeteer for reliable browser automation. Handles dynamic content and JavaScript-rendered pages efficiently.

#### ✔ Safe & Controlled Processing

Built-in rate limiting (1-2 second delays), configurable pagination limits, and graceful error handling to respect Google Scholar's infrastructure.

### 💼 Use Cases

- **Literature reviews** — Collect and analyze academic papers for systematic reviews
- **Citation tracking** — Monitor citation counts and track research impact
- **Publication monitoring** — Track new publications in specific research areas
- **Dataset building** — Create structured datasets for academic research or AI training
- **Competitive research** — Monitor competitor publications and research trends
- **Academic analysis** — Analyze publication patterns, author networks, and citation trends
- **PDF collection** — Automatically collect PDF links for offline research

### 📊 Supported Data

- **Article titles** — Full publication titles
- **Authors** — Complete author lists (up to 10 authors per article)
- **Publication years** — Extracted from metadata
- **Citation counts** — Number of citations for each article
- **Abstracts** — Article abstracts when available
- **PDF links** — Direct links to PDF files when available
- **Google Scholar links** — Direct links to article pages on Google Scholar

### ⚙️ How It Works

1. Enter your search query (e.g., "machine learning", "quantum computing")
2. Optionally set year range filters and pagination limits
3. Configure proxy settings for reliable access
4. Run the Actor
5. Download clean, structured academic datasets

### 🧩 Input Configuration

#### Example JSON Input

```json
{
  "query": "machine learning",
  "maxPages": 1,
  "startYear": 2020,
  "endYear": 2026,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

#### Key Options

- **query** — Search query string (required, e.g., "machine learning", "neural networks")
- **maxPages** — Maximum number of result pages to scrape (default: 1, recommended: 1-5)
- **startYear** — Filter results by minimum publication year (optional, 1900-2100)
- **endYear** — Filter results by maximum publication year (optional, 1900-2100)
- **proxyConfiguration** — Proxy settings for anti-bot protection (default: uses Apify Proxy)

#### Search Query Tips

- Use specific terms for better results (e.g., "deep learning neural networks" instead of "AI")
- Combine keywords with quotes for exact phrases: `"transfer learning"`
- Use Boolean operators: `machine learning AND computer vision`
- Filter by author: `author:"John Smith" machine learning`
- Filter by publication: `source:"Nature" quantum computing`

### 📂 Output Dataset

All articles are stored in the default Apify dataset with the following structure:

#### Example Output Record

```json
{
  "title": "Machine learning",
  "authors": [
    "ZH Zhou"
  ],
  "year": 2021,
  "citations": 3301,
  "abstract": "… from data is called learning or training. The … machine learning is to find or approximate ground-truth. In this book, models are sometimes called learners, which are machine learning …",
  "pdfLink": null,
  "scholarLink": "https://books.google.com/books?hl=en&lr=&id=ctM-EAAAQBAJ&oi=fnd&pg=PR6&dq=machine+learning&ots=o_OnT7Rv3p&sig=bH9TGnw_ZdZYH4lSLmKun7xX6Cs"
}
```

#### Output Fields

- **title** (string, required) — Article title
- **authors** (array, required) — List of author names (up to 10 authors)
- **year** (integer|null) — Publication year
- **citations** (integer|null) — Number of citations
- **abstract** (string|null) — Article abstract when available
- **pdfLink** (string|null) — Direct link to PDF file when available
- **scholarLink** (string, required) — Link to Google Scholar article page

#### Multiple Authors Example

```json
{
  "title": "A guide to machine learning for biologists",
  "authors": [
    "JG Greener",
    "SM Kandathil"
  ],
  "year": 2022,
  "citations": 2020,
  "abstract": "… A machine learning task is an objective specification for what we want a machine learning model to accomplish…",
  "pdfLink": "https://discovery.ucl.ac.uk/id/eprint/10134478/1/NRMCB-review-accepted-forRPS.pdf",
  "scholarLink": "https://www.nature.com/articles/s41580-021-00407-0"
}
```

#### Input File Example

Create `storage/key_value_stores/default/INPUT.json`:

```json
{
  "query": "quantum computing",
  "maxPages": 2,
  "startYear": 2020,
  "endYear": 2024
}
```

### 📈 Performance

- **Processing Speed** — ~3-4 seconds per page (depending on results)
- **Rate Limiting** — Built-in 1-2 second delays between requests
- **Concurrency** — Single request at a time for reliability
- **Scalability** — Handles 1-5 pages optimally (up to 50 articles per run)
- **Success Rate** — High reliability with proper proxy configuration

### 🔧 Advanced Configuration

#### Year Range Filtering

Filter results by publication year:

```json
{
  "query": "artificial intelligence",
  "startYear": 2020,
  "endYear": 2026
}
```

#### Multiple Pages

Scrape multiple pages for comprehensive results:

```json
{
  "query": "deep learning",
  "maxPages": 5
}
```

#### Proxy Configuration

Use Apify Proxy for reliable access:

```json
{
  "query": "neural networks",
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

### 📧 Support

- **Issues** — Use Apify Issues tab for bug reports
- **Documentation** — Check Apify documentation for platform features
- **Community** — Join Apify community for discussions

**Tags:** Google Scholar, academic research, literature review, citation analysis, research data, paper scraping, academic scraping, research automation, citation tracking, publication monitoring, academic dataset, research tools, scholarly articles, PDF extraction

***

**Built with ❤️ on Apify**

# Actor input Schema

## `query` (type: `string`):

Search query string (e.g., 'machine learning')

## `maxPages` (type: `integer`):

Maximum number of result pages to scrape (default: 1, max: 5 for MVP)

## `startYear` (type: `integer`):

Filter results by minimum publication year (optional)

## `endYear` (type: `integer`):

Filter results by maximum publication year (optional)

## `proxyConfiguration` (type: `object`):

Proxy settings for anti-bot protection

## Actor input object example

```json
{
  "query": "machine learning",
  "maxPages": 1,
  "startYear": 2025,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `overview` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("primeparse/google-scholar-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("primeparse/google-scholar-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call primeparse/google-scholar-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,primeparse/google-scholar-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/srM9zIfClIsIOjKes/builds/VmmD4tS4TW4RotqDL/openapi.json