# GitHub Scraper (`automation-lab/github-scraper`) Actor

Extract data from GitHub — repository details, developer profiles, trending repos, and search results. Stars, forks, languages, topics, and more. No API key needed.

- **URL**: https://apify.com/automation-lab/github-scraper.md
- **Developed by:** [Automation Lab](https://apify.com/automation-lab) (community)
- **Categories:** Developer tools, Lead generation
- **Features:** Limited permissions
- **Stats:** 93 total users, 12 monthly users, 98.6% runs succeeded, 1 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $1.80 / 1,000 repository or profile extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## GitHub Scraper

Scrape data from [GitHub](https://github.com) — repositories, repository contributors, developer profiles, trending repos, and search results. Get stars, forks, contribution counts, languages, topics, licenses, followers, and more.

### What does GitHub Scraper do?

GitHub Scraper extracts structured data from GitHub using its public GitHub API and web pages. It supports six modes:

- **Repository details** — Full metadata for specific repos (stars, forks, topics, license, dates)
- **Developer profiles** — Bio, followers, location, company, repos count for any user
- **Repository contributors** — Turn public repository URLs into developer lists with contribution counts
- **Trending repositories** — Today's/week's/month's hottest repos with star velocity
- **Search repositories** — Find repos by keyword, sorted by stars
- **Search users** — Find users with GitHub qualifiers and enrich every result into the full profile format

No GitHub API token is required. Anonymous access is limited to 60 API requests/hour; an optional secret token raises the standard GitHub allowance to 5,000 requests/hour.

### Who is it for?

GitHub Scraper is for developer-tool founders, recruiters, open-source researchers, venture analysts, and data teams that need structured repository and profile data without maintaining GitHub scraping infrastructure.

### Why scrape GitHub?

GitHub hosts 400M+ repositories and 100M+ developers. It's the primary source for:

- 📊 **Tech trend analysis** — Track which languages and frameworks are gaining traction
- 🔍 **Competitive intelligence** — Monitor competitor repos, stars growth, and release cadence
- 📈 **Developer recruiting** — Find active developers by language, location, and contribution history
- 🏗️ **Open source research** — Analyze licensing, dependency patterns, and community health
- 📰 **Content creation** — Curate trending repos for newsletters and social media

### Use cases

- **Newsletter creators** sharing weekly trending repos
- **VCs and investors** tracking open-source momentum
- **Hiring managers** building candidate lists from active contributors
- **Researchers** studying open-source ecosystem dynamics
- **DevRel teams** monitoring mentions and competitive landscape
- **Developers** discovering new tools and libraries

### How to scrape GitHub

1. Go to [GitHub Scraper](https://apify.com/automation-lab/github-scraper) on Apify Store
2. Choose a mode: repos, profiles, contributors, trending, repository search, or user search
3. Enter repository/profile URLs or a search query depending on the mode
4. Set the max results limit
5. Click **Start** and wait for results
6. Download data as JSON, CSV, or Excel

### Data you can extract

#### Repository data

| Field | Type | Description |
|-------|------|-------------|
| `fullName` | string | Owner/repo (e.g., `facebook/react`) |
| `description` | string | Repo description |
| `stars` | number | Star count |
| `forks` | number | Fork count |
| `watchers` | number | Watcher count |
| `openIssues` | number | Open issue count |
| `language` | string | Primary language |
| `topics` | array | Topic tags |
| `license` | string | License (MIT, Apache-2.0, etc.) |
| `isArchived` | boolean | Whether the repo is archived |
| `createdAt` | string | Creation date |
| `updatedAt` | string | Last update date |
| `size` | number | Repo size in KB |

#### Profile data

| Field | Type | Description |
|-------|------|-------------|
| `username` | string | GitHub username |
| `name` | string | Display name |
| `bio` | string | Profile bio |
| `company` | string | Company |
| `location` | string | Location |
| `followers` | number | Follower count |
| `following` | number | Following count |
| `publicRepos` | number | Public repo count |
| `blog` | string | Website URL |
| `twitterUsername` | string | X/Twitter handle |

#### Contributor data

Contributor mode uses GitHub's public `/repos/{owner}/{repo}/contributors` API endpoint. It emits one row per unique repository and username, preserving the order of your repository list.

| Field | Type | Description |
|-------|------|-------------|
| `repositoryFullName` | string | Repository in `owner/repo` format |
| `username` | string | Contributor's GitHub username |
| `profileUrl` | string | Public GitHub profile URL |
| `avatarUrl` | string or null | Public avatar URL |
| `contributions` | number | Contributions reported by GitHub for this repository |
| `contributorType` | string or null | GitHub account type, such as `User` or `Bot` |
| `repositoryUrl` | string | Canonical public repository URL |
| `sourceUrl` | string | Repository URL supplied in the input |
| `scrapedAt` | string | UTC extraction timestamp |

#### Trending repo data

| Field | Type | Description |
|-------|------|-------------|
| `fullName` | string | Owner/repo |
| `description` | string | Repo description |
| `language` | string | Primary language |
| `stars` | number | Total stars |
| `starsToday` | number | Stars gained in the period |
| `forks` | number | Fork count |
| `builtBy` | array | Top contributors with avatars |

### Input parameters

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `mode` | string | `"trending"` | Mode: `repos`, `profiles`, `contributors`, `trending`, `search`, `users` |
| `urls` | array | `[]` | Public HTTPS GitHub URLs (for repos, profiles, or contributors mode) |
| `searchQuery` | string | `""` | GitHub query (for repository `search` or `users` search) |
| `githubToken` | string | — | Optional encrypted GitHub PAT for higher API limits; sent only to `api.github.com` |
| `trendingSince` | string | `"daily"` | Trending period: `daily`, `weekly`, `monthly` |
| `trendingLanguage` | string | `""` | Filter by language (e.g., `python`) |
| `maxResults` | integer | `25` | Max results to return |

#### Input example

```json
{
    "mode": "users",
    "searchQuery": "location:berlin followers:>1000",
    "maxResults": 25
}
```

GitHub's [user search qualifiers](https://docs.github.com/en/search-github/searching-on-github/searching-users) can filter by location, followers, repositories, language, and account creation date. User search returns the same enriched fields as profile mode and charges the existing `profile` event once per emitted row.

Existing inputs that omit `mode` still use `trending`, and all existing modes keep their current output fields and pricing events.

### Output example

#### Repository

```json
{
    "name": "react",
    "fullName": "facebook/react",
    "owner": "facebook",
    "description": "The library for web and native user interfaces.",
    "url": "https://github.com/facebook/react",
    "homepageUrl": "https://react.dev",
    "language": "JavaScript",
    "stars": 243711,
    "forks": 50668,
    "watchers": 243711,
    "openIssues": 1150,
    "topics": ["declarative", "frontend", "javascript", "library", "react", "ui"],
    "license": "MIT",
    "isArchived": false,
    "isFork": false,
    "defaultBranch": "main",
    "createdAt": "2013-05-24T16:15:54Z",
    "updatedAt": "2026-03-08T09:17:07Z",
    "pushedAt": "2026-03-05T15:52:24Z",
    "size": 942058,
    "scrapedAt": "2026-03-08T09:45:54.218Z"
}
```

#### Profile

```json
{
    "username": "torvalds",
    "name": "Linus Torvalds",
    "bio": null,
    "company": "Linux Foundation",
    "location": "Portland, OR",
    "followers": 289246,
    "following": 0,
    "publicRepos": 11,
    "avatarUrl": "https://avatars.githubusercontent.com/u/1024025?v=4",
    "url": "https://github.com/torvalds",
    "scrapedAt": "2026-03-08T09:45:50.123Z"
}
```

### How much does it cost to scrape GitHub?

GitHub Scraper uses pay-per-event pricing:

| Event | Price |
|-------|-------|
| Run started | $0.005 |
| Repository or profile extracted | $0.003 per item |

#### Cost examples

| Scenario | Items | Cost |
|----------|-------|------|
| Daily trending (25 repos) | 25 | ~$0.08 |
| 10 repo details | 10 | ~$0.04 |
| 50 contributor rows | 50 | ~$0.16 |
| Search (50 results) | 50 | ~$0.16 |

Apify's free plan includes $5/month in platform credits — enough for ~60 trending scrapes.

### Using GitHub Scraper with the Apify API

#### Node.js

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });

const run = await client.actor('automation-lab/github-scraper').call({
    mode: 'trending',
    trendingSince: 'daily',
    maxResults: 25,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach(repo => {
    console.log(`${repo.fullName}: ${repo.stars} stars (+${repo.starsToday} today)`);
});
```

#### Python

```python
from apify_client import ApifyClient

client = ApifyClient('YOUR_API_TOKEN')

run = client.actor('automation-lab/github-scraper').call(run_input={
    'mode': 'trending',
    'trendingSince': 'daily',
    'maxResults': 25,
})

dataset = client.dataset(run['defaultDatasetId']).list_items().items
for repo in dataset:
    print(f"{repo['fullName']}: {repo['stars']} stars (+{repo['starsToday']} today)")
```

#### cURL

```bash
curl "https://api.apify.com/v2/acts/automation-lab~github-scraper/run-sync-get-dataset-items?token=YOUR_API_TOKEN" \
  -X POST -H "Content-Type: application/json" \
  -d '{"mode": "trending", "trendingSince": "daily", "maxResults": 10}'
```

### Integrations

GitHub Scraper works with all Apify integrations:

- **Scheduled runs** — Track trending repos daily or weekly
- **Webhooks** — Get notified when a scrape completes
- **Google Sheets** — Export repos and profiles to spreadsheets
- **Slack** — Post trending repos to your team's channel
- **Zapier / Make** — Automate workflows with GitHub data

### Tips

- 📈 **Track trending daily** — Schedule runs to build a history of trending repos
- 🔍 **Use search for competitive analysis** — Search for keywords in your domain
- 👥 **Profile scraping** — Great for building lists of developers by location or company
- 🤝 **Contributor discovery** — Turn a public repository watchlist into developer leads ranked by repository contribution count
- 🏷️ **Filter by language** — Use `trendingLanguage` to focus on specific tech stacks
- ⚡ **Rate limits** — The actor follows GitHub's retry/reset headers for bounded retries; add a secret `githubToken` for larger jobs
- ✅ **Deterministic URLs** — URL modes require full public `https://github.com/...` URLs and deduplicate normalized targets in first-seen order
- 🚨 **Clear failures** — Exhausted API errors and empty/unparseable Trending pages fail the run instead of silently returning a misleading green empty result
- 📊 **Combine modes** — Run trending to discover repos, then repos mode for full details

### Legality

GitHub Scraper collects publicly available repository and profile information. Use the data responsibly, respect GitHub's terms, and avoid collecting private, authenticated, or personal data beyond your lawful purpose.

### Is it legal to scrape GitHub?

GitHub provides a public REST API specifically designed for programmatic access. This scraper uses that API and publicly accessible web pages. It does not bypass authentication, rate limits, or access private data. Always review [GitHub's Terms of Service](https://docs.github.com/en/site-policy/github-terms/github-terms-of-service) and [API usage policies](https://docs.github.com/en/rest/overview/resources-in-the-rest-api).

### FAQ

**Do I need a GitHub API token?**
No. The scraper works anonymously with GitHub's public API (60 requests/hour per IP). For larger jobs, set the optional `githubToken` secret input; Apify stores it encrypted, the Actor sends it only in the `Authorization` header to `api.github.com`, and it is never written to logs or datasets. A standard authenticated GitHub allowance is 5,000 requests/hour.

**How many trending repos are shown?**
GitHub's trending page shows up to 25 repos per language/period combination.

**Can I search for users or organizations?**
Yes. Use `mode: "users"` with qualifiers such as `location:berlin followers:>1000`; each match is enriched into a full public profile row. Repository search remains available as `mode: "search"`.

**How does contributors mode work?**
Provide one or more full public repository URLs such as `https://github.com/facebook/react`. The actor paginates the public GitHub API, deduplicates by `repositoryFullName` and `username`, and stops before exceeding `maxResults`. It does not enrich profiles or fetch commits, emails, followers, organizations, or authenticated data.

**What if a contributor has no public avatar or account type?**
Optional public values are returned as `null`; the actor does not infer missing data.

**What about private repos?**
The scraper only accesses public data. Private repos are not visible without authentication.

### Use with Claude AI (MCP)

This actor is available as a tool in Claude AI through the Model Context Protocol (MCP). Add it to Claude Desktop, Cursor, Windsurf, or any MCP-compatible client.

#### Setup for Claude Code

```bash
claude mcp add --transport http apify "https://mcp.apify.com?tools=automation-lab/github-scraper"
```

#### Setup for Claude Desktop, Cursor, or VS Code

Add this to your MCP config file:

```json
{
    "mcpServers": {
        "apify": {
            "url": "https://mcp.apify.com?tools=automation-lab/github-scraper"
        }
    }
}
```

#### Example prompts

- "What are the trending GitHub repositories today? Show me the top 25 with their star counts"
- "Search GitHub for the most starred machine learning repositories and summarize what each one does"
- "Get the GitHub profile and public repo stats for these developers: torvalds, gvanrossum, dhh"

Learn more in the [Apify MCP documentation](https://docs.apify.com/platform/integrations/mcp).

**The scraper returns an error about rate limiting.**
GitHub's public API allows 60 requests/hour per IP without authentication. The Actor respects `Retry-After` and `X-RateLimit-Reset` for bounded retries, then fails actionably if the limit remains exhausted. Add a secret `githubToken`, reduce `maxResults`, or split very large jobs. If failure occurs after some rows were emitted, those partial rows remain in the dataset and the run still reports failure.

**Profile data shows `null` for some fields like bio or company.**
Not all GitHub users fill in their profile details. The scraper returns `null` for fields the user hasn't set. This is expected behavior.

### Related actors

- [GitHub Trending Scraper](https://apify.com/automation-lab/github-trending-scraper) — Trending repositories from GitHub with star velocity
- [Hacker News Scraper](https://apify.com/automation-lab/hackernews-scraper) — Stories from Hacker News front page, newest, Ask HN, and more
- [Homebrew Scraper](https://apify.com/automation-lab/homebrew-scraper) — Homebrew formulas and casks with install counts
- [Stack Overflow Scraper](https://apify.com/automation-lab/stackoverflow-scraper) — Questions, answers, and tags from Stack Overflow
- [npm Scraper](https://apify.com/automation-lab/npm-scraper) — Package metadata from the npm registry
- [PyPI Scraper](https://apify.com/automation-lab/pypi-scraper) — Python package data from PyPI
- [Crates Scraper](https://apify.com/automation-lab/crates-scraper) — Rust crate metadata from crates.io
- [Hash Generator](https://apify.com/automation-lab/hash-generator) — Generate MD5, SHA-1, SHA-256, and SHA-512 hashes

# Changelog

This Actor's version history is a separate document: https://apify.com/automation-lab/github-scraper/changelog.md

# Actor input Schema

## `mode` (type: `string`):

What to scrape from GitHub.

## `urls` (type: `array`):

List of public GitHub repository or profile URLs. Used in 'repos', 'profiles', and 'contributors' modes.

## `searchQuery` (type: `string`):

GitHub search query. Used in 'search' for repositories and 'users' for public profiles; GitHub qualifiers are supported.

## `githubToken` (type: `string`):

Optional GitHub personal access token for higher API limits. Stored encrypted and sent only to api.github.com.

## `trendingSince` (type: `string`):

Time period for trending repos.

## `trendingLanguage` (type: `string`):

Filter trending repos by programming language (e.g., 'python', 'javascript'). Leave empty for all languages.

## `maxResults` (type: `integer`):

Maximum number of results to return.

## Actor input object example

```json
{
  "mode": "trending",
  "urls": [
    "https://github.com/facebook/react",
    "https://github.com/vercel/next.js"
  ],
  "searchQuery": "machine learning",
  "trendingSince": "daily",
  "trendingLanguage": "",
  "maxResults": 20
}
```

# Actor output Schema

## `repos` (type: `string`):

Repository detail, search, and trending rows.

## `profiles` (type: `string`):

Public GitHub developer profile rows.

## `contributors` (type: `string`):

Repository contributor rows with public profile URLs and contribution counts.

## `userSearch` (type: `string`):

Enriched public GitHub profiles returned by user search.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "trending",
    "urls": [
        "https://github.com/facebook/react",
        "https://github.com/vercel/next.js"
    ],
    "searchQuery": "machine learning",
    "trendingSince": "daily",
    "trendingLanguage": "",
    "maxResults": 20
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/github-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "trending",
    "urls": [
        "https://github.com/facebook/react",
        "https://github.com/vercel/next.js",
    ],
    "searchQuery": "machine learning",
    "trendingSince": "daily",
    "trendingLanguage": "",
    "maxResults": 20,
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/github-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "trending",
  "urls": [
    "https://github.com/facebook/react",
    "https://github.com/vercel/next.js"
  ],
  "searchQuery": "machine learning",
  "trendingSince": "daily",
  "trendingLanguage": "",
  "maxResults": 20
}' |
apify call automation-lab/github-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/github-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/SLzaZn1ea0Dd8p7i7/builds/k52dIgsfSYmOdm4s9/openapi.json