# 💎ESG Scraper: Sustainability Reports & PDF Disclosures (`primeparse/esg-content-scraper`) Actor

Powerful ESG scraper (Environmental, Social, and Governance) to automatically extract sustainability reports, PDF disclosures, articles, and content from any website. Get clean, AI-ready datasets with keyword filtering, metadata extraction, images, links, and full PDF support.

- **URL**: https://apify.com/primeparse/esg-content-scraper.md
- **Developed by:** [PrimeParse](https://apify.com/primeparse) (community)
- **Categories:** Automation, Lead generation, Developer tools
- **Features:** Limited permissions
- **Stats:** 18 total users, 0 monthly users, 100.0% runs succeeded, 1 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $2.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## 🌱 ESG Scraper: Sustainability Reports, Articles & PDF Disclosures Extractor

**Enterprise-grade ESG web scraper** that automatically extracts sustainability articles, corporate reports, climate news, and PDF disclosures — clean, structured, and ready for investors, compliance teams, or AI training.

**High-quality ESG & Sustainability Web Scraper for Investors, Analysts, and AI Teams**

Automatically collects **ESG articles, sustainability reports, corporate disclosures, climate news, and PDF reports** from any website — clean, structured, ready for analysis or AI.

Built for:

- Sustainable investors & analysts
- Compliance and risk teams
- AI/ML engineers building ESG models
- Researchers and NGOs tracking climate & governance trends

✅ Smart ESG keyword filtering
✅ Full clean article text extraction
✅ PDF sustainability reports parsing
✅ Rich metadata (date, author, description)
✅ ESG-relevant images and related links
✅ AI-ready dataset splitting (overview / full-text / images)

👉 **Runs on Apify • No code required • Pay only for compute used**

***

### 🚀 Why This Scraper

✔ **Purpose-Built for ESG Data**
Intelligently filters pages using custom ESG keywords (climate, emissions, governance, CSR, net zero, etc.).

✔ **Excellent PDF Handling**
Full text extraction from sustainability and ESG reports (PDF) with metadata where available.

✔ **Clean & Noise-Free Output**
Removes ads, navigation, scripts — only meaningful content remains.

✔ **Rich Structured Data**
Title, publication date, author, description, ESG keywords, internal links, relevant images.

✔ **AI & ML Ready**
Optional splitting into specialized datasets for RAG, LLM fine-tuning, or training.

✔ **Fast & Efficient**\
Powered by **Crawlee + Cheerio** — excellent for static and content-heavy sites (news, corporate pages, PDFs). For heavily JavaScript-rendered sites, results may vary.

✔ **Safe & Controlled Crawling**
Automatic domain restriction, depth limit (max 3 levels), request limits.

***

### 💼 Use Cases

- ESG portfolio screening and risk monitoring
- Training ESG-focused LLMs or RAG systems
- Regulatory compliance and disclosure tracking
- Competitive intelligence on corporate sustainability
- Academic research on climate and governance trends

***

### 📊 Supported Sources

- ESG news sections (Reuters, Bloomberg, FT, Guardian, etc.)
- Corporate sustainability / ESG pages
- Annual sustainability reports (PDF)
- Climate, emissions, governance disclosures

***

### ⚙️ How It Works

1. Provide start URLs (news sections, corporate pages, PDF links)
2. Set custom ESG keywords and limits
3. Run the Actor
4. Download clean, structured ESG datasets

***

### 🧩 Input Configuration

#### Example JSON Input

```json
{
"startUrls": [
  { "url": "https://www.reuters.com/sustainability/" },
  { "url": "https://www.weforum.org/stories/technological-innovation/" }
],
  "allowedDomains": ["reuters.com"],
  "useApifyProxy": false,
  "maxRequestsPerCrawl": 500,
  "esgKeywords": [
    "ESG",
    "sustainability",
    "climate",
    "emissions",
    "net zero",
    "governance"
  ],
  "extractContent": true,
  "extractMetadata": true,
  "followLinks": true,
  "useSeparateDatasets": true,
  "cleanDefaultDataset": true,
  "proxyUrls": [
    {
      "url": "http://user:pass@host:port"
    }
  ]
}
```

**Key Options**

- `startUrls` — one or more starting pages or direct PDF links (required)
- `allowedDomains` — restrict crawling to specific domains. If empty, automatically limited to domains from `startUrls`
- `maxRequestsPerCrawl` — control cost and crawl size
- `esgKeywords` — custom list for relevance filtering (default includes common ESG terms)
- `extractContent` / `extractMetadata` — toggle full text or metadata extraction
- `followLinks` — enable internal crawling (limited to depth 3 for safety)
- `useSeparateDatasets` — recommended for large runs and AI workflows
- `cleanDefaultDataset` — clear previous run data

***

### 📂 Output Datasets

**When `useSeparateDatasets: true` (recommended):**

- `esg-overview` (primary) — lightweight metadata for fast analysis
- `esg-full-content` — long articles (>5000 characters)
- `esg-images` — ESG-relevant images with context
- Default dataset — minimal preview records (for Apify UI visibility)

**When `useSeparateDatasets: false`**

- Single dataset with full detailed records

***

#### Example Output Record (Full Mode)

```json
{
  "url": "https://www.reuters.com/sustainability/example",
  "title": "Companies strengthen climate commitments",
  "scrapedAt": "2025-12-15T10:30:45Z",
  "publishedDate": "2025-12-10",
  "author": "Jane Doe",
  "description": "Major firms enhance ESG targets...",
  "content": "Full clean article text...\n\nParagraphs preserved...",
  "esgKeywords": ["climate", "emissions", "sustainability"],
  "relatedLinks": [
    {
      "url": "https://www.reuters.com/sustainability/esg-guide",
      "text": "ESG Explained"
    }
  ],
  "images": [
    {
      "url": "https://reuters.com/chart-netzero.jpg",
      "alt": "Net zero emissions progress"
    }
  ]
}
```

#### PDF Example

```json
{
  "url": "https://company.com/sustainability-2024.pdf",
  "title": "Annual Sustainability Report 2024",
  "content": "Full extracted report text...",
  "esgKeywords": ["sustainability", "carbon", "governance"],
  "type": "PDF",
  "author": "Corporate Sustainability Team",
  "publishedDate": "2024-03-15"
}
```

***

### 🏁 Getting Started

1. Click “Try for free” on Apify
2. Paste ESG/sustainability URLs or direct PDF links
3. Customize keywords and limits
4. Run and download your dataset

***

### 📧 Support

- Email: **kidaxxb@gmail.com**
- Response within 24 hours
- Issues: Use Apify Issues tab

**Tags:** ESG, sustainability, web scraping, PDF extraction, climate data, corporate governance, RAG, LLM training, sustainable investing, compliance monitoring

**Built with ❤️ on Apify**

# Actor input Schema

## `startUrls` (type: `array`):

URLs to start scraping ESG content from. Examples: ESG news sites, sustainability reports, ESG rating pages.

## `maxRequestsPerCrawl` (type: `integer`):

Maximum number of pages to scrape (0 = unlimited).

## `useApifyProxy` (type: `boolean`):

Whether to use Apify Proxy (recommended). Set to true only if your account has Proxy External Access.

## `allowedDomains` (type: `array`):

Restrict crawling to these domains (e.g., \['www.ft.com', 'www.reuters.com']). Leave empty for unlimited.

## `esgKeywords` (type: `array`):

Keywords to filter ESG-related content. Pages without these keywords will be skipped.

## `extractContent` (type: `boolean`):

Whether to extract the full text content of each page.

## `extractMetadata` (type: `boolean`):

Whether to extract metadata (publication date, author, description).

## `followLinks` (type: `boolean`):

Whether to follow and scrape linked pages.

## `cleanDefaultDataset` (type: `boolean`):

Delete all previous data before starting. Keep OFF to accumulate results across runs.

## `useSeparateDatasets` (type: `boolean`):

Split output into multiple datasets: esg-overview (light metadata), esg-full-content (long texts), esg-images (images only). Useful for large crawls or ML training.

## `proxyUrls` (type: `array`):

Newline-separated proxy URLs (e.g. http://user:pass@host:port). If provided, these will be used instead of Apify Proxy.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.reuters.com/sustainability/"
    },
    {
      "url": "https://www.weforum.org/stories/forum-institutional/"
    }
  ],
  "maxRequestsPerCrawl": 100,
  "useApifyProxy": false,
  "allowedDomains": [],
  "esgKeywords": [
    "ESG",
    "environmental",
    "social",
    "governance",
    "sustainability",
    "carbon",
    "emissions",
    "climate",
    "green",
    "renewable",
    "CSR",
    "corporate responsibility"
  ],
  "extractContent": true,
  "extractMetadata": true,
  "followLinks": true,
  "cleanDefaultDataset": false,
  "useSeparateDatasets": false
}
```

# Actor output Schema

## `message` (type: `string`):

No description

## `itemsScraped` (type: `string`):

No description

## `datasetId` (type: `string`):

No description

## `stats` (type: `string`):

No description

## `finishedAt` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.reuters.com/sustainability/"
        },
        {
            "url": "https://www.weforum.org/stories/forum-institutional/"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("primeparse/esg-content-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [
        { "url": "https://www.reuters.com/sustainability/" },
        { "url": "https://www.weforum.org/stories/forum-institutional/" },
    ] }

# Run the Actor and wait for it to finish
run = client.actor("primeparse/esg-content-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.reuters.com/sustainability/"
    },
    {
      "url": "https://www.weforum.org/stories/forum-institutional/"
    }
  ]
}' |
apify call primeparse/esg-content-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,primeparse/esg-content-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Q18mtke9TnO2aoOQy/builds/ylYI89NLz3byuTCg7/openapi.json