# Bulk Metadata, Open Graph & JSON-LD Extractor — SEO Tags (`haketa/bulk-metadata-extractor`) Actor

Extract on-page metadata from thousands of URLs at once: title, meta description, Open Graph & Twitter Card tags, canonical, favicon, hreflang, RSS feed and schema.org JSON-LD structured data. Fast, keyless bulk SEO metadata and structured-data extractor.

- **URL**: https://apify.com/haketa/bulk-metadata-extractor.md
- **Developed by:** [Haketa](https://apify.com/haketa) (community)
- **Categories:** SEO tools, Developer tools, Business
- **Features:** Limited permissions
- **Stats:** 3 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.60 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Bulk Metadata, Open Graph & JSON-LD Extractor — SEO Tags at Scale

[![Apify Actor](https://img.shields.io/badge/Apify-Actor-orange?style=flat-square\&logo=apify)](https://apify.com/haketa/bulk-metadata-extractor)
[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg?style=flat-square)](https://apify.com/haketa/bulk-metadata-extractor)
[![Data Export](https://img.shields.io/badge/Export-JSON%20%7C%20CSV%20%7C%20Excel-blue?style=flat-square)](https://apify.com/haketa/bulk-metadata-extractor)
[![SEO Tools](https://img.shields.io/badge/SEO-Metadata%20Audit-brightgreen?style=flat-square)](https://apify.com/haketa/bulk-metadata-extractor)
[![Scale](https://img.shields.io/badge/Scale-1%2C000%2B%20to%20100K%2B%20URLs-purple?style=flat-square)](https://apify.com/haketa/bulk-metadata-extractor)

**Extract on-page SEO metadata, Open Graph, Twitter Card and schema.org JSON-LD from thousands of URLs at once.** This **metadata scraper** reads the `<head>` of any webpage and returns, for each URL: **title, meta description, canonical, Open Graph & Twitter Card tags, favicon, hreflang, RSS feed and structured data** — no login, no coding, no API key. Paste a list of URLs and get a clean, comparable metadata inventory back. Whether you want to **scrape website metadata** for a handful of pages or audit an entire enterprise sitemap, this bulk **metadata scraper** does it in one run.

Export every result to **JSON, CSV, Excel, or API**.

> Built for **technical SEO specialists, content auditors, link-preview / unfurl developers, growth marketers, and data engineers enriching URL datasets** who need the metadata layer of the web in bulk.

***

### ⚡ At a glance

| Feature | Details |
| --- | --- |
| **Data Extracted** | Title, meta description, canonical, Open Graph (7 tags), Twitter Cards (6 tags), schema.org JSON-LD, hreflang, favicon, manifest |
| **JSON-LD Schema Scope** | Recursive extraction across all `@type` and `@graph` nodes (Product, Article, Organization, BreadcrumbList, etc.) |
| **Throughput & Speed** | High-concurrency Cheerio pool processing 500 URLs in ~3 minutes |
| **Scale & Capacity** | Supports individual page queries up to 100,000+ URL lists in a single job |
| **Engine Architecture** | Pure-HTTP lightweight pipeline bypassing browser rendering for maximum speed and cost efficiency |
| **Output Formats** | Clean datasets in JSON, CSV, Excel, XML, HTML table, and JSON Lines |
| **Integrations** | Apify REST API, webhooks, Python SDK, Node.js SDK, Make, Zapier, Google Sheets |
| **Proxy Flexibility** | Keyless direct HTTP by default; optional Apify Proxy for rate-limited corporate targets |

***

### 🔎 Why scrape website metadata?

Every webpage's HTML `<head>` encapsulates critical signals that dictate how that URL behaves across search engines, social platforms, and messaging applications. The `<title>`, `<meta name="description">`, and `<link rel="canonical">` define search ranking and SERP click-through rates. The Open Graph (`og:*`) and Twitter Card (`twitter:*`) tags govern how links unfurl in Slack, LinkedIn, Twitter/X, and WhatsApp. Furthermore, schema.org JSON-LD structured data instructs Google and AI search engines on rich snippet eligibility (`Product`, `FAQPage`, `Review`, `Recipe`, `Organization`).

Auditing these elements manually via browser developer tools or URL-by-URL browser extensions is painfully slow and impossible to scale across sitemaps or competitor catalogs.

This **bulk metadata scraper** automates the entire audit process. Paste a list of URLs or bare domains, and the extractor fetches each page over lightweight HTTP, parses `<head>` markup with Cheerio, resolves relative URLs to absolute links, walks JSON-LD schema graphs recursively, and outputs a complete, unified metadata audit record for each URL.

**Common reasons teams scrape website metadata:**

- **Technical SEO audits** — identify missing meta titles, truncated descriptions, duplicate canonicals, and missing meta robots across entire sites.
- **Structured data & rich snippet audits** — verify which product, blog, or category pages contain valid schema.org markup and which lack structured data.
- **Social link preview & unfurling systems** — power chat apps, CMS link unfurlers, and bookmark managers with real-time Open Graph titles and preview images.
- **Site migration & redesign QA** — capture metadata snapshots pre- and post-migration to verify zero loss of canonicals or title tags.
- **Competitor intelligence** — analyze how rival publications or e-commerce stores optimize their meta descriptions, OG images, and schema entities.
- **Lead & CRM enrichment** — enrich raw landing page URLs with company branding, favicons, site names, and descriptive titles.

***

### ✨ What this website metadata scraper does

- ✅ **Reads any list of URLs in bulk** — from a quick test of 10 links to **100,000+ URLs** fanned out across parallel workers.
- ✅ **Returns one clean record per URL** with 37 normalized fields ready for immediate analysis.
- ✅ **Standard SEO tags** — `title`, `metaDescription`, `canonical`, `robots`, `viewport`, `themeColor`, `author`, `keywords`, HTML `lang`, and `charset`.
- ✅ **Open Graph metadata** — `ogTitle`, `ogDescription`, `ogImage`, `ogType`, `ogSiteName`, `ogUrl`, `ogLocale`.
- ✅ **Twitter Card metadata** — `twitterCard`, `twitterTitle`, `twitterDescription`, `twitterImage`, `twitterSite`, `twitterCreator`.
- ✅ **Icons, feeds & manifests** — `favicon`, `appleTouchIcon`, web app `manifestUrl`, and RSS/Atom `feedUrl`.
- ✅ **Internationalization tracking** — extracts all alternate language versions from `link[rel=alternate][hreflang]`.
- ✅ **Deep schema.org JSON-LD parsing** — extracts all `script[type="application/ld+json"]` blocks, recursively indexes every `@type` (including nested `@graph` objects), and optionally exports raw JSON.
- ✅ **Diagnostic response telemetry** — records `statusCode`, final redirected `finalUrl`, MIME `contentType`, and detailed `error` messages for broken pages.
- ✅ **Fully resolved absolute URLs** — automatically converts relative favicon, image, and canonical paths into valid absolute URLs.
- ✅ **Resilient retry engine** — handles transient `429` and `5xx` errors with backoff retries and connection recycling.

***

### 🎯 Use cases & workflows

#### 1. Site-Wide Technical SEO Audit

Export your sitemap XML URLs and feed them into this Actor. Export the results to CSV and filter for:

- `title == ""` or `title.length > 60`
- `metaDescription == ""` or `metaDescription.length > 160`
- `canonical != finalUrl` (identifies improper canonicalization loops)

#### 2. Rich Snippet Schema Opportunity Analysis

Filter results for `schemaTypes == ""` to identify high-traffic e-commerce or editorial pages missing schema.org `Product` or `Article` tags that could qualify for enhanced Google search snippets.

#### 3. Automated Link-Preview Generation

Call the Actor via API whenever an internal user bookmarks or submits an external URL. Store the returned `ogTitle`, `ogDescription`, and `ogImage` to generate rich preview cards.

#### 4. Migration Regression Testing

Run an extraction on your staging environment before launching a site rebuild. Diff staging metadata against production metadata to ensure titles, descriptions, and hreflang tags remain intact.

***

### 🧩 Schema.org Structured Data Entity Reference

This Actor includes an advanced recursive JSON-LD parser that identifies schema.org entities even when nested multiple levels deep within `@graph` structures:

| Schema Entity | Typical URL Types | Key Attributes Monitored | Rich Result Eligibility |
| --- | --- | --- | --- |
| `Product` | E-commerce PDPs, SaaS pricing | `name`, `offers`, `aggregateRating`, `brand`, `sku` | Product snippets, price & stock badges |
| `Article` / `NewsArticle` | Editorial posts, news articles | `headline`, `author`, `datePublished`, `publisher` | Top Stories, author rich carousels |
| `BreadcrumbList` | Category and product taxonomies | `itemListElement`, `position`, `name`, `item` | Breadcrumb hierarchy in search snippets |
| `Organization` | Homepages, about pages, contact | `name`, `url`, `logo`, `contactPoint`, `sameAs` | Knowledge Graph brand panels |
| `WebSite` | Domain homepages | `name`, `url`, `potentialAction` (`SearchAction`) | Google Sitelinks Search Box |
| `FAQPage` | Help centers, product FAQs | `mainEntity` (`Question`, `acceptedAnswer`) | Expandable FAQ search accordions |
| `Review` | Testimonial pages, merchant hubs | `itemReviewed`, `reviewRating`, `author` | Star rating review snippets |
| `HowTo` | Tutorial and DIY documentation | `step`, `supply`, `tool`, `totalTime` | HowTo visual rich cards |
| `LocalBusiness` | Storefronts, branch directories | `address`, `telephone`, `openingHoursSpecification` | Local 3-Pack and Maps integration |
| `JobPosting` | Careers pages, job boards | `title`, `baseSalary`, `datePosted`, `employmentType` | Google for Jobs listings |

***

### 🔄 End-to-End Automation & Webhook Integration

Connect this Actor directly to automated marketing and CRM operations using Apify Webhooks and modern iPaaS tools:

#### Automated Sitemap Audit with Make (Integromat) or n8n

1. **Trigger on Run Succeeded**: Create an Apify webhook that activates whenever `bulk-metadata-extractor` completes.
2. **Fetch Dataset Items**: In Make or n8n, iterate through the resulting dataset records.
3. **Filter Missing Critical Tags**: Identify records where `title == ""` OR `metaDescription == ""` OR `canonical == ""`.
4. **Export to Jira / Asana**: Automatically generate technical SEO bug tickets for engineering or content teams with the exact URL and missing tags.
5. **Slack Alert Notification**: Post a summary message to `#seo-alerts` displaying the total audit count and missing tag percentages.

#### Sample CSV Export Structure

When exporting results to CSV or spreadsheet applications, the data formats into predictable columns:

```csv
url,finalUrl,statusCode,contentType,title,metaDescription,canonical,lang,charset,robots,viewport,ogTitle,ogDescription,ogImage,schemaTypes,jsonLdCount,error,scrapedAt
"https://kubernetes.io/","https://kubernetes.io/","200","text/html","Kubernetes","Production-Grade Container Orchestration","https://kubernetes.io/","en","utf-8","index, follow","width=device-width","Kubernetes","Open source container system...","https://kubernetes.io/images/k8s.png","Organization, SearchAction","1","","2026-11-10T16:04:00.000Z"
```

***

### 📥 Input parameters

Configure your run via the Apify Console or pass a JSON payload directly through the API. Every field is optional.

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `urls` | array (stringList) | `[]` | List of webpage URLs or bare domains (e.g., `theverge.com`). Leave empty for a popular demo sample. |
| `includeJsonLd` | boolean | `true` | Include the full schema.org JSON-LD string in `structuredData`. Set to false for a lighter output containing only `schemaTypes`. |
| `maxItems` | integer | `0` | Maximum URLs to process. `0` = unlimited (processes your entire list). Supports 1,000+ to 100K+ URLs. |
| `maxConcurrency` | integer | `20` | Parallel concurrent requests (1–100). Raise for large audits on high-bandwidth networks. |
| `proxyConfiguration` | object | `{ "useApifyProxy": false }` | Optional Apify Proxy settings for targets with aggressive anti-scraping firewalls. |

#### Example input — Targeted multi-site audit

```json
{
  "urls": [
    "https://www.theverge.com",
    "https://github.com",
    "https://stripe.com",
    "https://www.nytimes.com"
  ],
  "includeJsonLd": true,
  "maxConcurrency": 20
}
```

#### Example input — 1,000+ large scale sitemap audit

```json
{
  "urls": [
    "https://example.com/page-1",
    "https://example.com/page-2",
    "https://example.com/page-1000"
  ],
  "includeJsonLd": false,
  "maxItems": 1500,
  "maxConcurrency": 50
}
```

***

### 📤 Output structure

Each URL generates a comprehensive record containing SEO, social, and structured data attributes:

```json
{
  "url": "https://kubernetes.io/",
  "finalUrl": "https://kubernetes.io/",
  "statusCode": "200",
  "contentType": "text/html",
  "title": "Kubernetes",
  "metaDescription": "Kubernetes, also known as K8s, is an open source system for automating deployment, scaling, and management of containerized applications.",
  "canonical": "https://kubernetes.io/",
  "lang": "en",
  "charset": "utf-8",
  "robots": "index, follow",
  "viewport": "width=device-width,initial-scale=1,shrink-to-fit=no",
  "themeColor": "#326de6",
  "ogTitle": "Production-Grade Container Orchestration",
  "ogDescription": "Kubernetes, also known as K8s, is an open source system for automating deployment, scaling, and management of containerized applications.",
  "ogImage": "https://kubernetes.io/images/kubernetes-open-graph.png",
  "ogType": "website",
  "ogSiteName": "Kubernetes",
  "ogUrl": "https://kubernetes.io/",
  "ogLocale": "en",
  "twitterCard": "summary",
  "twitterTitle": "Production-Grade Container Orchestration",
  "twitterDescription": "Kubernetes, also known as K8s, is an open source system...",
  "twitterImage": "https://kubernetes.io/images/kubernetes-open-graph.png",
  "favicon": "https://kubernetes.io/images/kubernetes.png",
  "appleTouchIcon": "https://kubernetes.io/images/favicon.png",
  "manifestUrl": "https://kubernetes.io/manifest.json",
  "feedUrl": "https://kubernetes.io/feed.xml",
  "hreflang": "bn, zh-cn, fr, de, hi, id, it, ja, ko, fa, pl, pt-br, ru, es, uk, vi",
  "schemaTypes": "Organization, SearchAction",
  "jsonLdCount": "1",
  "structuredData": "[{\"@context\":\"https://schema.org\",\"@type\":\"Organization\",\"url\":\"https://kubernetes.io\"}]",
  "error": "",
  "scrapedAt": "2026-11-10T16:04:00.000Z"
}
```

***

### 📊 Extracted fields & data coverage

| Field | Coverage | Type | Description |
| --- | --- | --- | --- |
| 🔗 `url` | 🟢 High (100%) | String | Original input URL submitted for extraction |
| 📍 `finalUrl` | 🟢 High (~99%) | String | Final URL resolved after HTTP redirects |
| 📡 `statusCode` | 🟢 High (100%) | String | HTTP response code (e.g., `200`, `301`, `404`, `500`) |
| 📄 `contentType` | 🟢 High (~99%) | String | MIME type header (e.g., `text/html`, `application/pdf`) |
| 🏷️ `title` | 🟢 High (~98%) | String | Document `<title>` tag (falls back to `og:title`) |
| 📝 `metaDescription` | 🟢 High (~88%) | String | `meta[name="description"]` text content |
| 🎯 `canonical` | 🟢 High (~95%) | String | `link[rel="canonical"]` URL resolved to absolute form |
| 🌐 `lang` | 🟢 High (~90%) | String | HTML document language code (e.g., `en`, `es`, `fr`) |
| 🔤 `charset` | 🟢 High (~92%) | String | Document character encoding (e.g., `utf-8`) |
| 🤖 `robots` | 🟡 Conditional (~40%) | String | Meta robots directive (e.g., `index, follow`, `noindex`) |
| 📱 `viewport` | 🟢 High (~95%) | String | Mobile responsive viewport meta tag |
| 🎨 `themeColor` | 🟡 Conditional (~30%) | String | Mobile browser toolbar theme color |
| 🖼️ `ogTitle` | 🟢 High (~85%) | String | Open Graph social sharing title |
| 📄 `ogDescription` | 🟢 High (~82%) | String | Open Graph preview card description |
| 🌅 `ogImage` | 🟢 High (~80%) | String | Open Graph social thumbnail image URL |
| 🏷️ `ogType` | 🟢 High (~78%) | String | Open Graph object type (`website`, `article`, `product`) |
| 🏢 `ogSiteName` | 🟢 High (~75%) | String | Brand or website name declared via Open Graph |
| 🐦 `twitterCard` | 🟢 High (~70%) | String | Twitter Card format (`summary`, `summary_large_image`) |
| 🔖 `favicon` | 🟢 High (~99%) | String | Website favicon URL resolved to absolute path |
| 🌍 `hreflang` | 🟡 Conditional (~35%) | String | Comma-separated alternate localized language codes |
| 🧩 `schemaTypes` | 🟢 High (~65%) | String | Detected schema.org `@type` entities (comma-separated) |
| 🔢 `jsonLdCount` | 🟢 High (100%) | String | Number of valid JSON-LD script blocks identified |
| 📦 `structuredData` | 🟡 Conditional (~65%) | String | Serialized raw JSON-LD payload (when enabled) |
| ⚠️ `error` | ⚪ Optional | String | Diagnostic message if the page failed to load |
| ⏰ `scrapedAt` | 🟢 High (100%) | String | ISO 8601 UTC timestamp of execution |

***

### 🔁 Scale & 1,000+ deep parallel processing

This Actor is engineered for enterprise-grade bulk throughput:

- **1,000+ to 100K+ URLs per run**: Submit massive URL files or sitemaps without running out of memory.
- **Configurable worker concurrency**: Adjust `maxConcurrency` between 1 and 100 to balance speed and target server politeness.
- **Zero browser rendering overhead**: Bypassing Chromium eliminates massive RAM and CPU bottlenecks, cutting runtime costs by up to 90%.
- **De-duplicated input processing**: The Actor automatically de-duplicates input URL strings to prevent redundant network requests.

***

### 💻 Developer integration & code samples

#### Python SDK

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run_input = {
    "urls": [
        "https://www.theverge.com",
        "https://stripe.com",
        "https://github.com"
    ],
    "includeJsonLd": True,
    "maxConcurrency": 20
}

run = client.actor("haketa/bulk-metadata-extractor").call(run_input=run_input)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"URL: {item['url']} | Title: {item['title']} | Schema: {item['schemaTypes']}")
```

#### Node.js SDK

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('haketa/bulk-metadata-extractor').call({
    urls: ['https://stripe.com', 'https://github.com'],
    includeJsonLd: false,
    maxConcurrency: 20,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.table(items.map(i => ({ url: i.url, title: i.title, ogImage: i.ogImage, schema: i.schemaTypes })));
```

#### cURL API

```bash
curl -X POST "https://api.apify.com/v2/acts/haketa~bulk-metadata-extractor/runs?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://www.theverge.com", "https://stripe.com"],
    "includeJsonLd": true
  }'
```

***

### 💡 Practical tips for optimal metadata extraction

1. **Bare domains are supported**: You can submit `theverge.com` or `stripe.com`; the engine automatically prepends `https://`.
2. **Exclude raw JSON-LD for speed**: If you only need schema entity names (`schemaTypes`) and not raw JSON blocks, set `includeJsonLd: false` to reduce dataset size.
3. **Audit hreflang consistency**: Inspect the `hreflang` column to verify that regional sites correctly declare cross-border alternates.
4. **Inspect `contentType`**: Check `contentType` to confirm that your URLs returned standard `text/html` rather than redirected API responses or PDF documents.

***

### ❓ Frequently Asked Questions

**Does the scraper render client-side JavaScript?**\
No. It inspects server-rendered HTML markup where standard `<head>` tags and JSON-LD schema blocks are defined. This ensures ultra-fast execution and low operating costs.

**Can I analyze 20,000 URLs at once?**\
Yes. You can paste thousands of URLs into `urls`. The engine processes requests concurrently through an internal worker pool.

**Are relative image and favicon paths resolved?**\
Yes. All relative links for `og:image`, `twitter:image`, `canonical`, and `favicon` are converted to fully qualified absolute HTTP(S) URLs.

**What formats can I download?**\
Results can be downloaded in JSON, CSV, Excel, XML, HTML table, or JSON Lines via the Apify Console or API.

**How does error handling work?**\
If a target URL returns a `404`, `403`, or connection timeout, the Actor still outputs a record containing the `statusCode` and descriptive `error` message.

**Can I use this Actor to extract metadata for private intranet pages?**\
If your intranet or staging pages require specific HTTP authentication headers or are accessible only via a dedicated corporate proxy, configure your proxy credentials in `proxyConfiguration` to route traffic securely.

**How does the extractor handle multiple JSON-LD script tags on a single page?**\
Many modern CMS systems output separate JSON-LD tags for breadcrumbs, products, and organization info. The Actor parses every script tag on the page, combines unique entity types into `schemaTypes`, and consolidates all raw objects into `structuredData`.

**Does this scraper extract Open Graph audio or video tags?**\
The Actor focuses on the core Open Graph properties (`title`, `description`, `image`, `type`, `site_name`, `url`, `locale`) that power the vast majority of social cards and search engines.

**Is it safe to run across large corporate sites without causing server overload?**\
Yes. You can regulate the load on target servers using the `maxConcurrency` parameter (e.g. setting it to `5` or `10` for polite, throttled crawls).

***

### 🔗 Related SEO & data extraction scrapers

- [Bulk Tech Detector](https://apify.com/haketa/bulk-tech-detector) — detect website software stacks, CMS, frameworks, and analytics in bulk.
- [Sanctions & Watchlist Screening Scraper](https://apify.com/haketa/sanctions-screening) — screen names and corporate entities against OFAC, EU, UN, and UK regimes.
- [Treasury & Sovereign Debt Data Scraper](https://apify.com/haketa/treasury-data) — extract official US Treasury yield curves, debt metrics, and exchange rates.
- [Nevada Contractor License Scraper](https://apify.com/haketa/nevada-contractor-scraper) — extract licensed contractors, bonds, and phone contacts from the NSCB registry.

***

### 📋 Changelog

#### v1.2 (2026-11)

- **Unlocked 1,000+ to 100K+ batch processing**: confirmed unrestricted queue execution when `maxItems: 0` is set.
- **Enhanced input normalization**: added robust handling for `{ url: "..." }` object inputs and bare domain strings.
- **Upgraded documentation**: added visual badges, `⚡ At a glance` quick reference, field coverage tables, and Python/Node.js SDK examples.

#### v1.1 (2026-07)

- Added recursive schema.org JSON-LD parsing across nested `@graph` entities.
- Added absolute URL resolution for `ogUrl` and Twitter card images.

#### v1.0 (2026-06)

- Initial release with keyless Cheerio metadata extraction.

# Changelog

This Actor's version history is a separate document: https://apify.com/haketa/bulk-metadata-extractor/changelog.md

# Actor input Schema

## `urls` (type: `array`):

Paste the page URLs you want metadata for — one per line. Handles anything from a handful of pages to millions. Leave this empty to see a live sample across popular sites first.

## `includeJsonLd` (type: `boolean`):

Include the full schema.org JSON-LD blocks in a structuredData field. Turn off to keep only the detected schema types for a smaller, cleaner output.

## `maxItems` (type: `integer`):

Stop after this many URLs. Set to 0 for no limit and process your entire list.

## `maxConcurrency` (type: `integer`):

How many URLs to fetch at the same time. Higher is faster; lower is gentler on the target sites.

## `proxyConfiguration` (type: `object`):

Apify Proxy. Off by default; enable it if some target sites return errors on direct requests.

## Actor input object example

```json
{
  "urls": [
    "https://www.theverge.com",
    "https://github.com",
    "https://stripe.com",
    "https://www.nytimes.com",
    "https://www.airbnb.com",
    "https://www.shopify.com",
    "https://www.notion.so",
    "https://www.bbc.com"
  ],
  "includeJsonLd": true,
  "maxItems": 100,
  "maxConcurrency": 20,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `url` (type: `string`):

The input URL

## `finalUrl` (type: `string`):

URL after redirects

## `statusCode` (type: `string`):

HTTP status code

## `contentType` (type: `string`):

Response MIME type (e.g. text/html)

## `title` (type: `string`):

Page title

## `metaDescription` (type: `string`):

Meta description

## `canonical` (type: `string`):

Canonical URL

## `lang` (type: `string`):

HTML lang

## `charset` (type: `string`):

Charset

## `robots` (type: `string`):

Robots meta

## `viewport` (type: `string`):

Viewport meta

## `themeColor` (type: `string`):

Theme color

## `author` (type: `string`):

Author meta

## `keywords` (type: `string`):

Keywords meta

## `ogTitle` (type: `string`):

Open Graph title

## `ogDescription` (type: `string`):

Open Graph description

## `ogImage` (type: `string`):

Open Graph image

## `ogType` (type: `string`):

Open Graph type

## `ogSiteName` (type: `string`):

Open Graph site name

## `ogUrl` (type: `string`):

Open Graph URL

## `ogLocale` (type: `string`):

Open Graph locale

## `twitterCard` (type: `string`):

Twitter card type

## `twitterTitle` (type: `string`):

Twitter title

## `twitterDescription` (type: `string`):

Twitter description

## `twitterImage` (type: `string`):

Twitter image

## `twitterSite` (type: `string`):

Twitter site

## `twitterCreator` (type: `string`):

Twitter creator

## `favicon` (type: `string`):

Favicon URL

## `appleTouchIcon` (type: `string`):

Apple touch icon URL

## `manifestUrl` (type: `string`):

Web app manifest URL

## `feedUrl` (type: `string`):

RSS/Atom feed URL

## `hreflang` (type: `string`):

hreflang languages

## `schemaTypes` (type: `string`):

schema.org types found

## `jsonLdCount` (type: `string`):

Number of JSON-LD blocks

## `structuredData` (type: `string`):

Raw JSON-LD

## `error` (type: `string`):

Error, if any

## `scrapedAt` (type: `string`):

Timestamp

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://www.theverge.com",
        "https://github.com",
        "https://stripe.com",
        "https://www.nytimes.com",
        "https://www.airbnb.com",
        "https://www.shopify.com",
        "https://www.notion.so",
        "https://www.bbc.com"
    ],
    "maxItems": 100,
    "maxConcurrency": 20
};

// Run the Actor and wait for it to finish
const run = await client.actor("haketa/bulk-metadata-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": [
        "https://www.theverge.com",
        "https://github.com",
        "https://stripe.com",
        "https://www.nytimes.com",
        "https://www.airbnb.com",
        "https://www.shopify.com",
        "https://www.notion.so",
        "https://www.bbc.com",
    ],
    "maxItems": 100,
    "maxConcurrency": 20,
}

# Run the Actor and wait for it to finish
run = client.actor("haketa/bulk-metadata-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://www.theverge.com",
    "https://github.com",
    "https://stripe.com",
    "https://www.nytimes.com",
    "https://www.airbnb.com",
    "https://www.shopify.com",
    "https://www.notion.so",
    "https://www.bbc.com"
  ],
  "maxItems": 100,
  "maxConcurrency": 20
}' |
apify call haketa/bulk-metadata-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,haketa/bulk-metadata-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/WcHFl8i9KcvrNVT29/builds/520cS7Z9rad5HHxLp/openapi.json