Google News Scraper — Search & Topic Headlines

📰 Google News Scraper — Search & Topic Headlines

Extract live, real-time news articles, media headlines, publisher attributions, publication timestamps, and source links from Google News at scale. Search for any keyword, company name, brand, executive, product, or regulatory event, or monitor curated topic sections (BUSINESS, TECHNOLOGY, WORLD, HEALTH, SCIENCE) across any international language and country edition.

Whether powering automated PR clipping workflows, competitor media share-of-voice benchmarking, brand sentiment tracking, equity research signals, or Retrieval-Augmented Generation (RAG) pipelines for Large Language Models, this actor delivers clean, normalized news data in JSON, CSV, and Excel without API keys or browser bloat.

---

⚡ At a glance

Feature	Specification
Data Source	Google News Real-Time Feeds (Global Editions)
Search Modes	Keyword Query, Search Operators, Structured Topic Sections
Topic Sections	WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH
Global Locales	Any language code (hl, e.g. en-US, de, fr, es, tr) & country (gl, e.g. US, GB, DE, TR)
Throughput Speed	~400–800 articles in 10–20 seconds via parallelized feeds
Pagination & Scale	Unlocked multi-feed fan-out for 1,000+ articles (set maxItemsPerQuery: 0 for all feed items)
Field Completeness	100% fill rate across core headline, source, date, and link fields
Export Formats	Dataset (JSON, CSV, XML, Excel) & Key-Value Store OUTPUT mirror
Proxy Requirement	Apify Proxy Datacenter (enabled by default, fully supported)
Default Sample Run	Automatic dual-topic query (artificial intelligence, tesla) when run with {}

---

🔎 Why scrape Google News?

Google News aggregates tens of thousands of digital newspapers, wire services (Reuters, AP, Bloomberg), trade publications, regional journals, and broadcast networks into a single index.

However, Google does not offer an open public REST API for developers. Scraping the web UI via headless browsers is sluggish, expensive, and fragile against layout changes. This actor solves this challenge through a high-performance protocol-level HTTP architecture:

Strategic Value & Applications:

• Media Monitoring & Public Relations: Automatically track press mentions of your brand, executive leadership, or portfolio companies across thousands of publications without expensive enterprise media monitoring retainers.
• Competitor Intelligence & Share-of-Voice: Compare volume and sentiment of press mentions between rival firms, identifying major product announcements, executive departures, and legal proceedings.
• Financial & Market Research: Monitor earnings reports, M&A rumors, regulatory scrutiny, and commodity market news in real time to generate quantitative market signals.
• Crisis Communications & Early Warning: Detect breaking controversies, safety recalls, or data breaches the moment news outlets publish.
• LLM Context & RAG Pipelines: Supply up-to-the-minute factual news context to generative AI assistants, market summary bots, and automated newsletter curation systems.
• Cross-Border Media Research: Analyze how international outlets frame global events by pulling regional editions across different languages and geographic editions.

---

🌍 Supported topic sections & locales

Built-in Topic Feeds

You can query standard high-level topic feeds simply by passing the uppercase category names in topics:

• WORLD — Global geopolitical developments, international relations, and diplomacy.
• NATION — Domestic affairs, national politics, and government announcements.
• BUSINESS — Corporate earnings, financial markets, mergers & acquisitions, and macroeconomics.
• TECHNOLOGY — Software, artificial intelligence, hardware, venture capital, and consumer tech.
• ENTERTAINMENT — Film, music, television, gaming, and celebrity media.
• SPORTS — Professional leagues, tournament results, and athletic news.
• SCIENCE — Astronomy, environmental research, physics, and academic discoveries.
• HEALTH — Medical trials, public health policies, wellness, and pharmaceuticals.

Advanced Search Operators

The actor supports Google News search syntax in the queries field:

• Exact Match: "artificial intelligence"
• Boolean OR: Apple OR Microsoft OR Google
• Exclusion: Tesla -cars -automotive
• Site Specific: site:reuters.com "federal reserve"
• In-Title Match: intitle:merger

---

✨ Key features

• ⚡ Ultra-Fast Multi-Feed Concurrency: Queries multiple keyword feeds and topic sections concurrently, harvesting hundreds of articles per second.
• 🎯 Publisher & Source Normalization: Parses the publisher name cleanly (source) and isolates the publisher's root domain (sourceUrl), stripping redundant outlet names from article titles.
• 🕒 Standardized ISO 8601 Timestamps: Normalizes diverse publish date formats into standard UTC ISO timestamps (publishedAt), enabling chronological sorting and time-series bucketing.
• 🔓 Deep Multi-Feed Scaling (1,000+ Articles): Set maxItemsPerQuery: 0 (or leave empty) to capture all available headlines in every feed. Combine multiple keywords to assemble 1,000+ article corpora.
• 🌐 Global Country & Language Coverage: Seamlessly adjust interface language (hl) and edition jurisdiction (gl) for regional localization.
• 🛡️ Zero Browser Overhead: Pure lightweight protocol client architecture ensuring low memory footprint (256–512 MB) and zero headless browser crashes.

---

📥 Input parameters

The actor features streamlined, sensible defaults. All properties can be customized via the web console or JSON API payload.

Parameter	Type	Default	Description
queries	array	["artificial intelligence", "tesla"]	Search terms or search operator queries. Each query is retrieved independently.
topics	array	[]	Optional topic sections (WORLD, NATION, BUSINESS, TECHNOLOGY, etc.).
language	string	"en-US"	Interface/content language code (e.g. en-US, en-GB, de, fr, es, tr).
country	string	"US"	Two-letter country/edition code (e.g. US, GB, DE, TR, FR).
maxItemsPerQuery	integer	0	Maximum articles to retain per query/topic (0 = auto/unlimited; prefilled to 50 for fast validation).
maxConcurrency	integer	5	Maximum number of feeds to fetch concurrently (1–15).
proxyConfiguration	object	{ "useApifyProxy": true }	Proxy configuration. Standard datacenter proxies are enabled by default.

---

📊 Data fields & output structure

Every output record represents a single news article containing headline, publisher, publication date, and direct navigation links.

📰 Article Metadata

Field	Type	Coverage	Description
title	string	🟢 High (100%)	Clean article headline (with publisher suffix stripped)
source	string	🟢 High (100%)	Publisher or media brand name (e.g. Reuters, CNBC)
sourceUrl	string	🟢 High (100%)	Base URL of publisher website
publishedAt	string	🟢 High (100%)	ISO 8601 standardized UTC publication date
pubDate	string	🟢 High (100%)	Original RFC-822 publication date string
link	string	🟢 High (100%)	Google News direct destination link
description	string	🟢 High (>98%)	Article snippet or summary teaser text
type	string	🟢 High (100%)	Query classification (search or topic)
query	string	🟢 High (100%)	Search keyword or topic name that produced the article
language	string	🟢 High (100%)	Target language code (hl)
country	string	🟢 High (100%)	Target country edition code (gl)
scrapedAt	string	🟢 High (100%)	ISO 8601 scrape timestamp

---

💡 Example output record

{
  "title": "Anthropic Unveils Next-Generation Claude 3.5 Sonnet Model",
  "source": "TechCrunch",
  "sourceUrl": "https://techcrunch.com",
  "publishedAt": "2026-11-10T14:20:00.000Z",
  "pubDate": "Tue, 10 Nov 2026 14:20:00 GMT",
  "link": "https://news.google.com/rss/articles/CBMiVWh0dHBzOi8vdGVjaGNydW5jaC5jb20vMjAyNi8xMS8xMC9hbnRocm9waWMtdW52ZWlscy1jbGF1ZGUtMy01LXNvbm5ldC1tb2RlbC11cGRhdGUv0gEA?oc=5",
  "description": "Anthropic announced major upgrades to its Claude 3.5 Sonnet frontier AI model, introducing improved coding capabilities and multimodal processing benchmarks.",
  "type": "search",
  "query": "artificial intelligence",
  "language": "en-US",
  "country": "US",
  "scrapedAt": "2026-11-10T16:50:00.000Z"
}

---

🎯 Practical input recipes

Recipe 1: Tech Sector & Frontier AI Intelligence

Monitor breaking artificial intelligence and semiconductor developments:

{
  "queries": [
    "artificial intelligence",
    "generative AI",
    "OpenAI OR Anthropic",
    "Nvidia GPU chips"
  ],
  "language": "en-US",
  "country": "US",
  "maxItemsPerQuery": 0
}

Recipe 2: Macroeconomic & Global Markets Overview

Extract top business and geopolitical headlines across US and European editions:

{
  "topics": [
    "BUSINESS",
    "WORLD"
  ],
  "language": "en-US",
  "country": "US",
  "maxItemsPerQuery": 100
}

Recipe 3: European Regional News in German

Track electric vehicles and automotive news in Germany:

{
  "queries": [
    "Elektroauto",
    "Volkswagen OR BMW OR Porsche"
  ],
  "language": "de",
  "country": "DE",
  "maxItemsPerQuery": 50
}

Recipe 4: Crisis Management & Brand Protection

Track negative sentiment indicators or litigation involving specific corporate entities:

{
  "queries": [
    "\"Company Name\" lawsuit OR investigation OR fine",
    "\"Company Name\" breach OR outage OR recall"
  ],
  "maxItemsPerQuery": 0
}

Recipe 5: Climate Change & ESG Policy Tracker

Follow multilateral climate agreements, carbon taxation, and green energy developments:

{
  "queries": [
    "\"carbon capture\" OR \"green hydrogen\"",
    "\"climate accord\" OR \"emissions regulation\""
  ],
  "language": "en-US",
  "country": "US",
  "maxItemsPerQuery": 100
}

Recipe 6: Biotechnology & Clinical Trial Announcements

Track FDA drug approvals, oncology breakthroughs, and pharmaceutical patents:

{
  "queries": [
    "FDA approval pharmaceutical",
    "clinical trial phase 3 oncology"
  ],
  "language": "en-US",
  "country": "US",
  "maxItemsPerQuery": 50
}

---

💡 Optimization & best practices

• Query Granularity vs. Breadth: A single Google News feed typically returns up to ~100 of the most relevant recent headlines. To harvest thousands of articles, break your topic down into 10–20 targeted sub-queries rather than one generic keyword.
• Deduplicating Across Related Feeds: When querying overlapping terms (e.g. Tesla and Elon Musk), deduplicate downstream by link or by combining title and source to maintain unique story counts.
• Time-Series Analysis: Because publishedAt is provided in ISO 8601 UTC format, you can easily group articles by hour or day in pandas or SQL to measure sudden spikes in press interest.
• Handling Redirect Links: Google News URLs automatically redirect to the publisher's actual webpage upon navigation. For downstream full-text extraction, you can feed these URLs directly into the Article Content Extractor.
• Scheduled Automated Ingestion: Use Apify's Scheduler to trigger hourly or daily runs. Combined with webhook notifications or Zapier, you can push instant Slack or email alerts when high-impact stories break.

---

🚀 How to use & integrate

1. Configure Queries & Topics: Add your search terms or select topic sections.
2. Select Regional Edition: Set your target language (hl) and country code (gl).
3. Execute Run: Trigger via Apify Console or execute via API:

   curl --request POST \
     --url "https://api.apify.com/v2/acts/haketa~google-news-scraper/runs?token=YOUR_API_TOKEN" \
     --header 'Content-Type: application/json' \
     --data '{"queries": ["tesla earnings"], "language": "en-US", "country": "US"}'
   

4. Export & Automate:
   • Python: Ingest into Pandas for NLP sentiment classification.
   • AI Pipelines: Stream articles into LangChain or LlamaIndex vector databases for RAG question answering.
   • Spreadsheets: Automatically sync new news items into Google Sheets.

---

❓ Frequently asked questions

Is scraping Google News legal?

Yes. Google News syndicates public news headlines via open protocol feeds. This scraper extracts publicly available metadata and links intended for syndication.

Do I need a Google News API key?

No. There is no API key, login, or Google Cloud registration required. The actor works immediately out of the box.

How many articles can I collect in a single run?

Each search query or topic feed provides up to ~100 articles. By passing multiple queries, you can extract 1,000+ articles in a single run within 20–30 seconds.

How fresh is the news coverage?

Feeds are updated continuously in real time as media outlets publish stories. Articles published minutes ago are indexed and available.

Can I extract news in different languages?

Yes. Set language to the ISO language code (e.g. fr for French, ja for Japanese, es for Spanish) and country to the two-letter ISO country code.

Can I extract the full text of each article?

Google News indexes headlines and short summaries. To extract the complete body text and images of individual articles, route the output links to our Article Content Extractor.

Why are article destination URLs encrypted?

Google News formats article outbound links through its redirection proxy (news.google.com/rss/articles/...) to track referral metrics. When opened in any standard browser or HTTP client, these links immediately redirect with an HTTP 302 code to the canonical publisher article URL.

What happens if I query a non-existent topic?

If an invalid topic string is supplied, the endpoint returns an empty set without error. Stick to standard valid topics: WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH.

How far back does Google News coverage go?

Google News focuses primarily on recent and breaking developments (typically the past 7–30 days). For multi-year historical archives, schedule periodic extraction to build an ongoing historical news repository.

Can I connect the output to an automated Slack alert?

Yes. Using Apify Webhooks or the Apify integration on Zapier or Make, you can trigger a webhook whenever a run finishes that posts matching high-impact headlines directly into your company Slack or Microsoft Teams channel.

---

🛠️ Step-by-step code integration examples

Python Integration with Pandas

from apify_client import ApifyClient
import pandas as pd

client = ApifyClient("YOUR_APIFY_TOKEN")

# Trigger Google News Scraper run
run = client.actor("haketa/google-news-scraper").call(
    run_input={
        "queries": ["artificial intelligence", "machine learning"],
        "language": "en-US",
        "country": "US",
        "maxItemsPerQuery": 50,
    }
)

# Fetch dataset items and load into DataFrame
items = client.dataset(run["defaultDatasetId"]).list_items().items
df = pd.DataFrame(items)

# Group by media source
source_counts = df["source"].value_counts()
print("Top Media Outlets:")
print(source_counts.head(10))

Node.js / JavaScript Integration

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('haketa/google-news-scraper').call({
    queries: ['artificial intelligence'],
    language: 'en-US',
    country: 'US',
    maxItemsPerQuery: 50,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Extracted ${items.length} news headlines.`);

---

🔗 Related intelligence scrapers

• Article Content Extractor — Extract full article text, author, and media from any URL.
• RSS Feed Reader — Parse any custom RSS or Atom syndication feed.
• Google Maps Business Leads Scraper — Extract business listings with enriched email and social profiles.
• Substack Scraper — Scrape newsletters, thought leadership, and independent journalism.

---

📋 Changelog

v1.2 (2026-11)

• Deep Pagination & Scale Unlocked: Removed artificial 100-item caps; setting maxItemsPerQuery: 0 now preserves all available headlines across multi-query batches for 1,000+ article corpora.
• BuildTag Standardized: Added explicit "buildTag": "latest" configuration in actor manifest.
• Fast Prefill Optimization: Adjusted prefill limits to 50 records for rapid sub-15s sample runs.
• Default Fallback Robustness: Added safe fallback defaults (AI, Tesla) ensuring runs with empty input payloads execute reliably with ~100 sample headlines.
• Publisher Parsing Enhancements: Improved entity decoding and clean title extraction, stripping trailing publisher strings across international editions.

v1.1 (2026-10)

• Parallel Feed Ingestion: Added configurable concurrency controls to process up to 15 feeds simultaneously.
• ISO Date Formatting: Standardized timestamp conversion into ISO 8601 format.

v1.0 (2026-09)

• Initial release featuring keyword search, topic sections, and international edition support.