# Lovable Sites Scraper - Find & Enrich lovable.app Apps (`makework36/lovable-sites-scraper`) Actor

Discover sites built with Lovable.dev. Enumerates \*.lovable.app subdomains from public sources (CT logs, RapidDNS, hackertarget) and enriches each with title, description, Open Graph tags and custom domain detection. Perfect for lead-gen, competitive intel and market research on AI-built apps.

- **URL**: https://apify.com/makework36/lovable-sites-scraper.md
- **Developed by:** [deusex machine](https://apify.com/makework36) (community)
- **Categories:** Developer tools, Lead generation, AI
- **Features:** Limited permissions
- **Stats:** 49 total users, 10 monthly users, 100.0% runs succeeded, 2 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Lovable Sites Scraper — Find & Enrich `.lovable.app` Apps

Discover every public site built with **Lovable.dev** (the AI app builder by GPT Engineer). This actor enumerates live `*.lovable.app` subdomains from multiple public sources, then enriches each URL with HTTP metadata — title, description, Open Graph tags, favicon, canonical URL and **custom-domain detection** — so you can turn the raw list into a searchable, filterable dataset of AI-built apps.

Perfect for **lead generation**, **competitive intelligence**, **market research on AI-built products**, **design inspiration** and **agency prospecting**. If you're selling to founders who ship with AI, this is your radar.

***

### Why this actor exists

Lovable.dev is one of the hottest AI app builders on the market — thousands of founders, indie hackers and agencies use it daily to ship full-stack apps in minutes. Every published Lovable project gets a forced subdomain on `*.lovable.app`, and optionally a custom domain on top. That forced subdomain is the reason we can enumerate the entire public surface of Lovable: if someone shipped it and hit Publish, it's discoverable.

But there's no official directory. No search engine. No public API. If you want to know:

- Which agencies are shipping client work on Lovable?
- What SaaS niches are being built with AI right now?
- Who just bought a custom domain (signal: they're serious, have budget)?
- What landing pages are converting in your market?
- Which Lovable sites are dead vs. live vs. placeholder?

…you had to scrape it yourself. Until now.

This actor does the heavy lifting: multi-source subdomain enumeration, concurrent HTTP enrichment, dead-site filtering, custom-domain detection, and keyword search across the whole result set. You get a clean, structured dataset ready to import into your CRM, your BI tool, or your cold-outreach workflow.

***

### What you get — output fields

Each row in the output dataset contains:

| Field | Type | Description |
|---|---|---|
| `subdomain` | string | The full `xxx.lovable.app` hostname |
| `url` | string | Canonical `https://` URL |
| `status` | integer | HTTP status code returned (200, 404, 500…) |
| `isLive` | boolean | `true` if the site responds 200 and is NOT the Lovable placeholder page |
| `isDefault` | boolean | `true` if the response is Lovable's "project not found" / "not deployed yet" page |
| `title` | string | `<title>` tag, unescaped and trimmed to 300 chars |
| `description` | string | `<meta name="description">`, 600 chars |
| `ogTitle` | string | `<meta property="og:title">` |
| `ogDescription` | string | `<meta property="og:description">` |
| `ogImage` | string | `<meta property="og:image">` URL — great for thumbnails |
| `ogUrl` | string | `<meta property="og:url">` |
| `canonical` | string | `<link rel="canonical">` href |
| `favicon` | string | `<link rel="icon">` URL (absolute) |
| `customDomain` | string | Detected custom domain (from canonical / og:url hostname) — empty if none |
| `hasCustomDomain` | boolean | `true` if `customDomain` is set — **key lead-gen filter** |
| `contentLength` | integer | Response body size in bytes |
| `emails` | string\[] | Extracted email addresses (only when `extractContacts: true`) |
| `phones` | string\[] | Extracted phone numbers in E.164 / US format (only when `extractContacts: true`) |
| `whatsapp` | string | WhatsApp link (`wa.me/...` or `api.whatsapp.com/send`) (only when `extractContacts: true`) |
| `social` | object | Social media URLs by platform: `linkedin`, `twitter`, `instagram`, `facebook`, `youtube`, `tiktok`, `github` (only when `extractContacts: true`) |
| `scrapedAt` | string | ISO-8601 UTC timestamp when the row was enriched |

The `hasCustomDomain` field is the money field for lead-gen: a Lovable user who went through the effort of wiring up DNS is several orders of magnitude more likely to be a paying, serious customer than someone with a placeholder.

#### 🎯 Pro tip — B2B lead generation combo

Combine `mustHaveCustomDomain: true` + `extractContacts: true` for the highest-quality lead set:

- **~56% of qualified sites return at least one contact channel** (email / phone / WhatsApp / social).
- Real samples extracted in production: emails like `info@avoskills.legal`, phones like `+33 6 24750760`, WhatsApp links, plus LinkedIn / Instagram / Facebook profile URLs.
- Diverse geographies — US, Brazil, France, UK, Mexico — depending on the discovery batch.

Combined with the custom-domain qualifier, you typically get **10–30 contact-rich leads per 1,000 candidate subdomains**, ready for cold outreach.

> **Honest limitation**: Lovable.dev sites are React SPAs by default. The remaining ~44% are pure SPAs whose contact info only renders client-side via JavaScript — HTTP-based scrapers (this actor included) cannot see it without a full browser. The qualifier filter already biases the dataset toward sites that have moved off the default Lovable shell, which is why the 56% rate is achievable here.

***

### How it works

#### 1) Discovery — multi-source subdomain enumeration

The actor queries up to three public sources in parallel:

- **crt.sh** — Certificate Transparency logs. Every SSL certificate issued for `*.lovable.app` is logged here. The Lovable platform mostly uses a wildcard cert, so CT coverage is partial, but many projects get their own cert issued.
- **hackertarget.com** — Free passive DNS / host search. Covers a large chunk of the surface with recent data.
- **rapiddns.io** — Aggregated subdomain database. Pulls from passive DNS, CT and DNS brute-force.

By default all three are enabled (`sources: ["crtsh", "hackertarget", "rapiddns"]`). You can disable any of them via the input. Combining sources gives the broadest coverage — each individual source has blind spots.

> **How much coverage to expect, honestly.** hackertarget and rapiddns each cap their free tier at **50 results per query**, so crt.sh is what actually determines the size of a run: with it, the pool is typically several hundred candidates; without it, it collapses to about 100. crt.sh returns `502` under load fairly often, so the actor now **retries it up to 3 times with backoff** and logs a clear warning if it still can't be reached — if a run comes back thin, that warning in the log is why, and re-running later usually fixes it.

#### 2) Cleaning

Raw results are deduplicated, wildcard entries (`*.lovable.app`) are dropped, and `www.<project>.lovable.app` duplicates of the bare form are collapsed by default (toggle with `includeWww: true`).

#### 3) Sampling — random by default

The discovery sources return more subdomains than a typical run's `maxSites` cap, so the actor has to choose which ones to keep — and that choice matters more than it looks.

By default (`resultOrder: "random"`) the candidate pool is **shuffled before the cap is applied**, so every run returns a genuinely different sample of the Lovable surface. Run it three times with `maxSites: 100` and you'll have explored 300 different sites, not the same 100 three times over.

Set `resultOrder: "alphabetical"` when you need a **reproducible** list — the same input always yields the same sites, which is what you want for regression testing or for a stable, citable dataset. Note that with alphabetical order a small `maxSites` will always hand you the same slice near the start of the alphabet.

Need a random sample you can reproduce later? Set `randomSeed` to any number. Same seed + same sources = same sample.

#### 3b) Advanced search — pick exactly which sites you want

Everything in this group runs **before** any HTTP request, so narrowing down costs you nothing and speeds the run up. All of them match the **slug** — the part before `.lovable.app` — so `nova.lovable.app` is matched as `nova`.

| Want to… | Use |
|---|---|
| Browse alphabetically — every site starting with `n` | `subdomainStartsWith: "n"` |
| Find a naming convention — everything ending in `-app` | `subdomainEndsWith: "-app"` |
| Find a word anywhere in the name | `subdomainContains: "shop"` |
| Look up **one specific site** you already know | `exactSubdomains: ["my-shop"]` |
| Anything more complex | `subdomainRegex: "^(shop\|store)"` |
| Cap how many you get back | `maxSites: 50` |

Filters combine (they AND together), and `maxSites` always applies last — so `subdomainStartsWith: "n"` + `maxSites: 50` gives you 50 sites starting with `n`, randomly sampled from all the ones that matched.

**`exactSubdomains` behaves differently on purpose**: when you already know the site names, discovery is pointless, so the actor skips the sources entirely and goes straight to fetching them. It's much faster, and it's the only way to reach a site so new that no public source has indexed it yet. The other search filters are ignored in that mode — you already said exactly what you want.

If a filter is too tight and nothing matches, the run logs a clear warning instead of finishing silently with an empty dataset. An invalid `subdomainRegex` fails the run immediately with the reason.

#### 4) Enrichment (optional, on by default)

For each surviving candidate, the actor fires a concurrent HTTPS GET with:

- 10-second timeout
- Redirect follow (so we catch sites that `301` to a custom domain)
- Desktop-class User-Agent
- Configurable concurrency (`concurrency: 15` by default, up to 50)

The HTML is parsed with lightweight regex (no Playwright / no heavy browser) — fast and cheap. We extract title, meta tags, OG tags, canonical, favicon. We also detect the Lovable placeholder / "not deployed" page and set `isLive=false` for those so your dataset isn't polluted with dead projects.

#### 5) Filtering

Two filters run at the end:

- **`onlyLive: true`** (default) — drop dead / placeholder sites
- **`searchQuery`** — case-insensitive match across subdomain + title + description + OG tags + customDomain. Great for niching down: `"crypto"`, `"saas"`, `"real estate"`, `"restaurant"`, `"ai"`.

#### 6) Charging

The actor bills per **result pushed** ($0.005 per site), not per candidate discovered. If `onlyLive=true` and 80% of candidates are dead, you only pay for the 20% that landed in your dataset.

***

### Input parameters

| Field | Type | Default | Description |
|---|---|---|---|
| `maxSites` | integer | `100` | Maximum sites to return. Range: 1–5000 |
| `resultOrder` | string | `random` | `random` = different sample of sites every run (recommended). `alphabetical` = deterministic, always the same first N sites |
| `randomSeed` | integer | — | Optional. With `resultOrder: random`, fixes the sample so you can reproduce an exact run later |
| `enrichHtml` | boolean | `true` | Fetch each site and extract metadata. Turn off for pure subdomain enumeration (cheapest) |
| `onlyLive` | boolean | `true` | Skip 4xx/5xx/timeouts/placeholder pages |
| `includeWww` | boolean | `false` | Keep `www.*` variants even when the bare form is also found |
| `searchQuery` | string | `""` | Keyword filter (case-insensitive) across subdomain + title + description + OG + customDomain |
| `mustHaveCustomDomain` | boolean | `false` | Drop sites without a custom domain. Pre-filters for serious users / paying customers |
| `subdomainContains` | string | `""` | Pre-enrichment filter on subdomain string. Saves cost. E.g. `"shop"`, `"ai"`, `"crypto"` |
| `subdomainStartsWith` | string | `""` | Only sites whose name starts with this. A single letter browses alphabetically: `"n"` → nova, nutrition, nextstep… |
| `subdomainEndsWith` | string | `""` | Only sites whose name ends with this. E.g. `"-app"`, `"-ai"`, `"-store"` |
| `exactSubdomains` | array | `[]` | Look up these exact sites and skip discovery entirely. Accepts `"my-shop"`, `"my-shop.lovable.app"` or the full URL |
| `subdomainRegex` | string | `""` | Regex against the slug, case-insensitive. E.g. `"^n"`, `"^(shop\|store)"`, `"\\d{2,}$"` |
| `titleContains` | string | `""` | Filter sites whose `<title>` contains this substring (case-insensitive) |
| `excludeKeywords` | array | `[]` | Blacklist — drop sites whose subdomain/title/description matches any of these |
| `extractContacts` | boolean | `false` | Extract emails, phone numbers, WhatsApp and social media URLs from each site. Adds ~1–2s per site. Best paired with `mustHaveCustomDomain: true` for ~56% extraction rate |
| `concurrency` | integer | `15` | Parallel HTTP requests during enrichment (1–50) |
| `sources` | array | all 3 | Any combination of `crtsh`, `hackertarget`, `rapiddns` |

#### Sample inputs

**🎯 B2B lead-gen — qualified contacts (recommended):**

```json
{
  "maxSites": 1000,
  "mustHaveCustomDomain": true,
  "extractContacts": true,
  "onlyLive": true,
  "excludeKeywords": ["test", "demo", "staging"]
}
```

Returns sites with custom domains AND extracted email / phone / WhatsApp / social profiles in one shot. ~56% of qualified results return at least one contact channel — ready for cold outreach.

**Lead-gen: find Lovable sites with custom domains (filter client-side):**

```json
{
  "maxSites": 2000,
  "enrichHtml": true,
  "onlyLive": true,
  "concurrency": 20
}
```

Run → filter dataset where `hasCustomDomain = true` → that's your qualified lead list.

**Niche research: find crypto apps built on Lovable:**

```json
{
  "maxSites": 500,
  "searchQuery": "crypto",
  "enrichHtml": true,
  "onlyLive": true
}
```

**Pure enumeration (fastest / cheapest):**

```json
{
  "maxSites": 5000,
  "enrichHtml": false
}
```

No HTTP enrichment — you get a clean list of subdomains in seconds. Good for feeding into your own downstream pipeline.

**Alphabetical browsing — 50 sites starting with "n":**

```json
{
  "maxSites": 50,
  "subdomainStartsWith": "n",
  "onlyLive": true
}
```

**Check specific sites you already know (skips discovery — fastest):**

```json
{
  "exactSubdomains": [
    "my-shop",
    "acme-landing.lovable.app",
    "https://client-portal.lovable.app/"
  ],
  "extractContacts": true
}
```

Any of the three formats works. Ideal for monitoring your own projects or a competitor's — and it finds sites too new to appear in the public sources.

**Reproducible sample (same sites every run):**

```json
{
  "maxSites": 300,
  "resultOrder": "random",
  "randomSeed": 42,
  "onlyLive": true
}
```

Random coverage across the whole Lovable surface, but pinned — re-run it next month with the same seed and you're comparing like with like. Drop `randomSeed` (or set `resultOrder: "alphabetical"` for a strict A→Z list) depending on which kind of repeatability you need.

**Competitive intel on a specific vertical:**

```json
{
  "maxSites": 1000,
  "searchQuery": "saas",
  "sources": ["crtsh", "hackertarget", "rapiddns"],
  "onlyLive": true,
  "concurrency": 25
}
```

***

### Use cases

#### 1. Lead generation for AI dev agencies

You build custom apps for clients. Your ICP is founders who've already validated with a no-code AI tool but hit the ceiling. Scrape Lovable → filter `hasCustomDomain=true` → enrich with the custom-domain owner (Clearbit / Apollo) → send a personalized cold email: *"Saw you shipped `acme.com` on Lovable — once you need real auth + Stripe + multi-tenant, here's what we do."*

Conversion rates on this kind of cold outreach are typically 3–8× higher than generic lists because you're qualifying on **intent + budget + tech stack** all at once.

#### 2. Competitive intelligence for SaaS founders

You're building a SaaS. You want to know what's being shipped in your space right now — not six months ago when Crunchbase got around to indexing it. Filter `searchQuery` by your vertical keyword, inspect titles + descriptions, collect patterns. Which painpoints are recurring? Who's pricing what? Which are gaining traction (check the custom domain → check DNS age → check backlinks)?

#### 3. Market research on AI-built products

Investors, analysts, journalists: Lovable is one of the primary funnels where AI-generated software becomes production software. Enumerating this surface gives you a weekly pulse on what's being built with AI, what verticals are hot, what's dying in the graveyard of placeholder pages.

#### 4. Design and UX inspiration

Need examples of how AI builders design their landing pages? Filter by vertical, pull the `ogImage` field, build a mood board of 500 AI-generated homepages in an afternoon.

#### 5. Agency prospecting for Lovable itself

If you work at Lovable or a compatible tool (Vercel, Supabase, Clerk, Stripe), this is your list of current users. Segment by custom-domain vs. placeholder, prioritize the serious ones, reach out with case studies and integration guides.

#### 6. Historical tracking / weekly deltas

Schedule the actor to run weekly. Diff the datasets. What got published this week? What died? What graduated from placeholder to custom domain? That's a trend chart nobody else has.

***

### Code examples

#### Python — Apify client

```python
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run = client.actor("makework36/lovable-sites-scraper").call(run_input={
    "maxSites": 500,
    "enrichHtml": True,
    "onlyLive": True,
    "searchQuery": "saas",
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["subdomain"], "→", item["title"], "| custom:", item.get("customDomain") or "-")
```

#### Node.js

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });

const run = await client.actor('makework36/lovable-sites-scraper').call({
    maxSites: 1000,
    enrichHtml: true,
    onlyLive: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();

const withCustomDomain = items.filter(i => i.hasCustomDomain);
console.log(`${withCustomDomain.length} qualified leads out of ${items.length} live sites`);
```

#### cURL (sync run-and-wait)

```bash
curl -X POST "https://api.apify.com/v2/acts/makework36~lovable-sites-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"maxSites": 200, "searchQuery": "crypto", "enrichHtml": true}'
```

#### PHP

```php
$url = "https://api.apify.com/v2/acts/makework36~lovable-sites-scraper/run-sync-get-dataset-items?token=$token";
$payload = json_encode(["maxSites" => 500, "onlyLive" => true]);
$ch = curl_init($url);
curl_setopt_array($ch, [
    CURLOPT_POST => true,
    CURLOPT_POSTFIELDS => $payload,
    CURLOPT_HTTPHEADER => ["Content-Type: application/json"],
    CURLOPT_RETURNTRANSFER => true,
]);
$data = json_decode(curl_exec($ch), true);
```

#### Zapier / Make / n8n

Apify has native connectors for all three. Drop this actor in as a scheduled trigger (weekly/daily), pipe the dataset to Google Sheets, Airtable, Notion, HubSpot or your warehouse of choice. Typical setup: weekly run → filter `hasCustomDomain=true` → push new rows to your CRM as leads → enrich with Clearbit → trigger outreach sequence.

***

### Pricing

**$0.005 per site returned**, plus $0.00005 per actor start.

Billing model: **pay-per-result**. You only pay for rows that end up in your dataset — not for candidates that were dead, placeholder, or filtered out by your search settings. This is why the advanced search filters run *before* enrichment: narrowing your search makes a run cheaper, never pricier.

Indicative totals:

| Run size | Cost |
|---|---|
| 100 sites | $0.50 |
| 500 sites | $2.50 |
| 1,000 sites | $5.00 |
| 5,000 sites | $25.00 |

The actor has no monthly subscription, no minimum spend, and no proxy/compute overhead charges — the $0.005/site is fully-loaded.

Compare to building this yourself:

- Hiring a scraping dev: 2–5 days at $500/day = **$1,000–$2,500**
- Scraping stack (proxy + compute + maintenance): **$50–$200/month** ongoing
- Keeping it alive as sources change: ongoing engineering time

A one-off $10 run replaces weeks of work.

***

### FAQ

**How fresh is the data?**
Every run hits the sources live — no stale cache. crt.sh updates within minutes of a new cert. hackertarget and rapiddns refresh passive DNS daily-to-weekly. If a site was published this morning and has its own cert, it'll show up.

**Can this find sites that haven't been published / are private?**
No. By design, this actor only sees what's publicly reachable. If a Lovable project was never shared publicly, it's invisible to all three sources.

**What about sites with only custom domains and no `lovable.app` hostname?**
Lovable currently forces a `*.lovable.app` subdomain on every published project — the custom domain is added on top, not instead. So every live Lovable site has both. We discover via the forced `*.lovable.app` and surface the custom domain in the `customDomain` field.

**Why are some results missing a title / description?**
Some Lovable projects are single-page apps rendered client-side with minimal SSR. The HTML we fetch is the shell; the actual content loads from JS. In those cases we extract what's in the shell and leave the rest empty. If you need rendered content, pipe the URLs into a Playwright-based enrichment actor as a second pass.

**I keep getting the same sites on every run — how do I get new ones?**
Make sure `resultOrder` is `random` (the default since v1.1). The sources return thousands of subdomains and `maxSites` only keeps a slice of them — with `alphabetical` ordering that slice is always the same one. With random ordering each run samples a different part of the surface. To sweep the whole thing systematically instead of sampling, raise `maxSites` (up to 5000) rather than re-running.

**Will random ordering give me duplicates across runs?**
Some overlap is expected — it's sampling without memory, not a queue. If you're building a cumulative database, dedupe on the `subdomain` field as you import; it's the stable unique key. For scheduled weekly runs, dedupe-on-import also gives you your "what's new this week" delta for free.

**How do I filter for very recent sites?**
crt.sh returns certificate issuance dates — a future version of this actor may surface that. For now, run the actor weekly and diff against your previous dataset to find what's new.

**Can I get phone / email of site owners?**
Not directly — this actor surfaces what's public on the site. For contact enrichment, combine the output with Clearbit, Apollo, Hunter.io or a WHOIS lookup on the `customDomain` field.

**Does this work for other AI app builders (Bolt, v0, Replit)?**
This one is Lovable-specific. If you need `bolt.new`, `v0.app` or Replit public surface scraping, check our other actors or request one.

**What's the difference between `isLive` and `status = 200`?**
`status = 200` just means the server responded. `isLive = true` also means the response wasn't Lovable's default "project not found / not deployed yet" page. Many dead Lovable URLs return 200 + placeholder, which is worse than a 404 because you'd waste outreach on them. `isLive` cleans that up.

**Why is `enrichHtml=true` so much slower?**
Because it's actually fetching each URL. 1000 sites at concurrency=15 takes roughly 2–4 minutes. Turn it off if you only need the subdomain list.

**Can I re-run this on just a single subdomain to re-check it?**
Yes — since v1.2, put it in `exactSubdomains` (e.g. `["my-shop"]`). Discovery is skipped and the site is fetched and enriched directly, so it's fast and cheap. Handy for monitoring your own or a client's project on a schedule.

**Can I get only sites starting with a specific letter?**
Yes — `subdomainStartsWith: "n"`. Combine it with `maxSites` to cap the volume. For finer control there's `subdomainEndsWith`, `subdomainContains` and `subdomainRegex`. All of them filter before any HTTP request, so they make the run cheaper and faster rather than more expensive.

**Do the search filters match the full hostname or just the project name?**
The project name (slug) — `nova.lovable.app` is matched as `nova`. That's why `subdomainStartsWith: "n"` behaves the way you'd expect and isn't thrown off by a `www.` prefix.

***

### Troubleshooting

**Run finishes with very few results.**
Check: (1) `maxSites` isn't too low, (2) at least two sources are enabled, (3) `onlyLive=true` might be aggressive if Lovable had an outage — try `onlyLive=false` to see raw reachability, (4) if `searchQuery` is set, loosen it.

**HTTP 429 / rate-limit warnings.**
Drop `concurrency` from 15 → 5 or 8. Sources occasionally tighten their free tiers.

**crt.sh returns errors.**
crt.sh is flaky under load (502 Bad Gateway, timeouts) — measured failing on the majority of attempts during some windows. Since v1.2 the actor retries it up to 3 times with backoff before giving up, then continues with the other sources and logs `crt.sh unavailable after retries`. Since the other two sources are capped at 50 results each on their free tiers, a run without crt.sh is much narrower — if you see that warning and the result count looks thin, re-run later rather than assuming the Lovable surface shrank.

**`customDomain` empty on a site I know has one.**
Custom-domain detection relies on `<link rel="canonical">` or `<meta property="og:url">`. If the Lovable project doesn't set either to the custom domain, we can't detect it from the HTML alone. A v2 of this actor may add DNS / HTTP redirect-chain detection.

**Some subdomains return `contentLength: 0`.**
Means the server returned an empty body (rare — usually a 3xx → 2xx redirect chain ending at an empty page, or HEAD-like response). Treat those as dead.

**I want `og:image` URLs absolutized.**
They're returned as-is from the HTML. If a Lovable site uses a relative `og:image`, you'll get the relative path. Prepend the `url` field to resolve it.

***

### Related actors from the same author

- **Airbnb Scraper — Listings, Prices, Photos & Hosts API** — full Airbnb enumeration for travel / lead-gen
- **Airbnb Market Analytics — ADR, RevPAR & Occupancy** — short-term rental market metrics
- **Airbnb MCP Server — Claude, Cursor & AI Agents** — conversational Airbnb search for LLM agents
- **VRBO Scraper** — vacation-rental competitive data
- **Skyscanner scrapers** — flight + hotel discovery

Check my Apify profile (`makework36`) for the full catalog of 70+ production scrapers.

***

### Changelog

**1.4 — 2026-08-22** — **Billing cleanup.** Removed an internal `charge()` call for a custom event that was not part of the actor's live pricing: it billed nothing, but emitted one warning per row in the run log. Billing is unchanged and handled by the platform per dataset item. Also removed a stale local pricing file that no longer matched the published price.

**1.3 — 2026-08-22** — **crt.sh retries.** crt.sh is the source that determines how many candidates a run finds (the other two are capped at 50 each on their free tiers), and it returns 502 under load often enough to noticeably shrink runs. It's now retried up to 3 times with backoff, with an explicit warning in the log when coverage was reduced. Pricing and coverage figures in this README were re-measured and corrected.

**1.2 — 2026-08-22** — **Advanced search.** New optional filters to choose exactly which sites come back, all applied before any HTTP request (so they make runs cheaper, not pricier): `subdomainStartsWith` (browse alphabetically — every site starting with `n`), `subdomainEndsWith`, `subdomainRegex` for power users, and `exactSubdomains` to look up sites you already know by name — which skips discovery entirely and reaches sites too new to be indexed by the public sources. Filters match the project slug, combine with `maxSites`, warn clearly when nothing matches, and reject an invalid regex up front.

**1.1 — 2026-08-22** — **Random sampling.** Previously the candidate pool was sorted alphabetically before the `maxSites` cap was applied, so every run returned the same slice of sites near the start of the alphabet — repeat runs explored no new ground. Candidates are now shuffled before the cap (`resultOrder: "random"`, the new default), so each run samples a different part of the Lovable surface. Added `randomSeed` to reproduce a specific sample, and `resultOrder: "alphabetical"` to restore the old deterministic behaviour.

**1.0 — 2026-04-21** — Initial release. Multi-source discovery (crt.sh + hackertarget + rapiddns), HTTP metadata enrichment, custom-domain detection, `onlyLive` + `searchQuery` filters, pay-per-result billing.

***

### Legal / ethics note

This actor only reads **publicly available** information: Certificate Transparency logs (a legal requirement for every issued SSL cert, mandated by browsers), passive DNS databases (publicly queried), and live HTTP GETs to publicly-published URLs. No authentication bypass, no private data, no rate-limit evasion on Lovable's own infrastructure. If you use the output for cold outreach, follow CAN-SPAM / GDPR / whatever jurisdiction applies to your recipient list.

***

**Built by makework36.** Questions, feature requests, or bug reports → open an issue on the actor page or DM on Apify.

# Actor input Schema

## `maxSites` (type: `integer`):

Maximum number of Lovable sites to extract and enrich. Each site counts as one billable result.

## `enrichHtml` (type: `boolean`):

If true, fetch each site and extract title, description, Open Graph tags and detect custom domain. Slower but much more useful. Turn off for pure subdomain enumeration.

## `onlyLive` (type: `boolean`):

If true, skip sites that return 4xx/5xx or timeout. Requires enrichHtml.

## `includeWww` (type: `boolean`):

If false, drop duplicated www.<project>.lovable.app when the bare form is already captured.

## `searchQuery` (type: `string`):

Optional — only keep sites whose subdomain, title or description contains this keyword (case-insensitive). E.g. 'crypto', 'airbnb', 'saas'.

## `mustHaveCustomDomain` (type: `boolean`):

If true, drop sites that still run on a bare \*.lovable.app URL. Great for qualified lead-gen (custom domain = intent + budget).

## `subdomainContains` (type: `string`):

Optional — only keep sites whose subdomain (slug) contains this substring. Applied BEFORE enrichment (saves cost). E.g. 'shop', 'ai', 'app', 'crypto'.

## `subdomainStartsWith` (type: `string`):

Optional — only keep sites whose name starts with this. Use a single letter to browse alphabetically (e.g. 'n' → nova, nutrition, nextstep…) or a full prefix (e.g. 'shop-').

## `subdomainEndsWith` (type: `string`):

Optional — only keep sites whose name ends with this. Useful for naming conventions, e.g. '-app', '-ai', '-store'.

## `exactSubdomains` (type: `array`):

Optional — look up these exact sites instead of discovering. Accepts 'my-shop', 'my-shop.lovable.app' or the full URL. When set, discovery sources are SKIPPED: it's much faster and it also reaches brand-new sites the public sources haven't indexed yet. All other search filters are ignored.

## `subdomainRegex` (type: `string`):

Optional — regular expression matched against the slug, case-insensitive. E.g. '^n' (starts with n), '^(shop|store)' (starts with either), '\d{2,}$' (ends in 2+ digits). An invalid pattern fails the run with a clear message instead of silently returning nothing.

## `titleContains` (type: `string`):

Optional — only keep sites whose <title> tag contains this substring (case-insensitive). Requires enrichHtml. E.g. 'SaaS', 'Restaurant'.

## `excludeKeywords` (type: `array`):

Optional — blacklist. Drop sites containing any of these keywords in subdomain/title/description (case-insensitive). E.g. \['test', 'demo', 'staging', 'example'].

## `extractContacts` (type: `boolean`):

If true, also extract emails, phone numbers, WhatsApp links and social media (LinkedIn, Twitter, Instagram, Facebook, YouTube, TikTok, GitHub) from each site. Adds ~1s per site.

## `resultOrder` (type: `string`):

How candidates are picked before the Max sites cap. 'Random' returns a different sample of sites on every run (recommended for lead-gen and for discovering new sites). 'Alphabetical' always returns the same first N sites — use it only when you need a reproducible list.

## `randomSeed` (type: `integer`):

Optional — only used when Result order is 'random'. Set a fixed number to reproduce the exact same random sample later. Leave empty for a truly different sample each run.

## `concurrency` (type: `integer`):

Parallel HTTP requests when enriching. Higher = faster but more IP load.

## `sources` (type: `array`):

Public subdomain discovery sources to query. Combine for better coverage.

## Actor input object example

```json
{
  "maxSites": 100,
  "enrichHtml": true,
  "onlyLive": true,
  "includeWww": false,
  "searchQuery": "",
  "mustHaveCustomDomain": false,
  "subdomainContains": "",
  "subdomainStartsWith": "",
  "subdomainEndsWith": "",
  "exactSubdomains": [],
  "subdomainRegex": "",
  "titleContains": "",
  "excludeKeywords": [],
  "extractContacts": false,
  "resultOrder": "random",
  "concurrency": 15,
  "sources": [
    "crtsh",
    "hackertarget",
    "rapiddns"
  ]
}
```

# Actor output Schema

## `dataset` (type: `string`):

All scraped items in the default dataset (JSON).

## `csv` (type: `string`):

Default dataset formatted as CSV for spreadsheets (Google Sheets, Excel).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("makework36/lovable-sites-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("makework36/lovable-sites-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call makework36/lovable-sites-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,makework36/lovable-sites-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/j2f5kiKVMr7GtWOEc/builds/pJVd728hHNzyk8nok/openapi.json