Website Classifier & Text Classifier - Industry & Sentiment

Website Classifier & Text Classifier: industry, page type, IAB, sentiment

What it does

Give it URLs, bare domains or texts. It reads each one and labels it with AI. You can classify by industry and business model (B2B/B2C), page type, topic, IAB category, sentiment, content moderation or your own custom labels. Every row has a label, a confidence level, a one-sentence reason and the language.

Example. Input: https://stripe.com/pricing, https://docs.python.org/3/tutorial/index.html, https://www.bbc.com/news, https://www.allbirds.com/products/mens-tree-runners with Classify by = Page type. Output from a real run: pricing, documentation, news and product, all with high confidence.

Price: $4.00 per 1,000 items at Standard depth. Quick is $3.00 and Deep is $8.00 per 1,000. There is no start fee. Pages that cannot be read and bad input lines are free.

Input: URLs, bare domains, your own texts or another Actor's dataset. Output: one row per item with the label, confidence, reason and language (industry also gives business model and a company summary). Price: $4.00 per 1,000 items at the standard depth ($3.00 Quick, $8.00 Deep), no start fee; unreadable pages and bad input lines are free.

• 🏢 Industry + business model: 30 industries (SaaS, e-commerce, financial services, healthcare, marketing agencies, local services…), plus businessModel (B2B, B2C, B2B2C, nonprofit, government) and a one-sentence companySummary. Built for lead lists. Martech and adtech platforms count as marketing, hosting and domain companies as IT services, and dictionaries and other reference sites as media, even when they sell through an app.
• 🏷️ Page type: homepage, product, category, article, news, documentation, pricing, about, contact, careers, login, legal, forum, video, event, recipe, search, landing, profile, other
• 🗂️ Topic (23 topics) or IAB content categories (30 tier-1 categories used in ad tech), with several labels per page when they apply
• 💬 Text presets: Sentiment (positive, neutral, negative, mixed) for reviews, comments and support tickets, and Content moderation (safe, adult, violence, hate, spam, self-harm; several can apply)
• 🏠 Almost-empty home pages handled: when a home page or bare domain has less than 300 characters of text, the site's About page is read too (free), and pagesRead shows which pages were used
• ✍️ Custom labels in any language: e.g. Competitor, Potential customer, Irrelevant, each with an optional description to guide the AI
• 🔗 Domains, URLs, texts or another Actor's dataset: stripe.com works, and so do reviews, posts or company descriptions
• 💬 Explainable: a short reason for every label, so you can spot-check quickly
• 🌐 Reads protected and JavaScript sites: plain HTTP first, a second network if a site blocks data centers, a real browser when needed
• 🅿️ Parked and placeholder sites flagged for free: parked or for-sale domains, "coming soon" pages and default server pages (e.g. "Welcome to nginx") come back with label: null and a siteStatus, without an AI call and without a charge
• 🧹 Duplicates removed, rows map back to your list: stripe.com and HTTPS://www.Stripe.com/ are one site, classified and charged once; every row has input (your line) and inputs (all lines merged into it)
• 💸 $4 per 1,000 pages, no start fee, and pages that cannot be read are free

Who is it for?

• Sales and lead generation: enrich a list of company websites with industry, B2B/B2C and a summary, then keep only the leads that fit
• SEO and content teams: map a whole site by page type, find thin landing pages or misplaced content
• Data and AI teams: label URLs or texts before scraping, building datasets or routing pages to the right parser
• Ad tech and brand safety: classify publisher pages into IAB categories, including sensitive topics

How much does it cost?

Analysis depth	What the AI reads	Price
Quick	first 4,000 characters	$3 / 1,000 pages
Standard (default)	first 12,000 characters	$4 / 1,000 pages
Deep	up to 32,000 characters	$8 / 1,000 pages

Example: classifying 1,000 company websites costs $4.00 at the standard depth, $3.00 with Quick or $8.00 with Deep.

No start fee. Texts are charged like pages. Pages that cannot be read (blocked, errors) are reported with an errorType but not charged. Higher Apify plans get volume discounts (see the Pricing tab).

For comparison, the dedicated website-industry classifiers we found on Apify Store charge $4 per 1,000 sites, the same as our standard price (checked September 2026). This Actor adds page types, topics, IAB, custom labels, confidence and reasons at that price.

Control your cost

• What is charged: one event per page or text that got a label.
• What is free: lines that are not a URL, private or local addresses, pages robots.txt disallows, duplicates of a site already in the list, parked / placeholder / default server pages, domains that do not exist, blocked pages, pages that only show a bot check, an "enable JavaScript" notice, a cookie-consent notice or (except with page-type labels) a login wall, image-only URLs and pages without readable text, empty texts, errors and time-outs, and results reused from your dedupe store (fromCache: true). Every row has charged: true/false, and failed rows say "(not charged)".
• Before the run starts, the log and the status message show the plan: how many items, the worst-case cost and your max charge per run (also saved as costPlan in SUMMARY).
• Max charge per run: set Maximum cost per run in the run options (or maxTotalChargeUsd in the API). When it is reached, the run stops starting new items and finishes cleanly: SUMMARY.status is LIMIT_REACHED. SUMMARY.notProcessed remains {count} without input values; SUMMARY.remainingInput gives the private key-value store key REMAINING_INPUT and the total pending count. Pending items are not written to the results dataset.
• Continue in a new run: open this run's key-value store and copy REMAINING_INPUT.urls into the new input's domains or urls, and REMAINING_INPUT.texts into texts. Copy its taxonomy and analysisDepth, plus customLabels, multiLabel or readAboutPage when present. Re-enter any other options you need, including extra instructions; they are not copied into this record. The saved lists retain their input order and omit completed items, so a new run does not repeat their results or charges. The record includes totalCount, savedCount and truncated; it holds at most 1,000 pending items and 256 KiB of JSON. If truncated is true (or settingsTruncated is true), use the original input list to add the omitted tail (or custom settings) before continuing. The saved items form a prefix of the pending list. If the original input came from a dataset, the saved values are copied into urls and texts rather than retaining the dataset link.
• If Apify restarts the run (server migration or Resurrect), items already finished are skipped and not charged again (SUMMARY.resumedSkipped).

Large lists: how to run thousands of domains

1. Paste the whole list in one run (up to 50,000 URLs or domains). Duplicates of a site are merged and charged once.
2. Passing failures are retried for you. A page that timed out, lost its connection or answered HTTP 408, 429, 500, 502, 503, 504, 522 or 524 is read once more at the end of the run, with half as many pages in parallel. A site that answered 429 (rate limit) is read again no sooner than 30 seconds after its first try; if it still answers 429 the free row says rate_limited. It still gets exactly one row: the result (charged once, retried: true) or, if it fails again, its free error row (retried: true). Domains that do not exist, blocked pages, 404s, robots.txt refusals and parked sites are not retried; they fail the same way again. SUMMARY.retryPass counts attempted, recovered, stillFailed and skipped (not retried because the run's timeout or your max charge was near). With Retry unreachable websites through a second network (networkRetry, on by default) a connection failure or time limit is read on that retry through our Cloudflare WARP network (via: "warp") instead of the same routes. Refused and unreachable websites get one second try. A website that refuses every request (HTTP 401, 403, 406 or 451, not a CAPTCHA or bot-check page) is asked once more right away through our WARP network with a standard browser User-Agent (Retry refused websites through a second network, warpRetry, on by default; CAPTCHAs are never solved and a bot-check page is never retried). A domain the run's DNS cannot find, or an unreachable site (Cloudflare 530), is checked with public DNS (DNS-over-HTTPS): a domain that exists after all is read (or retried at the end of the run); one that has no address in public DNS either is a free dns_not_found row (the website is gone) and is not retried. A recovered page is a normal result at the normal price; one that still fails keeps its first errorType, gets retried: true and is free. SUMMARY.recovered counts them: warp, network and dns.
3. Interrupted runs continue without paying twice. If Apify migrates or restarts the run, finished items are skipped, items waiting for their retry keep waiting, and nothing is charged again. Timeout and abort telemetry. The Actor sends its anonymous run counter before an announced abort or timeout. A migration keeps the counter for the resumed final run. A sudden process kill can still prevent delivery.
4. Run the failures again in one click. The key-value store record FAILED_DOMAINS lists every URL or domain that still failed, one per line exactly as you entered it (a one-column CSV without a header). SUMMARY.failedByType tells you why they failed (dns_not_found for a domain that does not exist, not_found for a page that answers 404/410, blocked, rate_limited…), and SUMMARY.failureSummary groups the same failures by whose side they are on: siteGone (the domain or page no longer exists: drop these, a rerun gives the same answer), siteRefused, siteError, temporary (connection, time limit, rate limit: worth running again later), classificationService, invalidInput and other. Filter the free failure rows on retryable: true before pasting them into a new run. Confirmed dead sites have failureCategory: "site_gone"; network, gateway and HTTP errors carry a safe failureKind and are marked temporary only when a second try could help. An unconfirmed DNS or gateway error is not proof that a website is gone. A failure row also has httpStatus when the website answered with an HTTP error (404, 429, 503…). A row whose error starts with classification service temporarily unavailable means the page was read but our AI classification service did not answer, even on the end-of-run retry: the website is fine, so run those items again later (SUMMARY.classificationServiceUnavailable counts them).
5. Lists you run again and again: skip what you already have. Set Skip items classified in earlier runs (dedupeStoreName, e.g. "my-leads"). Each charged result is also kept in a key-value store named classifier-cache-my-leads in your own Apify account. A later run with the same store name and the same settings (taxonomy, custom labels, several labels, extra instructions, depth, About page; for URLs also the page reading mode, route and proxy) returns the earlier result for a URL or text it already classified within dedupeMaxAgeDays (default 30), with fromCache: true and cachedAt, not charged. Change any of those settings and items are classified (and charged) again. Off by default; delete the store in Storage to start over.
6. Check the counts. SUMMARY.byResult has one count per result type: classified (charged), fromCache, placeholderSites, failed, invalidInput and notProcessed (stopped at your max charge; see REMAINING_INPUT).

{ "domains": ["stripe.com", "notion.so", "…5,000 more…"], "taxonomy": "industry", "dedupeStoreName": "my-leads", "dedupeMaxAgeDays": 30 }

How to use it

1. Paste URLs or bare domains (one per line), or add texts. A line that is not a URL becomes one free invalid_input row; the rest of the run continues.
2. Choose Classify by: Page type, Topic, Industry + business model, IAB or Custom labels.
3. For custom labels, write one label per line, optionally with a description after a colon.
4. Click Start and export the results as CSV, Excel or JSON. The Summary counts items per label.

Input example: industry of company websites

{
    "domains": ["stripe.com", "allbirds.com", "redcross.org", "www.gov.uk", "hubspot.com"],
    "taxonomy": "industry"
}

Output example (real results)

[
    { "url": "https://stripe.com", "label": "financial-services", "businessModel": "B2B", "confidence": "high",
      "companySummary": "Stripe is a financial services platform that helps businesses accept payments, build flexible billing models, and manage money movement." },
    { "url": "https://allbirds.com", "label": "consumer-products", "businessModel": "B2C", "confidence": "high",
      "companySummary": "Allbirds is a company that designs and sells comfortable and sustainable shoes made from natural materials." },
    { "url": "https://redcross.org", "label": "nonprofit", "businessModel": "nonprofit", "confidence": "high" },
    { "url": "https://www.gov.uk", "label": "government-public-sector", "businessModel": "government", "confidence": "high" },
    { "url": "https://hubspot.com", "label": "software-saas", "businessModel": "B2B", "confidence": "high",
      "companySummary": "HubSpot provides software and tools for businesses to manage marketing, sales, customer service, and CRM" }
]

A text works the same way: "We are a family-run plumbing company serving Leeds since 1998…" came back as local-home-services, B2C. Business models are the AI's reading of the page; for mixed cases (marketplaces, platforms) check them where it matters. Sites that refuse automated access (in our test, doordash.com returned HTTP 403 even in a browser) come back with errorType: "blocked" and are not charged.

Parked domains and placeholder pages

Lead lists often contain domains with no business behind them. When a short page matches a parked or for-sale domain, a "coming soon" / under-construction page, an example page or a default web server page, the row has success: true, label: null, siteStatus (parked, placeholder or default_server_page) and a reason, and it is not charged (no AI call). Real sites get siteStatus: "active". Real result: example.com → siteStatus: "placeholder", not charged. A domain that does not exist fails in a few seconds with errorType: "dns_not_found" (checked with public DNS too), free.

Lead-generation workflow: scraper → classifier → filter

1. Run a lead source, for example a Google Maps or company-directory scraper, that outputs each company's website.
2. Start this Actor with that run's dataset:

{
    "datasetId": "<dataset ID of the scraper run>",
    "datasetField": "website",
    "taxonomy": "industry",
    "analysisDepth": "quick"
}

3. In the results, filter by label and businessModel (e.g. only software-saas + B2B), or use Custom labels such as Good fit: B2B software company with 10+ employees / Poor fit: anything else. Quick depth ($3 / 1,000) is enough for home pages.

With a dataset, values that look like URLs or domains are read as pages; other values (e.g. review texts) are classified as text. Leave Dataset field empty to use the first URL field found.

Compare labels with a previous classification run

For a recurring list, set previousDatasetId to the Dataset ID of an earlier run of this classifier:

{ "domains": ["stripe.com", "notion.so"], "taxonomy": "industry", "previousDatasetId": "<previous classifier dataset ID>" }

The Actor reads up to 100,000 earlier dataset rows in pages, matching the emitted canonical url and taxonomy. Each new classified result has previousLabel and labelChanged: true when the label differs, false when it matches, or null when there is no matching earlier URL/taxonomy. Texts without a URL also have null. SUMMARY.changed, SUMMARY.unchanged and SUMMARY.new count those classified results; free failures and placeholder pages are excluded. A prior Dataset that cannot be read or is over the limit produces a free invalid_input row and NO_RESULTS before classification starts.

The prior Dataset never skips classification. With the optional dedupe store enabled, its separate cache may still return a free earlier result. A changed label means only that the two output labels differ; it does not prove the website changed or that either label is correct.

For scheduled runs, read CLASSIFICATION_CHANGES from this run's default key-value store (defaultKeyValueStoreId) after the run finishes. It is a JSON array containing only successfully classified URLs whose label changed or whose URL was absent from the previous Dataset. Each entry has url, taxonomy, label, previousLabel and changeStatus (changed or new); a new URL has previousLabel: null. An existing URL with only a different taxonomy is not a new URL. Unchanged results, cached earlier results, text, placeholders and failed pages are excluded. With a valid previousDatasetId, an empty array means there were no fresh changes; without it, this record is not created. The full results Dataset and SUMMARY.changed / unchanged / new remain available. Successfully reclassified pages still incur the normal page-classified event at standard depth (or the existing Quick / Deep event); this change list adds no event or charge.

Industry codes for your CRM: NAICS 2022 and SIC

With "taxonomy": "industry", set Industry codes (industryCodes) to naics, sic or both (default none: no code columns) to add standard codes to each row: naicsCode (6-digit NAICS 2022) and naicsTitle, sicCode (4-digit SIC 1987) and sicTitle, and codeConfidence. The codes come from a fixed lookup table of the 30 industry labels, not from another AI call, so they add no time and no charge. For example, every row labelled software-saas gets:

{ "label": "software-saas", "naicsCode": "513210", "naicsTitle": "Software Publishers", "sicCode": "7372", "sicTitle": "Prepackaged Software", "codeConfidence": "high" }

Each label maps to one default code, so codeConfidence says how far to trust it: high (the label nearly always means that code, e.g. legal-services → 541110 Offices of Lawyers), medium (the most common code; some companies belong to a neighbouring one), low (the label spans several sectors, e.g. automotive → 441110 New Car Dealers, though makers and repair shops have other codes; review before use) and none (no single code fits: consumer-products, other, a label outside the list and parked sites get empty codes). It is never higher than the AI's own confidence. The codes describe the industry label, not a registry record of the company.

Input example: page types

{
    "domains": [
        "https://stripe.com/pricing",
        "https://docs.python.org/3/tutorial/index.html",
        "https://www.bbc.com/news",
        "https://www.allbirds.com/products/mens-tree-runners"
    ],
    "taxonomy": "pageType"
}

Real results for this input: pricing, documentation, news and product, all with high confidence.

{
    "url": "https://www.ikea.com/us/en/cat/billy-bookcases-58288/",
    "success": true,
    "label": "category",
    "confidence": "high",
    "reason": "The page lists several products, all variations of the BILLY bookcase, with prices and details.",
    "language": "en",
    "unmatchedLabel": null,
    "taxonomy": "pageType",
    "analysisDepth": "standard",
    "mode": "fast",
    "via": "backend"
}

IAB categories

With "taxonomy": "iab" (real results, quick depth): bbc.com/sport/football → Sports + News and Politics; allbirds.com → Shopping + Style & Fashion; stripe.com → Business and Finance + Technology & Computing.

Almost-empty home pages: the About page is read too

Some company home pages are a logo and a video. When a home page (or bare domain) gives less than 300 characters of text, the Actor follows the site's About / Company link (or tries /about and /about-us) and classifies both pages together, at no extra cost. Real result for palantir.com read over plain HTTP (the home page gave 14 characters of text):

{
    "url": "https://palantir.com",
    "label": "software-saas",
    "businessModel": "B2B",
    "confidence": "high",
    "companySummary": "Palantir builds software that empowers organizations to integrate their data, decisions, and operations.",
    "pagesRead": ["https://www.palantir.com/", "https://www.palantir.com/about"]
}

This is used with every taxonomy except Page type (where the home page itself is what you want labelled). Turn it off with "readAboutPage": false.

Text presets: sentiment and content moderation

{
    "texts": [
        "Arrived two days early and the sound quality is amazing. Best headphones I have owned.",
        "Great screen, but the keyboard feels cheap and the fan is loud."
    ],
    "taxonomy": "sentiment"
}

Real results: the first text → positive, the second → mixed ("The text mentions both a positive and a negative aspect of the product."); a complaint about a dead battery and unanswered emails → negative; "The package contains one charger and a USB-C cable." → neutral. All four with confidence: high.

With "taxonomy": "contentModeration", five test texts came back as: a thank-you comment on a recipe → safe; a news sentence about a city budget → safe; "CLICK HERE to win a FREE iPhone!!! ... enter your card details" → spam; an insult telling people to leave the country → hate; a message about planning to end one's life → self-harm. The labels field lists every category that applies when there is more than one. Moderation labels are an aid for review queues, not a replacement for human moderators.

Input example: your own labels

{
    "domains": ["https://asana.com/", "https://www.atlassian.com/software/jira", "https://www.allbirds.com/"],
    "taxonomy": "custom",
    "customLabels": [
        "Competitor: sells project management or team collaboration software",
        "Potential customer: software or technology company that may need a project management tool",
        "Irrelevant: anything else"
    ]
}

Labels can be in any language; results use your labels exactly as written. With Topic and IAB, the labels field lists every label that applies.

Output fields

Field	Meaning
input, inputs, inputIndex	The line you entered, every line merged into this site (duplicates), and its position in your list
label	The best label (null for parked or placeholder sites)
siteStatus	active, or parked, placeholder, default_server_page (not charged)
charged	Whether this row was charged
datasetIndex	With Dataset ID: position of the value in the source dataset
previousLabel, labelChanged	With Previous classification dataset: earlier label (or null) and whether the emitted label differs (true, false, or null without a comparable earlier URL/taxonomy); classified results only
labels	All labels that apply (only with several labels per page)
pagesRead	Only when the About page was read too: the home page and the About page used
businessModel, companySummary	Only with Industry: B2B / B2C / B2B2C / nonprofit / government, and one sentence about the company
naicsCode, naicsTitle, sicCode, sicTitle, codeConfidence	Only with Industry and Industry codes set to naics, sic or both: the NAICS 2022 / SIC 1987 code and title for the label, and high, medium, low or none (codes empty)
confidence	high, medium or low: the AI's own estimate of how clearly the page fits
reason	One sentence explaining the choice
language	Main language of the page (ISO 639-1 code)
unmatchedLabel	Filled only in the rare case the AI answered with a label that is not in your list
via	How the page was fetched: backend, direct (Apify's network), vm (our second server), browser, or warp (our Cloudflare WARP network, on a second try)
source, textPreview	For texts: text-0, dataset-12… and the start of the text
error, errorType	For items that failed: invalid_input, robots_disallowed, blocked, rate_limited, dns_not_found, unreachable, network, not_found, http_error, timeout, no_text… (not charged). An error that starts with classification service temporarily unavailable means the page was read but not classified: try it again later
failureCategory, failureKind, retryable	Free failure rows distinguish confirmed dead sites (site_gone), temporary faults and unconfirmed errors. failureKind is a safe subtype of network, unreachable or http_error. retryable means a later run may help, even if this run's second attempt failed or had no time.
httpStatus	On a failure row, the HTTP status the website answered (404, 429, 503…), when it answered one
retried	true when the row comes from a second try: the end-of-run retry of a passing failure, or (on a free error row) the WARP retry of a refused page that was refused again
fromCache, cachedAt	Only with Skip items classified in earlier runs: true when the result was reused from your dedupe store (not charged), and when it was saved there

The key-value store record SUMMARY has status (SUCCESS, PARTIAL_RESULTS, FAILED, NO_RESULTS, LIMIT_REACHED), byResult (one count per result type), failedByType (failures per errorType), failureSummary (the same failures by group: siteGone, siteRefused, siteError, temporary, classificationService, invalidInput, other), the count per label and, when robots.txt excluded pages, skippedByRobotsTxt; when the classification service stayed unavailable for some items, classificationServiceUnavailable (their count). When pages were retried it has retryPass; recovered: {warp, network, dns} counts the items read on a second try (refused, then read through WARP; read on the end-of-run retry; found by public DNS); when some still failed, failedDomains: {key, count} points to the FAILED_DOMAINS record; with a dedupe store, dedupe: {store, maxAgeDays, fromCache}. A charge-limited run also has notProcessed: {count} and remainingInput: {key, count}; the private REMAINING_INPUT record contains the pending input values.

Use with AI agents (MCP)

Connect Apify's MCP server (https://mcp.apify.com?tools=tidytools/ai-page-classifier) to Claude, Cursor or any MCP client, then ask e.g. "What industry are these 20 companies in, and which of them sell to businesses?".

{ "domains": ["stripe.com", "notion.so"], "taxonomy": "industry" }

Failed items are not charged and carry an errorType, so an agent can tell a blocked site from a real answer.

Use it from code and integrations

Run it from your own code with the Apify API. This call waits for the run and returns the results as JSON (replace YOUR_TOKEN with your Apify API token):

curl -X POST "https://api.apify.com/v2/acts/tidytools~ai-page-classifier/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domains":["https://stripe.com/pricing","https://docs.python.org/3/tutorial/index.html","https://www.bbc.com/news","https://www.allbirds.com/products/mens-tree-runners"]}'

The synchronous endpoint waits up to 5 minutes. For bigger runs, start the run with POST https://api.apify.com/v2/acts/tidytools~ai-page-classifier/runs and read the dataset when it finishes, or use the apify-client package for JavaScript or Python.

Schedules and integrations: run it daily or weekly with Apify Schedules, get a webhook when a run finishes, or send the results to Zapier, Make, n8n, Google Sheets, Slack and other apps with Apify integrations. Results can be exported as JSON, CSV, Excel or XML.

Tips

• Descriptions make custom labels much more accurate: say what belongs in each label, not just its name.
• Use Extra instructions (up to 1,000 characters) for edge cases, e.g. "Our own website is never a competitor."
• For company classification, use home pages or bare domains; a single blog post may be about a different topic than the company.
• confidence is the model's self-assessment. Spot-check low and medium results, and a sample of high ones for important decisions.
• If a site blocks our servers, the Actor retries from Apify's network automatically, and a site that refuses every route is asked once more through our Cloudflare WARP network. For stubborn sites, set Plain HTTP requests from to Apify's network and add a proxy under Advanced settings.
• To extract specific fields (prices, names, dates) instead of a label, use AI Web Data Extractor (by TidyTools).

How accurate is it?

We labelled a sample ourselves and ran the Actor on it (October 3, 2026, Standard depth, default settings). Pages the Actor could not read (blocked sites, errors) are free and are not counted below.

Classify by	Items sent	Read and labelled	Matches our label	Accuracy	Not readable (free)
Industry + business model (company home pages)	82	63	61	97%	19 (mostly bot-blocked sites)
Page type (one page of each kind)	22	21	20	95%	1
Sentiment (short reviews and notes)	12	12	12	100%	0
All	116	96	93	97%	20

The three mismatches are arguable: cisco.com came back as software-saas (we had said telecommunications), tyson.com as consumer-products (we had said agriculture-food-production), and the GPL 3.0 license text as documentation (we had said legal).

Read this as a rough guide, not a benchmark: one person wrote the labels, the sample is small, mostly well-known sites and clear-cut cases, and the closest label is a judgment call for companies that fit two industries. Smaller or unusual sites, short pages and mixed-purpose pages will score lower. 1 in 5 of the company sites in our industry sample could not be read at all, which is why a free blocked row is not an error in the AI. Spot-check with the one-sentence reason on every row.

Limitations

• The AI reads the first part of each page (4,000, 12,000 or 32,000 characters depending on the depth); details further down a long page are not considered.
• Some sites block automated access; those pages are reported with an error and not charged. The Actor identifies itself (TidyToolsFetch; on the one WARP retry of a refused page, a standard browser User-Agent that still ends with our TidyTools token) and does not get around bot protection: CAPTCHA and bot-check pages are never solved or retried.
• robots.txt is respected for every page it reads (including the About page and the page a redirect lands on): a page the site's robots.txt disallows is not read and comes back as a free robots_disallowed row. Texts you paste are never checked.
• Private, local and reserved addresses (localhost, 127.0.0.1, 192.168.x.x, 169.254.x.x, *.local…) are not public web pages: each becomes a free invalid_input row and is never requested. Cloudflare 526 identifies an invalid certificate; 525/527 are gateway failures and 530 is ambiguous until public DNS is checked. All are free; only public DNS confirming no address becomes dns_not_found.
• Input mistakes never fail the run: with nothing usable (no valid URL, text or dataset, or fewer than 2 custom labels), the run ends normally with free invalid_input rows (charged: false) and SUMMARY.status NO_RESULTS. An empty entry in Texts to classify gets its own free row.
• Time limit per item (Advanced settings, default 180 seconds) stops a page that takes too long; it comes back as a free timeout row.
• As with any AI model, a small share of labels can be wrong, especially for pages that mix several purposes.
• Runs on a paid Apify plan use a higher-accuracy AI model for the labels; runs on the free plan use a standard model, so their labels can be somewhat less accurate. The price per item is the same.

FAQ

Does the Actor collect usage data? The Actor sends one anonymous usage counter per run (input type, item count range, outcome) to improve the tool; no URLs, content or account data.

Can this website classifier tell me each company's industry? Yes. Set Classify by to Industry + business model to get one of 30 industries, plus businessModel (B2B, B2C, B2B2C, nonprofit, government) and a one-sentence companySummary. Bare domains like stripe.com work, and when a home page has less than 300 characters of text, the site's About page is read too at no extra cost.

Can I get NAICS or SIC codes for my CRM? Yes, with the Industry taxonomy: set Industry codes to NAICS, SIC or both. Each industry label maps to a NAICS 2022 and a SIC code from a fixed table, with codeConfidence saying how well the label fits one code. No extra charge.

Can I use it as a text classifier for sentiment analysis or content moderation? Yes. Add texts in Texts to classify (optional) and choose Sentiment (positive, neutral, negative, mixed) or Content moderation (safe, adult, violence, hate, spam, self-harm; several can apply). Moderation labels are an aid for review queues, not a replacement for human moderators.

Does it support IAB categories and my own labels? Yes. IAB uses the 30 tier-1 content categories used in ad tech, with several labels per page when they apply. With Custom labels, write one label per line in any language, optionally with a description after a colon; results use your labels exactly as written.

What is charged and what is free? One event per page or text that got a label: $4 / 1,000 pages at the Standard Analysis depth, $3 with Quick or $8 with Deep, with no start fee. Lines that are not a URL, private or local addresses, pages robots.txt disallows, duplicates of a site already in the list, parked / placeholder / default server pages, domains that do not exist, blocked pages, image-only URLs and pages without readable text, empty texts, errors and time-outs are free.

What if a website refuses the request or cannot be reached? When a website refuses the request or cannot be reached for a moment, it is automatically tried once more (through our Cloudflare WARP network, or with a public DNS check); a row that still fails is not charged. A recovered page is charged like any other result, once. CAPTCHA and bot-check pages are never solved or retried, and a domain that does not exist (dns_not_found) is not retried. A site that answers 429 (rate limit) is asked again on the end-of-run retry, at least 30 seconds later. Turn this off with Retry refused websites through a second network (warpRetry) and Retry unreachable websites through a second network (networkRetry) under Advanced settings.

Does it respect robots.txt? Yes. Before reading a page the Actor checks the site's robots.txt (as TidyToolsFetch); a disallowed page is not read, comes back as a free robots_disallowed row and is counted in SUMMARY.skippedByRobotsTxt.

What happens if my input is wrong? The run ends normally. An unusable URL or private address such as localhost:3000 and each empty text get free invalid_input rows (input: null, charged: false) with a fixed reason and, where available, the source field and entry number; usable items continue. An unknown Classify by value or fewer than 2 usable custom labels instead stops processing with one free invalid_input row and SUMMARY.status NO_RESULTS.

Related Actors

More tools by TidyTools that work well with this one:

• Web Page Summarizer: summarize pages or texts with key points and checked quotes
• AI Web Data Extractor: turn any page into structured JSON with the fields you choose
• Company Website Enrichment: add company details to a list of domains
• Website Tech Stack Detector: find the CMS, shop platform and tools a site uses

Support

Open an issue in the Issues tab with an example URL. Issues are checked regularly.