# Semrush Scraper (`radeance/semrush-scraper`) Actor Access Semrush Domain Authority (DA), Desktop / Mobile Traffic, Keyword Difficulty, Backlink profiles, trending sites & competitor insights from the enterprise-level SEO platform . Supports bulk analysis with historical trends. Download as CSV, JSON, Excel, XML & more. - **URL**: https://apify.com/radeance/semrush-scraper.md - **Developed by:** [Radeance](https://apify.com/radeance) (community) - **Categories:** SEO tools, Automation, Developer tools - **Features:** Limited permissions - **Stats:** 1,797 total users, 81 monthly users, 100.0% runs succeeded, 83 bookmarks - **User rating**: 4.99 out of 5 stars ## Pricing from $17.50 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## πŸ’Ž Semrush SEO Scraper The **Semrush Scraper** turns domains and keywords into structured, export-ready SEO data. Analyze **domain authority**, **website traffic and historical trends**, **backlink and organic search insights**, **keyword difficulty with competing SERP pages**, and **top website rankings by country or industry** β€” across multiple domains or keywords in one run. Built for competitor research, SEO reporting, and dashboards, results are ready to use in JSON, CSV, XLSX, and other Apify-supported formats. ### ❢ Key Features - **πŸ” Comprehensive SEO Data Extraction:**
- βœ… Choose from **Domain Authority**, **Domain Overview**, **Website Traffic**, **Keyword Difficulty**, and **Top Websites Ranking** searches - βœ… Extract **authority scores**, **website rankings**, **traffic analytics**, **organic keyword data**, and **backlink insights** - βœ… Access current metrics and historical trends for deeper SEO and competitive analysis - **πŸ“ˆ Advanced Traffic Analytics:**
- βœ… Capture **visits**, **unique visits**, **bounce rate**, **time on site**, and **pages per visit** - βœ… Compare **desktop and mobile traffic**, geographical traffic shares, and historical organic performance - βœ… Analyze inbound and outbound traffic sources, top organic pages, keywords, and competitors - **πŸ–‡οΈ Backlink & Authority Intelligence:**
- βœ… Retrieve **Semrush Authority Score**, **Moz Domain Authority**, and **Moz Spam Score** - βœ… Track backlink and referring-domain totals, changes, and historical trends - βœ… Inspect top backlinks with source and target URLs, anchor text, link attributes, and page authority - **πŸ“ Keyword Difficulty & SERP Competition:**
- βœ… Evaluate **multiple keywords** for a selected country in a single run - βœ… Retrieve a clear **keyword difficulty index** for every keyword - βœ… Review competing SERP pages with authority, backlink, referring-domain, traffic, and organic keyword metrics - **πŸ† Trending Website Data:**
- βœ… Retrieve up to 100 **top-ranking websites** globally or for a selected country and industry - βœ… Compare traffic, desktop and mobile shares, and each website's leading traffic source - βœ… Measure **month-over-month** and **year-over-year** traffic changes and historical trends - **πŸ“Š Flexible Data Output:**
- βœ… Export SEO data seamlessly in multiple formats: CSV, XLSX, JSON, JSONL, XML, and RSS ### ❷ Output #### Field Reference Refer to this [Output Schema for Semrush Scraper](https://apify.com/radeance/semrush-scraper/output-schema) for a complete list of all the fields returned in the output. #### Domain Authority Overview ###### Requires: `Domain or URL` ###### Fields: `authority_score`, `moz_domain_authority`, `moz_spam_score`, `referral_domains`, `backlinks`, `dofollow_percentage`, `domain_age` ```json { "data_captured_at": "2026-09-27T12:00:00.000000", "type": "domain_authority", "domain": "openai.com", "authority_score": 96, "moz_domain_authority": 92, "moz_spam_score": 0.01, "referral_domains": 491840, "backlinks": 70350000, "dofollow_percentage": 0.74, "domain_age": 19 } ``` #### Domain Overview ###### Requires: `Domain or URL` ###### Fields: `categories`, `visits`, `visits_history`, `bounce_rate`, `pages_per_visit`, `time_on_site`, `global_rank`, `category_rank`, `country_rank`, `authority_score`, `backlinks_google_penalty_risk`, `traffic_organic`, `traffic_paid`, `traffic_history`, `traffic_by_country_history`, `traffic_by_device_history`, `inbound_traffic_sources`, `outbound_traffic_sources`, `total_backlinks`, `total_backlinks_last_month`, `total_backlinks_change`, `backlinks_history`, `backlinks_last_updated`, `referral_domains`, `referral_domains_last_month`, `referral_domains_change`, `referral_domains_history`, `top_organic_keywords`, `top_organic_keywords_last_updated`, `competitors` ```json { "data_captured_at": "2026-06-14T22:22:01.780939", "type": "domain_overview", "last_updated": "2026-06-12T00:00:00", "domain": "google.com", "country": "us", "categories": [ { "category": "Online Services" } ], "visits": 101172727926, "visits_history": [ { "date": "2026-05-01T00:00:00", "value": 101172727926 }, { "date": "2026-04-01T00:00:00", "value": 93834069957 }, { "date": "2026-03-01T00:00:00", "value": 111753386145 } ], "bounce_rate": 0.3669, "pages_per_visit": 5.6419, "time_on_site": 799.0, "global_rank": { "category": "Global", "rank": 1, "change": 0, "rank_last_month": 1 }, "category_rank": { "category": "Online Services", "rank": 1, "change": 0, "rank_last_month": 1 }, "country_rank": { "country": "US", "rank": 1, "change": 0, "rank_last_month": 1 }, "authority_score": 100, "backlinks_google_penalty_risk": "Low", "traffic_organic": 5235637014, "traffic_paid": 21228492, "traffic_history": [ { "date": "2026-05-15T00:00:00", "organic_traffic": 5235637014, "paid_traffic": 21228492, "search_traffic": 5256865506 }, { "date": "2026-04-15T00:00:00", "organic_traffic": 4628706977, "paid_traffic": 23417013, "search_traffic": 4652123990 }, { "date": "2026-03-15T00:00:00", "organic_traffic": 4638180545, "paid_traffic": 21650526, "search_traffic": 4659831071 }, { "date": "2026-02-15T00:00:00", "organic_traffic": 4793343366, "paid_traffic": 21571469, "search_traffic": 4814914835 }, { "date": "2026-01-15T00:00:00", "organic_traffic": 4790672330, "paid_traffic": 18204040, "search_traffic": 4808876370 }, { "date": "2025-12-15T00:00:00", "organic_traffic": 4917115520, "paid_traffic": 23052763, "search_traffic": 4940168283 } ], "traffic_by_country_history": [ { "country": "US", "traffic_share": 0.2405, "traffic": 24331115522, "desktop_share": 0.2794, "mobile_share": 0.7206 }, { "country": "JP", "traffic_share": 0.0632, "traffic": 6389773210, "desktop_share": 0.1857, "mobile_share": 0.8143 }, { "country": "IN", "traffic_share": 0.0542, "traffic": 5487266135, "desktop_share": 0.2773, "mobile_share": 0.7227 }, { "country": "BR", "traffic_share": 0.0501, "traffic": 5067029603, "desktop_share": 0.2751, "mobile_share": 0.7249 }, { "country": "DE", "traffic_share": 0.0343, "traffic": 3471013237, "desktop_share": 0.3288, "mobile_share": 0.6712 } ], "traffic_by_device_history": [ { "date": "2026-05-01T00:00:00", "value": 101172727926, "desktop_visits": 29349576296, "mobile_visits": 71823151630, "desktop_share": 0.2900937525127022, "mobile_share": 0.7099062474872978 }, { "date": "2026-04-01T00:00:00", "value": 93834069957, "desktop_visits": 27351478846, "mobile_visits": 66482591111, "desktop_share": 0.2914877171856019, "mobile_share": 0.7085122828143982 }, { "date": "2026-03-01T00:00:00", "value": 111753386145, "desktop_visits": 32321386007, "mobile_visits": 79432000138, "desktop_share": 0.2892206412883365, "mobile_share": 0.7107793587116635 }, { "date": "2026-02-01T00:00:00", "value": 88460065425, "desktop_visits": 25528803784, "mobile_visits": 62931261641, "desktop_share": 0.2885912830987486, "mobile_share": 0.7114087169012514 }, { "date": "2026-01-01T00:00:00", "value": 94757767992, "desktop_visits": 27958159934, "mobile_visits": 66799608058, "desktop_share": 0.29504873876261406, "mobile_share": 0.7049512612373859 }, { "date": "2025-12-01T00:00:00", "value": 95362815483, "desktop_visits": 26615919234, "mobile_visits": 68746896249, "desktop_share": 0.2791016508813619, "mobile_share": 0.720898349118638 } ], "inbound_traffic_sources": [ { "source": "Direct", "visits": 27186577924, "visits_last_month": 25344605557, "share": 0.92630223 }, { "source": "youtube.com", "visits": 243299470, "visits_last_month": 219053012, "share": 0.00828971 }, { "source": "bing.com", "visits": 50097916, "visits_last_month": 46479615, "share": 0.00170694 }, { "source": "chatgpt.com", "visits": 46273515, "visits_last_month": 40206382, "share": 0.00157664 }, { "source": "web.whatsapp.com", "visits": 43977047, "visits_last_month": 41992850, "share": 0.00149839 } ], "outbound_traffic_sources": [ { "source": "youtube.com", "visits": 1574608577, "visits_last_month": 1449034666, "share": 0.0383161 }, { "source": "reddit.com", "visits": 1268530601, "visits_last_month": 1179403913, "share": 0.03086808 }, { "source": "wikipedia.org", "visits": 790621068, "visits_last_month": 753363172, "share": 0.01923876 }, { "source": "facebook.com", "visits": 634252563, "visits_last_month": 593022231, "share": 0.01543373 }, { "source": "instagram.com", "visits": 556469880, "visits_last_month": 500397434, "share": 0.01354099 } ], "total_backlinks": 50277073038, "total_backlinks_last_month": 50401117901, "total_backlinks_change": -0.25, "backlinks_history": [ { "date": "2026-05-01T00:00:00", "backlinks": 50277073038 }, { "date": "2026-04-01T00:00:00", "backlinks": 50401117901 }, { "date": "2026-03-01T00:00:00", "backlinks": 51513015121 }, { "date": "2026-02-01T00:00:00", "backlinks": 51146670544 }, { "date": "2026-01-01T00:00:00", "backlinks": 51675883893 }, { "date": "2025-12-01T00:00:00", "backlinks": 51930531506 } ], "backlinks_last_updated": "2026-05-01T00:00:00", "referral_domains": 51300049, "referral_domains_last_month": 51986210, "referral_domains_change": -1.32, "referral_domains_history": [ { "date": "2026-05-01T00:00:00", "referringDomains": 51300049 }, { "date": "2026-04-01T00:00:00", "referringDomains": 51986210 }, { "date": "2026-03-01T00:00:00", "referringDomains": 52629411 }, { "date": "2026-02-01T00:00:00", "referringDomains": 52374318 }, { "date": "2026-01-01T00:00:00", "referringDomains": 51920588 }, { "date": "2025-12-01T00:00:00", "referringDomains": 51836255 } ], "top_organic_keywords": [ { "keyword": "google", "traffic_share": 0.0004, "search_volume": 83100000, "cpc": 20.72 }, { "keyword": "google flights", "traffic_share": 0.0003, "search_volume": 20400000, "cpc": 0.75 }, { "keyword": "gmail", "traffic_share": 0.0003, "search_volume": 55600000, "cpc": 1.64 }, { "keyword": "facebook", "traffic_share": 0.0003, "search_volume": 101000000, "cpc": 7.17 }, { "keyword": "google maps", "traffic_share": 0.0003, "search_volume": 16600000, "cpc": 0.42 } ], "top_organic_keywords_last_updated": "2026-05-15T00:00:00", "competitors": [ { "domain": "apple.com", "relevance": 0.28, "visits": 704638097, "favicon": null }, { "domain": "microsoft.com", "relevance": 0.13, "visits": 1222875095, "favicon": "https://static.semrush.com/power-pages/media/favicons/microsoft-com-favicon-12e3dac8.ico" }, { "domain": "wikipedia.org", "relevance": 0.09, "visits": 4182321063, "favicon": "https://static.semrush.com/power-pages/media/favicons/wikipedia-org-favicon-904ce6bd.ico" } ] } ``` #### Website Traffic Search ###### Requires: `Domain or URL` ###### Fields: `organic_search_traffic`, `organic_search_traffic_history`, `mobile_organic_search_traffic`, `mobile_organic_search_traffic_history`, `organic_traffic_value`, `organic_traffic_value_history`, `organic_keywords_total`, `organic_keywords_history` ```json { "data_captured_at": "2026-10-06T16:26:52.611685", "type": "website_traffic", "last_updated": "2026-10-15T00:00:00", "domain": "openai.com", "organic_search_traffic": 16985060, "organic_search_traffic_history": [ { "date": "2026-09-15T00:00:00", "value": 16750837 }, { "date": "2026-10-15T00:00:00", "value": 16985060 } ], "mobile_organic_search_traffic": 9152828, "mobile_organic_search_traffic_history": [ { "date": "2026-09-15T00:00:00", "value": 9158688 }, { "date": "2026-10-15T00:00:00", "value": 9152828 } ], "organic_traffic_value": 17783217, "organic_traffic_value_history": [ { "date": "2026-09-15T00:00:00", "value": 19216167 }, { "date": "2026-10-15T00:00:00", "value": 17783217 } ], "organic_keywords_total": 946052, "organic_keywords_history": [ { "date": "2026-09-15T00:00:00", "value": 793119 }, { "date": "2026-10-15T00:00:00", "value": 946052 } ] } ``` #### Keyword Difficulty Search ###### Requires: `Keyword` `Country` ###### Fields `keyword` `country` `keyword_difficulty_index` `serp_results` ```json { "data_captured_at": "2026-09-15T17:24:46.761497", "type": "keyword_difficulty", "keyword": "playstation", "country": "us", "keyword_difficulty_index": 100, "serp_results": [ { "domain": "www.playstation.com", "url": "https://www.playstation.com/en-us/", "authority_score": 88, "referral_domains": 3966, "backlinks": 584710, "organic_traffic": 1254309, "organic_keywords": 8082 }, { "domain": "x.com", "url": "https://x.com/PlayStation", "authority_score": 69, "referral_domains": 295, "backlinks": 2687, "organic_traffic": 26037, "organic_keywords": 806 }, { "domain": "en.wikipedia.org", "url": "https://en.wikipedia.org/wiki/PlayStation", "authority_score": 66, "referral_domains": 1546, "backlinks": 10204, "organic_traffic": 26642, "organic_keywords": 2436 }, { "domain": "blog.playstation.com", "url": "https://blog.playstation.com/" }, { "domain": "www.playstation.com", "url": "https://www.playstation.com/en-gb/ps5/" }, { "domain": "www.instagram.com", "url": "https://www.instagram.com/playstation/?hl=en" }, { "domain": "www.youtube.com", "url": "https://www.youtube.com/user/PlayStation" }, { "domain": "store.playstation.com", "url": "https://store.playstation.com/" }, { "domain": "www.youtube.com", "url": "https://www.youtube.com/user/PlayStation" }, { "domain": "www.gamestop.com", "url": "https://www.gamestop.com/popular-consoles/playstation" } ] } ``` #### Top Websites Ranking Search ###### Requires: `Top Website Industry` `Top Website Country` ###### Fields: `total_traffic` `total_traffic_history` `ranks` (up to 100 results) ```json { "data_captured_at": "2025-10-09T20:03:08.157973", "type": "top_trending_websites", "last_updated": "2025-08-01T00:00:00", "country": "global", "industry": "all", "total_traffic": 253261802635, "total_traffic_history": [ { "date": "2024-09-01T00:00:00", "value": 255234250991 }, { "date": "2024-10-01T00:00:00", "value": 268751094934 }, { "date": "2024-11-01T00:00:00", "value": 247929534176 }, ... { "date": "2025-06-01T00:00:00", "value": 240830956236 }, { "date": "2025-07-01T00:00:00", "value": 257868271205 }, { "date": "2025-08-01T00:00:00", "value": 254846321495 } ], "ranks": [ { "domain": "google.com", "rank": 1, "traffic": 98223803338, "traffic_month_over_month_change_percentage": -3.082808, "traffic_year_over_year_change_percentage": 0.02475095, "desktop_traffic": 26987570738, "mobile_traffic": 71236232600, "desktop_share": 0.274755912730568, "mobile_share": 0.7252440872694321, "top_source": "direct", "traffic_month_over_month_change": -3028051267, "traffic_year_over_year_change": 24311324 }, { "domain": "youtube.com", "rank": 2, "traffic": 48112196329, "traffic_month_over_month_change_percentage": 0.02100302, "traffic_year_over_year_change_percentage": -4.286362, "desktop_traffic": 14752923970, "mobile_traffic": 33359272359, "desktop_share": 0.30663584487219847, "mobile_share": 0.6933641551278015, "top_source": "direct", "traffic_month_over_month_change": 10105014, "traffic_year_over_year_change": -2062262900 }, { "domain": "facebook.com", "rank": 3, "traffic": 9676753447, "traffic_month_over_month_change_percentage": -2.6834202, "traffic_year_over_year_change_percentage": -11.636927, "desktop_traffic": 4746416117, "mobile_traffic": 4930337330, "desktop_share": 0.49049675007184257, "mobile_share": 0.5095032499281574, "top_source": "direct", "traffic_month_over_month_change": -259667956, "traffic_year_over_year_change": -1126076734 }, ... { "domain": "mercadolivre.com.br", "rank": 98, "traffic": 216763258, "traffic_month_over_month_change_percentage": 0.416558, "traffic_year_over_year_change_percentage": 0.11037884999999999, "desktop_traffic": 126577134, "mobile_traffic": 90186124, "desktop_share": 0.583941832060856, "mobile_share": 0.416058167939144, "top_source": "direct", "traffic_month_over_month_change": 902944, "traffic_year_over_year_change": 239260 }, { "domain": "dailymail.co.uk", "rank": 99, "traffic": 213389734, "traffic_month_over_month_change_percentage": -10.592063, "traffic_year_over_year_change_percentage": -25.038593, "desktop_traffic": 76267580, "mobile_traffic": 137122154, "desktop_share": 0.35740978991988437, "mobile_share": 0.6425902100801157, "top_source": "direct", "traffic_month_over_month_change": -22602375, "traffic_year_over_year_change": -53429787 }, { "domain": "scribd.com", "rank": 100, "traffic": 213064083, "traffic_month_over_month_change_percentage": 0.022294247, "traffic_year_over_year_change_percentage": 0.19249655000000002, "desktop_traffic": 79302603, "mobile_traffic": 133761480, "desktop_share": 0.3722007101497253, "mobile_share": 0.6277992898502748, "top_source": "search", "traffic_month_over_month_change": 47501, "traffic_year_over_year_change": 410141 } ] } ``` ### ❸ Input #### Semrush Search Parameters Semrush Scraper Sample Input

| Parameter | Type | Default | Description | | ----------------------- | -------------- | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | `urls` | array (string) | `["make.com","zapier.com"]` | List of domains or full URLs to analyze on Semrush. Supports bulk analysis (e.g. `["make.com","example.com"]`). | | `keyword` | string | `ai automation` | Keyword for Keyword Difficulty Search | | `country` | string | `us` | Country for Keyword Difficulty Search | | `include_overview` | boolean | `false` | Includes basic domain overview metrics such as authority score, backlinks, organic traffic, and general stats. | | `include_authority` | boolean | `true` | Includes Semrush Authority Score, Moz Domain Authority and Spam Score, plus available referring-domain, backlink, dofollow and domain-age metrics. | | `include_traffic` | boolean | `true` | Includes website traffic insights from Semrush | | `include_top_websites` | boolean | `false` | Includes top trending websites by industry and country with global ranking insights. | | `industry_top_websites` | string | `all` | Industry filter for top websites search (e.g. `all`, `accounting-and-auditing`, `advertising-and-marketing`, `banking`, `education`). | | `country_top_websites` | string | `global` | Country filter for top websites search (e.g. `global`, `us`, `uk`, `de`, `fr`, `jp`, `br`, `ru`). | #### Supported URL Formats | Link | Supported | | -------------------------------------------------- | --------- | | | βœ… | | [https://www.n8n.io](https://www.make.com) | βœ… | | | βœ… | | [pipedream.com](pipedream.com) | βœ… | #### JSON Input Sample JSON input if you use the apify api via CURL, Python, JS etc. ```json { "urls": ["https://www.make.com/"], "keyword": "ai automation", "country": "us", "include_overview": true, "include_authority": true, "include_top_websites": true, "industry_top_websites": "all", "country_top_websites": "global" } ``` ### Available Search Types Each run can include up to 5 different search types: - `overview` - Basic Domain metrics and stats like authority score, backlinks, organic traffic, inbound / outbound links, and general stats - `authority` - Domain authority metrics like domain rating, backlinks, referral domains, and authority indicators - `traffic` - Website traffic insights like visits, unique visits, bounce rate, time on site, pages per visit, backlinks, referral domains, organic search traffic and keywords - `keyword_difficulty` - Keyword difficulty and SERP results for a specific keyword and country - `top_websites` - Top 100 trending websites worldwide or in a specific industry and country ### ❹ Use Cases - **SEO Specialists & Digital Marketers**: Gather detailed authority metrics, keyword data, traffic analytics, and backlink insights to refine SEO strategies and outperform competitors. - **Business Intelligence Teams**: Analyze **website performance trends**, **market positioning**, and **competitive landscapes** for data-driven decision-making. - **Enterprise SEO Teams**: Monitor **domain health**, **backlink evolution**, **traffic changes**, and **keyword visibility** across multiple websites and markets. - **Competitive Analysts**: Track **trending domains**, **industry benchmarks**, and **global traffic movements** to understand digital share of voice. - **Agencies & Consultants**: Provide clients with comprehensive **SEO audits**, **authority tracking**, and **market opportunity assessments**. - **Data Scientists & Researchers**: Collect large-scale SEO datasets for **trend forecasting**, **correlation analysis**, and **competitive intelligence modeling**. The **Semrush Data Scraper** streamlines the collection and analysis of in-depth SEO metrics, delivering enterprise-grade insights for professionals who need **complete competitive visibility** and **data-rich performance tracking**. ### ❺ Usage Limits This service has different usage limits depending on your subscription status: | User Type | Monthly Runs | # Bulk URLs Supported | Reset Period | | --------- | ------------ | --------------------- | ------------ | | **Free** | 5 runs | 1 per run | 30 days | | **Paid** | Unlimited | Unlimited | N/A | #### How Limits Work - **Free users**: Limited to 3 runs per 30-day period from your first usage - **Paid users**: No limits on the number of runs - **All users**: Can select any combination of the available search types to them in a single run - Usage resets automatically 30 days after your first run (for free users) ### ❻ FAQ #### How do i get started? Getting started is easy! Just click on the "Try for free" button at the top of the page, sign up for an Apify account if you don't have one, and you'll be taken to the Apify Console where you can run the Semrush Scraper Actor. You can input the parameters for your search, select the search types you want to include, and then start the run. Once the run is complete, you can view and download your results directly from the Apify Console. #### How long does a run take? The runtime depends on the number of search types you select and the number of URLs you include in your input. A single search type for a single URL typically takes around 15-30 seconds. If you select multiple search types or include multiple URLs, the runtime will increase accordingly. #### What format are the results in? The results are provided in JSON format, which is easy to read and can be easily integrated into your data analysis workflows. You can also view the output as a csv table. Each search type will have its own section in the output view, containing the relevant metrics and data for that search. ### ❼ While the scraper is running During the run, the actor will output log messages letting you know what is going on at any point. Each message always contains specific information about the process including which url / page the actor is working on. If you provide invalid inputs to the actor, it will immediately stop with a failure state and output log messages explaining what is wrong. If you are unsure what went wrong feel free to open up an issue in the issue tab. ### ❽ Legality of web scraping The **Semrush Scraper** is designed to ethically extract **only publicly available job data and company information**, and it **does not** scrape private user data such as personal email addresses or personal identifiers. Our services are ethical and **do not extract any private user data**. They only extract **what individuals or companies chose to share publicly**. We therefore believe that our services, when used for ethical purposes by our users, are safe to use. However, you should be aware that your results could contain personal data. Personal data is protected by the **GDPR** in the European Union and by other regulations around the world. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers. For more information you can read this [blog post](https://blog.apify.com/is-web-scraping-legal/) on the legality of web scraping from Apify. ### ❾ Feedback and Support **Your satisfaction** is **important** to us! Therefore we are constantly striving to enhance the performance of our services. If you have any technical feedback or encounter any bugs within the **Semrush Scraper**, please create an issue in the Actor’s Issues tab on the Apify Console to let us know about it. We will look into it as soon as possible. You can also contact us directly for **general help** on **issues** or **integrations** at suppport@radeance.com.
For **custom projects**, **general suggestions** or **new use cases** feel free to reach out to us at business@radeance.com # Changelog This Actor's version history is a separate document: https://apify.com/radeance/semrush-scraper/changelog.md # Actor input Schema ## `urls` (type: `array`): A list of URLs or Domains to search for on Semrush. You can specify multiple URLs by clicking on the Add button or in bulk by clicking on the Bulk edit button. Maximum Allowed URLs is 5 for free users and unlimited for paying users, others will be disregarded. ## `keywords` (type: `array`): Keywords to look up in Moz Keyword Explorer. ## `include_authority` (type: `boolean`): Needs Domain / Website. Includes Semrush Authority Score, Moz Domain Authority and Moz Spam Score, plus referring domains, backlinks, dofollow share and domain age when available. ## `include_overview` (type: `boolean`): Needs Domain / Website. If checked, the search will include website stats overview data like rank, category, visits, bounce rate, traffic from Semrush. ⚠️ This search will only return results on websites with high traffic. ## `include_traffic` (type: `boolean`): Needs Domain / Website. Includes US desktop and mobile organic search traffic, monthly histories, organic traffic value, and keyword totals. This search currently uses US data regardless of the Country selection. Visits, engagement, backlink data, and detailed keywords, pages, or competitors are currently unavailable. ## `include_keyword_difficulty` (type: `boolean`): Needs Keyword and Country. If checked, the search will include keywords data from Semrush. ## `country` (type: `string`): Select the country for the search on Semrush. Required for Keyword Difficulty search. ## `include_top_websites` (type: `boolean`): If checked, the search will include top websites data from Semrush from the Top websites ranking. ## `industry_top_websites` (type: `string`): Select the category for the search on Semrush Top Websites Ranking. ## `country_top_websites` (type: `string`): Select the country for the search on Semrush Top Websites Ranking. ## Actor input object example ```json { "urls": [ "openai.com" ], "keywords": [ "agentic ai" ], "include_authority": true, "include_overview": true, "include_traffic": false, "include_keyword_difficulty": false, "country": "us", "include_top_websites": false, "industry_top_websites": "all", "country_top_websites": "global" } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "urls": [ "openai.com" ], "keywords": [ "agentic ai" ] }; // Run the Actor and wait for it to finish const run = await client.actor("radeance/semrush-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "urls": ["openai.com"], "keywords": ["agentic ai"], } # Run the Actor and wait for it to finish run = client.actor("radeance/semrush-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "urls": [ "openai.com" ], "keywords": [ "agentic ai" ] }' | apify call radeance/semrush-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,radeance/semrush-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/yUvj8W0M4T0HhZKyA/builds/t1nGD428dCnvtvRqP/openapi.json