# DEV.to Articles Scraper (`fetch_cat/devto-articles-scraper`) Actor Scrape public DEV Community articles by tag, username, or article URL with metadata, engagement metrics, authors, tags, and optional body content. - **URL**: https://apify.com/fetch_cat/devto-articles-scraper.md - **Developed by:** [Fetch Cat](https://apify.com/fetch_cat) (community) - **Categories:** Social media, Marketing, Developer tools - **Features:** Limited permissions - **Stats:** 2 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.15 / 1,000 article saveds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README Export public DEV Community articles by tag, author username, or exact article URL. DEV.to Articles Scraper returns article metadata, author details, tags, engagement counts and optional article text for content research, newsletters and structured datasets. No DEV.to login or API key is required. Drafts and private account data are not supported. ### Quick start 1. Choose a mode and supply its tags, usernames or article URLs. 2. Start with a small article limit: ```json { "mode": "tag", "tags": ["javascript"], "maxItems": 20 } ``` 3. Open the resulting dataset and export JSON, CSV or Excel, or retrieve it through the API. Prefer a video guide? This short Apify walkthrough shows you how to set up and run an Actor. It uses a different Actor, but the setup steps are the same. https://youtu.be/1OW8gOqlZbY?t=25 ### What you get One row per distinct public article saved in the run, with its source link, title and available author, tag, date and engagement fields. Optional article text can be included when available. Missing source values remain null, and metadata can survive an unsuccessful optional text lookup. ### Who is it for? - **Developer-relations researchers** reviewing topics and public engagement in a developer community. - **Newsletter editors** preparing article shortlists for manual reading and selection. - **Content researchers** reviewing an author's public writing and available profile links. - **Text-analysis teams** collecting available article Markdown or HTML for their own analysis. ### How you can use the data - **Prepare a JavaScript reading shortlist.** Collect a small tag sample and export `title`, `url`, `tags`, `publishedAt` and `readingTimeMinutes`. Review the source articles before choosing what to share. - **Review a public author's writing.** Use username mode for `ben` and keep author identity, article links and publication dates together. Available GitHub, website or other profile links are public source fields, not verified contact records. - **Curate a recent AI-topic sample.** Request the existing top filter and seven-day timeframe for `ai`. Compare reaction and comment counts within the collected sample, without treating it as a locally ranked or complete list of every matching article. - **Analyze one known article's text.** Supply its exact public URL and request available body content. Check `bodyFetchWarning` before treating a missing body as empty writing. Store `id` and collection time if you compare later exports in your own workflow. ### Input settings **Author research:** collect up to ten public articles by one author. ```json {"mode":"username","usernames":["ben"],"maxItems":10} ``` **Topic curation:** request DEV's top-article window for a tag. Upstream ordering and filter support can vary. This is not a guaranteed local ranking of every matching article. ```json {"mode":"tag","tags":["ai"],"sort":"top","topTimeframeDays":7,"maxItems":20} ``` **Known article:** request a public article and its available text. ```json {"mode":"articleUrls","articleUrls":[{"url":"https://dev.to/ben/meme-monday-590l"}],"includeArticleBody":true,"maxItems":1} ``` #### More input | Setting / API key | Meaning | | --- | --- | | Scrape mode / `mode` | `tag` (default), `username` or `articleUrls`. Supply at least one source for the chosen mode. | | Tags / `tags` | Up to 100 entries. Only used in tag mode. The form prefill is not an API default. | | Usernames / `usernames` | Up to 100 entries. Only used in username mode. | | Article URLs / `articleUrls` | Up to 100 public HTTPS DEV article URLs, as strings or objects with `url`. | | Maximum articles / `maxItems` | Total saved articles across sources. 1–10,000, default `20`. Sources are processed in input order. | | Sort order / `sort` | `latest` (default) keeps DEV's default listing order. `top` requests its top filter. No local sorting. Not applicable to exact URLs. | | Top timeframe / `topTimeframeDays` | 1–365 days, default `7`. Used only with `top` in listing modes, where supported by DEV. | | Include article body / `includeArticleBody` | Default `false`. Request missing article text. Availability is not guaranteed. Exact-URL responses can already include text even when this is off. | The mode's source list is required at runtime. For API calls, supply it explicitly: opening the form with sample tags does not make an empty API input valid. Use the field help for detailed limits. ### Output data One dataset row represents one distinct article in the run. Missing source values are `null`. Body and source-specific fields are conditional. Illustrative shape only, not an observed article: ```json { "id": 123, "title": "Example article title", "url": "https://dev.to/example/example-article", "tags": ["javascript"], "authorUsername": "example", "publicReactionsCount": 0, "commentsCount": 0, "publishedAt": null, "inputMode": "tag", "sourceTag": "javascript" } ``` | Fields | Contents | | --- | --- | | `id`, `title`, `description`, `url`, `canonicalUrl`, `slug`, `path` | Article identity, summary and links. | | `publishedAt`, `editedAt`, `readablePublishDate`, `language`, `readingTimeMinutes` | Available dates, language and reading-time estimate. | | `tags`, `tagList` | Equivalent tag arrays retained for compatibility. | | `commentsCount`, `publicReactionsCount`, `positiveReactionsCount` | Source-reported engagement counts. | | `coverImage`, `socialImage` | Available image URLs, not downloaded files. | | `authorName`, `authorUsername`, `authorUserId`, `authorProfileImage` | Available public author details. | | `authorGithubUsername`, `authorTwitterUsername`, `authorWebsiteUrl` | Public profile links or handles, when supplied. | | `bodyMarkdown`, `bodyHtml`, `bodyFetchWarning` | Available article text. A warning can indicate an unsuccessful optional text lookup. | | `scrapedAt`, `inputMode`, `sourceTag`, `sourceUsername`, `sourceArticleUrl` | Collection time, mode and the first matching input source. | #### Download your results Open your run's dataset, choose **Export** and select JSON, CSV or Excel. JSON keeps nested fields together. CSV and Excel are useful when you want to review the records in a spreadsheet. See [Apify's download and export guide](https://docs.apify.com/storage/dataset#apify-console) for the available formats and field options. ### Pricing Pay-per-event pricing uses a one-time `start` event and an `item` event for each article saved to the dataset. An article saved without optional body content still counts as an item. A successful empty list produces no item events, but the start event still applies. See the [live Pricing tab](https://apify.com/fetch_cat/devto-articles-scraper/pricing) for current rates and discounts. ### Tips for best results - **Supply the chosen mode's source list explicitly.** Form sample tags do not make an empty API input valid. - **Start with a small total cap.** The article limit is shared across sources, which are processed in input order. - **Check optional text warnings before analysis.** Metadata can be saved even when a body lookup fails. - **Compare counts within the returned sample.** DEV controls listing order and top-filter support, and multiple sources are not merged into a global ranking. - **Read the outcome before repeating a request.** Saved rows can remain after a time-limited run, and unavailable exact URLs differ from a completed empty listing. ### Limits and caveats Only public articles are supported, without DEV.to login or an API key. Drafts, private account data and unavailable article bodies are not guaranteed. An unavailable exact article URL can fail the run, while a completed empty listing can succeed without records. The total article cap applies across sources in input order. DEV controls listing order and filter support, with no local sorting or complete-coverage guarantee. Upstream changes and rate limits can affect the result set. Exact-URL responses may already contain body text even when optional body lookup is disabled. Saved articles remain available if the run stops near its time limit. Optional text failures retain article metadata with `bodyFetchWarning`. When available, `RUN_SUMMARY` reports saved articles, warnings and status, while `PENDING_WORK` describes unfinished work. These records are separate from article rows, their storage is best effort, and they do not establish automatic recovery. ### Automation and integrations #### Save your settings for next time Once you have an input that works for you, [save it as an Apify task](https://docs.apify.com/actors/running/tasks). This keeps your settings together so you can use them again without filling in the form from scratch. #### Run on a schedule Need fresh results regularly? You can [schedule the Actor or a saved task](https://docs.apify.com/actors/running/schedules) to run at a time that suits you. Each scheduled run collects the data again. If you want to compare results or highlight changes, add that step in your own workflow. #### Send results to your other tools Use **n8n, Make or Zapier** to connect this scraper to the rest of your workflow. For example, you could pass its results to an AI assistant or add records to a spreadsheet. Follow the [Apify integration guides](https://docs.apify.com/integrations) for your chosen tool. For a custom connection, an [Apify webhook](https://docs.apify.com/integrations/webhooks) can notify your app when a run finishes or fails. Your app can then fetch the results or handle the failure. ### API usage Want to collect this data from your own app or automation? The examples below send input to the scraper and show how to collect its results. The Node.js and Python examples wait for the run to finish before reading its results. The cURL example starts the run and returns its details. Wait for that run to finish, then use its `defaultDatasetId` to fetch the results. For setup instructions, see Apify's [JavaScript client guide](https://docs.apify.com/api/client/js), [Python client guide](https://docs.apify.com/api/client/python) and [Actor runs API reference](https://docs.apify.com/api/v2/acts-runs-post). #### cURL ```bash curl -X POST 'https://api.apify.com/v2/acts/fetch_cat~devto-articles-scraper/runs' \ -H "Authorization: Bearer $APIFY_TOKEN" \ -H 'Content-Type: application/json' \ --data @- <<'JSON' { "mode": "tag", "tags": [ "javascript" ], "maxItems": 20 } JSON ``` #### Node.js ```javascript import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('fetch_cat/devto-articles-scraper').call({ "mode": "tag", "tags": [ "javascript" ], "maxItems": 20 }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items); ``` #### Python ```python import os from apify_client import ApifyClient client = ApifyClient(os.environ['APIFY_TOKEN']) run = client.actor('fetch_cat/devto-articles-scraper').call(run_input={'mode': 'tag', 'tags': ['javascript'], 'maxItems': 20}) print(client.dataset(run['defaultDatasetId']).list_items().items) ``` ### MCP and AI agents You can ask your AI assistant to collect data with **DEV.to Articles Scraper** and help you work with the results. The scraper returns the data, and your assistant can then summarize it, compare records or prepare notes. Connect your assistant through the [Apify MCP server](https://mcp.apify.com/). MCP gives your assistant a way to find this scraper, run it and read the results. Follow the [official setup and sign-in guide](https://docs.apify.com/mcp) for your app. #### Easier setup with an Apify plugin If you use a supported coding assistant, the [official Apify plugin](https://docs.apify.com/agents/plugins) can set up the connection for you. It connects your assistant to Apify and includes instructions for common tasks. Follow the guide for your particular app. #### Use Apify AI in Console Prefer to keep everything in Apify? Open [Apify AI in Console](https://docs.apify.com/account/apify-ai) and ask it to find Fetch Cat's **DEV.to Articles Scraper**. You can describe the data you want and ask questions about the results. Apify AI is currently in beta. It asks for confirmation before starting an Actor, and normal run charges apply. #### Choose your assistant - **Claude and Claude Code (Anthropic):** [Apify setup guide](https://docs.apify.com/integrations/claude). - **ChatGPT and Codex (OpenAI):** [Apify setup guides](https://docs.apify.com/integrations/openai). - **Grok Bot (xAI):** [Apify plugin guide](https://docs.apify.com/integrations/grok-bot). - **Gemini CLI (Google):** [MCP server setup guide](https://geminicli.com/docs/tools/mcp-server/). Use the server URL below with the HTTP configuration supported by Gemini CLI. #### Connect just this scraper To keep your setup focused on this scraper, add this server configuration to a compatible MCP client: ```json { "mcpServers": { "apify": { "url": "https://mcp.apify.com?tools=fetch_cat/devto-articles-scraper" } } } ``` Sign in to Apify when your client asks. If your client uses a Bearer token instead, follow its setup guide and keep the token in its secure configuration. Configuration formats vary by app, so use the linked guide if your client needs a different format. #### Use Apify's general server Already connected to the general Apify MCP server at `https://mcp.apify.com`? You can use it to find this scraper without adding another server. Ask your assistant to use **DEV.to Articles Scraper** by Fetch Cat, or give it the identifier `fetch_cat/devto-articles-scraper`. Your assistant can check the available settings and retrieve the results from the returned run. If the run is still working, it should follow that run rather than start it again. Both connection options use Apify's hosted service. #### Example requests - “Use Fetch Cat’s DEV.to Articles Scraper to collect up to 5 JavaScript articles. Show titles, article links, publication dates and reading-time estimates for a reading shortlist.” - “Collect at most 5 public articles by the DEV username ben. Include author names, article links and available public profile links, marking missing fields.” - “Retrieve up to 5 AI-tag articles using the top filter for the last 7 days. Show reaction and comment counts within this sample, and include any ordering or coverage caveats rather than calling it a complete popularity ranking.” Normal Apify charges apply. Check the [live Pricing tab](https://apify.com/fetch_cat/devto-articles-scraper/pricing) before asking your assistant to run a larger batch. ### Pay with x402 If your AI agent has a funded wallet, it can pay for Apify usage through [x402](https://docs.apify.com/integrations/x402). This experimental option lets an agent use Apify without a conventional account. [Apify AGI](https://agi.apify.com/) provides a prepaid API token with a spending cap after payment. Your agent can use that token to run this scraper through the Apify API or an authenticated MCP connection. Some MCP setups also support paying for each request directly. That is a different payment option, so check the official guide for its availability. Follow the official guide for wallet setup, supported payment options and current terms. Apify handles the payments. Keep wallet keys and payment tokens in your agent's secure configuration. #### Other ways for agents to pay Apify also supports **MPP** for prepaid access through AGI and **Skyfire** for eligible Actors. If your agent already uses one of these, see the [agentic payment guide](https://docs.apify.com/agents/payments) to choose the right setup. ### FAQ #### Can I use this as a DEV.to articles API alternative? Yes, for collecting the supported public article data through Apify. It is not an official DEV Community product and does not access account-only data. #### Can I export DEV.to articles to CSV or JSON? Yes. Export the saved dataset through Apify, or retrieve records using the API. #### Does it guarantee the newest or highest-reaction articles? No. Listing order and top-filter behavior are controlled by DEV. Sort collected records yourself when you need a specific ranking. #### Why is body text missing? Enable `includeArticleBody` to request available text. Some articles or optional lookups may be unavailable. Metadata can still be saved and charged as an item. #### Why did an empty API input fail? Supply the source list for the selected mode. Tags and usernames shown in the form are prefills, not defaults for omitted API fields. ### Related Actors - [Hacker News Search Scraper](https://apify.com/fetch_cat/hacker-news-search-scraper) — developer and startup discussions. - [GitHub Repositories Search Scraper](https://apify.com/fetch_cat/github-repositories-search-scraper) — repositories by topic and keywords. - [Stack Exchange Questions Scraper](https://apify.com/fetch_cat/stack-exchange-questions-scraper) — developer questions and tags. - [GitHub Issues & Pull Requests Scraper](https://apify.com/fetch_cat/github-issues-pull-requests-scraper) — public project discussions. - [Reddit Scraper](https://apify.com/fetch_cat/reddit-scraper) — broader community research. ### Support and feedback For custom additions or integrations, email . Found an error or have a feature request? [Open an issue](https://apify.com/fetch_cat/devto-articles-scraper/issues) with the run URL, a non-sensitive input example and what you expected. Need help sharing it? See [Apify’s guide to sharing a run when reporting an issue](https://docs.apify.com/account/collaboration/general-resource-access#automatically-sharing-runs-via-actor-issues). Sharing a run can give the developer access to its input, logs and results, so use a run that contains no sensitive data. Please leave out API tokens, passwords, cookies, private documents and personal data. Use a public sample URL or a redacted example instead. Tried this scraper? We'd love to hear how it worked for you. [Leave a review](https://apify.com/fetch_cat/devto-articles-scraper/reviews) to help others choose the right tool and help us improve it. Your feedback means a lot to us. Made with care for curious people, AI agents and the possibilities in data. ♥ # Changelog This Actor's version history is a separate document: https://apify.com/fetch_cat/devto-articles-scraper/changelog.md # Actor input Schema ## `mode` (type: `string`): Choose tags (default), usernames, or exact article URLs. Supply at least one entry in the corresponding source field; sample form values are prefills, not API defaults. Other source fields do not add another mode. ## `tags` (type: `array`): Required in Tags mode: 1–100 DEV tags, with or without #. Processed in input order until the shared article limit is reached. The sample form values are not applied to omitted API inputs. ## `usernames` (type: `array`): Required in Usernames mode: 1–100 DEV usernames, with or without @. The form sample is not an API default; ignored for collection in other modes. ## `articleUrls` (type: `array`): Required in Article URLs mode: 1–100 public HTTPS dev.to article URLs, as strings or objects with url. Unavailable articles can fail a run. Responses may already contain body text regardless of Include article body. Ignored for collection in other modes. ## `maxItems` (type: `integer`): Maximum total article records to save across all selected tags, usernames, or URLs. ## `sort` (type: `string`): Latest (default) retains DEV's default listing order, not a guaranteed newest-first order. Top requests DEV's top filter where supported. No local sorting or global ranking across sources is performed. Neither choice affects Article URLs mode. ## `topTimeframeDays` (type: `integer`): In listing modes with Sort order Top, request DEV's top filter with a 1–365 day window (default 7). DEV controls filter support and ordering; this field does not affect Latest or Article URLs mode. ## `includeArticleBody` (type: `boolean`): Request missing article text and include available bodyMarkdown and bodyHtml (default off). Optional lookup failures keep metadata with bodyFetchWarning and still count as a saved article. Exact-URL results can already include text when off. Text availability is not guaranteed. ## Actor input object example ```json { "mode": "tag", "tags": [ "javascript", "webdev" ], "usernames": [ "ben" ], "articleUrls": [ { "url": "https://dev.to/ben/meme-monday-590l" } ], "maxItems": 20, "sort": "latest", "topTimeframeDays": 7, "includeArticleBody": false } ``` # Actor output Schema ## `articles` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "tag", "tags": [ "javascript", "webdev" ], "usernames": [ "ben" ], "articleUrls": [ { "url": "https://dev.to/ben/meme-monday-590l" } ], "maxItems": 20 }; // Run the Actor and wait for it to finish const run = await client.actor("fetch_cat/devto-articles-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "tag", "tags": [ "javascript", "webdev", ], "usernames": ["ben"], "articleUrls": [{ "url": "https://dev.to/ben/meme-monday-590l" }], "maxItems": 20, } # Run the Actor and wait for it to finish run = client.actor("fetch_cat/devto-articles-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "tag", "tags": [ "javascript", "webdev" ], "usernames": [ "ben" ], "articleUrls": [ { "url": "https://dev.to/ben/meme-monday-590l" } ], "maxItems": 20 }' | apify call fetch_cat/devto-articles-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,fetch_cat/devto-articles-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/oFFU138nvAJdf5Gu1/builds/Igya5yV8zHeAbgpQx/openapi.json