# Shopify API Scraper (`rl1987/shopify-api-scraper`) Actor

API scraper to get product data from almost any Shopify store!

- **URL**: https://apify.com/rl1987/shopify-api-scraper.md
- **Developed by:** [R.L.](https://apify.com/rl1987) (community)
- **Categories:** E-commerce
- **Features:** Limited permissions
- **Stats:** 19 total users, 1 monthly users, 100.0% runs succeeded, 3 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.25 / 1,000 product variant rows

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### Shopify API Scraper

Most eCommerce stores based on the Shopify platform expose a RESTful API to get
product data, e.g. the sneaker boutique [Social Status](https://socialstatuspgh.com):

- https://socialstatuspgh.com/products.json — the whole catalogue
- https://socialstatuspgh.com/collections/sneakers/products.json — one collection

Other sneaker and apparel stores that work the same way:

- https://solefly.com/products.json
- https://www.allbirds.com/products.json
- https://pelacase.myshopify.com/products.json

Note, however, that in some cases this trick won't work with primary store
domain. In that case one should inspect web site source with View Source
feature of Chrome browser and look for `*.myshopify.com` subdomain being
referenced in JavaScript code, e.g.:

```javascript
Shopify.shop = "pelacase.myshopify.com";
```

#### Proxy — required in practice

Shopify rate-limits `products.json` **per client IP, across stores**: once the
limiter trips, unrelated storefronts start answering
`HTTP 429 {"errors":[{"message":"You have sent too many requests..."}]}`.
Measured on one host, 9 of 18 unrelated stores returned 429 after a short burst,
and the same request was 429 through Apify's **datacenter** pool while
succeeding through **residential**:

```
direct       HTTP 200  products=250
datacenter   HTTP 429
residential  HTTP 200  products=250
```

So the scraper takes a **fresh exit IP for every request** and defaults to
Apify Proxy residential. A full 10-page catalogue crawl then completes with no
429s at all. You can switch pools or disable the proxy in the input, but
expect throttling past a few pages without one.

#### Input example

```json
{
  "start_urls": [
    {
      "url": "https://socialstatuspgh.com/products.json"
    }
  ],
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}
```

#### Output example

This scraper outputs one row per product variant, e.g.:

```json
{
  "product_id": 7985713840170,
  "title": "THE WHITAKER GRP SUEDE CODA ECHO POINTS (CHAMBRAY BLUE/ALPINE SNOW/BUTTERCREAM)",
  "description": "The Whitaker Grp introduces PUMA CODA a fresh take on an everyday classic. Crafted from soft premium suede, this low-top sneaker pairs a clean neutral upper with textured pink terry laces for an unexpected pop of color. Designed with subtle contrast stitching and a durable rubber sole, it delivers effortless comfort while making a statement with every step.\n\nDetails\n\n\nPremium suede upper\nPlush terry laces\nLow-top silhouette\nContrast stitching throughout\nCushioned footbed for all-day comfort\nDurable rubber outsole\nTrue to size\n\n\nSKU: 40874401",
  "handle": "the-whitaker-grp-coda-echo-points-chambray-blue-alpine-snow-buttercream",
  "web_url": "https://socialstatuspgh.com/products/the-whitaker-grp-coda-echo-points-chambray-blue-alpine-snow-buttercream",
  "vendor": "PUMA CODA",
  "product_type": "SNEAKERS",
  "tags": [
    "81026",
    "CODA",
    "FinalSale",
    "fw26",
    "maincolor-blue/cream",
    "mens",
    "new arrivals",
    "nosale",
    "puma",
    "sneakers"
  ],
  "images": [
    "https://cdn.shopify.com/s/files/1/0262/2475/files/PUMACODABLU-1.jpg?v=1786378574",
    "https://cdn.shopify.com/s/files/1/0262/2475/files/PUMACODABLU-2.jpg?v=1786378574",
    "https://cdn.shopify.com/s/files/1/0262/2475/files/PUMACODABLU-3.jpg?v=1786378574",
    "..."
  ],
  "featured_image": null,
  "variant_id": 44682235478058,
  "variant_title": "4",
  "option1": "4",
  "option2": null,
  "option3": null,
  "variant_sku": "40874401",
  "variant_available": false,
  "price": "130.00",
  "compare_at_price": null
}
```

#### When something goes wrong

The run status tells you whose problem it is, and the status message says what
happened. A run never finishes green and empty without an explanation:

| What happened | Run status | What you do |
|---|---|---|
| No usable `products.json` URL in the input | **Succeeded**, "Nothing to scrape: ..." | fix the URL |
| Store is password-protected, or the endpoint is disabled (401/403/404, or an HTML page) | **Succeeded**, "the catalogue is not public" | nothing — the store does not serve it |
| Rate-limited after 4 attempts on rotating IPs | **Succeeded**, "still failing after 4 attempts" | enable/raise the residential proxy, retry later |
| Run timeout hit mid-catalogue | **Succeeded**, "the run timeout was reached" | raise the run timeout; rows already fetched are kept |
| The endpoint changed shape, or an unexpected bug | **Failed** | report it — this one is ours |

Partial results are always delivered: rows are flushed as pages complete, so a
run that is throttled or cut short still hands over everything it fetched, and a
page that fails does not discard the pages that succeeded. A run that hits any
error also writes an `ERROR_SUMMARY` record to the key-value store with the
counts and a sample of each.

#### Notes

- Some of the heavily customised Shopify stores (e.g. Supreme) do not expose
  the API this scraper relies on.
- It is recommended to disable timeout for larger scraping jobs that span
  across multiple stores or cover large amount of products.

### Global marketplace toolkit

Part of the **Global marketplace toolkit** — product and pricing data across international B2B/B2C marketplaces and classifieds sites:

- [DHgate API Scraper](https://apify.com/rl1987/dhgate-api-scraper) — Scrapes DHgate product search results and detail pages via mobile API.
- [Made-in-China.com API Scraper](https://apify.com/rl1987/made-in-china-api-scraper) — Scrapes product search results and details from Made-in-China.com's mobile app API.
- [Back Market API Scraper](https://apify.com/rl1987/backmarket-api-scraper) — Scrape refurbished-device listings and full product detail from Back Market's API.
- [Senukai.lt API Scraper](https://apify.com/rl1987/senukai-api-scraper) — Scrape Senukai.lt product listings and full detail — prices, discounts, EAN/GTIN, images.
- [Kleinanzeigen API Scraper](https://apify.com/rl1987/kleinanzeigen-api-scraper) — Scrape Kleinanzeigen (Germany) classified-ad listings and full details via mobile API.
- [Algolia API Scraper](https://apify.com/rl1987/algolia-api-scraper) — Generalised scraper for any Algolia-powered search index — scrape hits or discover facets.

### Did you find this useful?

⭐ Rate this actor on Apify! Your feedback helps other users find it and helps us keep improving it.

# Actor input Schema

## `start_urls` (type: `array`):

Shopify <code>products.json</code> endpoints to scrape, one per store or collection. Use <code>https://\<store>/products.json</code> for a whole catalogue, or <code>https://\<store>/collections/\<handle>/products.json</code> for one collection. The scraper pages through the catalogue itself, so no page or limit parameter is needed.

## `proxyConfiguration` (type: `object`):

Shopify rate-limits <code>products.json</code> per client IP <b>across stores</b>, so a proxy is effectively required: the scraper takes a fresh exit IP for every request. Residential is the default because Apify's datacenter pool is measurably rate-limited by Shopify where residential is not. Running with no proxy works for a handful of pages and then starts returning HTTP 429.

## Actor input object example

```json
{
  "start_urls": [
    {
      "url": "https://socialstatuspgh.com/products.json"
    }
  ],
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "start_urls": [
        {
            "url": "https://socialstatuspgh.com/products.json"
        }
    ],
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("rl1987/shopify-api-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "start_urls": [{ "url": "https://socialstatuspgh.com/products.json" }],
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("rl1987/shopify-api-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "start_urls": [
    {
      "url": "https://socialstatuspgh.com/products.json"
    }
  ],
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}' |
apify call rl1987/shopify-api-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,rl1987/shopify-api-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/7DjHCwC6CvWmhd3X0/builds/XVbICuM1KygZQAbHp/openapi.json