Shopify Scraper: Sale Prices, Compare-at & Stock avatar

Shopify Scraper: Sale Prices, Compare-at & Stock

Pricing

from $0.72 / 1,000 products

Go to Apify Store
Shopify Scraper: Sale Prices, Compare-at & Stock

Shopify Scraper: Sale Prices, Compare-at & Stock

Shopify product scraper for any store: price vs compare-at price, onSale flag, variants, SKUs, stock and images from products.json. Bulk stores, CSV/JSON.

Pricing

from $0.72 / 1,000 products

Rating

0.0

(0)

Developer

MRSLM Data

MRSLM Data

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

8 days ago

Last modified

Features

Limited permissions

Share

Get every public product, price, variant and stock flag from any Shopify store as JSON, CSV or Excel. $0.90 per 1,000 products, no login, no browser.

This Shopify scraper works on any Shopify store you name: you do not need the store's API key or an app install. It is built for one job, finding out what a competitor sells and what is on sale today, so the output puts price, compareAtPrice and an onSale flag side by side.

Point it at one store or a thousand. It reads each store's own public product feed, adds the store's currency, and hands you one clean row per product (or per variant if you want size/colour rows for a spreadsheet).

Quick start

  1. Press Start. The form is already filled with this example input, so the first run returns real rows in under a minute:
{
"stores": [
"allbirds.com",
"kith.com"
],
"maxProductsPerStore": 50
}
  1. Swap in your own values, or call the Actor through the Apify API, Make, Zapier, n8n or the Python/JS clients with the same JSON.
  2. Download the results as JSON, CSV or Excel from the Output tab, or schedule the run to repeat.

Output fields

Every row has: store, productId, handle, title, url, vendor, productType, tags, currency, priceMin, priceMax, compareAtPriceMax, onSale, available, variantCount, options, imageUrl, imageCount, createdAt, updatedAt, publishedAt, scrapedAt, optional description, and a variants list (id, title, sku, price, compareAtPrice, available, option1-3, grams). In variant mode each variant becomes its own row with variantId, variantTitle, sku, price, compareAtPrice, variantAvailable, option1-3 and weightGrams as flat columns.

The full guide below is also published as "How to Monitor a Competitor's Shopify Prices and Stock (Step-by-Step Tutorial with Real Data)".

Who it is for

  • Shopify and DTC merchants: watch competitors' prices, compare-at (sale) prices and size-level stock every morning.
  • Brands and MAP teams: check the price your products are listed at across Shopify retailers, matched on SKU or vendor.
  • Dropshippers and wholesale buyers: pull a supplier's full catalog with SKUs, weights and images into a sheet, and see when items go out of stock.
  • Analysts and AI builders: clean, structured product data from many stores for market research, product matching or a RAG index.

What you can and cannot see in a competitor's Shopify store

Before building anything, it helps to know exactly what the public feed contains, because it decides what you can monitor.

Every Shopify storefront answers GET /products.json unless the merchant has switched it off with an app or a custom proxy. Each product in the feed has:

  • the product ID, handle (the URL slug), title, vendor, product type and tags
  • created, updated and published timestamps
  • a list of variants: each size or colour combination with its own ID, SKU, price, compare-at price (the "was" price), weight and an available flag
  • option names and values (for example Size: 8, 8.5, 9...)
  • image URLs
  • the description as HTML (body_html)

That covers the three things most people want to monitor:

  1. Price. The current selling price of every variant.
  2. Markdowns. When a merchant puts something on sale, Shopify shows the old price as the compare-at price. A compare-at price above the price means "on sale", and the gap is the discount.
  3. Stock. Each variant is either available or not. When a size flips to unavailable, it sold out (or the merchant pulled it).

What the feed does not give you:

  • Exact stock quantities. Shopify only publishes the yes/no flag. Nobody outside the company can see "12 left". Be suspicious of any tool that claims to show a Shopify competitor's inventory counts.
  • Currency. Prices are plain numbers in the store's base currency. The currency code lives in a second file, /meta.json. The Actor reads it for you and adds a currency column to every row.
  • Sales numbers, traffic or reviews. None of that is in the feed. (For reviews of Shopify apps, see our Shopify App Store Reviews Scraper.)
  • Localised prices. Some stores show visitors from other countries converted or different prices. The feed holds the base price.

Two limits of the feed trip up almost everyone who writes their own script:

  1. Page size. The feed returns at most 250 products per request (?limit=250). You have to page with &page=2, &page=3 and so on.
  2. The 25,000 cap. Shopify refuses requests where page × limit goes past 25,000 (HTTP 400). For stores bigger than that, fetch collection by collection (/collections/{handle}/products.json); each collection has its own cap.

A third, less known issue: some stores return a page with fewer than 250 products before the real end of the catalog. A script that stops at the first short page silently loses products, and in a monitoring setup that looks exactly like "products removed". The Actor keeps paging until it gets an empty page, and skips any product ID it has already seen, so a store that ignores the page parameter can never make you pay for duplicates or report fake removals.

The use case: a running-shoe shop watching Allbirds

Say you sell men's casual sneakers and Allbirds is the competitor your customers compare you with. You want to know, every morning:

  • Did they change the price of any men's shoe?
  • Did they start a sale, and how deep is it?
  • Which sizes did they sell out of? (A sold-out size 11 at a competitor is a customer who might buy from you today.)
  • Did they launch or remove a colour?

Allbirds groups its men's shoes in a collection with the handle mens (from the URL allbirds.com/collections/mens). On 28 October 2026 that collection held 146 products. That is the scope we monitor.

How to find a competitor's collection handle: open their store, click the category you care about, and look at the address bar. In https://allbirds.com/collections/mens, the handle is mens. In https://kith.com/collections/mens-footwear, it is mens-footwear. If you want the whole store, leave the collection empty.

Is it a Shopify store at all? Add /products.json to the store's address in your browser. If you see a wall of JSON that starts with {"products":[, it is Shopify and the feed is on. If you get a page not found, it is not Shopify or the feed is off.

Step 1: Choose the exact inputs

Open the Actor and press Try for free. You need a free Apify account; the free plan's monthly platform credit covers thousands of products. Then fill in the form, or switch the input to JSON and paste this:

{
"stores": ["allbirds.com"],
"collection": "mens",
"outputMode": "variant",
"maxProductsPerStore": 0
}

Here is why each field is set the way it is.

stores: the competitor's domain. Any form works: allbirds.com, https://www.allbirds.com, or any page URL from the store (only the domain is used). You can list several competitors in one run, one per line. Each store is paged separately and politely: at most a few requests at a time and about one request every 2 seconds per store.

collection: the part of the catalog you compete on. Watching only mens keeps the run small and cheap, and avoids alerts about products you do not sell. It is also the way around the 25,000-product cap on very large stores: run once per collection.

outputMode: variant for stock tracking. This is the most important choice in a monitoring setup:

  • product (the default) gives one row per product: priceMin, priceMax, compareAtPriceMax, onSale, available (true if any size is in stock), variantCount, plus the full variants list nested inside the row.
  • variant gives one row per size or colour: variantId, variantTitle, sku, price, compareAtPrice, variantAvailable, option1–option3 and weightGrams as flat columns, plus the product columns repeated on each row.

If you only care whether a shoe is on sale, product mode is enough and costs about 14 times less for a shoe store (one row per product instead of one per size). If you care which sizes sold out, you need variant mode, because a product stays "available" as long as a single size is left. In our run, 141 of 346 sizes were sold out while 24 of the 25 products still said available: true.

maxProductsPerStore: 0 for monitoring. Zero means "the whole collection". Do not cap a monitoring run at, say, 50 products. The feed does not always come back in the same order, and a capped run compares a different slice each day. We saw this happen for real: on Kith (another Shopify store) the first 50 products of the feed on 28 October and on 29 October had no products in common, because Kith's feed starts with its newest arrivals and Kith adds products every day. A capped comparison would have reported 50 "removed" and 50 "new" products that never changed. Cap with maxResults only as a cost safety net, set well above the collection size.

Optional fields:

  • maxResults: a hard cap on rows saved (and paid for) across all stores. The run stops cleanly when it is reached.
  • includeDescription: adds the description as plain text, cut at 2,000 characters. Leave it off for price monitoring; it only makes the dataset bigger.
  • proxyConfiguration: not needed for most stores. Turn on Apify Proxy only if a store rate-limits you.

Step 2: Run it and read the real output

Press Start. For this tutorial we ran a 25-product slice of the same collection first (the same input with "maxProductsPerStore": 25) so the output is easy to read. Result on 29 October 2026: 25 products became 346 variant rows in 4.4 seconds, charged as 346 product events. That run cost $0.31 plus the $0.00005 start fee.

One real row, exactly as it came out (the tags and options arrays are shortened here):

{
"store": "allbirds.com",
"productId": 7199602016336,
"handle": "mens-wool-cruiser",
"title": "Men's Wool Cruiser - Natural Grey (Natural White Sole)",
"url": "https://allbirds.com/products/mens-wool-cruiser",
"vendor": "Allbirds",
"productType": "Shoes",
"tags": ["allbirds::gender => mens", "allbirds::material => wool", "allbirds::price-tier => msrp", "..."],
"currency": "USD",
"priceMin": 105,
"priceMax": 105,
"compareAtPriceMax": null,
"onSale": false,
"available": true,
"variantCount": 14,
"options": [{ "name": "Size", "values": ["8", "8.5", "9", "...", "15"] }],
"imageUrl": "https://cdn.shopify.com/s/files/1/1104/4168/files/A11724_25Q3_Wool-Cruiser-Natural-Grey-Natural-White-Sole_PDP_LEFT__1.png?v=1754504830",
"imageCount": 5,
"createdAt": "2025-07-23T13:05:56-07:00",
"updatedAt": "2026-10-29T09:38:50-07:00",
"publishedAt": "2026-09-25T15:55:10-07:00",
"scrapedAt": "2026-10-29T16:38:51.123Z",
"variantId": 41243415609424,
"variantTitle": "8",
"sku": "A11723M080",
"price": 105,
"compareAtPrice": null,
"variantAvailable": true,
"option1": "8",
"option2": null,
"option3": null,
"weightGrams": 685
}

How to read it:

  • variantId is the key for monitoring. It stays the same for the life of that size and colour, so it is what you match day to day. Product titles and handles can change; variant IDs normally do not.
  • price 105, compareAtPrice null: full price, not on sale. If this shoe goes on sale, you will see something like price: 79, compareAtPrice: 105, and onSale turns true.
  • variantAvailable true: size 8 is in stock. For this exact shoe, sizes 13.5, 14 and 15 were false on both 28 and 29 October.
  • updatedAt is Shopify's own "last edited" time for the product. It changes for many reasons (inventory sync, tags, images), so do not use it alone as a "price changed" signal. Compare the actual price fields instead.
  • tags often carry merchant-internal information. Here Allbirds tags the price tier (msrp), material and gender, which is handy for filtering in a sheet.

The same data as a spreadsheet, using the CSV export with only the columns that matter for monitoring (real first rows):

storetitlevariantTitleskupricecompareAtPricevariantAvailable
allbirds.comMen's Wool Cruiser - Natural Grey (Natural White Sole)8A11723M080105true
allbirds.comMen's Wool Cruiser - Natural Grey (Natural White Sole)8.5A11723M085105true
allbirds.comMen's Wool Cruiser - Natural Grey (Natural White Sole)9A11723M090105true
allbirds.comMen's Wool Cruiser - Natural Grey (Natural White Sole)9.5A11723M095105true
allbirds.comMen's Wool Cruiser - Natural Grey (Natural White Sole)10A11723M100105true

What one run already tells you

Even before you compare two days, a single variant-mode run is useful competitive intelligence. From those 346 rows:

  • Pricing is flat by model. Every Wool Cruiser and Cruiser colour was $105, every Tree Runner NZ was $100, and nothing in this slice was marked down. Allbirds' men's sale items sit elsewhere in the catalog.
  • 41% of sizes were sold out. 141 of 346 size rows had variantAvailable: false.
  • The gaps are at the edges of the size run. Size 13.5 was sold out in 19 of 24 colours, size 15 in 18 of 22 and size 8 in 18 of 25, while size 10 was out in only 4 of 25. If you stock big and small sizes, that is a gap you can advertise into.
  • One colour was completely gone but still listed. Men's Tree Runner NZ - Natural White had all 14 sizes unavailable. In product mode that row shows available: false, a typical sign of a product about to be dropped or waiting for a restock.

And the default product-mode run over two stores ({"stores": ["allbirds.com", "kith.com"], "maxProductsPerStore": 50}) on 29 October returned 100 products in 4.2 seconds: 13 of the 50 Allbirds products were on sale, and Kith had one deep markdown in its newest 50, the GUIZIO Low Rise Paillette Skirt - Chocolate at $89.40 with a compare-at price of $298 (70% off).

Step 3: Save it as a Task

A Task is a saved input for an Actor. Monitoring needs one, because schedules, integrations and the "last run" links below all point at a Task.

  1. In the Actor's input page, after setting the JSON from Step 1, click Save as a new task (or Actions → Create task).
  2. Give it a clear name such as allbirds-mens-prices. Its API name becomes your-username~allbirds-mens-prices.
  3. In the Task's Settings, you can lower the memory to 512 MB if you like. The run reads JSON only; it does not need a browser.
  4. Optional but recommended: in the run options, set Maximum cost per run (for example $1.00). The Actor stops cleanly at that budget and you are never billed past it.

To monitor several competitors, either list them all in one Task's stores field (one dataset, a store column tells them apart), or make one Task per competitor (cleaner sheets and separate alerts). If they need different collections, use separate Tasks, because collection applies to every store in a run.

Step 4: Schedule a daily run

  1. In Apify Console, open Schedules and click Create new.
  2. Set the frequency with a cron expression. 0 7 * * * runs every day at 07:00. Pick your own timezone in the schedule settings so "7 in the morning" means your morning.
  3. Under Actors and tasks, add the Task allbirds-mens-prices.
  4. Save. Each run creates a fresh dataset, so you keep a full history: every past day is still there to compare against, for as long as your plan keeps run data.

How often should you run it? Daily is right for most stores; prices on Shopify rarely change more than once a day, and a markdown usually lasts days. Hourly makes sense only around big sale events (Black Friday, end-of-season sales) or for limited releases that sell out fast. Remember that each run is paid per row, so an hourly variant-mode run costs 24 times a daily one. A good pattern is a daily schedule all year plus a temporary hourly schedule for the week of Black Friday.

Step 5: Export to CSV and Google Sheets

You have four ways to get the rows out, from simplest to most automated.

Option A: download a CSV by hand

Open a run, go to the Output tab, and click Export. Choose CSV, Excel, JSON, XML or HTML. In the export dialog you can pick only the columns you need, which keeps the file small. For monitoring, these are enough: store, title, variantTitle, sku, price, compareAtPrice, variantAvailable, url, variantId.

Apify can always give you the dataset of a Task's latest successful run at one fixed address:

https://api.apify.com/v2/actor-tasks/YOUR_USERNAME~allbirds-mens-prices/runs/last/dataset/items?status=SUCCEEDED&format=csv&clean=1&fields=store,title,variantTitle,sku,price,compareAtPrice,variantAvailable,url,variantId

Open it with your API token in the Authorization: Bearer header (from code), or add &token=... for tools that only take a URL. The fields parameter selects and orders the columns; clean=1 drops empty and hidden fields. We tested the same call on a run's dataset; the first lines come back exactly as the table in Step 2:

"store","title","variantTitle","sku","price","compareAtPrice","variantAvailable","url"
"allbirds.com","Men's Wool Cruiser - Natural Grey (Natural White Sole)","8","A11723M080","105",,"true","https://allbirds.com/products/mens-wool-cruiser"
"allbirds.com","Men's Wool Cruiser - Natural Grey (Natural White Sole)","8.5","A11723M085","105",,"true","https://allbirds.com/products/mens-wool-cruiser"

Google Sheets in one formula: put the link (with &token=) in a cell and use =IMPORTDATA(A1) in another sheet. Sheets refreshes it about every hour. Be careful: anyone who can open the sheet can read the token in the cell. Only do this with a sheet you do not share, and treat that token like a password; you can replace it in Apify Console under Settings → API & Integrations at any time.

Option C: push rows into Google Sheets after every run

For a sheet you share with your team, do not put a token in it. Instead, let Apify push the rows:

  • Make or Zapier: use the Apify app. Trigger "Watch Actor runs" / "Finished Task run" on allbirds-mens-prices, then "Get dataset items", then Google Sheets "Add rows" (or "Bulk add rows"). Make and Zapier both have ready-made templates for "Apify dataset to Google Sheets".
  • n8n: the Apify node ("Run task" or a webhook trigger) followed by the Google Sheets node.
  • Inside Apify: the Google Sheets Import & Export Actor can run after your Task (add it as an integration on the Task) in replace mode to keep one sheet tab equal to the latest run, or append mode to build a history tab. It asks you to connect your Google account on first use.

A practical layout is two tabs: today (replaced on every run) and yesterday (filled by copying today's tab before the replace, or by importing the previous run's dataset). That is all the change formulas below need.

Option D: pull it from code

The Apify Python client (pip install apify-client, version 3.x) takes the same JSON input as the form:

import os
from apify_client import ApifyClient
client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("mrslm_data/shopify-products").call(run_input={
"stores": ["allbirds.com"],
"collection": "mens",
"outputMode": "variant",
"maxProductsPerStore": 0,
})
for item in client.dataset(run.default_dataset_id).iterate_items():
print(item["title"], item["variantTitle"], item["price"], item["variantAvailable"])

In apify-client 3.x a run is an object, so use run.default_dataset_id. Older 1.x/2.x examples written as run["defaultDatasetId"] raise an error in 3.x. The JavaScript client (apify-client on npm) works the same way with client.actor(...).call(input) and client.dataset(run.defaultDatasetId).listItems().

Step 6: Find what changed since yesterday

A daily copy of a competitor's catalog is nice. A daily list of only the changes is what you actually read. Every change you care about comes down to comparing two rows with the same variantId:

changerule (yesterday → today)
price dropprice went down
price riseprice went up
new markdown / markdown endedcompareAtPrice changed (empty → a number means a sale started; a number → empty means it ended)
sold outvariantAvailable true → false
back in stockvariantAvailable false → true
new product or colourvariantId is in today's data but not yesterday's
removedvariantId was in yesterday's data but is gone today

In Google Sheets

With the two tabs from Option C (columns in the same order, variantId in column I), add these formulas in the today tab, row 2, and fill them down:

J2 (yesterday's price): =IFERROR(XLOOKUP($I2, yesterday!$I:$I, yesterday!$E:$E), "NEW")
K2 (change): =IF(J2="NEW", "new variant",
IF(E2 < J2, "price drop",
IF(E2 > J2, "price rise",
IF(F2 <> XLOOKUP($I2, yesterday!$I:$I, yesterday!$F:$F), "compare-at changed",
IF(G2 <> XLOOKUP($I2, yesterday!$I:$I, yesterday!$G:$G), IF(G2, "back in stock", "sold out"), "")))))

Columns here follow the export order from Option A: A store, B title, C variantTitle, D sku, E price, F compareAtPrice, G variantAvailable, H url, I variantId. Then filter column K to non-empty values, or add conditional formatting (red for "sold out", green for "price drop"). For "removed" items, put =IF(ISNA(MATCH($I2, today!$I:$I, 0)), "removed", "") in the yesterday tab and fill it down.

In Python

This script compares the last two successful runs of your Task and writes every change to changes.csv. It needs only pip install apify-client and your token in the APIFY_TOKEN environment variable:

"""Compare the last two successful runs of a saved Apify Task (shopify-products, variant mode)
and write every price, markdown and stock change to changes.csv. Needs: pip install apify-client (3.x)."""
import csv, os, sys
from apify_client import ApifyClient
client = ApifyClient(os.environ["APIFY_TOKEN"])
TASK = sys.argv[1] # your task, e.g. "your-username~allbirds-mens-prices"
runs = client.task(TASK).runs().list(desc=True, limit=2, status="SUCCEEDED").items
if len(runs) < 2:
sys.exit("Run the task at least twice first.")
new_run, old_run = runs[0], runs[1]
def load(run):
return {r["variantId"]: r for r in client.dataset(run.default_dataset_id).iterate_items()}
new, old = load(new_run), load(old_run)
changes = []
for vid, n in new.items():
name = f'{n["title"]} / {n["variantTitle"]}'
o = old.get(vid)
if o is None:
changes.append(("new variant", name, "", n["price"], n["url"]))
continue
if n["price"] != o["price"]:
kind = "price drop" if n["price"] < o["price"] else "price rise"
changes.append((kind, name, o["price"], n["price"], n["url"]))
if (n["compareAtPrice"] or 0) != (o["compareAtPrice"] or 0):
changes.append(("compare-at changed", name, o["compareAtPrice"], n["compareAtPrice"], n["url"]))
if n["variantAvailable"] != o["variantAvailable"]:
kind = "back in stock" if n["variantAvailable"] else "sold out"
changes.append((kind, name, o["variantAvailable"], n["variantAvailable"], n["url"]))
for vid, o in old.items():
if vid not in new:
changes.append(("removed", f'{o["title"]} / {o["variantTitle"]}', o["price"], "", o["url"]))
with open("changes.csv", "w", newline="") as f:
w = csv.writer(f)
w.writerow(["change", "product / variant", "before", "after", "url"])
w.writerows(sorted(changes, key=lambda c: (c[0], c[1])))
print(f"{len(changes)} changes between {old_run.started_at:%Y-%m-%d %H:%M} and {new_run.started_at:%Y-%m-%d %H:%M} UTC")

Run it with python shopify_price_diff.py your-username~allbirds-mens-prices. We tested it against two real runs of the 25-product slice taken two minutes apart on 29 October; it correctly printed 0 changes, which is what you should see most of the time on a short gap. If you run it every morning after the schedule, an empty file means "nothing to do today" and a non-empty one is your to-do list. Swap the CSV for a Slack or email message if you want alerts instead of a file.

If you need the changes at product level only (product mode), use productId as the key and compare priceMin, priceMax, compareAtPriceMax, onSale and available instead.

What a real day-over-day comparison found

We compared the default two-store run from 28 October 2026 with the same input on 29 October 2026, about 24 hours apart:

  • Allbirds (50 products): the same 50 products both days, no price changes, no new or ended markdowns, and one sell-out: Women's Wool Runner Mizzle - Natural Grey (Cream Sole), on sale at $62 (compare-at $125), went from available: true to available: false. The last sizes of a half-price shoe selling out in a day is exactly the kind of signal this setup exists for.
  • Kith (50 products): zero products in common between the two days, because Kith's feed starts with its newest products and the run was capped at 50. This is the trap described in Step 1. For a store like Kith, monitor a fixed collection with maxProductsPerStore: 0.
  • Allbirds men's slice (variant mode, 42 size rows of 3 shoes on 28 October vs. the same sizes on 29 October): no price or stock changes at all.

Most days look like that: almost nothing moves, and then a markdown or a run of sell-outs appears. That is why reading only the changes, not the whole catalog, matters.

Don't want to write the comparison yourself?

Our Shopify & WooCommerce Price Drop & Restock Watcher does Step 6 for you. It keeps its own copy of the last run and saves only the change rows (price drops, restocks, sold-outs, new and removed products), so a schedule gives you a dataset of changes directly. Use this Shopify Scraper when you want the full catalog every day (for a price history, reporting or a sheet), and the watcher when you only want alerts.

What it costs

You pay per row saved: $0.90 per 1,000 products (Apify Silver plan $0.81, Gold $0.72), plus a $0.00005 start fee per run. In variant mode each variant row counts as one product. Stores that are not Shopify, or that have their feed turned off, cost nothing.

Price per event

Current pay-per-event prices for this Actor on Apify, by Apify subscription plan:

Event (per 1,000)Free / BronzeSilverGold / Platinum / Diamond
Product (product)$0.90$0.81$0.72
Actor start (once per run at the default memory, not per 1,000)$0.00005$0.00005$0.00005

You are charged only for the events in this table: nothing for compute, and nothing for searches that return no rows beyond the start fee.

For the Allbirds setup in this tutorial:

setuprows per runcost per runcost per month (daily)
men's collection, product mode146$0.13about $4
men's collection, variant modeabout 2,000 (146 products × 13.8 sizes on average)about $1.80about $54
25-product slice, variant mode (our test)346$0.31about $9

Other worked examples: 3 competitors × 2,000 products in product mode = 6,000 rows = $5.40 per run. The default two-store example (100 products) costs $0.09.

Ways to keep it cheap:

  • Use product mode unless you really need size-level stock. It is about 14 times cheaper for shoes and apparel.
  • Watch the collection you compete in, not the whole store.
  • Run daily, not hourly, except during big sales.
  • Set Max results and Maximum cost per run as a safety net.

Shopify scraper vs. doing it by hand

Checking product pages by eye works for five products. It does not work for size-level stock on 146 shoes every morning, and people miss things.

Opening products.json in a browser works for a quick look at a small store. It stops being practical when you need more than 250 products (manual paging), more than one store, a CSV, or anything on a schedule. You also get prices with no currency.

Writing your own Python script is a reasonable weekend project. The loop is simple; the edge cases are what take time:

  • stopping correctly (empty page, not short page) and handling the 25,000 cap
  • stores on custom domains, www redirects and stores that turned the feed off
  • adding currency from /meta.json
  • flattening variants into rows and handling compare_at_price that is null, "0.00" or lower than the price
  • polite rate limits so you are not blocked, retries on timeouts, robots.txt
  • somewhere to run it every day, store every day's results and compare them

Using the Actor costs $0.90 per 1,000 products, and those cases are already handled. It runs in the cloud on a schedule and connects to Sheets, Make, Zapier and n8n. If you only need one export, once, from one small store, the free browser method is fine and you do not need us.

Other things people monitor with Shopify product data

  • MAP and brand protection. Brands check the price their products are listed at across many Shopify retailers, matching on sku or vendor.
  • Assortment and launches. Count products by productType and vendor, see which brands a retailer carries, and spot new launches by publishedAt or new variantIds.
  • Dropshipping and wholesale buying. Pull a supplier's catalog with SKUs, weights and images straight into a sheet, and get told when a supplier's item goes out of stock before your customers order it.
  • Resale and limited releases. Watch a collection hourly during a drop and see which sizes sell out first.
  • Market research and AI datasets. Clean, structured product data for analysis, product matching or a RAG index, without writing or maintaining a scraper.

Limits

  • Only Shopify stores that serve their public product feed work. Stores that turned it off, or are not Shopify, are reported in the run summary and cost nothing.
  • Shopify stops paging at 25,000 products per feed. For bigger stores, run once per collection.
  • Stock is in stock / out of stock per variant. Exact unit counts are private to the merchant.
  • Prices are in the store's base currency. Prices shown to visitors in other countries (Shopify Markets) can differ.
  • The feed has no sales figures, reviews or traffic data.
  • Each run is a snapshot. To see what changed, compare two runs as shown in Step 6.

FAQ

Is this a Shopify scraper for one store or many? Both. Put one store or hundreds in stores; each store is read in turn and every row carries the store's domain, so one dataset can hold a whole competitor set.

Is there an official Shopify API for watching other stores? No. Shopify's Admin and Storefront APIs need the store owner's access token. The public products.json feed is the only open source of a store's catalog, and it is what this Actor reads.

Which stores work? Any Shopify storefront that serves its public product feed, which is most of them. If a store turned it off, the run says so for that store in its RUN_SUMMARY record ("Not a Shopify store, or its public product feed is turned off") and carries on with the others. Failed stores cost nothing.

Can I see how many units a competitor has left? No. Shopify only publishes an in-stock yes/no per variant. Exact counts are private to the merchant.

Will the competitor know I am watching? The Actor reads the same public feed every visitor's browser and many apps read, at a polite rate (about one request every 2 seconds per store). It does not log in, fill carts or place orders.

How do I monitor a store with more than 25,000 products? Shopify blocks paging past 25,000 products in one feed. Run once per collection (mens, womens, sale and so on). Each collection has its own 25,000 limit.

Why did my comparison show lots of new and removed products that did not really change? Almost always because the run was capped (maxProductsPerStore above 0) on a store whose feed order changes. Set it to 0 and limit the scope with collection instead.

Are prices in USD? Prices are in the store's base currency, and the currency column (read from /meta.json) tells you which: USD, EUR, GBP and so on. Prices shown to visitors in other countries can differ from the base price in the feed.

How fast is it? In our runs, 100 products from two stores took 4.2 seconds and 346 variant rows took 4.4 seconds. Large catalogs page at 250 products per request, about one request every 2 seconds per store.

Does it work with Make, Zapier, n8n or Google Sheets? Yes. Use the Apify app in Make, Zapier or n8n ("Run Actor" or "Run task", then "Get dataset items"), the Google Sheets Import & Export Actor as a Task integration, or the CSV link in Step 5.

How do I keep costs predictable? Set Max results, or Maximum cost per run in the run options. The Actor stops cleanly at the cap and you are never billed past it.

Is it legal to monitor a competitor's prices? Reading prices a store publishes to every visitor is standard competitive research. The Actor only reads the public feed without a login, follows robots.txt, keeps a polite request rate and collects no personal data. You are responsible for how you use the data: follow the site's terms, copyright and the privacy laws that apply to you.

Can AI agents use it through MCP? Yes. Apify's hosted MCP server can expose this Actor as a tool to any MCP client (Claude Desktop, Cursor, VS Code and others). Add https://mcp.apify.com/?tools=mrslm_data/shopify-products as a remote MCP server and sign in with your Apify account or API token. The agent fills in the same input fields shown above and gets the same rows back, billed at the same per-event prices.

This Actor only reads pages and feeds that the website itself publishes to every visitor without a login. It obeys each site's robots.txt, keeps a polite request rate, and never tries to get past logins, paywalls, captchas or bot checks. It does not collect personal data such as emails, phone numbers or names of private individuals. You are responsible for how you use the data: follow the source site's terms, copyright and the privacy laws that apply to you (GDPR, CCPA, etc.). This tool is independent and not affiliated with or endorsed by the platforms it reads.

⌂ 换网址