# Socrata Open Data Scraper: SODA datasets to CSV/JSON (`datamule/socrata-soda-extractor`) Actor

Export rows from any Socrata open-data portal (NYC, Chicago, WA, SF) by domain + dataset ID, with optional SoQL filters. Also lists a portal's datasets or a dataset's columns. Returns JSON/CSV/Excel rows, no API key. $0.0005 per record (about $0.50 per 1,000).

- **URL**: https://apify.com/datamule/socrata-soda-extractor.md
- **Developed by:** [Datamule](https://apify.com/datamule) (community)
- **Categories:** Developer tools, Business, Other
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 79.6% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.35 / 1,000 dataset items

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Socrata Open Data Scraper: export any city or state SODA dataset

Exports rows from any Socrata open-data portal (NYC, Chicago, Washington, San Francisco and hundreds more) without an API key. Give it a portal domain and a dataset ID, optionally add SoQL filters, and get clean JSON/CSV/Excel rows. It can also list every dataset on a portal, or return a dataset's column schema. Built for analysts, journalists, civic-tech developers and AI agents that need public-sector data. **Price: $0.0005 per returned record (about $0.50 per 1,000; lower on paid Apify plans).**

Running it with an empty input `{}` returns 1,000 recent NYC 311 service requests (about $0.50 at the Free-plan price). Set `maxRecords` first if you want to spend less.

### Example input

```json
{ "mode": "rows", "domain": "data.cityofnewyork.us", "datasetId": "erm2-nwe9",
  "where": "complaint_type = 'Noise - Residential'", "order": "created_date DESC", "maxRecords": 100 }
```

### Example output (real run, mode `rows`)

```json
{
  "_domain": "data.cityofnewyork.us",
  "_datasetId": "erm2-nwe9",
  "_mode": "rows",
  "data": {
    "unique_key": "70691427",
    "created_date": "2026-10-09T02:56:50.000",
    "agency": "DOT",
    "complaint_type": "Street Condition",
    "descriptor": "Pothole",
    "incident_zip": "11415",
    "status": "Open",
    "borough": "QUEENS"
  }
}
```

(Trimmed: `data` holds every column the portal returns for that row.)

Real `catalog` output (one record per dataset): `{"_domain":"data.cityofchicago.org","_mode":"catalog","datasetId":"ydr8-5enu","name":"Building Permits","type":"dataset","category":"Buildings","updatedAt":"2026-10-10T12:06:20.000Z","permalink":"https://data.cityofchicago.org/d/ydr8-5enu","link":"https://data.cityofchicago.org/Buildings/Building-Permits/ydr8-5enu"}`

### Modes

| `mode` | Returns | Needs |
|---|---|---|
| `rows` (default) | One record per dataset row; row columns are inside `data` | `domain`, `datasetId` |
| `catalog` | One record per dataset on the portal | `domain` (optional `q`, `category`) |
| `metadata` | One record with the dataset's columns (name, fieldName, type) | `domain`, `datasetId` |

### Input

| Field | Description |
|---|---|
| `mode` | `rows`, `catalog` or `metadata`. Default `rows`. |
| `domain` | Portal host, e.g. `data.cityofnewyork.us`. Default `data.cityofnewyork.us`. |
| `datasetId` | The `xxxx-xxxx` code in the dataset URL, e.g. `erm2-nwe9`. Default `erm2-nwe9`. |
| `select`, `where`, `order` | Optional SoQL clauses (rows mode). |
| `q` | Full-text search (rows) or keyword filter (catalog). |
| `category` | Dataset category filter (catalog mode). |
| `maxRecords` | Stop after N records; default 1000; `0` = no limit (can be millions of rows, set a cap first). |
| `pageSize` | Rows per request, 1-50000; default 1000. |
| `appToken` | Optional Socrata app token; never required. |

### Output fields

- All modes: `_domain`, `_mode`.
- rows: `_datasetId`, `data` (object with the dataset's own columns).
- catalog: `datasetId`, `name`, `description`, `type`, `category`, `updatedAt`, `createdAt`, `permalink`, `link`, `pageViewsTotal` (when the portal reports it). Note: the catalog `rowCount` field is derived from the portal's column list, so treat it as a column count, not a row count.
- metadata: `datasetId`, `name`, `description`, `category`, `attribution`, `createdAt`, `rowsUpdatedAt`, `viewCount`, `downloadCount`, `columnCount`, `columns[]`.

### Pricing

Pay per event: one `dataset-item` event of $0.0005 per record returned (Free plan; lower on Bronze and above), whether it is a row, a catalog entry or a metadata record. No charge for runs that return nothing.

### Limits

- Public datasets only; private or login-required datasets fail. Socrata (SODA2) portals only.
- A run that returns no records fails with exit code 91.
- Unauthenticated portals may rate-limit; the Actor retries with backoff. Add `appToken` or lower `pageSize` if it persists.
- Default memory is 256 MB (measured peak about 76 MB).

#### Maximum charge per run

If you set a maximum total charge, each page request asks the portal only for what the remaining budget and `maxRecords` can cover. When the budget runs out the run stops with the records already delivered and logs "Max total charge reached". Each record is written and billed in one step, and a restarted run resumes from the saved offset, so nothing is written or billed twice.

### Use from an AI agent / API

Python:

```python
from apify_client import ApifyClient
client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("datamule/socrata-soda-extractor").call(
    run_input={"domain": "data.cityofchicago.org", "datasetId": "ijzp-q8t2",
               "order": "date DESC", "maxRecords": 50})
rows = list(client.dataset(run["defaultDatasetId"]).iterate_items())
```

MCP (Claude, Cursor and other agents): add `https://mcp.apify.com?tools=datamule/socrata-soda-extractor` as a remote MCP server with your Apify token. The Actor then appears as a tool whose arguments are the Input fields above.

Agent tips: to find a dataset ID, run `mode: "catalog"` with `q`; to learn column names before writing `where`, run `mode: "metadata"`; always set `maxRecords`.

### FAQ

**Do I need a Socrata account or API key?** No.

**Where do I find the dataset ID?** It is the `xxxx-xxxx` code at the end of the dataset URL, or use `catalog` mode.

**Why are my rows nested under `data`?** Every portal has different columns, so the row's own columns are kept together under `data`.

**Can I get everything?** Set `maxRecords` to `0`. Large datasets cost proportionally ($0.50 per 1,000 rows).

**Does it work on non-US portals?** Any portal running the Socrata SODA API works; give its host as `domain`.

# Actor input Schema

## `mode` (type: `string`):

What to return. "rows" (default) = records of one dataset, needs domain + datasetId. "catalog" = one row per dataset published on the domain (needs only domain). "metadata" = one record with a dataset's column names and types (needs domain + datasetId). Example: "rows".

## `domain` (type: `string`):

Hostname of the Socrata open-data portal, no path. Required. Scheme and path are stripped if pasted. Example: "data.cityofnewyork.us" (others: data.cityofchicago.org, data.wa.gov, data.sfgov.org). Defaults to data.cityofnewyork.us so an empty input returns real data.

## `datasetId` (type: `string`):

Dataset ID: the 9-character xxxx-xxxx code at the end of the dataset URL on the portal. Needed for rows and metadata modes, ignored in catalog mode. Example: "erm2-nwe9" (NYC 311 service requests). Defaults to erm2-nwe9.

## `select` (type: `string`):

Optional SoQL $select: comma-separated column names to keep (rows mode). Leave empty for all columns. Example: "unique_key, created_date, complaint_type, borough".

## `where` (type: `string`):

Optional SoQL $where filter (rows mode). Use column field names from metadata mode; strings in single quotes. Example: "complaint_type = 'Noise - Residential' AND created_date > '2026-01-01T00:00:00'".

## `order` (type: `string`):

Optional SoQL $order sort (rows mode). Add DESC for newest first. Example: "created_date DESC". Without it row order is the portal's default.

## `q` (type: `string`):

Optional full-text search. Rows mode: matches any text column (SoQL $q). Catalog mode: keyword filter on dataset names/descriptions. Example: "pothole".

## `category` (type: `string`):

Optional catalog-mode filter on the portal's dataset category; ignored in rows/metadata. Must match the portal's own category label exactly. Example: "Public Safety".

## `maxRecords` (type: `integer`):

Maximum records to return and be charged for. Default 1000 (= about $0.50). 0 = no limit, fetch the whole dataset (can be millions of rows, so set a cap first). Example: 100.

## `pageSize` (type: `integer`):

Rows per request to the portal, 1-50000. Default 1000 is fine; lower it only if the portal rate-limits. Example: 1000.

## `appToken` (type: `string`):

Optional Socrata app token for higher portal rate limits. Not needed for normal use. Stored as a secret. Example: leave empty.

## Actor input object example

```json
{
  "mode": "rows",
  "domain": "data.cityofnewyork.us",
  "datasetId": "erm2-nwe9",
  "select": "unique_key, created_date, complaint_type, borough",
  "where": "complaint_type = 'Noise - Residential'",
  "order": "created_date DESC",
  "q": "pothole",
  "category": "Public Safety",
  "maxRecords": 100,
  "pageSize": 1000
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domain": "data.cityofnewyork.us",
    "datasetId": "erm2-nwe9"
};

// Run the Actor and wait for it to finish
const run = await client.actor("datamule/socrata-soda-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "domain": "data.cityofnewyork.us",
    "datasetId": "erm2-nwe9",
}

# Run the Actor and wait for it to finish
run = client.actor("datamule/socrata-soda-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domain": "data.cityofnewyork.us",
  "datasetId": "erm2-nwe9"
}' |
apify call datamule/socrata-soda-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,datamule/socrata-soda-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/MAsq5cA192TWKaqg7/builds/V8V22fZkGtFhQVazT/openapi.json