# jq Helper – Transform, Filter & Reshape JSON with jq (`rl1987/jq-helper`) Actor

Run any jq 1.8 program over JSON and get the results back as a dataset. Paste inline JSON or JSONL, or point it at another Actor's dataset to add a transform step to a pipeline: map, select, flatten, rename keys, group_by, reduce, unique_by. Inline runs are free. Nothing to install.

- **URL**: https://apify.com/rl1987/jq-helper.md
- **Developed by:** [R.L.](https://apify.com/rl1987) (community)
- **Categories:** Developer tools
- **Features:** Limited permissions
- **Stats:** 79 total users, 28 monthly users, 84.4% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## jq Helper – Transform, Filter & Reshape JSON with jq

**Run any [jq](https://jqlang.github.io/jq/) program over your JSON and get clean, structured results back.** Reshape, filter, flatten, enrich, and aggregate JSON — from a pasted snippet or straight from another Apify Actor's dataset. The fast, no-code way to add a JSON transformation step to any scraping or data pipeline.

> Map · filter · select · flatten · rename keys · merge fields · group & aggregate — all with battle-tested jq 1.8 syntax.

***

### What does jq Helper do?

jq Helper is a tiny, blazing-fast Apify Actor that puts the full power of **jq** — the most popular command-line JSON processor — into a hosted, chainable step. Instead of writing a throwaway script every time a scraper returns messy JSON, drop in a jq filter and let this Actor do the transformation in the cloud.

It's built to slot directly into the **Apify ecosystem**: point it at the output dataset of any scraper and use it as a JSON **post-processing, cleanup, and enrichment** stage in your automation.

### What can it do?

- **Full jq 1.8 support** — the complete language: `select`, `map`, `group_by`, `reduce`, `sort_by`, `unique`, string ops, math, conditionals, and more.
- **Two input sources** — paste **inline JSON / JSONL**, or read items from a **linked Apify dataset** by ID (perfect for chaining Actors).
- **Two apply modes** — transform **each item** independently (mapping/filtering/enrichment) or process the **whole array at once** (aggregations, grouping, dedupe).
- **Pipeline-ready** — writes results to its own dataset so the next Actor or integration (Zapier, Make, Google Sheets, webhooks…) can pick them up.
- **Robust by default** — errors on a single bad record are skipped and logged (or fail-fast if you prefer), and results are pushed in efficient batches.
- **Zero setup** — no servers, no `jq` install, no dependencies to manage.

### What do people use jq Helper for?

- **Clean up scraped data** — drop nulls, rename fields, normalize values, flatten nested objects.
- **Filter datasets** — keep only the records that match your criteria (`select(.price < 100)`).
- **Reshape for export** — turn a verbose API response into a tidy, flat table for CSV/Excel/Sheets.
- **Enrich records** — derive new fields (domains from emails, full location strings, computed flags).
- **Aggregate & summarize** — group by a field, count, average, sum across an entire dataset.
- **Glue Actors together** — sit between a scraper and your destination as a no-code transformation step.

### How do I transform JSON with jq Helper?

1. Choose your data source: paste JSON into **`jsonInput`**, or set **`datasetId`** to an existing dataset (e.g. an upstream scraper's run).
2. Write your transformation in the **`filter`** field using jq syntax.
3. Pick an **`applyMode`** (`perItem` or `wholeInput`).
4. Run it — transformed records land in the Actor's default dataset, ready to export or pass downstream.

#### Per-item vs. whole-input (read this first)

This is the one thing to get right:

| Mode | What the filter receives | Reach a field with |
|------|--------------------------|--------------------|
| **`perItem`** (default) | each array element, one at a time, as a single object | `.websiteUrl` |
| **`wholeInput`** | the entire array as one value | `.[].websiteUrl` |

In **per-item** mode the array is iterated for you, so **don't** prefix with `.[]` — `.[].websiteUrl` will fail with *"Cannot index string with string"*. Use a leading `.[]` only in **whole-input** mode.

### What inputs does it take?

| Field | Type | Description |
|-------|------|-------------|
| `filter` | string (required) | The jq program. jq 1.8 syntax. May emit 0, 1, or many values per input — each becomes one output item. |
| `applyMode` | `perItem` | `wholeInput` | `perItem` (default): run on each element. `wholeInput`: feed the whole array as a single value. |
| `datasetId` | string | Apify dataset to read items from (provide either this **or** `jsonInput`, not both). **Map this field to chain Actors in an integration.** Charged per successful run; see the **Pricing** tab. Your API token must have access to the dataset (owned by you, or explicitly shared) — if it cannot read it, the run ends with no rows and the status message says why. |
| `jsonInput` | string | Raw JSON (object/array/scalar) or JSONL (one value per line). Free — for prototyping. |
| `wrapKey` | string | Non-object outputs are wrapped as `{ <wrapKey>: value }` (dataset items must be objects). Default `value`. |
| `failOnError` | boolean | `true` stops the run and marks it **FAILED** on any jq runtime error; default `false` skips + logs the bad record and delivers the rest. |

### What does the output look like?

Each value emitted by your filter becomes one item in the Actor's default dataset. Object outputs are stored as-is (their keys become columns); scalars and arrays are wrapped under `wrapKey`. Export the dataset as **JSON, CSV, Excel, HTML, or RSS**, or hand it to the next step in your workflow.

### What do common jq filters look like?

**Reshape & build a location string (per-item):**

```jq
{fullName, currentEmployer, location: ([.city, .state, .country] | map(select(.)) | join(", "))}
```

**Keep only verified records (per-item):**

```jq
select(.clearVerified == true)
```

**Pull one field as a clean column (per-item):**

```jq
{websiteUrl}
```

**Group and summarize the whole dataset (whole-input):**

```jq
group_by(.state) | map({state: .[0].state, count: length}) | .[]
```

**Deduplicate by a key (whole-input):**

```jq
unique_by(.profileUrl) | .[]
```

### Which jq function do I need?

Pick the task, copy the function. Everything here is standard jq and runs as-is.

| I want to… | jq | Mode |
| --- | --- | --- |
| Keep only some keys | `{name, price, url}` | per item |
| Drop keys | `del(.rawHtml, .debug)` | per item |
| Rename a key | `{title: .name} + del(.name)` | per item |
| Keep matching records | `select(.price < 100)` | per item |
| Drop empty/null fields | `with_entries(select(.value != null and .value != ""))` | per item |
| Derive a new field | `. + {domain: (.email \| split("@")[1])}` | per item |
| Flatten a nested object | `. + .address \| del(.address)` | per item |
| Join an array into a string | `. + {tags: (.tags \| join(", "))}` | per item |
| Explode an array into rows | `.variants[] + {parentId: .id}` | per item |
| Numbers from strings | `.price \|= (gsub("[^0-9.]";"") \| tonumber)` | per item |
| Group and count | `group_by(.state) \| map({state: .[0].state, count: length}) \| .[]` | whole input |
| Sum or average | `[{total: (map(.price) \| add), avg: (map(.price) \| add / length)}] \| .[]` | whole input |
| Deduplicate | `unique_by(.url) \| .[]` | whole input |
| Sort | `sort_by(-.price) \| .[]` | whole input |
| Top N | `sort_by(-.rating) \| .[:10] \| .[]` | whole input |
| Count records | `[{count: length}] \| .[]` | whole input |

Aggregations end in `.[]` because each emitted value becomes one dataset item — without it you get a single item holding the whole array.

### How do I chain it after another Actor?

Run **jq Helper** automatically whenever an upstream scraper finishes, and pass it that run's dataset.

**In the Console (no code):**

1. Open the **upstream** Actor's run config → **Integrations** tab → **Connect Actor or Task**.
2. Choose **jq Helper** to run on success (event `ACTOR.RUN.SUCCEEDED`).
3. In jq Helper's input, set the **`datasetId`** field to the upstream run's dataset using the variable:
   ```
   {{resource.defaultDatasetId}}
   ```
   Leave `jsonInput` empty (the two sources are mutually exclusive), then set your `filter` and `applyMode`.

The transformed items land in jq Helper's own dataset — ready for the next Actor, a webhook, or an export to Sheets/Make/Zapier.

> Other variables from the same run you can drop into string fields: `{{resource.id}}` (run ID), `{{resource.actId}}`, `{{resource.defaultKeyValueStoreId}}`, `{{resource.status}}`.

**In code (parent orchestrator):**

```python
run = await Actor.call(actor_id='you/scraper', run_input={...})
await Actor.call(
    actor_id='rl1987/jq-helper',
    run_input={
        'filter': '.websiteUrl',
        'applyMode': 'perItem',
        'datasetId': run.default_dataset_id,
    },
)
```

> Chaining always uses the `datasetId` path, so each chained run is a paid conversion. Inline `jsonInput` stays free for prototyping.

### FAQ

**Do I need to install jq, and which version is it?** Nothing to install — the engine is bundled with the Actor and runs in the cloud. It is jq 1.8, the full language, including 1.8 additions like `abs` and `pick`.

**What does it cost?** Runs on inline `jsonInput` are **free**, so you can prototype a filter at no cost. A run that reads a dataset (`datasetId`) is charged a flat fee whatever the dataset's size — the same to transform ten rows or a million. See the **Pricing** tab for the current price.

**Does it handle JSONL?** Yes. `jsonInput` is parsed as a single JSON document first; if that fails it is parsed as JSONL, one JSON value per non-empty line.

**My filter returns a string/number — why is it `{"value": ...}`?** Dataset items must be JSON objects, so scalars and arrays are wrapped. Set `wrapKey` to rename the key, or emit an object (e.g. `{websiteUrl}`) for named columns.

**How do I aggregate across all records?** Use `applyMode: wholeInput` and reach elements with `.[]`. Per-item mode never sees more than one record at a time, so `group_by`, `unique_by`, `add` and `sort_by` need whole-input mode.

**What if one record is malformed?** By default it's skipped, logged as a warning, and counted in the run's status message, so one bad row never loses the rest. Set `failOnError: true` to stop the run and mark it FAILED instead — use that when a downstream integration must not receive partial output.

**My run is green but the dataset is empty — why?** Check the run's **status message**: every run says what it did. The four reasons a run delivers nothing are *the filter errored on every input* (the error is in the log), *the filter ran fine and matched nothing* (e.g. a `select` no record satisfies), *the input had no items*, and *the run hit its max-cost setting before the conversion could be billed*. The commonest by far is a leading `.[]` in per-item mode — see the mode table above.

**Why didn't a bad input fail the run?** Because a mistake in your input is not a crash. A missing or unparseable `filter`, malformed `jsonInput`, both sources at once, or a `datasetId` this Actor's token cannot read all end the run **SUCCEEDED with no rows** and put the reason in the status message. You are not billed, and your Apify dashboard does not fill up with red runs while you iterate on a filter.

**Can I schedule it or trigger it automatically?** Yes. Use Apify **Schedules** for a recurring run, or the upstream Actor's **Integrations** tab to run it on every successful scrape — see the chaining section above.

### Can I use it from n8n?

Use the [`n8n-nodes-jq-helper`](https://www.npmjs.com/package/n8n-nodes-jq-helper) community node ([source](https://github.com/rl1987/n8n-nodes-jq-helper)) to run this Actor directly from an n8n workflow.

### How do I run it locally?

```bash
pip install -r requirements.txt
apify run   # reads storage/key_value_stores/default/INPUT.json
```

***

Built with the [Apify SDK for Python](https://docs.apify.com/sdk/python/) and the [`jq` bindings](https://github.com/mwilliamson/jq.py) (bundles libjq — no system `jq` required).

> **Disclaimer:** This is an independent, unofficial project. It is **not affiliated with, endorsed by, or otherwise associated with the [jq project](https://jqlang.github.io/jq/) or any of its developers.** "jq" is used here only to describe the JSON-processing language this Actor runs; all rights to jq belong to its respective authors.

### Data pipeline toolkit

Part of the **Data pipeline toolkit** — small, chainable Actors for cleaning, transforming, and generating data inside a larger pipeline:

- [DuckDB Helper – SQL over CSV, JSON, Parquet, Excel, SQLite](https://apify.com/rl1987/duckdb-wrapper) — Run a DuckDB SQL query over remote/local files, push results to a dataset.
- [Regex Helper](https://apify.com/rl1987/regex-helper) — Apply named regular expressions to strings, extract structured matches.
- [URL Wrangler](https://apify.com/rl1987/url-wrangler) — Join, decompose, and rewrite URLs and query params in batch.
- [ZIP Code Helper](https://apify.com/rl1987/zip-code-helper) — Resolves US ZIP codes into city, state, county, and more.
- [Postal Address Normaliser](https://apify.com/rl1987/postal-address-normaliser) — Parses and normalises postal addresses using libpostal.
- [Phone Number Wrangler](https://apify.com/rl1987/phone-number-wrangler) — Validate, format, and parse phone numbers using libphonenumber.
- [UUID Generator](https://apify.com/rl1987/uuid-generator) — Generate bulk UUIDs (v1, v3, v4, v5, v7) on demand.
- [Secure Password & Passphrase Generator](https://apify.com/rl1987/password-generator) — Generate secure passwords and diceware passphrases per NIST guidance.
- [Thumbnail Maker](https://apify.com/rl1987/thumbnail-maker) — Generates thumbnails from image URLs using ImageMagick.
- [Katana Web Crawler (ProjectDiscovery)](https://apify.com/rl1987/pd-katana) — Crawl websites with Katana, stream results as JSONL.
- [ProjectDiscovery Notify](https://apify.com/rl1987/pd-notify) — Stream records to Slack, Discord, Telegram, Email, and more.

### Did you find this useful?

Rate this actor on Apify! Your feedback helps other users find it and helps us keep improving it.

# Actor input Schema

## `filter` (type: `string`):

The jq program to run (standard jq 1.8 syntax). Compiled once and applied per the apply mode below. A filter may emit zero, one, or many values per input — each becomes one output item; non-object values are wrapped under the scalar wrap key. IMPORTANT — do NOT prefix with `.[]` in per-item mode: each item is already a single object, so `.websiteUrl` is correct and `.[].websiteUrl` errors with 'Cannot index string with string'. Use a leading `.[]` only in whole-input mode, where the whole array is fed in at once.

## `applyMode` (type: `string`):

How the filter maps over the input. Per-item (default): the array is iterated for you and the filter runs on each element as a single object — write `.field`, NOT `.[].field`. Best for mapping/filtering/enriching rows. Whole-input: the entire array is fed to the filter as one value, so use `.[]` to reach elements. Best for group_by, reduce, sort, unique, and other cross-record aggregations.

## `jsonInput` (type: `string`):

Raw JSON (object, array, or scalar) or JSONL (one JSON value per line) to transform. Provide either this or a Dataset ID, not both. In per-item mode, a top-level array is iterated element by element; a single object/scalar is treated as one item. Runs on inline input are FREE — prototype your jq snippets here at no cost.

## `datasetId` (type: `string`):

ID of an Apify dataset to read items from (e.g. the default dataset of an upstream actor run). Provide either this or inline JSON, not both. This is the field to map when wiring this actor into an integration/workflow. Processing a dataset is a paid conversion, charged once per run whatever the dataset's size - see the Pricing tab. The API token running this actor must be able to read the dataset (your own, or one explicitly shared with you); if it cannot, the run finishes with no rows and says so in its status message.

## `wrapKey` (type: `string`):

Dataset items must be JSON objects. When the filter emits a non-object value (string, number, array, boolean, null), it is wrapped as { <wrapKey>: value }.

## `failOnError` (type: `boolean`):

If enabled, a jq runtime error on any input stops the run and marks it FAILED - the signal to use when a downstream integration must not receive partial output. If disabled (default), the erroring input is skipped and logged as a warning, the rest are still delivered, and the run's status message reports how many errored.

## Actor input object example

```json
{
  "filter": "{name, isAdult: (.age >= 18)}",
  "applyMode": "perItem",
  "jsonInput": "[{\"name\": \"alice\", \"age\": 30}, {\"name\": \"bob\", \"age\": 16}]",
  "wrapKey": "value",
  "failOnError": false
}
```

# Actor output Schema

## `results` (type: `string`):

Rows produced by the run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "filter": "{name, isAdult: (.age >= 18)}",
    "jsonInput": "[{\"name\": \"alice\", \"age\": 30}, {\"name\": \"bob\", \"age\": 16}]"
};

// Run the Actor and wait for it to finish
const run = await client.actor("rl1987/jq-helper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "filter": "{name, isAdult: (.age >= 18)}",
    "jsonInput": "[{\"name\": \"alice\", \"age\": 30}, {\"name\": \"bob\", \"age\": 16}]",
}

# Run the Actor and wait for it to finish
run = client.actor("rl1987/jq-helper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "filter": "{name, isAdult: (.age >= 18)}",
  "jsonInput": "[{\\"name\\": \\"alice\\", \\"age\\": 30}, {\\"name\\": \\"bob\\", \\"age\\": 16}]"
}' |
apify call rl1987/jq-helper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,rl1987/jq-helper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/KK10Q0gjD9WIY7yGF/builds/8Y59SZyQd4RjrCPWp/openapi.json