Short answer: put one guard step in front of every LLM call and every outbound action. The guard hashes the action into an idempotency key, checks a Postgres ledger for duplicates, checks the client's month-to-date LLM spend against a cap, checks an actions-per-hour limit, and stops the workflow with an alert if any check fails.
Why this keeps happening (this week's threads)
- A dev.to post titled "I let an agent run unattended overnight. At 3am it emailed 400 customers the wrong thing." drew 48 comments.
- On r/automation, "My automation opened 83 tickets while I was asleep" drew 13 comments.
- On r/n8n, a thread asks how to log the token cost of every LLM call to Postgres, tagged by client and project, and stop the workflow when a monthly budget is hit.
- Another r/n8n thread asks how to monitor a large number of n8n workflows in production.
The common root causes are retries without idempotency, loops with no rate ceiling, and no per-client spend ledger.
The guard pattern (4 checks)
| Check | What it blocks | How |
|---|---|---|
| Idempotency key | Duplicate emails, tickets, payments on retry | SHA-256 of client, action type, recipient, payload |
| Monthly budget | Runaway token spend per client |
SUM(cost_usd) by client and month vs cap |
| Rate cap | Overnight loops |
COUNT(*) of actions in last hour vs cap |
| Kill switch | Everything above | Slack alert, then Stop and Error node |
Ledger schema
CREATE TABLE IF NOT EXISTS guard_actions (
idempotency_key CHAR(64) PRIMARY KEY,
client_id TEXT NOT NULL,
project TEXT,
action_type TEXT,
month CHAR(7) NOT NULL,
cost_usd NUMERIC(12,6) DEFAULT 0,
created_at TIMESTAMPTZ DEFAULT NOW()
);
Cost estimate formula
Estimated cost per call is (input_tokens x input_price + max_tokens x output_price) / 1,000,000, with input tokens approximated as characters divided by 4. Replace it with the provider's returned usage after the call for exact accounting.
Where to run it for $0 to $5
- Self-host n8n plus Postgres on Railway or a small Hetzner box, or start on n8n Cloud.
- If you prefer a visual builder, the same pattern works in Make.com with a Data Store as the ledger.
FAQ
Does this add latency? One indexed Postgres read and one insert per action, typically a few milliseconds on the same host.
Does it work with agents (LangChain, CrewAI)? Yes, call the guard as a tool or HTTP webhook before any side-effecting tool.
Disclosure: Some links are affiliate links — I may earn a commission at no extra cost to you.
Top comments (5)
The root-cause list is the valuable part: retries without idempotency, loops with no ceiling, and no spend ledger are three separate failures that happen to produce the same 3 a.m. headline. Hashing the action itself — client, type, recipient, payload — into the key is the right construction, because a retry regenerates everything except the intent. The one addition I would make is recording the guard's decision, not just the action: when the kill switch fires, 'we stopped it, and here is the row that proves it' is a much better morning-after answer than reconstructing it from the absence of emails.
Thanks, and I fully agree on logging the guard's decision. In practice I'd keep it in the same ledger table with a status column (sent / blocked_duplicate / blocked_budget / killed) plus the reason, so the morning-after question "what did the guard stop?" is a single query instead of guesswork.
Using a SHA-256 key as the primary key in the ledger is a neat way to let Postgres reject duplicate retries for you. One thing I'd wonder about is whether the key should include a date bucket, so a legitimate repeat email to the same recipient next week isn't blocked.
Good question! I'd tie the key to the intent rather than the calendar. For transactional email the business ID (order ID, ticket ID) is already in the payload, so a new order next week produces a new key automatically. For campaigns or scheduled sends I add the campaign/run ID to the hash. A plain date bucket also works, but it can let two sends through if a retry happens to cross midnight, so a run ID is a bit safer.
the same guard works one step earlier on data pulls: ask for a free count before any paid call, pass a max charge on the call itself, and feed last run's dataset id as an exclusion so a retry never pays for the same person twice dev.to/kevinhall/clay-alternative-...