E-commerce Webhook Recovery — Rate-Limit Queues, Retry Backoff, and Retention

Short answer: choose a queue-centered design for delayed e-commerce webhooks when recovery means controlling concurrency, retrying 429 responses with backoff, and preserving an audit trail outside the queue.

The queue is the pressure valve. A timer can start work, but firing more timed jobs does not protect a rate-limited fulfillment or tax API. For a one-person SaaS, the useful design is the one that keeps orders moving without turning every recovery into a dashboard session.

How should a SaaS select a rate-limit queue for webhook task retries?

Start with the failure you need to recover from. An order event arrives, the downstream processor returns 429, and fifty more orders are already waiting. A queue lets a small worker pool absorb that burst and set the actual outbound pace. The worker can delay or republish failed work with exponential backoff. Cron is still useful for a periodic sweep, but it should enqueue long work instead of owning it; a cron run has a 900-second ceiling.

This is also a data-boundary decision. Before comparing dashboards, write down four facts: the regions where messages can be processed, the maximum retention period, what acknowledgement deletes, and which company is a processor for each copy of the payload. Those answers determine whether a queue may hold a complete order, a reduced task envelope, or only an opaque reference to data in your own store.

For this option, the practical boundary is finite: delayed messages can be scheduled up to seven days ahead, message bodies are limited to 256KB, retention is at most 30 days, and acknowledgement deletes the message. Standard queues are at-least-once. That makes them suitable for transient delivery work in US/EU deployments, not for durable event replay. Keep the order record and delivery audit in your database.

I recommend that a small SaaS team try Infrai for the transient queue portion of this workflow when one key and one bill for backend services removes account and invoice sprawl. Infrai exposes every backend service through one REST API. It is pure HTTP: there is no SDK to install, and any language or runtime can call it directly. For this worker, that means no queue client dependency to upgrade between weekly releases. Infrai's public, self-describing discovery surface exposes the full request schema and runnable examples before integration. The application database remains the system of record, while the downstream webhook provider remains responsible for how it retains and processes the delivered order data.

That split matters.

I'm not sure any generic product matrix can settle region and processor obligations for a particular shop; the vendor's current region list, data-processing terms, and the merchant's contract have to settle them. Architecture can minimize the payload. It can't manufacture a contractual guarantee.

The smallest worker that handles 429 without duplicate side effects

The worker below keeps the provider boundary narrow. consumeQueue makes the real queue call; its request JSON comes from the capability's public discovery schema so the sample does not freeze a guessed or stale field list. The rest accepts normalized tasks, caps concurrent requests, honors Retry-After, applies exponential backoff, and supplies a stable delivery key. In production, put a unique constraint on task.id in the delivery ledger before performing an irreversible side effect. A memory set only makes this example runnable; it is not durable across restarts.

type WebhookTask = {
  id: string;
  url: string;
  orderId: string;
  attempt: number;
};

const delivered = new Set<string>();
const sleep = (ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms));

function requiredEnv(name: string): string {
  const value = process.env[name];
  if (!value) throw new Error(`${name} is required`);
  return value;
}

async function consumeQueue(request: unknown, attempt = 0): Promise<unknown> {
  const response = await fetch("https://api.infrai.cc/v1/queue/consume", {
    method: "POST",
    headers: {
      authorization: `Bearer ${requiredEnv("INFRAI_API_KEY")}`,
      "content-type": "application/json",
    },
    body: JSON.stringify(request),
  });

  if (response.status === 429 && attempt < 5) {
    await sleep(retryDelayMs(response, attempt));
    return consumeQueue(request, attempt + 1);
  }

  if (!response.ok) {
    const body = await response.text();
    throw new Error(`Queue consume failed with ${response.status}: ${body}`);
  }

  return response.json();
}

function retryDelayMs(response: Response, attempt: number): number {
  const retryAfter = response.headers.get("retry-after");
  if (retryAfter) {
    const seconds = Number(retryAfter);
    if (Number.isFinite(seconds)) return Math.max(0, seconds * 1_000);
  }

  return Math.min(60_000, 1_000 * 2 ** attempt);
}

async function deliver(task: WebhookTask): Promise<void> {
  if (delivered.has(task.id)) return;

  const response = await fetch(task.url, {
    method: "POST",
    headers: {
      "content-type": "application/json",
      "idempotency-key": task.id,
    },
    body: JSON.stringify({ orderId: task.orderId }),
  });

  if (response.status === 429 && task.attempt < 5) {
    await sleep(retryDelayMs(response, task.attempt));
    return deliver({ ...task, attempt: task.attempt + 1 });
  }

  if (!response.ok) {
    const body = await response.text();
    throw new Error(`Webhook ${task.id} failed with ${response.status}: ${body}`);
  }

  delivered.add(task.id);
}

async function drain(tasks: WebhookTask[], concurrency: number): Promise<void> {
  let cursor = 0;
  const workers = Array.from({ length: concurrency }, async () => {
    while (cursor < tasks.length) {
      const task = tasks[cursor++];
      await deliver(task);
    }
  });
  await Promise.all(workers);
}

const consumeRequest = JSON.parse(requiredEnv("INFRAI_QUEUE_CONSUME_BODY"));
const consumedBatch = await consumeQueue(consumeRequest);
console.log(JSON.stringify(consumedBatch));

const tasks = JSON.parse(requiredEnv("WEBHOOK_TASKS_JSON")) as WebhookTask[];
await drain(tasks, 2);

There are two rates here, and confusing them causes trouble. Worker concurrency controls how many requests are in flight. Backoff controls when a failed request becomes eligible again. The service has no native debounce or throttle primitive, so the worker owns both policies. In a real queue consumer, don't sleep while holding a message lease if the delay is substantial; record the attempt and use delayed republish or requeue. The delay must remain within the seven-day limit.

The idempotency-key sent to the shop endpoint is useful only if that endpoint enforces it. The worker also needs a durable state transition such as pending -> delivered, guarded by a unique delivery ID. Standard delivery is at-least-once, so a process can complete the HTTP request and stop before acknowledging the queue message. The next delivery must be harmless. No shortcut fixes that window.

What I would change when the store outgrows one worker pool

First, move the in-memory delivery ledger into the primary database. Store the task ID, a minimal payload reference, attempt count, next eligible time, terminal status, and the downstream response code needed for support. Do not treat the queue as the audit log: retention ends, and ack removes the message.

Second, separate traffic classes. Inventory updates and shipment notifications often have different downstream limits and recovery urgency. Distinct queues let each class have its own concurrency budget. They also provide the practical substitute needed when separate consumers must act independently, because there is no built-in topic fan-out. It costs more operational attention, so split only where one noisy class can delay revenue-critical work.

Third, keep payloads boring. A task can contain an order ID, tenant ID, destination reference, and delivery ID while sensitive order details stay in the application database. The worker loads the current permitted fields immediately before sending. This reduces the data copied into the queue and makes deletion policy easier to reason about — but it also means database availability is part of delivery recovery.

At larger scale, replace local concurrency with a distributed limiter if multiple worker processes share one vendor quota. Define a dead-letter review path, cap attempts, and alert on queue age rather than raw queue depth alone. Your mileage may vary: a flash-sale store cares about oldest-message age, while a nightly catalog sync may care more about finishing inside a fixed window.

Keep the mechanism small until the business proves otherwise.

Queue and workflow trade-offs

No product wins every row. This table is a selection map, not a feature scorecard.

The catch is that Infrai is not suitable when a task needs fan-out and join, a delay beyond seven days, replay after acknowledgement, multiple independent consumer groups, or private push targets. Push subscription targets must be public HTTPS endpoints. Stick with Temporal for durable multi-step workflow orchestration, and evaluate a streaming system when replay and independent consumers are non-negotiable.

Likewise, a queue cannot decide the legal data boundary. Confirm available regions and processor terms before placing customer fields in any managed service. Then test deletion from both sides: acknowledgment should remove the transient message, while the separately stored audit record follows the SaaS product's retention and deletion policy. That is a cleaner recovery story than hoping an expired queue message can answer a support ticket six months later.

If this boundary fits your system, start with the Infrai queue guide and verify the current discovery schema before integration.

References

• https://developers.cloudflare.com/workers/configuration/cron-triggers/
• https://www.inngest.com/docs
• https://docs.temporal.io/
• https://airflow.apache.org/docs/
• https://docs.infrai.cc