Skip to content

A zero next_retry_delay fails the activity with RETRY_STATE_TIMEOUT #13010

Open

Description

@wuchris-ch

Expected Behavior

An activity fails with ApplicationFailureInfo.next_retry_delay set to 0s, and the server retries it with no delay. The proto comment says the field overrides the retry interval calculated by the retry policy, and sdk-core already retries a local activity right away for a zero.

Actual Behavior

The activity is not retried. It closes after the first attempt with RETRY_STATE_TIMEOUT, with unlimited attempts and no schedule-to-close timeout.

The activity fails once. The retry policy is initial interval 4s, backoff coefficient 1.

next_retry_delay Result
not set second attempt after 4s
1s second attempt after 1s
0s no second attempt, retry_state=RETRY_STATE_TIMEOUT

One way to get a zero: the TypeScript sample in the Delayed Retry docs passes ${retryAfterSeconds}s, so a Retry-After: 0 response fails the activity.

RetryActivity replaces the maximum interval with the requested delay (mutable_state_impl.go#L7052-L7057). With a zero delay the interval and the maximum interval are both zero, and nextBackoffInterval returns RETRY_STATE_TIMEOUT for that pair (retry.go#L100-L102).

The other retry paths handle the same zero differently:

  • Workflow retry: the policy's maximum interval is kept, so retry.go#L104-L106 turns the zero into the maximum interval. With initial interval 2s and maximum interval 6s, the next run started 6s after a failure with a zero delay.
  • Standalone activity: hasEnoughTimeForRetry reads <= 0 as not set, so the 4s policy interval was used.

Steps to Reproduce the Problem

  1. temporal server start-dev

  2. Run a workflow that calls this activity with { startToCloseTimeout: '30 seconds', retry: { initialInterval: '4 seconds', backoffCoefficient: 1 } }. This is TypeScript SDK 1.25.0. The Java, .NET and Ruby SDKs also send an explicit zero.

    import { ApplicationFailure, activityInfo } from '@temporalio/activity';
    
    export async function flaky(): Promise<number> {
      if (activityInfo().attempt === 1) {
        throw ApplicationFailure.create({
          message: 'retry now',
          nextRetryDelay: 0,
        });
      }
      return activityInfo().attempt;
    }
  3. The activity fails after one attempt with ActivityFailure: Activity task failed, retryState=TIMEOUT, which fails the workflow.

The same result at the function level on main. RetryActivity passes the delay as the maximum interval:

zero := time.Duration(0)
interval, state := nextBackoffInterval(
	now, 1, 0,
	durationpb.New(4*time.Second), // initial interval
	durationpb.New(zero),          // maximum interval
	nil, 1.0,
	backoff.MakeBackoffAlgorithm(&zero),
)
// interval == backoff.NoBackoff
// state == RETRY_STATE_TIMEOUT

Specifications

  • Version: 1.32.2 (dev server from Temporal CLI 1.10.0). The same logic is on main at 862b2db.
  • Platform: macOS 27, arm64
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    ⌂ 换网址