Documentation
¶
Overview ¶
Package retry provides exponential backoff retry mechanisms for transient failures.
This package wraps github.com/cenkalti/backoff/v5 with application-specific retry patterns and error handling semantics. It offers several retry strategies:
- Execute: Retries an operation until it succeeds or times out
- ExecuteWithResult: Generic version that returns a result on success
- ExecuteSQL: Specialized retry for SQL operations
- WaitForCondition: Polls a condition until it becomes true
- Sustain: Keeps a long-running operation alive, restarting on transient failures
The package defines special error types to control retry behavior:
- ErrNonRetryable: Wraps permanent failures that should stop retries immediately
- ErrBackOff: Indicates transient errors that should trigger exponential backoff
Retry behavior is configured through the Profile struct, which defines:
- InitialInterval: Starting wait time between retries
- RandomizationFactor: Jitter to prevent thundering herd
- Multiplier: Factor by which wait time increases each retry
- MaxInterval: Maximum wait time between retries
- MaxElapsedTime: Total time budget for all retries (nil = default, 0 = unlimited)
Example Usage:
// Simple retry with default profile
err := retry.Execute(ctx, nil, func() error {
return someOperation()
})
// Retry with custom profile and result
profile := &retry.Profile{
InitialInterval: 100 * time.Millisecond,
MaxElapsedTime: 30 * time.Second,
}
result, err := retry.ExecuteWithResult(ctx, profile, func() (*Data, error) {
return fetchData()
})
// Keep a service running with automatic restart
err := retry.Sustain(ctx, profile, func() error {
return runService() // Returns ErrBackOff on transient errors
})
Index ¶
- Variables
- func Execute(ctx context.Context, p *Profile, o func() error, terminalErrors ...error) error
- func ExecuteSQL(ctx context.Context, p *Profile, e executor, sqlStmt string, args ...any) error
- func ExecuteWithResult[T any](ctx context.Context, p *Profile, operation func() (T, error), ...) (T, error)
- func Sustain(ctx context.Context, p *Profile, op func() error) error
- func WaitForCondition(ctx context.Context, p *Profile, condition func() bool) bool
- type Profile
Constants ¶
This section is empty.
Variables ¶
var ( // ErrNonRetryable represents an error that should not trigger a retry. // It's used to wrap an underlying error condition when an operation // fails and retrying would be useless. // The code performing retries will check for this error type // (e.g., using errors.Is) and stop the retry loop if found. ErrNonRetryable = errors.New("cannot recover from error") // ErrBackOff is returned for transient errors specifically on an existing connection, // implying the initial connection was successful. Retry the operation after a backoff delay. ErrBackOff = errors.New("backoff required before retrying") )
Functions ¶
func Execute ¶
Execute executes the given operation repeatedly until it succeeds or a timeout occurs. Errors matching terminalErrors stop retries immediately. It returns nil on success, or the error returned by the final attempt on timeout.
func ExecuteSQL ¶
ExecuteSQL executes the given SQL statement until it succeeds or a timeout occurs.
func ExecuteWithResult ¶
func ExecuteWithResult[T any]( ctx context.Context, p *Profile, operation func() (T, error), terminalErrors ...error, ) (T, error)
ExecuteWithResult executes the given operation repeatedly until it succeeds or a timeout occurs. Errors matching terminalErrors stop retries immediately.
func Sustain ¶
Sustain attempts to keep a continuous operation `op` running indefinitely. Unlike Execute which retries until success, Sustain is designed for long-running operations that should continue running until explicitly stopped or a permanent failure occurs.
Operation Behavior:
- If op returns nil: Operation is running smoothly, Sustain resets backoff and retries immediately
- If op returns ErrBackOff: Transient error, Sustain applies exponential backoff before retrying
- If op returns ErrNonRetryable: Permanent failure, Sustain stops and returns the error
- If op returns any other error: Retries immediately (treated as transient)
Sustain stops and returns when:
- op returns an error wrapping ErrNonRetryable (permanent failure),
- The context ctx is cancelled (returns context error),
- The backoff strategy times out after MaxElapsedTime of continuous ErrBackOff errors.
func WaitForCondition ¶
WaitForCondition repeatedly evaluates the given condition function until it returns true or a timeout occurs. It uses exponential backoff between attempts as defined by the retry profile. Returns nil when the condition is satisfied, or an error if the context is cancelled or max elapsed time is reached.
Types ¶
type Profile ¶
type Profile struct {
InitialInterval time.Duration `mapstructure:"initial-interval"`
RandomizationFactor float64 `mapstructure:"randomization-factor"`
Multiplier float64 `mapstructure:"multiplier"`
MaxInterval time.Duration `mapstructure:"max-interval"`
MaxElapsedTime *time.Duration `mapstructure:"max-elapsed-time" validate:"omitempty,gte=0"`
}
Profile can be used to define the backoff properties for retries.
MaxElapsedTime controls the total time budget for retries:
- nil (unset): the default (15 minutes) is used.
- 0: retries never stop (unlimited); the backoff runs until the operation succeeds, the context is cancelled, or a permanent error occurs.
- positive: the backoff returns the underlying error after this duration.
Note: the unlimited (0) budget only removes the time limit from the backoff. gRPC connection retries remain bounded (see connection.MakeGrpcRetryPolicyJSON); unbounded service reconnection is provided by Sustain.
This is used as a workaround for known issues:
- Dropping a database with proximity to accessing it. See: https://support.yugabyte.com/hc/en-us/articles/10552861830541-Unable-to-Drop-Database.
- Creating/dropping tables immediately after creating a database. See: https://github.com/yugabyte/yugabyte-db/issues/14519.
func (*Profile) NewBackoff ¶
func (p *Profile) NewBackoff() *backoff.ExponentialBackOff
NewBackoff creates a new backoff.ExponentialBackOff instance with this profile.
Note: MaxElapsedTime is not set on the backoff instance itself. In backoff/v5, MaxElapsedTime is passed separately to backoff.Retry() as a configuration option via backoff.WithMaxElapsedTime(). This allows the same backoff instance to be reused with different timeout policies.
func (*Profile) WithDefaults ¶
WithDefaults returns a clone of this profile with default values.
A nil MaxElapsedTime is replaced with the default (15 minutes). A non-nil value is preserved as-is, including an explicit 0 which requests unlimited retries. This makes WithDefaults idempotent for MaxElapsedTime.
Source Files