outbox

package module
v3.2.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 7, 2026 License: MIT Imports: 31 Imported by: 0

README

Melody outbox integration (v3)

A transactional-outbox implementation for the messagebus package over Bun: business writes and the messages they produce commit in one database transaction, and a relay drains the outbox table to any messagebus/contract.Transport (for example the AMQP integration) with retry, exponential backoff and a dead-letter terminal state.

Installation

go get github.com/precision-soft/melody/integrations/outbox/v3
import outbox "github.com/precision-soft/melody/integrations/outbox/v3"

Requirements

Claiming is atomic through SELECT ... FOR UPDATE SKIP LOCKED, so the backing database must support it: PostgreSQL or MySQL 8+. Two relay instances never publish the same row even without a distributed lock.

The table

Store.EnsureSchema creates the melody_outbox table for demos and tests (production schemas belong in bunorm/migrate migrations):

Column Purpose
id Row id; also the stable message id (melody-outbox-<id>) consumers deduplicate on.
type_name Codec type name used to rebuild the Go message.
payload Codec-encoded message body.
status pending → inflight → sent / dead.
attempts Send failures — drives backoff and the MaxAttempts dead-letter.
delivery_attempts Every claim — drives the MaxDeliveryAttempts crash-poison cap.
available_at When the row is next due (backoff scheduling + visibility timeout).
last_error Last send/decode failure WITH its cause chain (capped to fit the narrowest default column), for operators.
claim_token Fencing token, rewritten on every claim. Every transition write (RecordDeliveryAttempt, MarkSent, MarkDead, the retry reschedule) is guarded on claim_token = ? as well as the in-flight status, so a stale run whose claim already lapsed and was re-claimed by another instance matches no row and cannot clobber the new owner. Guarding on status alone is not enough: a re-claim returns the row to the very in-flight state the stale run also holds.

Usage

Encode/decode your messages

The application supplies the MessageCodec because only it knows its message types — a JSON codec over a small type registry is the typical implementation:

type Codec struct{}

func (instance Codec) Encode(message any) (string, []byte, error) { /* switch on your types */ }

func (instance Codec) Decode(typeName string, payload []byte) (any, error) { /* inverse */ }
Enqueue inside your transaction
store := outbox.NewStore(database, Codec{})

transactionErr := database.RunInTx(ctx, nil, func(ctx context.Context, tx bun.Tx) error {
    if insertErr := tx.NewInsert().Model(&order).Scan(ctx); nil != insertErr {
        return insertErr
    }

    /* commits atomically with the business write — the point of the pattern */
    return store.Enqueue(ctx, tx, OrderPlaced{OrderId: order.Id})
})
Wire the relay through the module
relay := outbox.NewRelay(outbox.RelayConfig{
    Repository: store,
    Transport:  amqpTransport,
    Codec:      Codec{},
    /* optional: Locker (e.g. the rueidis locker) as a single-drainer optimization; SKIP LOCKED already makes concurrent relays safe */
    /* optional overrides: BatchSize, MaxAttempts, InitialBackoff, MaxBackoff, BackoffFactor, VisibilityTimeout, MaxDeliveryAttempts, LockName, LockTtl */
})

app.RegisterModule(
    outbox.NewModule(outbox.ModuleConfig{
        Store: store,
        Relay: relay,
    }),
)

The module registers the store and relay under the canonical container names service.outbox.store / service.outbox.relay (resolvable through StoreMustFromContainer / RelayMustFromContainer) and exposes the relay lifecycle command.

The relay sends through its transport and never closes it, and the registered *Relay has no Close of its own — so the transport behind a prebuilt relay is closed by nobody unless the composition root registers it with the container, through messagebus.RegisterTransports or as a service of its own, for the ordered teardown to close. The same rule the ModuleConfig GoDoc gives for factories holds here: a transport (or database) opened privately and handed to the relay lives exactly as long as the process. The framework's v3/.example registers its outbox transport as service.outbox.transport for that reason.

Relay defaults

Every RelayConfig tunable is optional; a non-positive value resolves to the default below:

Field Default Meaning
BatchSize 100 Rows claimed per RunOnce; clamped from above too (100000), because it flows into a slice pre-allocation. Store.ClaimDueMessages applies the same cap to the limit it is handed and refuses a non-positive one by name, since bun writes no LIMIT for it and the claim would take the whole table.
MaxAttempts 12 Send failures after which the row is dead-lettered.
MaxDeliveryAttempts 2 × MaxAttempts (24) Claims after which the row is dead-lettered as crash-poison. Must exceed MaxAttempts — every retry re-claim counts toward it, so a value at or below it (including an unset zero) is raised to the default rather than dead-lettering a normally-retrying row.
InitialBackoff 15s Delay before the first retry.
MaxBackoff 10m Cap on the grown delay.
BackoffFactor 2.0 Growth per prior attempt. A value below 1 resolves to the default.
VisibilityTimeout 5m How long a claimed row stays hidden before it re-surfaces — the net that recovers rows whose claimer crashed. Must comfortably exceed the time to drain one batch.
LockTtl 30s Lease TTL for the optional single-drainer Locker; the lease is refreshed at half of it. A refresh that fails is not retried: the claimed batch is drained under its claim tokens, to its end or to a failed send, then the failure is reported.
LockName melody:outbox:relay Lock name used with that Locker.

Locker itself has no default — without one, SKIP LOCKED alone already makes concurrent relays safe.

Run the relay
melody:outbox:relay [--interval 1s] [--idle-backoff 1s] [--limit 0]

The command drains batches until interrupted (SIGINT/SIGTERM stop the loop; a signal that lands mid-batch aborts the drain and whatever stayed claimed re-surfaces after VisibilityTimeout): --interval is the poll cadence, --idle-backoff the sleep after an empty batch (the interval when absent), --limit N stops after N batches (for cron-style invocations). Consecutive batch failures back off exponentially (capped at one minute) so a repository outage does not tight-loop. A cancellation exits clean, but a genuine batch failure that merely coincides with the parent context's deadline is returned — a supervisor sees the failed drain. Relay.RunOnce stays available for embedding the drain into a custom loop.

Delivery semantics

  • At-least-once: a transport success followed by a crash before MarkSent redelivers the row. Every publish carries the deterministic messagebus.MessageIdStamp melody-outbox-<id> (the AMQP transport round-trips it as the message id) so consumers can deduplicate.
  • Retry: a failed send reschedules the row with exponential backoff (InitialBackoff × BackoffFactor, capped at MaxBackoff, overflow-safe) and dead-letters it at MaxAttempts, and, once that write succeeded, writes one outbox message send failed ERROR record naming the row, its type and the attempt it was charged, with the transport's cause; a contained send panic keeps its own outbox delivery panicked record instead.
  • A failed send ends the batch: the relay cannot tell a broker outage from a message the broker refused, so it stops at the first failed send rather than charge every claimed row an attempt on every pass of an outage. The rows the batch had not reached go back to pending uncharged, available again when the failed row is tried again (or would have been, when the failure dead-lettered it), with a last_error naming the row that held them. A claim takes the lowest id first, so during an outage the first row of each claimed batch is the only one charged, pass after pass, until it is dead-lettered after about 66 minutes at the default settings and the next row of that batch takes its place. Rows enqueued during the outage are due at once, so every poll that finds them claims a new batch and charges its first row: the rows charged grow with the polls that find fresh rows, not only with the backlog. The price is head-of-line blocking: once the broker is back, the held rows wait out the failed row's current backoff (at most MaxBackoff), and a message the broker refuses holds every row behind it for its own backoff each time it is due, until it reaches MaxAttempts.
  • Poison: an undecodable row dead-letters immediately; a row that keeps crashing or hanging the relay between claim and resolve dead-letters once the delivery attempts the relay records for it — one each time a claim's run reaches the row, a batch-mate it never reached charged nothing — exceed MaxDeliveryAttempts (default 2 × MaxAttempts; the attempt after the cap is the one that buries it). A panic raised by the codec's Decode or the transport's Send is contained per row and charged as the failure the collaborator would have returned — a decode panic dead-letters the row, a send panic takes the retry path — with the panic in last_error under a panic: prefix and an outbox delivery panicked record in the process log; after a decode panic the batch goes on, a send panic ends it like any failed send. A panic raised by the repository is not contained.
  • Crash recovery: a claimed row re-surfaces after VisibilityTimeout if its claimer died.
  • Clocks: the visibility window is computed and compared on the application hosts' wall clocks, not the database's, so inter-replica clock skew subtracts directly from VisibilityTimeout — a replica running ahead can re-claim early and duplicate a publish. Keep the fleet's clocks synced; the MessageIdStamp dedup covers the residual duplicates the at-least-once contract already admits.
  • Locker (optional): with a lock/contract.Locker only one instance drains at a time; the lease is refreshed mid-batch (twice per LockTtl). The lease is an optimisation, not what keeps two instances apart — the claim tokens are — so a lease lost mid-batch does not abort the drain: the claimed rows stay this run's, the batch is drained to its end or to a failed send, and the refresh failure is reported after it, for the command's loop to log and back off on. A failed Acquire is a different answer from a lease another instance holds: the lease store unreachable, the run returns the error before it claims a row, so with a Locker wired an outage of the lease store stops delivery. An application that would rather deliver through it, the claim tokens keeping concurrent relays apart, wraps its Locker so that an Acquire error answers acquired, logging it, and the Release and Refresh of such a lease do nothing.

Documentation

Index

Constants

View Source
const (
	StatusPending  = "pending"
	StatusInFlight = "inflight"
	StatusSent     = "sent"
	StatusDead     = "dead"
)
View Source
const ServiceRelay = "service.outbox.relay"
View Source
const ServiceStore = "service.outbox.store"
View Source
const TableName = "melody_outbox"

TableName is the outbox table the Store reads and writes.

Variables

This section is empty.

Functions

func RegisterRelayService

func RegisterRelayService(registrar ServiceRegistrar, relay *Relay)

func RegisterStoreService

func RegisterStoreService(registrar ServiceRegistrar, store *Store)

Types

type Message

type Message struct {
	bun.BaseModel `bun:"table:melody_outbox"`

	Id               int64     `bun:"id,pk,autoincrement"`
	TypeName         string    `bun:"type_name,notnull"`
	Payload          []byte    `bun:"payload,notnull"`
	Status           string    `bun:"status,notnull"`
	Attempts         int       `bun:"attempts,notnull"`
	DeliveryAttempts int       `bun:"delivery_attempts,notnull"`
	AvailableAt      time.Time `bun:"available_at,notnull"`
	CreatedAt        time.Time `bun:"created_at,notnull"`
	LastError        string    `bun:"last_error,nullzero"`
	ClaimToken       string    `bun:"claim_token,nullzero"`
}

Message is the outbox table row. The relay-facing fields (status, attempts, delivery_attempts, available_at) drive retry scheduling; payload + type_name are what the codec needs to rebuild the message. Attempts counts send failures (drives backoff and the MaxAttempts dead-letter); delivery_attempts counts every delivery the relay actually attempts for this row (drives the MaxDeliveryAttempts crash-poison cap) — it is advanced per row at delivery time, not for the whole batch at claim time, so a batch-mate the relay never reached is never charged. claim_token is a fencing token rewritten on every claim: each transition write carries the token of the claim it belongs to, so a stale run whose claim has lapsed and been re-claimed by another instance matches no row and cannot clobber the new owner (guarding on status alone is not enough, because a re-claim returns the row to the in-flight state the stale run also holds).

type MessageCodec

type MessageCodec interface {
	Encode(message any) (typeName string, payload []byte, err error)

	Decode(typeName string, payload []byte) (any, error)
}

MessageCodec encodes a domain message into a durable form for the outbox and decodes it back when the relay publishes it. The application supplies it because only the application knows its message types; a JSON codec over a small type registry is the typical implementation.

type Module

type Module struct {
	// contains filtered or unexported fields
}

func NewModule

func NewModule(config ModuleConfig) *Module

func (*Module) Description

func (instance *Module) Description() string

func (*Module) Name

func (instance *Module) Name() string

func (*Module) RegisterCliCommands

func (instance *Module) RegisterCliCommands(kernelInstance kernelcontract.Kernel) []clicontract.Command

func (*Module) RegisterServices

func (instance *Module) RegisterServices(registrar applicationcontract.ServiceRegistrar)

type ModuleConfig

type ModuleConfig struct {
	Store *Store

	Relay *Relay

	StoreFactory func(resolver containercontract.Resolver) (*Store, error)

	RelayFactory func(resolver containercontract.Resolver) (*Relay, error)
}

ModuleConfig wires the outbox services: a prebuilt Store and Relay, or a StoreFactory and RelayFactory registered as the service providers and run at first resolution, for a database not available before Boot; setting both a prebuilt object and its factory panics at RegisterServices. Only the *Store or *Relay a factory returns is registered, so a factory resolves container-owned services rather than opening private connections. The relay never closes its transport, so the composition root registers the transport with the container, through messagebus.RegisterTransports or as a service of its own.

type Pending

type Pending struct {
	Id               int64
	TypeName         string
	Payload          []byte
	Attempts         int
	DeliveryAttempts int
	ClaimToken       string
}

Pending is a stored outbox row ready to be relayed: the codec type name and payload needed to rebuild the message, plus the retry counters. Attempts is how many sends have failed (drives backoff and the MaxAttempts dead-letter); DeliveryAttempts is the row's stored delivery-attempt count at claim time — the relay advances it per row at delivery time (RecordDeliveryAttempt) and the post-increment value, not this snapshot, drives the MaxDeliveryAttempts crash-poison cap. ClaimToken is the fencing token stamped on the row by the claim that produced this Pending; the relay passes it back to every transition write so a stale run cannot clobber a row another instance has since re-claimed.

type Relay

type Relay struct {
	// contains filtered or unexported fields
}

func NewRelay

func NewRelay(config RelayConfig) *Relay

func RelayMustFromContainer

func RelayMustFromContainer(serviceContainer containercontract.Container) *Relay

func RelayMustFromResolver

func RelayMustFromResolver(resolver containercontract.Resolver) *Relay

func (*Relay) RunOnce

func (instance *Relay) RunOnce(runtimeInstance runtimecontract.Runtime) (int, error)

RunOnce drains one batch of due messages and returns how many were published; when a Locker is configured and another instance holds the lease, it returns 0. The lease is an optimisation: every claimed row carries this run's claim token, so a lease lost mid-batch hands no claimed row to the new holder, and the batch is drained to its end, or to a failed send, with the refresh failure reported after it. A repository failure mid-batch ends the run where it happened, and the visibility timeout re-surfaces the unreached rows. A failed send ends the batch too: the relay cannot tell a transport outage from a message the broker refused, and going on would charge every batch-mate an attempt on every pass of an outage, so the failed row alone is charged and the unreached rows are handed back uncharged, held until the failed row is tried again: the lowest id comes first in every claim, so through an outage that row is the only one charged, pass after pass.

type RelayCommand

type RelayCommand struct {
	// contains filtered or unexported fields
}

func NewRelayCommand

func NewRelayCommand(relay *Relay) *RelayCommand

func NewRelayCommandFromResolver added in v3.1.0

func NewRelayCommandFromResolver(resolver containercontract.Resolver) *RelayCommand

NewRelayCommandFromResolver resolves service.outbox.relay on the first batch, opening no database at boot

func (*RelayCommand) Description

func (instance *RelayCommand) Description() string

func (*RelayCommand) Flags

func (instance *RelayCommand) Flags() []clicontract.Flag

func (*RelayCommand) Name

func (instance *RelayCommand) Name() string

func (*RelayCommand) Run

func (instance *RelayCommand) Run(
	runtimeInstance runtimecontract.Runtime,
	commandContext *clicontract.CommandContext,
) error

type RelayConfig

type RelayConfig struct {
	Repository Repository

	/* Transport is what every row is published through. The relay only sends through it and never closes it: on the module's prebuilt path the composition root registers the transport with the container — through messagebus.RegisterTransports, or as a service of its own — so the container's ordered teardown closes it, and on the factory path the factory resolves a container-owned one, as ModuleConfig says. A transport that is neither closes with the process, abruptly. */
	Transport messagebuscontract.Transport

	Codec MessageCodec

	Locker lockcontract.Locker

	LockName string

	LockTtl time.Duration

	/* VisibilityTimeout is how long a claimed (in-flight) row stays hidden from other claimers before it re-surfaces — the safety net that recovers rows an instance claimed but crashed before resolving. It must comfortably exceed the time to drain one batch; defaults to defaultVisibilityTimeout. */
	VisibilityTimeout time.Duration

	BatchSize int

	MaxAttempts int

	/* MaxDeliveryAttempts caps how many delivery attempts the relay records for a single row (RecordDeliveryAttempt, once per row it reaches after a claim) before it is dead-lettered as poison, regardless of whether its sends ever returned an error. It is the safety net for a row that crashes (or hangs past the visibility timeout) the relay between claim and resolve: such a row's send-failure attempt count never advances, so without this cap it would re-surface forever and never reach MaxAttempts. It must exceed MaxAttempts because every recorded attempt — including a normal retry's — counts toward it: a value at or below MaxAttempts, an unset zero included, is raised to defaultMaxDeliveryAttemptsFactor * MaxAttempts, saturating at math.MaxInt. */
	MaxDeliveryAttempts int

	InitialBackoff time.Duration

	MaxBackoff time.Duration

	BackoffFactor float64
}

RelayConfig configures the outbox relay — the loop that drains pending rows to the message transport with retry, exponential backoff and a dead-letter terminal state. Every batch is claimed atomically (FOR UPDATE SKIP LOCKED), so two instances do not claim the same row while its VisibilityTimeout holds, even without a Locker; the relay stays at-least-once — a send still running when the timeout lapses, or a crash between the send and MarkSent, lets the row be claimed and published again —, and this requires a backend that supports SELECT … FOR UPDATE SKIP LOCKED (PostgreSQL, or MySQL 8+). A Locker is still useful as an additional optimization: supply one (for example the Redis locker) so only one instance does any work at a time in a multi-instance deployment.

type Repository

type Repository interface {
	ClaimDueMessages(ctx context.Context, limit int, visibility time.Duration) ([]Pending, error)

	/* RecordDeliveryAttempt increments and persists a single row's delivery_attempts and returns the post-increment count. It is called for each row as the relay attempts it, not for the batch at claim time, so a row that crashes or hangs the relay advances only its own crash-poison counter. The claimToken fences the write: claimed is false when this claim does not hold the row any more, re-claimed by another instance or already resolved, and the relay skips it. */
	RecordDeliveryAttempt(ctx context.Context, id int64, claimToken string) (deliveryAttempts int, claimed bool, err error)

	MarkSent(ctx context.Context, id int64, claimToken string) error

	Reschedule(ctx context.Context, id int64, attempts int, availableAt time.Time, lastError string, claimToken string) error

	MarkDead(ctx context.Context, id int64, attempts int, lastError string, claimToken string) error
}

Repository is the persistence the relay drives. The bun-backed Store implements it; abstracting it keeps the relay's retry/dead-letter logic unit-testable without a database.

func NewLazyRepository added in v3.1.0

func NewLazyRepository(resolver containercontract.Resolver) Repository

NewLazyRepository returns a Repository that resolves the registered service.outbox.store on first use; the underlying Store is resolved once and reused, and a failed resolution is retried on the next call.

type ServiceRegistrar

type ServiceRegistrar interface {
	RegisterService(serviceName string, provider any, options ...containercontract.RegisterOption)
}

type Store

type Store struct {
	// contains filtered or unexported fields
}

func NewStore

func NewStore(database *bun.DB, codec MessageCodec) *Store

func StoreMustFromContainer

func StoreMustFromContainer(serviceContainer containercontract.Container) *Store

func StoreMustFromResolver

func StoreMustFromResolver(resolver containercontract.Resolver) *Store

func (*Store) ClaimDueMessages

func (instance *Store) ClaimDueMessages(ctx context.Context, limit int, visibility time.Duration) ([]Pending, error)

ClaimDueMessages atomically claims a batch of due rows so that concurrent relay instances — even without a shared Locker — never grab the same row. It requires a backend that supports SELECT … FOR UPDATE SKIP LOCKED (PostgreSQL, or MySQL 8+). It selects due rows FOR UPDATE SKIP LOCKED inside a transaction (so a row another instance is claiming is skipped, not blocked on) and flips them to the in-flight state with available_at pushed out by the visibility timeout. A claimed row is therefore invisible to every other claimer until either the relay resolves it (sent/rescheduled/dead) or the visibility timeout lapses — which re-surfaces rows an instance claimed but crashed before resolving. A due row is one that is pending, or already in-flight but past its visibility deadline. Claiming does NOT touch delivery_attempts — that counter is advanced per row when the relay actually attempts delivery (RecordDeliveryAttempt), so a row the relay never reaches (a batch-mate behind a crashing row) is not charged a delivery attempt it never received.

func (*Store) Enqueue

func (instance *Store) Enqueue(ctx context.Context, executor bun.IDB, message any) error

Enqueue serializes a message and inserts it as a pending outbox row using the given executor — pass the caller's bun.Tx so the outbox write commits atomically with the business write, which is the whole point of the pattern.

func (*Store) EnsureSchema

func (instance *Store) EnsureSchema(ctx context.Context) error

EnsureSchema creates the outbox table for demos and tests. Production code would express this through the bunorm migrate package.

func (*Store) MarkDead

func (instance *Store) MarkDead(ctx context.Context, id int64, attempts int, lastError string, claimToken string) error

func (*Store) MarkSent

func (instance *Store) MarkSent(ctx context.Context, id int64, claimToken string) error

MarkSent resolves the row as sent. Every resolution write is guarded on status in-flight and on the claim's fencing token, since a lapsed claim can be re-claimed back to in-flight by another instance and a status-only write would clobber that owner; with the token a stale run's write matches no row.

func (*Store) RecordDeliveryAttempt

func (instance *Store) RecordDeliveryAttempt(ctx context.Context, id int64, claimToken string) (int, bool, error)

RecordDeliveryAttempt increments a single in-flight row's delivery_attempts and returns the post-increment count, charging only a row the relay actually reached. The read and write run in one transaction under FOR UPDATE, so a concurrent claimer cannot lose an increment. A row whose claim_token does not match this claim, re-claimed by another instance or already resolved, returns claimed=false so the relay skips it; the token, not the status, tells "still mine" from "re-claimed", since a re-claim returns the row to the same in-flight state.

func (*Store) Reschedule

func (instance *Store) Reschedule(
	ctx context.Context,
	id int64,
	attempts int,
	availableAt time.Time,
	lastError string,
	claimToken string,
) error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址