pg

package module
v2.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 14, 2026 License: MIT Imports: 25 Imported by: 1

README

eventsourced PostgreSQL event store

GoReportCard Go Reference

Package pg provides a PostgreSQL implementation of eventsourced.EventStore with automatic schema migrations.

Install

go get codeberg.org/eventsourced/pg/v2

Prerequisites

  • PostgreSQL 12+
  • The lib/pq driver (imported for side effects)

Getting Started

import (
    "database/sql"

    _ "github.com/lib/pq"

    "codeberg.org/eventsourced/eventsourced"
    "codeberg.org/eventsourced/pg/v2"
)

// Open database connection
db, err := sql.Open("postgres", "postgres://localhost/mydb?sslmode=disable")
if err != nil {
    log.Fatal(err)
}

// Create the event store with registered event types
store, err := pg.New(ctx, db,
    pg.WithEventTypes(
        &Deposited{},
        &Withdrawn{},
    ),
)
if err != nil {
    log.Fatal(err)
}

// Use with a command handler
handler, err := eventsourced.NewHandler(ctx, &Account{}, store,
    eventsourced.WithEventPublisher(publisher),
)

Event Type Registration

All event types that will be stored or replayed must be registered using WithEventTypes. Events must be passed as pointers and must implement eventsourced.Event:

store, err := pg.New(ctx, db,
    pg.WithEventTypes(
        &AccountCreated{},
        &Deposited{},
        &Withdrawn{},
    ),
)

JSON struct tags are validated at registration to catch duplicate field names early.

Database Schema

The store automatically creates and migrates three tables:

Table Purpose
events Append-only event log
aggregates Registry of aggregate roots
snapshots Periodic aggregate state snapshots
events
Column Type Description
global_sequence_no serial Unique, ordered ID across all events
name text Event type name (Go type string)
aggregate_id text Aggregate root ID
aggregate_name text Aggregate type name
aggregate_sequence_no int Per-aggregate sequence number
payload jsonb JSON-serialized event data
tstamp timestamp When the event occurred
aggregates
Column Type Description
id text Aggregate root ID
name text Aggregate type name
snapshots
Column Type Description
id serial Snapshot ID
name text Aggregate type name
aggregate_id text Aggregate root ID
sequence_no int Aggregate sequence at snapshot time
payload jsonb JSON-serialized aggregate state
tstamp timestamp When the snapshot was taken

Migrations

Schema migrations are embedded in the binary and run automatically when New() is called. Migration state is tracked in the goose_db_version_pg table (separate from any application-level goose migrations).

Configuration Options

Option Default Description
WithEventTypes (required) Register event types for serialization
WithTraceHandler no-op Add distributed tracing
WithLogger slog.Default() Set custom logger
WithIgnoreUndefinedEventTypesDuringReplay false Skip unrecognized event types during replay
WithEventTypeAliases none Read events stored under another type name (e.g. across a package move)
WithCommittedReads false MaxGlobalSequenceNo and Events stop below in-flight inserts, so a read view back fill never skips an event that commits after a higher one

Error Handling

Error Condition
eventsourced.ErrStaleAggregate Concurrent modification (sequence number conflict)
eventsourced.ErrAggregateAlreadyExists Duplicate aggregate creation

Snapshot Behavior

Snapshots are managed by the command handler (not the store). The store provides RestoreSnapshot and StoreSnapshot methods. By default, the command handler takes a snapshot every 10 events (configurable via eventsourced.WithSnapshotInterval).

Renamed Event Types

An event's stored name is its Go type's reflect.Type.String(). For a generic type this includes the full import path of its type argument (eventsourced.ExternalEvent[*example.org/app/handlers.OrderPlaced]), so moving a package renames its stored events. WithEventTypeAliases lets the store read events under another name: each alias maps a name to a type registered with WithEventTypes. Events are always stored under their type's current name; aliases are only read. Snapshot and aggregate names are pkg.Type without the path, so a move doesn't rename them (unless the aggregate type itself is generic).

Moving a package without breaking a rolling deploy or a rollback takes two releases, because every running version must read both names:

  1. Release N, before the move: alias the future name to the current type.
  2. Release N+1, the move: alias the old name to the moved type. Roll back no further than N.
// Release N+1, after moving example.org/old/handlers to example.org/app/handlers:
store, err := pg.New(ctx, db,
    pg.WithEventTypes(&eventsourced.ExternalEvent[*handlers.OrderPlaced]{}),
    pg.WithEventTypeAliases(map[string]any{
        "eventsourced.ExternalEvent[*example.org/old/handlers.OrderPlaced]": &eventsourced.ExternalEvent[*handlers.OrderPlaced]{},
    }),
)

Keeping the aliases needs no data change. To drop them later, first rename the stored names in events.name, event_outbox.event_name and command_idempotency.event_name once nothing older than N+1 runs.

Ignoring Undefined Event Types

When evolving a system, you may remove event types that still exist in the database. Use WithIgnoreUndefinedEventTypesDuringReplay to skip these during aggregate replay instead of returning an error:

store, err := pg.New(ctx, db,
    pg.WithEventTypes(&CurrentEvent{}),
    pg.WithIgnoreUndefinedEventTypesDuringReplay(),
)

Documentation

Overview

Package pg provides a PostgreSQL-backed implementation of eventsourced.EventStore.

The store persists events in an append-only table with global sequence numbers, aggregate metadata in a separate table, and optional snapshots for performance optimization. Database schema is managed automatically via embedded goose migrations.

Usage

db, err := sql.Open("postgres", connStr)
if err != nil {
    log.Fatal(err)
}

store, err := pg.New(ctx, db,
    pg.WithEventTypes(&MyEvent{}),
    pg.WithTraceHandler(tracer),
)
if err != nil {
    log.Fatal(err)
}

handler, err := eventsourced.NewHandler(ctx, &MyAggregate{}, store,
    eventsourced.WithEventPublisher(publisher),
)

Event Type Registration

All event types must be registered with WithEventTypes so the store can serialize and deserialize them. Event types must be pointers that implement eventsourced.Event. JSON struct tags are validated for uniqueness at registration.

Database Schema

The store creates and manages three tables:

  • events: Append-only event log with global and per-aggregate sequence numbers
  • aggregates: Registry of aggregate roots by ID and type
  • snapshots: Periodic aggregate state snapshots for faster replay

Migrations are tracked in a goose_db_version_pg table to avoid conflicts with application migrations.

Configuration

Use functional options to configure the store:

Index

Constants

This section is empty.

Variables

View Source
var ErrNoLogger = fmt.Errorf("logger must be set to a non-nil value")

ErrNoLogger is returned by WithLogger if a nil logger is provided.

View Source
var ErrOutboxAlreadyStarted = errors.New("outbox processor already started")

ErrOutboxAlreadyStarted is returned by OutboxProcessor.Start if it has already been started once. A single processor instance must not be started twice — create a new one instead.

Functions

This section is empty.

Types

type BackoffStrategy

type BackoffStrategy interface {
	// NextDelay calculates the delay before the next retry attempt
	NextDelay(attempt int) time.Duration
}

BackoffStrategy defines how retry delays are calculated

type ExponentialBackoff

type ExponentialBackoff struct {
	InitialDelay time.Duration
	MaxDelay     time.Duration
	Multiplier   float64
	Jitter       bool
}

ExponentialBackoff implements exponential backoff with jitter

func DefaultBackoffStrategy

func DefaultBackoffStrategy() *ExponentialBackoff

DefaultBackoffStrategy returns a sensible default exponential backoff strategy - Initial delay: 1 second - Max delay: 5 minutes - Multiplier: 2 (doubles each time) - Jitter: enabled

func (*ExponentialBackoff) NextDelay

func (e *ExponentialBackoff) NextDelay(attempt int) time.Duration

NextDelay calculates the exponential backoff delay for the given attempt. Attempt counting starts at 0 (first failure).

type FixedBackoff

type FixedBackoff struct {
	Delay time.Duration
}

FixedBackoff implements a fixed delay backoff (legacy behavior)

func (*FixedBackoff) NextDelay

func (f *FixedBackoff) NextDelay(_ int) time.Duration

NextDelay returns the same fixed delay for all attempts

type IdempotencyCleanup

type IdempotencyCleanup struct {
	KeysDeleted int
	Duration    time.Duration
}

IdempotencyCleanup records that the background cleanup deleted some number of expired idempotency records.

func (IdempotencyCleanup) IsMetric

func (IdempotencyCleanup) IsMetric()

IsMetric implements eventsourced.Metric.

type Logger

type Logger interface {
	Info(msg string, args ...any)
	Error(msg string, args ...any)
}

Logger defines the logging interface used by the event store for migration and operational messages.

type Option

type Option func(store *Store) error

Option is used to modify Store

func WithCommittedReads added in v2.2.0

func WithCommittedReads() Option

WithCommittedReads makes MaxGlobalSequenceNo and Events see only events below every in-flight insert. Use it for a store that read views back fill from.

A global sequence number is taken when an event is inserted, not when it commits, so 102 can commit before 101. Without this option a back fill that reads while 101 is in flight advances past it, and the read view then drops 101 as already processed.

Each call to MaxGlobalSequenceNo or Events reads the position under a SHARE lock on the events table. The lock waits for in-flight inserts and queues new ones behind it for at most 2 s per attempt; a call makes up to 10 attempts 500 ms apart (about 25 s in all) before returning the lock timeout error. It needs UPDATE, DELETE or TRUNCATE (or MAINTAIN on PostgreSQL 17+) privilege on events, and the primary: a standby can't take the lock or see the primary's in-flight inserts.

While the lock waits it cancels an autovacuum of events, and anything holding events longer than the lock timeout (a data migration updating events, a statement_timeout under 2 s) makes the call fail after its attempts: don't run such migrations while read views back fill.

It relies on the events sequence having CACHE 1 (the default), and New fails if it doesn't: a cached sequence hands out numbers before any insert holds the lock. Numbers taken outside an INSERT (an explicit nextval, a COPY with explicit values) aren't covered either.

func WithEventTypeAliases added in v2.4.0

func WithEventTypeAliases(aliases map[string]any) Option

WithEventTypeAliases lets the store read events stored under another name than their type's current one, e.g. after the type's package moved. Each key is a name as stored (for a generic event such as eventsourced.ExternalEvent[*example.org/old/handlers.T] that includes the type argument's full import path), each value a pointer to an event type registered with WithEventTypes to decode it into. Aliases are only read: events are always stored under their type's current name, and a type that is only reachable through an alias can't be stored. New fails if an alias is empty, equals a registered name, or points to an unregistered type.

func WithEventTypes

func WithEventTypes(eventTypes ...interface{}) Option

WithEventTypes registers event types with the event store

func WithIgnoreUndefinedEventTypesDuringReplay

func WithIgnoreUndefinedEventTypesDuringReplay() Option

WithIgnoreUndefinedEventTypesDuringReplay ignores undefined event types in the event store

func WithLogger

func WithLogger(logger Logger) Option

WithLogger sets the logger used by the event store for migration and operational logging. Defaults to slog.Default().

func WithMetrics

func WithMetrics(metrics eventsourced.MetricsRecorder) Option

WithMetrics sets the metrics recorder to use on the store. The metrics recorder will receive callbacks for event storage, snapshot operations, and other store operations to enable monitoring and observability.

func WithOutbox

func WithOutbox(enabled bool) Option

WithOutbox enables the transactional outbox pattern for reliable event publishing. When enabled, events are written to an outbox table in the same transaction as the event store, ensuring at-least-once delivery semantics. A background processor must be started separately to publish events from the outbox.

func WithOutboxAggregateOrder added in v2.3.0

func WithOutboxAggregateOrder() Option

WithOutboxAggregateOrder makes the outbox processor publish one aggregate's events in the order they were stored. Without it an entry that failed and backs off, or one in another processor's batch, is overtaken by its aggregate's later events. Use it when consumers depend on that order, e.g. one that drops an update for something whose create it hasn't seen.

The cost: an entry that keeps failing holds back its aggregate's later events until it is published or reaches the retry limit, so pick the retry limit with that in mind and alert on old unpublished entries. Every processor must use the same retry limit: one with a higher limit retries an entry the others gave up on, after its successors went out. Each batch takes one entry per aggregate, and the processor runs batches back to back while they find entries.

It relies on the events sequence having CACHE 1 (the default), and New fails if it doesn't. It has no effect without WithOutbox(true).

func WithTraceHandler

func WithTraceHandler(traceHandler eventsourced.TraceHandler) Option

WithTraceHandler sets the trace handler to use on the store

type OutboxBatch

type OutboxBatch struct {
	EntriesProcessed int
	Duration         time.Duration
	Errors           int
}

OutboxBatch records the result of one outbox polling cycle: how many entries were picked up, how long the whole batch took (including all per-entry publishes and the surrounding transaction), and how many entries errored along the way.

func (OutboxBatch) IsMetric

func (OutboxBatch) IsMetric()

IsMetric implements eventsourced.Metric.

type OutboxCleanup

type OutboxCleanup struct {
	EntriesDeleted int
	Duration       time.Duration
}

OutboxCleanup records that the background cleanup deleted some number of already-published entries.

func (OutboxCleanup) IsMetric

func (OutboxCleanup) IsMetric()

IsMetric implements eventsourced.Metric.

type OutboxProcessor

type OutboxProcessor struct {
	// contains filtered or unexported fields
}

OutboxProcessor polls the outbox table and publishes events

func NewOutboxProcessor

func NewOutboxProcessor(store *Store, publisher eventsourced.EventPublisher, opts ...OutboxProcessorOption) *OutboxProcessor

NewOutboxProcessor creates a new outbox processor

func (*OutboxProcessor) Start

func (p *OutboxProcessor) Start(ctx context.Context) error

Start begins processing the outbox in the background. It blocks until the context is cancelled or Stop is called. Calling Start more than once on the same processor returns ErrOutboxAlreadyStarted.

func (*OutboxProcessor) Stop

func (p *OutboxProcessor) Stop()

Stop gracefully stops the outbox processor. It is safe to call multiple times (subsequent calls are no-ops) and blocks until any in-flight processBatch / cleanup has finished. A Stop call before Start signals the processor to exit immediately when Start is eventually invoked.

type OutboxProcessorOption

type OutboxProcessorOption func(*OutboxProcessor)

func WithOutboxBackoffStrategy

func WithOutboxBackoffStrategy(strategy BackoffStrategy) OutboxProcessorOption

WithOutboxBackoffStrategy sets the backoff strategy for retrying failed entries (default: exponential backoff)

func WithOutboxBatchSize

func WithOutboxBatchSize(size int) OutboxProcessorOption

WithOutboxBatchSize sets how many events to process per batch (default: 100)

func WithOutboxCleanupAfter

func WithOutboxCleanupAfter(duration time.Duration) OutboxProcessorOption

WithOutboxCleanupAfter sets how long to keep published events before cleanup (default: 7 days)

func WithOutboxCleanupInterval

func WithOutboxCleanupInterval(interval time.Duration) OutboxProcessorOption

WithOutboxCleanupInterval sets how often the cleanup task runs (default: 1h)

func WithOutboxMaxRetries

func WithOutboxMaxRetries(max int) OutboxProcessorOption

WithOutboxMaxRetries sets maximum retry attempts before giving up (default: 10)

func WithOutboxMetrics

func WithOutboxMetrics(metrics eventsourced.MetricsRecorder) OutboxProcessorOption

WithOutboxMetrics sets the metrics recorder for the outbox processor. The recorder is the same eventsourced.MetricsRecorder used by Store; the processor's records (pg.OutboxBatch, pg.OutboxPublish, pg.OutboxRetry, pg.OutboxCleanup) flow through the single Record method.

func WithOutboxPollInterval

func WithOutboxPollInterval(interval time.Duration) OutboxProcessorOption

WithOutboxPollInterval sets how often to poll for unpublished events (default: 1s)

func WithOutboxRetryBackoff

func WithOutboxRetryBackoff(backoff time.Duration) OutboxProcessorOption

WithOutboxRetryBackoff sets the backoff duration between retries (default: 30s)

type OutboxPublish

type OutboxPublish struct {
	EventType string
	Success   bool
	Duration  time.Duration
}

OutboxPublish records a single outbox publish attempt. Success is true when the publisher accepted the event without error.

func (OutboxPublish) IsMetric

func (OutboxPublish) IsMetric()

IsMetric implements eventsourced.Metric.

type OutboxRetry

type OutboxRetry struct {
	EventType  string
	RetryCount int
	Permanent  bool
}

OutboxRetry records that an outbox entry has been scheduled for retry (or marked permanently failed). RetryCount is the new error count after this failure.

func (OutboxRetry) IsMetric

func (OutboxRetry) IsMetric()

IsMetric implements eventsourced.Metric.

type Payload

type Payload json.RawMessage

Payload is a JSON raw message type that implements database/sql Scanner and Valuer interfaces for transparent JSON serialization between Go and PostgreSQL.

func (*Payload) Scan

func (p *Payload) Scan(value interface{}) error

Scan implements the sql.Scanner interface for reading from database results.

func (Payload) Value

func (p Payload) Value() (driver.Value, error)

Value implements the driver.Valuer interface for database storage.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is a PostgreSQL-backed implementation of eventsourced.EventStore. It persists events, aggregate metadata, and snapshots in PostgreSQL tables, with automatic schema migrations via goose.

func New

func New(ctx context.Context, db *sql.DB, opts ...Option) (*Store, error)

New instantiates an event store using the provided SQL-connection (PostgreSQL) and event types

func (Store) CheckIdempotency

func (p Store) CheckIdempotency(ctx context.Context, aggregate eventsourced.Aggregate, idempotencyKey string) (eventsourced.Event, error)

CheckIdempotency checks if a command with the given idempotency key has already been processed. Returns the cached event if found, nil otherwise.

func (Store) CleanupExpiredIdempotencyKeys

func (p Store) CleanupExpiredIdempotencyKeys(ctx context.Context) (int64, error)

CleanupExpiredIdempotencyKeys removes expired idempotency records.

func (Store) Events

func (p Store) Events(ctx context.Context, fromGlobalSequenceNo int, c chan eventsourced.Event, aggregateTypes ...eventsourced.Aggregate) error

Events retrieves all events since the given global sequence number, optionally filtered by aggregate types, and sends them to the provided channel. Results are fetched in batches of 10,000 and the channel is closed when all events have been sent. With WithCommittedReads the stream ends at the committed position read when it starts.

func (Store) GetAggregateRoots

func (p Store) GetAggregateRoots(ctx context.Context, aggregateType reflect.Type) ([]eventsourced.ID, error)

GetAggregateRoots retrieves the IDs of all aggregate roots matching the given type from PostgreSQL.

func (Store) GetEvents

func (p Store) GetEvents(ctx context.Context, aggregate eventsourced.Aggregate, sinceSnapshot eventsourced.Snapshot) ([]eventsourced.EventWrapper, error)

GetEvents retrieves all events for the given aggregate since the provided snapshot. If sinceSnapshot is nil, all events for the aggregate are returned.

func (Store) MaxGlobalSequenceNo

func (p Store) MaxGlobalSequenceNo(ctx context.Context) (int, error)

MaxGlobalSequenceNo returns the highest global sequence number across all events, or 0 if no events exist. With WithCommittedReads it returns the highest one with no in-flight insert below it.

func (Store) RestoreSnapshot

func (p Store) RestoreSnapshot(ctx context.Context, aggregate eventsourced.Aggregate) (eventsourced.Snapshot, error)

RestoreSnapshot retrieves the latest snapshot for the given aggregate and unmarshals it into the aggregate. Returns nil if no snapshot exists.

func (Store) StoreEvent

func (p Store) StoreEvent(ctx context.Context, aggregate eventsourced.Aggregate, event eventsourced.Event, aggregateSequenceNo int, newAggregate bool) error

StoreEvent persists an event for the given aggregate in a transaction. If newAggregate is true, the aggregate is also registered. Returns ErrStaleAggregate on sequence number conflicts and ErrAggregateAlreadyExists on duplicate aggregate creation.

func (Store) StoreIdempotency

func (p Store) StoreIdempotency(ctx context.Context, aggregate eventsourced.Aggregate, idempotencyKey string, event eventsourced.Event, ttl time.Duration) error

StoreIdempotency stores the idempotency key and resulting event for a processed command.

func (Store) StoreSnapshot

func (p Store) StoreSnapshot(ctx context.Context, aggregate eventsourced.Aggregate, aggregateSequenceNo int) error

StoreSnapshot serializes the aggregate to JSON and stores it as a snapshot at the given sequence number.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址