retention

package
v0.31.39 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 13, 2026 License: Apache-2.0 Imports: 25 Imported by: 0

Documentation

Overview

Package retention enforces opt-in concept version lifetimes. Log lines keep their dedicated chunk-aware sweeper; all graph concepts use this bounded path.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Archive

type Archive interface {
	Upload(context.Context, string, string, []byte, string) (string, error)
	DownloadWithLimit(context.Context, string, string, int64) ([]byte, error)
}

Archive verifies the bytes from object storage, not only an upload receipt.

type ChunkReport added in v0.31.18

type ChunkReport struct {
	Chunk            string    `json:"chunk"`
	Hypertable       string    `json:"hypertable"`
	RangeStart       time.Time `json:"rangeStart"`
	RangeEnd         time.Time `json:"rangeEnd"`
	TotalBytes       int64     `json:"totalBytes"`
	IndexBytes       int64     `json:"indexBytes"`
	LiveBytes        int64     `json:"liveBytes"`
	LiveRows         int64     `json:"liveRows"`
	ReclaimableBytes int64     `json:"reclaimableBytes"`
	AfterBytes       int64     `json:"afterBytes,omitempty"`
	ReclaimedBytes   int64     `json:"reclaimedBytes,omitempty"`
	Attempts         int       `json:"attempts,omitempty"`
	DurationMs       int64     `json:"durationMs,omitempty"`
	// Vacuum is "done", or why the VACUUM after a rewrite did not run
	// (autovacuum then sets the visibility map in its own time).
	Vacuum string `json:"vacuum,omitempty"`
}

ChunkReport is one chunk, as measured and, after a rewrite, as it became.

type CompactOptions added in v0.31.18

type CompactOptions struct {
	// DryRun measures and reports what a pass would do, without rewriting a
	// chunk or writing the progress table.
	DryRun bool
	// MinReclaimBytes and MinReclaimPercent: a chunk is worth rewriting only when
	// it would give back at least this many bytes AND at least this share of its
	// total size. Both, so neither a huge chunk with a sliver of free space nor a
	// tiny chunk that is mostly free is rewritten.
	MinReclaimBytes   int64
	MinReclaimPercent int64
	// MeasureChunks bounds how many chunks one pass scans to (re)measure.
	MeasureChunks int
}

CompactOptions is one pass's policy: the thresholds the DSL chose, and whether to rewrite at all.

type CompactionReport added in v0.31.18

type CompactionReport struct {
	DryRun bool `json:"dryRun"`
	// Outcome is compacted, planned (a dry run that would rewrite Chunk),
	// skipped (nothing rewritten; Reason says why) or failed (a rewrite was
	// attempted and did not finish; the chunk is as it was).
	Outcome string `json:"outcome"`
	// Reason, for skipped and failed: disabled, already_running,
	// reorder_unavailable, no_candidate, room_unknown, no_room, not_owner,
	// wal_backlog, wal_unknown, no_reorder_index, out_of_time; lock_timeout,
	// statement_timeout, cancelled, error.
	Reason string `json:"reason,omitempty"`
	Detail string `json:"detail,omitempty"`
	// Chunk is the chunk chosen (and, for compacted, what became of it).
	Chunk *ChunkReport `json:"chunk,omitempty"`
	// Closed is how many closed, uncompressed chunks the pass considered,
	// Measured those it (re)measured, MeasureFailures those it could not, and
	// Eligible those over both thresholds.
	Closed          int           `json:"closed"`
	Measured        []ChunkReport `json:"measured,omitempty"`
	MeasureFailures []string      `json:"measureFailures,omitempty"`
	Eligible        int           `json:"eligible"`
	Room            *Room         `json:"room,omitempty"`
	WAL             *WAL          `json:"wal,omitempty"`
	DurationMs      int64         `json:"durationMs"`
}

CompactionReport is what a pass did, and why when it did nothing.

type Compactor added in v0.31.18

type Compactor struct {
	// Session opens the dedicated maintenance session every statement of a pass
	// runs on (PluginContext.OpenMaintenanceSession). Required.
	Session func(context.Context) (*sql.Conn, func(), error)
	// Storage reads the database-health report; nil or an error means none.
	Storage func(context.Context) (StorageReading, error)
	// VolumeBytes is MEMQL_CHUNK_COMPACTION_VOLUME_BYTES: the data volume's size,
	// used only when there is no fresh report. 0 means unset.
	VolumeBytes int64
	// Tables are the hypertables a pass considers, resolved through the
	// session's search_path.
	Tables []string
	Logger *slog.Logger
	// Now is the pass's clock for every decision and every stored time; nil is
	// time.Now. The closure cutoff is passed to the database from it too.
	Now func() time.Time
	// Attempts and RetryPause override maxAttempts and defaultRetryPause (tests).
	Attempts   int
	RetryPause time.Duration
	// Disabled is the operator's stop switch (MEMQL_CHUNK_COMPACTION_ENABLED,
	// plugin.go), with the reason it reads as disabled. A disabled pass that is
	// not a dry run answers skipped/disabled before it opens a session; a dry run
	// still measures and reports, so the effect of turning it back on can be
	// seen first.
	Disabled       bool
	DisabledReason string
}

Compactor runs a closed-chunk compaction pass. See the header.

func (*Compactor) Run added in v0.31.18

Run executes one pass. It returns an error only when the pass could not start or could not read the catalog -- nothing was decided and nothing changed. Every outcome after that, a failed rewrite included, is the report's.

type Integration

type Integration struct {
	// contains filtered or unexported fields
}

func (*Integration) Capabilities

func (i *Integration) Capabilities() []memql.IntegrationCapability

func (*Integration) IntegrationName

func (*Integration) IntegrationName() string

type Report

type Report struct {
	Scanned   int      `json:"scanned"`
	Eligible  int      `json:"eligible"`
	Archived  int      `json:"archived"`
	Deleted   int      `json:"deleted"`
	Oversized int      `json:"oversized"`
	Objects   []string `json:"objects,omitempty"`
}

type Room added in v0.31.18

type Room struct {
	// Source is where capacity came from: databaseHealth (the monitor's fresh
	// report) or configured (MEMQL_CHUNK_COMPACTION_VOLUME_BYTES).
	Source        string `json:"source"`
	CapacityBytes int64  `json:"capacityBytes"`
	UsedBytes     int64  `json:"usedBytes"`
	// UsableFreeBytes is capacity less the reserve PostgreSQL cannot use, less
	// what is used.
	UsableFreeBytes int64 `json:"usableFreeBytes"`
	// NeedBytes is twice the chosen (or, when none fits, the smallest) eligible
	// chunk's live data and indexes.
	NeedBytes int64 `json:"needBytes,omitempty"`
	// WALFreeBytes is the WAL volume's usable free space, when a fresh report
	// gave it.
	WALFreeBytes *int64     `json:"walFreeBytes,omitempty"`
	ObservedAt   *time.Time `json:"observedAt,omitempty"`
}

Room is the data volume as the pass judged it.

type StorageReading added in v0.31.18

type StorageReading struct {
	ObservedAt        time.Time
	DataBytes         int64
	DataCapacityBytes int64
	WALBytes          int64
	WALCapacityBytes  int64
	Issues            []string
}

StorageReading is the database-health monitor's report, as the room check uses it.

type Sweeper

type Sweeper struct {
	DB         *bun.DB
	Concepts   memorynodes.Registry
	Admit      func(context.Context, memorynodes.MemoryNode) bool
	Invalidate func(string)
	Archive    Archive
	Container  string
	Now        func() time.Time
}

func (*Sweeper) Run

func (s *Sweeper) Run(ctx context.Context, dry bool) (Report, error)

Run divides a bounded pass fairly among declarations. Policy-specific cursor keys include the policy contents, so changing TTL cannot strand older rows.

type WAL added in v0.31.18

type WAL struct {
	ArchiveMode     string     `json:"archiveMode"`
	PendingSegments int64      `json:"pendingSegments"`
	LastArchivedAt  *time.Time `json:"lastArchivedAt,omitempty"`
}

WAL is the archive as the pass judged it.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址