kv

package
v0.202611.36 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 14, 2026 License: GPL-3.0 Imports: 6 Imported by: 0

Documentation

Overview

Package kv defines the transactional, ordered key-value store that holds Amp's pristine: the channel graphs, the channel logs, and the working-copy tables. The pristine is a cache that can be rebuilt from the changes, but it must stay consistent, so every update happens in a transaction.

Two implementations exist: disk.Store (package kv/disk), persistent trees on a medium such as files, and Memory, for tests and for embedding.

Index

Constants

This section is empty.

Variables

View Source
var ErrConflict = errors.New("kv: the transaction conflicts with a concurrent one")

ErrConflict is returned by Commit when another transaction published a conflicting write after the transaction's snapshot. Nothing of the transaction is published; the caller runs its operation again.

View Source
var ErrEmptyValue = errors.New("kv: empty value")

ErrEmptyValue is returned by Put for an empty value. Get returns nil for a missing key, so values must not be empty.

View Source
var ErrOwnerMoving = fmt.Errorf("%w: this process no longer owns the store", ErrUnavailable)

ErrOwnerMoving marks an operation that the store didn't execute, because this process no longer owns the store: another process takes it over. A retry is safe for every operation. It matches ErrUnavailable too, so that a read raises it as a read error.

View Source
var ErrReadOnly = errors.New("kv: transaction is read-only")

ErrReadOnly is returned by write operations in a read-only transaction.

View Source
var ErrSnapshotLost = fmt.Errorf("%w: the snapshot's data went away", ErrUnavailable)

ErrSnapshotLost marks a read of a snapshot whose data went away: GC of a newer owner removed it. A retry reads a new snapshot. It matches ErrUnavailable too.

View Source
var ErrUnavailable = errors.New("kv: the store's data is unavailable now")

ErrUnavailable marks a read that a store couldn't do now, such as a read of a remote medium that timed out or was throttled after its retries: the store isn't damaged, and the operation can run again later. A store on files never returns it.

View Source
var ErrUnsupported = errors.New("kv: this process doesn't support the store's format")

ErrUnsupported marks a store whose format this process doesn't support, such as during an upgrade.

Functions

func Catch added in v0.202611.9

func Catch(f func() error) (err error)

Catch runs f, and returns the read error that it raised, if any.

func CatchRead added in v0.202611.9

func CatchRead(err *error)

CatchRead recovers a panic of a *ReadError into *err; other panics go on. Defer it where a request or a background job starts.

func CommitWithoutSync

func CommitWithoutSync(t Txn) error

CommitWithoutSync commits t as Commit does, but a store that can doesn't wait for the commit to reach the disk: a crash can lose it, and the store then has the state before it. A later commit that syncs makes it durable. It is for a commit whose loss recovery repairs, such as the end of an operation whose record recovery runs again.

func DiffEntries added in v0.202611.5

func DiffEntries(t Txn, a, b string, f func(key, value []byte) bool)

DiffEntries is DiffKeys that also calls f on the keys that both buckets have with different values (with their values in a), with the same costs.

func DiffKeys added in v0.202611.5

func DiffKeys(t Txn, a, b string, f func(key, value []byte) bool)

DiffKeys calls f on the keys of bucket a that bucket b doesn't have, with their values in a, in key order, until f returns false. A store whose buckets share subtrees, as the copies of disk.Store do, skips each subtree that both buckets share: the cost is that of the parts of the trees that differ, not of their sizes. The result never depends on the sharing: two buckets with the same keys can have different trees. Other stores, and buckets that the transaction wrote, merge the two buckets' keys with cursors.

func MergeDiffEntries added in v0.202611.5

func MergeDiffEntries(a, b Cursor, f func(key, value []byte) bool)

MergeDiffEntries is DiffEntries of the buckets of two cursors.

func MergeDiffKeys added in v0.202611.5

func MergeDiffKeys(a, b Cursor, f func(key, value []byte) bool)

MergeDiffKeys is DiffKeys of the buckets of two cursors: it reads every key of a, and the keys of b up to a's last.

func RaiseRead added in v0.202611.9

func RaiseRead(err error)

RaiseRead panics with a *ReadError of err.

func UnionKeys

func UnionKeys(t Txn, names []string, f func(key []byte) bool)

UnionKeys calls f on the keys of the named buckets, until f returns false: on each key at least once, in no particular order. A store whose buckets share subtrees, as the copies of disk.Store do, reads each shared subtree once: the cost is that of the distinct data, not of the sum of the buckets.

func UnionKeysIn added in v0.202611.5

func UnionKeysIn(t Txn, names []string, lo, hi []byte, f func(key []byte) bool)

UnionKeysIn is UnionKeys for the keys k with lo <= k < hi; a nil hi doesn't bound them. A store whose buckets share subtrees reads only the subtrees that can hold such keys, each once.

func Update

func Update(s Store, f func(Txn) error) error

Update runs f in a write transaction, and commits if f returns nil.

func View

func View(s Store, f func(Txn) error) error

View runs f in a read transaction.

Types

type Bucket

type Bucket interface {
	// Get returns the value of a key, or nil if the key is missing.
	Get(key []byte) []byte
	Put(key, value []byte) error
	Delete(key []byte) error
	Cursor() Cursor
}

Bucket is an ordered map from keys to values.

type Committing

type Committing interface {
	Get(bucket string, key []byte) []byte
	// Last returns the last key of a bucket and its value, or a nil key.
	Last(bucket string) (key, value []byte)
	Put(bucket string, key, value []byte) error
}

Committing is the state that an OnCommit function reads and writes.

type Cursor

type Cursor interface {
	First() (key, value []byte)
	Last() (key, value []byte)
	// Seek moves to the first key that is greater than or equal to key.
	Seek(key []byte) (k, v []byte)
	Next() (key, value []byte)
	Prev() (key, value []byte)
}

Cursor iterates over the keys of a bucket in byte order. Each method returns the key and value at the new position, or a nil key at the end.

type Memory

type Memory struct {
	// contains filtered or unexported fields
}

Memory is a Store in memory. A transaction works on a copy-on-write snapshot of the store, so readers never see uncommitted writes.

func NewMemory

func NewMemory() *Memory

NewMemory returns an empty store.

func (*Memory) Begin

func (m *Memory) Begin(writable bool) (Txn, error)

func (*Memory) Close

func (m *Memory) Close() error

type ReadError added in v0.202611.9

type ReadError struct{ Err error }

ReadError is the error of a read of a store that failed. The reads of a Bucket and a Cursor return no error, so they panic with a *ReadError (RaiseRead), and the boundary of a request or of a background job turns it into its error (CatchRead). The deferred functions on the way out release the locks.

func (*ReadError) Error added in v0.202611.9

func (e *ReadError) Error() string

func (*ReadError) Unwrap added in v0.202611.9

func (e *ReadError) Unwrap() error

type Store

type Store interface {
	// Begin starts a transaction. A store allows any number of read
	// transactions, and one write transaction at a time.
	Begin(writable bool) (Txn, error)
	Close() error
}

Store is a transactional key-value store with ordered keys.

type Txn

type Txn interface {
	// Bucket returns the named bucket. In a write transaction, it creates
	// the bucket if it doesn't exist. In a read transaction, a missing
	// bucket is empty.
	Bucket(name string) Bucket
	// DeleteBucket removes a bucket and all of its keys.
	DeleteBucket(name string) error
	// CopyBucket replaces the bucket to with a copy of the bucket from. A
	// persistent store shares the tree, so the copy costs time
	// independent of the size of the bucket.
	CopyBucket(from, to string) error
	// Buckets lists the names of the buckets with the given prefix, in
	// order.
	Buckets(prefix string) []string
	Writable() bool
	// OnCommit adds f to a write transaction. Commit calls f when the
	// transaction can no longer conflict, with the store's current state
	// and the transaction's writes, and publishes what f writes together
	// with the transaction. f must write only keys that no transaction
	// writes outside OnCommit, such as the next key of a sequence; then
	// two transactions that append to one sequence don't conflict.
	OnCommit(f func(Committing) error)
	Commit() error
	Rollback() error
}

Txn is a transaction. Byte slices that a transaction returns are valid only until the transaction ends, and the caller must not modify them.

Source Files

  • kv.go
  • memory.go
  • readerr.go

Directories

Path Synopsis
Package disk is Amp's pristine store: persistent B+trees whose nodes never change once written, shared between snapshots and between buckets, so that copying a bucket costs nothing.
Package disk is Amp's pristine store: persistent B+trees whose nodes never change once written, shared between snapshots and between buckets, so that copying a bucket costs nothing.
meta
Package meta is the metadata store of a cloud store (kv/disk's cloud medium): the small, mutable state of each store, in a service that orders its writes and checks their conditions atomically, such as DynamoDB (package meta/dynamo) or Postgres (package meta/postgres).
Package meta is the metadata store of a cloud store (kv/disk's cloud medium): the small, mutable state of each store, in a service that orders its writes and checks their conditions atomically, such as DynamoDB (package meta/dynamo) or Postgres (package meta/postgres).
meta/dynamo
Package dynamo is a metadata store of the cloud medium (meta.Store) in one DynamoDB table, with the semantics of meta.Memory.
Package dynamo is a metadata store of the cloud medium (meta.Store) in one DynamoDB table, with the semantics of meta.Memory.
meta/metatest
Package metatest checks an implementation of meta.Store: the conformance tests that every metadata store passes.
Package metatest checks an implementation of meta.Store: the conformance tests that every metadata store passes.
meta/postgres
Package postgres is a metadata store (meta.Store) on Postgres, such as Aurora PostgreSQL.
Package postgres is a metadata store (meta.Store) on Postgres, such as Aurora PostgreSQL.
Package kvtest checks a kv.Store against a model: a sorted map per bucket.
Package kvtest checks a kv.Store against a model: a sorted map per bucket.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址