Documentation
¶
Overview ¶
Package kv defines the transactional, ordered key-value store that holds Amp's pristine: the channel graphs, the channel logs, and the working-copy tables. The pristine is a cache that can be rebuilt from the changes, but it must stay consistent, so every update happens in a transaction.
Two implementations exist: disk.Store (package kv/disk), persistent trees on a medium such as files, and Memory, for tests and for embedding.
Index ¶
- Variables
- func Catch(f func() error) (err error)
- func CatchRead(err *error)
- func CommitWithoutSync(t Txn) error
- func DiffEntries(t Txn, a, b string, f func(key, value []byte) bool)
- func DiffKeys(t Txn, a, b string, f func(key, value []byte) bool)
- func MergeDiffEntries(a, b Cursor, f func(key, value []byte) bool)
- func MergeDiffKeys(a, b Cursor, f func(key, value []byte) bool)
- func RaiseRead(err error)
- func UnionKeys(t Txn, names []string, f func(key []byte) bool)
- func UnionKeysIn(t Txn, names []string, lo, hi []byte, f func(key []byte) bool)
- func Update(s Store, f func(Txn) error) error
- func View(s Store, f func(Txn) error) error
- type Bucket
- type Committing
- type Cursor
- type Memory
- type ReadError
- type Store
- type Txn
Constants ¶
This section is empty.
Variables ¶
ErrConflict is returned by Commit when another transaction published a conflicting write after the transaction's snapshot. Nothing of the transaction is published; the caller runs its operation again.
ErrEmptyValue is returned by Put for an empty value. Get returns nil for a missing key, so values must not be empty.
var ErrOwnerMoving = fmt.Errorf("%w: this process no longer owns the store", ErrUnavailable)
ErrOwnerMoving marks an operation that the store didn't execute, because this process no longer owns the store: another process takes it over. A retry is safe for every operation. It matches ErrUnavailable too, so that a read raises it as a read error.
ErrReadOnly is returned by write operations in a read-only transaction.
var ErrSnapshotLost = fmt.Errorf("%w: the snapshot's data went away", ErrUnavailable)
ErrSnapshotLost marks a read of a snapshot whose data went away: GC of a newer owner removed it. A retry reads a new snapshot. It matches ErrUnavailable too.
ErrUnavailable marks a read that a store couldn't do now, such as a read of a remote medium that timed out or was throttled after its retries: the store isn't damaged, and the operation can run again later. A store on files never returns it.
ErrUnsupported marks a store whose format this process doesn't support, such as during an upgrade.
Functions ¶
func Catch ¶ added in v0.202611.9
Catch runs f, and returns the read error that it raised, if any.
func CatchRead ¶ added in v0.202611.9
func CatchRead(err *error)
CatchRead recovers a panic of a *ReadError into *err; other panics go on. Defer it where a request or a background job starts.
func CommitWithoutSync ¶
CommitWithoutSync commits t as Commit does, but a store that can doesn't wait for the commit to reach the disk: a crash can lose it, and the store then has the state before it. A later commit that syncs makes it durable. It is for a commit whose loss recovery repairs, such as the end of an operation whose record recovery runs again.
func DiffEntries ¶ added in v0.202611.5
DiffEntries is DiffKeys that also calls f on the keys that both buckets have with different values (with their values in a), with the same costs.
func DiffKeys ¶ added in v0.202611.5
DiffKeys calls f on the keys of bucket a that bucket b doesn't have, with their values in a, in key order, until f returns false. A store whose buckets share subtrees, as the copies of disk.Store do, skips each subtree that both buckets share: the cost is that of the parts of the trees that differ, not of their sizes. The result never depends on the sharing: two buckets with the same keys can have different trees. Other stores, and buckets that the transaction wrote, merge the two buckets' keys with cursors.
func MergeDiffEntries ¶ added in v0.202611.5
MergeDiffEntries is DiffEntries of the buckets of two cursors.
func MergeDiffKeys ¶ added in v0.202611.5
MergeDiffKeys is DiffKeys of the buckets of two cursors: it reads every key of a, and the keys of b up to a's last.
func RaiseRead ¶ added in v0.202611.9
func RaiseRead(err error)
RaiseRead panics with a *ReadError of err.
func UnionKeys ¶
UnionKeys calls f on the keys of the named buckets, until f returns false: on each key at least once, in no particular order. A store whose buckets share subtrees, as the copies of disk.Store do, reads each shared subtree once: the cost is that of the distinct data, not of the sum of the buckets.
func UnionKeysIn ¶ added in v0.202611.5
UnionKeysIn is UnionKeys for the keys k with lo <= k < hi; a nil hi doesn't bound them. A store whose buckets share subtrees reads only the subtrees that can hold such keys, each once.
func Update ¶
Update runs f in a write transaction, and commits if f returns nil.
Types ¶
type Bucket ¶
type Bucket interface {
// Get returns the value of a key, or nil if the key is missing.
Get(key []byte) []byte
Put(key, value []byte) error
Delete(key []byte) error
Cursor() Cursor
}
Bucket is an ordered map from keys to values.
type Committing ¶
type Committing interface {
Get(bucket string, key []byte) []byte
// Last returns the last key of a bucket and its value, or a nil key.
Last(bucket string) (key, value []byte)
Put(bucket string, key, value []byte) error
}
Committing is the state that an OnCommit function reads and writes.
type Cursor ¶
type Cursor interface {
First() (key, value []byte)
Last() (key, value []byte)
// Seek moves to the first key that is greater than or equal to key.
Seek(key []byte) (k, v []byte)
Next() (key, value []byte)
Prev() (key, value []byte)
}
Cursor iterates over the keys of a bucket in byte order. Each method returns the key and value at the new position, or a nil key at the end.
type Memory ¶
type Memory struct {
// contains filtered or unexported fields
}
Memory is a Store in memory. A transaction works on a copy-on-write snapshot of the store, so readers never see uncommitted writes.
type ReadError ¶ added in v0.202611.9
type ReadError struct{ Err error }
ReadError is the error of a read of a store that failed. The reads of a Bucket and a Cursor return no error, so they panic with a *ReadError (RaiseRead), and the boundary of a request or of a background job turns it into its error (CatchRead). The deferred functions on the way out release the locks.
type Store ¶
type Store interface {
// Begin starts a transaction. A store allows any number of read
// transactions, and one write transaction at a time.
Begin(writable bool) (Txn, error)
Close() error
}
Store is a transactional key-value store with ordered keys.
type Txn ¶
type Txn interface {
// Bucket returns the named bucket. In a write transaction, it creates
// the bucket if it doesn't exist. In a read transaction, a missing
// bucket is empty.
Bucket(name string) Bucket
// DeleteBucket removes a bucket and all of its keys.
DeleteBucket(name string) error
// CopyBucket replaces the bucket to with a copy of the bucket from. A
// persistent store shares the tree, so the copy costs time
// independent of the size of the bucket.
CopyBucket(from, to string) error
// Buckets lists the names of the buckets with the given prefix, in
// order.
Buckets(prefix string) []string
Writable() bool
// OnCommit adds f to a write transaction. Commit calls f when the
// transaction can no longer conflict, with the store's current state
// and the transaction's writes, and publishes what f writes together
// with the transaction. f must write only keys that no transaction
// writes outside OnCommit, such as the next key of a sequence; then
// two transactions that append to one sequence don't conflict.
OnCommit(f func(Committing) error)
Commit() error
Rollback() error
}
Txn is a transaction. Byte slices that a transaction returns are valid only until the transaction ends, and the caller must not modify them.
Source Files
¶
- kv.go
- memory.go
- readerr.go
Directories
¶
| Path | Synopsis |
|---|---|
|
Package disk is Amp's pristine store: persistent B+trees whose nodes never change once written, shared between snapshots and between buckets, so that copying a bucket costs nothing.
|
Package disk is Amp's pristine store: persistent B+trees whose nodes never change once written, shared between snapshots and between buckets, so that copying a bucket costs nothing. |
|
meta
Package meta is the metadata store of a cloud store (kv/disk's cloud medium): the small, mutable state of each store, in a service that orders its writes and checks their conditions atomically, such as DynamoDB (package meta/dynamo) or Postgres (package meta/postgres).
|
Package meta is the metadata store of a cloud store (kv/disk's cloud medium): the small, mutable state of each store, in a service that orders its writes and checks their conditions atomically, such as DynamoDB (package meta/dynamo) or Postgres (package meta/postgres). |
|
meta/dynamo
Package dynamo is a metadata store of the cloud medium (meta.Store) in one DynamoDB table, with the semantics of meta.Memory.
|
Package dynamo is a metadata store of the cloud medium (meta.Store) in one DynamoDB table, with the semantics of meta.Memory. |
|
meta/metatest
Package metatest checks an implementation of meta.Store: the conformance tests that every metadata store passes.
|
Package metatest checks an implementation of meta.Store: the conformance tests that every metadata store passes. |
|
meta/postgres
Package postgres is a metadata store (meta.Store) on Postgres, such as Aurora PostgreSQL.
|
Package postgres is a metadata store (meta.Store) on Postgres, such as Aurora PostgreSQL. |
|
Package kvtest checks a kv.Store against a model: a sorted map per bucket.
|
Package kvtest checks a kv.Store against a model: a sorted map per bucket. |