storage

package
v0.14.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 9, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// SHA256KeyBytes is the byte length of a SHA-256 content key.
	SHA256KeyBytes = sha256.Size
	// SHA256KeyHexBytes is the encoded length of a SHA-256 content key.
	SHA256KeyHexBytes = sha256.Size * 2
)
View Source 
const DefaultKeyPageSize = 256

DefaultKeyPageSize is used when a pager receives a non-positive limit.

Variables

View Source 
var (
	// ErrInvalidSHA256Key is returned when a key is not a SHA-256 digest.
	ErrInvalidSHA256Key = errors.New("storage: invalid sha256 key")
	// ErrInvalidSHA256HexKey is returned when a key is not a hex-encoded SHA-256 digest.
	ErrInvalidSHA256HexKey = errors.New("storage: invalid sha256 hex key")
	// ErrSHA256Mismatch is returned when data does not match its SHA-256 key.
	ErrSHA256Mismatch = errors.New("storage: sha256 key mismatch")
)
View Source 
var (
	// ErrInventoryKeyEmpty reports a malformed inventory entry.
	ErrInventoryKeyEmpty = errors.New("storage: inventory contains empty key")
	// ErrInventoryPagerInvalid reports a pager that violates BlobKeyPager's cursor contract.
	ErrInventoryPagerInvalid = errors.New("storage: invalid inventory pager")
)
View Source 
var (
	// ErrNotFound is returned when a key is missing.
	ErrNotFound = errors.New("storage: not found")
	// ErrKeyEmpty is returned for empty keys.
	ErrKeyEmpty = errors.New("storage: empty key")
)

Functions

func ParseSHA256KeyHex added in v0.4.0 

func ParseSHA256KeyHex(s string) ([]byte, error)

ParseSHA256KeyHex parses a hex-encoded SHA-256 key.

func SHA256Key added in v0.4.0 

func SHA256Key(data []byte) []byte

SHA256Key returns the content-addressed key for data.

func SHA256KeyHex added in v0.4.0 

func SHA256KeyHex(data []byte) string

SHA256KeyHex returns the hex-encoded content-addressed key for data.

func ValidateSHA256Key added in v0.4.0 

func ValidateSHA256Key(key []byte) error

ValidateSHA256Key verifies that key has the byte length of a SHA-256 digest.

func VerifySHA256Key added in v0.5.0 

func VerifySHA256Key(key, data []byte) error

VerifySHA256Key verifies that key is the SHA-256 digest of data.

Types

type BlobKeyLister added in v0.4.0 

type BlobKeyLister interface {
	ListKeys(ctx context.Context) ([][]byte, error)
}

BlobKeyLister is implemented by stores that can enumerate known blob keys.

type BlobKeyPager added in v0.12.0 

type BlobKeyPager interface {
	ListKeyPage(ctx context.Context, after []byte, limit int) (keys [][]byte, next []byte, err error)
}

BlobKeyPager returns at most limit keys strictly greater than after in bytewise order. The returned cursor is the last returned key; a zero-length key slice means enumeration is complete. Implementations should honor context cancellation while scanning and may reflect store mutations between page calls.

type BlobStore 

type BlobStore interface {
	Put(ctx context.Context, key []byte, data []byte) error
	Get(ctx context.Context, key []byte) ([]byte, error)
	Has(ctx context.Context, key []byte) (bool, error)
	Delete(ctx context.Context, key []byte) error
}

BlobStore is a minimal content-addressed blob API. Keys are typically hashes or opaque IDs.

type FileStore 

type FileStore struct {
	// contains filtered or unexported fields
}

FileStore stores blobs as files named by hex-encoded keys.

func NewFileStore 

func NewFileStore(dir string) (*FileStore, error)

NewFileStore opens or creates a directory-backed BlobStore.

func (*FileStore) Delete 

func (s *FileStore) Delete(ctx context.Context, key []byte) error

Delete removes key. Missing keys are not an error.

func (*FileStore) Get 

func (s *FileStore) Get(ctx context.Context, key []byte) ([]byte, error)

Get returns a copy of the value for key.

func (*FileStore) Has 

func (s *FileStore) Has(ctx context.Context, key []byte) (bool, error)

Has reports whether key exists.

func (*FileStore) ListKeyPage added in v0.12.0 

func (s *FileStore) ListKeyPage(ctx context.Context, after []byte, limit int) ([][]byte, []byte, error)

ListKeyPage returns the smallest keys strictly after after, bounded by limit. The process-local ordered index is rebuilt from the durable file names on first enumeration and then maintained after successful store mutations.

func (*FileStore) ListKeys added in v0.4.0 

func (s *FileStore) ListKeys(ctx context.Context) ([][]byte, error)

ListKeys returns all known keys in deterministic bytewise order.

func (*FileStore) Put 

func (s *FileStore) Put(ctx context.Context, key []byte, data []byte) error

Put stores data under key, replacing any existing value atomically.

type IntegritySummary added in v0.14.0 

type IntegritySummary struct {
	KeyCount             int
	KeyBytes             int64
	ContentAddressedKeys int
	VerifiedKeys         int
	VerifiedBytes        int64
	OpaqueKeys           int
	OpaqueBytes          int64
	CorruptKeys          int
	MissingKeys          int
}

IntegritySummary is a live aggregate of the blobs visible through an inventory lister. Content-addressed keys are verified against their stored bytes; opaque keys are counted without imposing a content-addressing contract on them.

func InspectInventory added in v0.14.0 

func InspectInventory(ctx context.Context, store BlobStore, lister BlobKeyLister) (IntegritySummary, error)

InspectInventory reads the current inventory and classifies each visible key. Native pagers are consumed one bounded page at a time; older listers use their complete-list compatibility path. The scan is live and does not claim a snapshot-consistent view while the store is changing.

type InventorySummary added in v0.14.0 

type InventorySummary struct {
	KeyCount int
	KeyBytes int64
	Digest   [sha256.Size]byte
}

InventorySummary is a live aggregate of the keys currently visible to a lister. Digest is SHA-256 over each key's uint32 big-endian length followed by its bytes, in bytewise key order.

func SummarizeInventory added in v0.14.0 

func SummarizeInventory(ctx context.Context, lister BlobKeyLister) (InventorySummary, error)

SummarizeInventory computes a deterministic aggregate without changing the add-only replication protocol. Native pagers are consumed one bounded page at a time; older listers use their complete-list compatibility path.

func (InventorySummary) DigestHex added in v0.14.0 

func (s InventorySummary) DigestHex() string

DigestHex returns the lowercase hexadecimal inventory fingerprint.

type MemoryStore 

type MemoryStore struct {
	// contains filtered or unexported fields
}

MemoryStore is an in-process BlobStore for tests and single-node demos.

func NewMemoryStore 

func NewMemoryStore() *MemoryStore

NewMemoryStore returns an empty MemoryStore.

func (*MemoryStore) Delete 

func (m *MemoryStore) Delete(ctx context.Context, key []byte) error

Delete removes key. Missing keys are not an error.

func (*MemoryStore) Get 

func (m *MemoryStore) Get(ctx context.Context, key []byte) ([]byte, error)

Get returns a copy of the value for key.

func (*MemoryStore) Has 

func (m *MemoryStore) Has(ctx context.Context, key []byte) (bool, error)

Has reports whether key exists.

func (*MemoryStore) Len 

func (m *MemoryStore) Len() int

Len returns the number of stored blobs (for metrics/tests).

func (*MemoryStore) ListKeyPage added in v0.12.0 

func (m *MemoryStore) ListKeyPage(ctx context.Context, after []byte, limit int) ([][]byte, []byte, error)

ListKeyPage returns the smallest keys strictly after after, bounded by limit.

func (*MemoryStore) ListKeys added in v0.4.0 

func (m *MemoryStore) ListKeys(ctx context.Context) ([][]byte, error)

ListKeys returns all known keys in deterministic bytewise order.

func (*MemoryStore) Put 

func (m *MemoryStore) Put(ctx context.Context, key []byte, data []byte) error

Put stores data under key, replacing any existing value.

func (*MemoryStore) Snapshot 

func (m *MemoryStore) Snapshot() map[string][]byte

Snapshot returns a shallow copy of keys for tests (order not stable).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.
⌂ 换网址