storage

package
v2.2.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 31, 2026 License: MIT Imports: 19 Imported by: 4

Documentation

Overview

Package storage provides a simple interface to store and retrieve data from multiple data stores.

Index

Constants

View Source 
const (
	DefaultMetricCounterLabel = "counter"
	Type                      = "storage"
)

Type is the type of the entity regarding the framework. It is used to for example, to identify the entity in the logs, metrics, and for tracing.

Variables

View Source 
var (
	// ErrRequiredPostHook is the error returned when the post-hook function is
	// missing.
	ErrRequiredPostHook = customerror.NewRequiredError("post-hook function", customerror.WithErrorCode("ERR_REQUIRED_POST_HOOK"))

	// ErrRequiredPreHook is the error returned when the pre-hook function is
	// missing.
	ErrRequiredPreHook = customerror.NewRequiredError("pre-hook function", customerror.WithErrorCode("ERR_REQUIRED_PRE_HOOK"))
)

Functions

func Close added in v2.2.3 

func Close(s IStorage) error

Close releases the resources held by s, if it holds any.

It returns nil when s does not implement ICloser, which makes it safe to call against any IStorage — including a third-party implementation written before ICloser existed, and against storage.Mock.

This matters most for the adapters whose leaked handle is more than a socket the OS reclaims at exit: bolt holds an exclusive file lock for as long as the database is open, so a storage that is never closed blocks every subsequent Open of the same path for the lifetime of the process.

func Count 

func Count(ctx context.Context, s IStorage, target string, prm *count.Count, options ...Func[*count.Count]) (int64, error)

Count data.

func CountFromMany 

func CountFromMany(
	ctx context.Context,
	m Map,
	target string,
	prm *count.Count,
	options ...Func[*count.Count],
) ([]int64, error)

CountFromMany counts documents concurrently against all DALs in the map.

func Create 

func Create[T any](ctx context.Context, s IStorage, id, target string, t T, prm *create.Create, options ...Func[*create.Create]) (string, error)

Create data.

func CreateIntoMany 

func CreateIntoMany[T any](
	ctx context.Context,
	m Map,
	id, target string,
	t T,
	prm *create.Create,
	options ...Func[*create.Create],
) ([]string, error)

CreateIntoMany creates one document concurrently against all DALs in the map.

func CreateMany 

func CreateMany[T any](
	ctx context.Context,
	str IStorage,
	target string,
	prm *create.Create,
	itemsMap map[string]T,
) ([]string, error)

CreateMany creates many documents concurrently against the same DAL.

func Delete 

func Delete(ctx context.Context, s IStorage, id, target string, prm *delete.Delete, options ...Func[*delete.Delete]) error

Delete data.

func DeleteFromMany 

func DeleteFromMany(
	ctx context.Context,
	m Map,
	id, target string,
	prm *delete.Delete,
	options ...Func[*delete.Delete],
) ([]bool, error)

DeleteFromMany deletes one documents concurrently against all DALs in the map.

func DeleteMany 

func DeleteMany(
	ctx context.Context,
	str IStorage,
	target string,
	prm *delete.Delete,
	ids ...string,
) ([]bool, error)

DeleteMany deletes many documents concurrently against the same DAL.

func Flatten2D 

func Flatten2D[T any](data [][]T) []T

Flatten2D takes a 2D slice and returns a 1D slice containing all the elements.

func List 

func List[T any](ctx context.Context, s IStorage, target string, prm *list.List, options ...Func[*list.List]) (T, error)

List data.

func ListFromMany 

func ListFromMany[T any](
	ctx context.Context,
	m Map,
	target string,
	prm *list.List,
	options ...Func[*list.List],
) ([]T, error)

ListFromMany lists documents concurrently against all DALs in the map.

NOTE: The results are flattened into a single slice.

func ParseToStruct 

func ParseToStruct(from, to any) error

ParseToStruct parses the given JSON (`from`) to struct (`to`).

func Retrieve 

func Retrieve[T any](ctx context.Context, s IStorage, id, target string, prm *retrieve.Retrieve, options ...Func[*retrieve.Retrieve]) (T, error)

Retrieve data.

func RetrieveFromMany 

func RetrieveFromMany[T any](
	ctx context.Context,
	m Map,
	id, target string,
	prm *retrieve.Retrieve,
	options ...Func[*retrieve.Retrieve],
) ([]T, error)

RetrieveFromMany retrieves one document concurrently against all DALs in the map.

func RetrieveMany 

func RetrieveMany[T any](
	ctx context.Context,
	str IStorage,
	target string,
	prm *retrieve.Retrieve,
	ids ...string,
) ([]T, error)

RetrieveMany retrieves many documents concurrently against the same DAL.

func Update 

func Update[T any](ctx context.Context, s IStorage, id, target string, t T, prm *update.Update, options ...Func[*update.Update]) error

Update data.

func UpdateIntoMany 

func UpdateIntoMany(
	ctx context.Context,
	m Map,
	id, target string,
	v any,
	prm *update.Update,
	options ...Func[*update.Update],
) ([]bool, error)

UpdateIntoMany updates one document concurrently against all DALs in the map.

func UpdateMany 

func UpdateMany[T any](
	ctx context.Context,
	str IStorage,
	target string,
	prm *update.Update,
	itemsMap map[string]T,
) ([]bool, error)

UpdateMany updates many documents concurrently against the same DAL.

Types

type Func 

type Func[T any] func(o *Options[T]) error

Func allows to set options.

func WithDatabase 

func WithDatabase[T any](db string) Func[T]

WithDatabase sets the database name.

func WithPostHook 

func WithPostHook[T any](fn HookFunc[T]) Func[T]

WithPostHook set the post-hook function.

func WithPreHook 

func WithPreHook[T any](fn HookFunc[T]) Func[T]

WithPreHook set the pre-hook function.

type HookFunc 

type HookFunc[T any] func(ctx context.Context, strg IStorage, id, target string, data any, param T) error

HookFunc specifies the function that will be called before and after the operation.

type ICloser added in v2.2.3 

type ICloser interface {
	// Close releases the resources held by the storage. It is idempotent.
	Close() error
}

ICloser is an optional interface an adapter may implement to release the resources it acquired at construction time.

It is deliberately NOT part of IStorage. Adding a method to IStorage would break every third-party implementation of it, and that is a v3 change. As a separate, single-method interface it is additive: callers that need to release resources type-assert for it, and callers that do not are unaffected.

Every adapter in this repository implements it, so the assertion always succeeds for storages built here. It is still worth asserting rather than depending on the concrete type, because that is what keeps a caller holding an IStorage able to clean up: 

if c, ok := strg.(storage.ICloser); ok {
	if err := c.Close(); err != nil { ... }
}

Or, more conveniently, via the Close helper below.

Implementations must be safe to call more than once: a second Close returns nil rather than an error, so a deferred Close alongside an explicit one is not a bug.

The in-process adapters (memory, file) hold no releasable resource and implement Close as a no-op purely so the assertion is uniform.

type IStorage 

type IStorage interface {
	// Count data.
	Count(ctx context.Context, target string, prm *count.Count, options ...Func[*count.Count]) (int64, error)

	// Delete data.
	Delete(ctx context.Context, id, target string, prm *delete.Delete, options ...Func[*delete.Delete]) error

	// Retrieve data.
	Retrieve(ctx context.Context, id, target string, v any, prm *retrieve.Retrieve, options ...Func[*retrieve.Retrieve]) error

	// List data.
	List(ctx context.Context, target string, v any, prm *list.List, options ...Func[*list.List]) error

	// Create data.
	Create(ctx context.Context, id, target string, v any, prm *create.Create, options ...Func[*create.Create]) (string, error)

	// Update data.
	Update(ctx context.Context, id, target string, v any, prm *update.Update, options ...Func[*update.Update]) error

	// GetType returns its type.
	GetType() string

	// GetClient returns the storage client. Use that to interact with the
	// underlying storage client.
	GetClient() any

	// GetLogger returns the logger.
	GetLogger() sypl.ISypl

	// GetName returns the storage name.
	GetName() string

	// GetCounterCounted returns the metric.
	GetCounterCounted() *expvar.Int

	// GetCounterCountedFailed returns the metric.
	GetCounterCountedFailed() *expvar.Int

	// GetCounterDeleted returns the metric.
	GetCounterDeleted() *expvar.Int

	// GetCounterDeletedFailed returns the metric.
	GetCounterDeletedFailed() *expvar.Int

	// GetCounterRetrieved returns the metric.
	GetCounterRetrieved() *expvar.Int

	// GetCounterRetrievedFailed returns the metric.
	GetCounterRetrievedFailed() *expvar.Int

	// GetCounterListed returns the metric.
	GetCounterListed() *expvar.Int

	// GetCounterListedFailed returns the metric.
	GetCounterListedFailed() *expvar.Int

	// GetCounterPingFailed returns the metric.
	GetCounterPingFailed() *expvar.Int

	// GetCounterCreated returns the metric.
	GetCounterCreated() *expvar.Int

	// GetCounterCreatedFailed returns the metric.
	GetCounterCreatedFailed() *expvar.Int

	// GetCounterUpdated returns the metric.
	GetCounterUpdated() *expvar.Int

	// GetCounterUpdatedFailed returns the metric.
	GetCounterUpdatedFailed() *expvar.Int
}

IStorage defines the data access layer interface.

The Data Access Layer (DAL) is generally more abstract and wider than the Data Access Object (DAO).

The DAL is responsible for providing a consistent and unified interface to access data from different data sources (such as databases, files, or web services) regardless of their underlying implementation details. It abstracts away the complexity of data access by providing a simple, unified interface that shields the rest of the application from the underlying data storage details.

On the other hand, the DAO is typically more specific to a particular data source, such as a particular database management system (such as MySQL or MongoDB) or a particular web service API. Its primary responsibility is to abstract away the details of the underlying data source and provide a simplified interface for performing CRUD (Create, Read, Update, Delete) operations on that data source.

So while both DAL and DAO are used to abstract away the complexities of data access, the DAL is generally wider and more abstract because it deals with multiple data sources, while the DAO is more specific and deals with a particular data source.

type Map 

type Map map[string]IStorage

Map is a map of strgs

func (Map) String 

func (m Map) String() string

String implements the Stringer interface.

func (Map) ToSlice 

func (m Map) ToSlice() []IStorage

ToSlice converts Map to Slice of IStorage.

type Mock 

type Mock struct {

	// Count data.
	MockCount func(ctx context.Context, target string, prm *count.Count, options ...Func[*count.Count]) (int64, error)

	// Delete data.
	MockDelete func(ctx context.Context, id, target string, prm *delete.Delete, options ...Func[*delete.Delete]) error

	// Retrieve data.
	MockRetrieve func(ctx context.Context, id, target string, v any, prm *retrieve.Retrieve, options ...Func[*retrieve.Retrieve]) error

	// List data.
	MockList func(ctx context.Context, target string, v any, prm *list.List, options ...Func[*list.List]) error

	// Create data.
	MockCreate func(ctx context.Context, id, target string, v any, prm *create.Create, options ...Func[*create.Create]) (string, error)

	// Update data.
	MockUpdate func(ctx context.Context, id, target string, v any, prm *update.Update, options ...Func[*update.Update]) error

	// GetType returns its type.
	MockGetType func() string

	// GetClient returns the storage client. Use that to interact with the underlying storage client.
	MockGetClient func() any

	// GetLogger returns the logger.
	MockGetLogger func() sypl.ISypl

	// GetName returns the storage name.
	MockGetName func() string

	// GetCounterCounted returns the metric.
	MockGetCounterCounted func() *expvar.Int

	// GetCounterCountedFailed returns the metric.
	MockGetCounterCountedFailed func() *expvar.Int

	// GetCounterDeleted returns the metric.
	MockGetCounterDeleted func() *expvar.Int

	// GetCounterDeletedFailed returns the metric.
	MockGetCounterDeletedFailed func() *expvar.Int

	// GetCounterRetrieved returns the metric.
	MockGetCounterRetrieved func() *expvar.Int

	// GetCounterRetrievedFailed returns the metric.
	MockGetCounterRetrievedFailed func() *expvar.Int

	// GetCounterListed returns the metric.
	MockGetCounterListed func() *expvar.Int

	// GetCounterListedFailed returns the metric.
	MockGetCounterListedFailed func() *expvar.Int

	// GetCounterPingFailed returns the metric.
	MockGetCounterPingFailed func() *expvar.Int

	// GetCounterCreated returns the metric.
	MockGetCounterCreated func() *expvar.Int

	// GetCounterCreatedFailed returns the metric.
	MockGetCounterCreatedFailed func() *expvar.Int

	// GetCounterUpdated returns the metric.
	MockGetCounterUpdated func() *expvar.Int

	// GetCounterUpdatedFailed returns the metric.
	MockGetCounterUpdatedFailed func() *expvar.Int
}

Mock is a struct which satisfies the storage.IStorage interface.

func (*Mock) Count 

func (m *Mock) Count(ctx context.Context, target string, prm *count.Count, options ...Func[*count.Count]) (int64, error)

Count data.

func (*Mock) Create 

func (m *Mock) Create(ctx context.Context, id, target string, v any, prm *create.Create, options ...Func[*create.Create]) (string, error)

Create data.

func (*Mock) Delete 

func (m *Mock) Delete(ctx context.Context, id, target string, prm *delete.Delete, options ...Func[*delete.Delete]) error

Delete data.

func (*Mock) GetClient 

func (m *Mock) GetClient() any

GetClient returns the storage client. Use that to interact with the underlying storage client.

func (*Mock) GetCounterCounted 

func (m *Mock) GetCounterCounted() *expvar.Int

GetCounterCounted returns the metric.

func (*Mock) GetCounterCountedFailed 

func (m *Mock) GetCounterCountedFailed() *expvar.Int

GetCounterCountedFailed returns the metric.

func (*Mock) GetCounterCreated 

func (m *Mock) GetCounterCreated() *expvar.Int

GetCounterCreated returns the metric.

func (*Mock) GetCounterCreatedFailed 

func (m *Mock) GetCounterCreatedFailed() *expvar.Int

GetCounterCreatedFailed returns the metric.

func (*Mock) GetCounterDeleted 

func (m *Mock) GetCounterDeleted() *expvar.Int

GetCounterDeleted returns the metric.

func (*Mock) GetCounterDeletedFailed 

func (m *Mock) GetCounterDeletedFailed() *expvar.Int

GetCounterDeletedFailed returns the metric.

func (*Mock) GetCounterListed 

func (m *Mock) GetCounterListed() *expvar.Int

GetCounterListed returns the metric.

func (*Mock) GetCounterListedFailed 

func (m *Mock) GetCounterListedFailed() *expvar.Int

GetCounterListedFailed returns the metric.

func (*Mock) GetCounterPingFailed 

func (m *Mock) GetCounterPingFailed() *expvar.Int

GetCounterPingFailed returns the metric.

func (*Mock) GetCounterRetrieved 

func (m *Mock) GetCounterRetrieved() *expvar.Int

GetCounterRetrieved returns the metric.

func (*Mock) GetCounterRetrievedFailed 

func (m *Mock) GetCounterRetrievedFailed() *expvar.Int

GetCounterRetrievedFailed returns the metric.

func (*Mock) GetCounterUpdated 

func (m *Mock) GetCounterUpdated() *expvar.Int

GetCounterUpdated returns the metric.

func (*Mock) GetCounterUpdatedFailed 

func (m *Mock) GetCounterUpdatedFailed() *expvar.Int

GetCounterUpdatedFailed returns the metric.

func (*Mock) GetLogger 

func (m *Mock) GetLogger() sypl.ISypl

GetLogger returns the logger.

func (*Mock) GetName 

func (m *Mock) GetName() string

GetName returns the storage name.

func (*Mock) GetType 

func (m *Mock) GetType() string

GetType returns its type.

func (*Mock) List 

func (m *Mock) List(ctx context.Context, target string, v any, prm *list.List, options ...Func[*list.List]) error

List data.

func (*Mock) Retrieve 

func (m *Mock) Retrieve(ctx context.Context, id, target string, v any, prm *retrieve.Retrieve, options ...Func[*retrieve.Retrieve]) error

Retrieve data.

func (*Mock) Update 

func (m *Mock) Update(ctx context.Context, id, target string, v any, prm *update.Update, options ...Func[*update.Update]) error

Update data.

type Operation 

type Operation string

Operation is the operation name. 

const (
	OperationCount    Operation = "count"
	OperationCreate   Operation = "create"
	OperationDelete   Operation = "delete"
	OperationList     Operation = "list"
	OperationRetrieve Operation = "retrieve"
	OperationUpdate   Operation = "update"
)

func (Operation) String 

func (o Operation) String() string

String implements the Stringer interface.

type Options 

type Options[T any] struct {
	// Database name.
	Database string `json:"database"`

	// PreHookFunc is the function which runs before the operation.
	PreHookFunc HookFunc[T] `json:"-"`

	// PostHookFunc is the function which runs after the operation.
	PostHookFunc HookFunc[T] `json:"-"`
}

Options for operations.

func NewOptions 

func NewOptions[T any]() (*Options[T], error)

NewOptions creates Options.

type Storage 

type Storage struct {
	// Logger.
	Logger sypl.ISypl `json:"-" validate:"required"`

	// Name of the storage type.
	Name string `json:"name" validate:"required,lowercase,gte=1"`
	// contains filtered or unexported fields
}

Storage definition.

func New 

func New(ctx context.Context, name string) (*Storage, error)

New returns a new Storage.

func (*Storage) GetCounterCounted 

func (s *Storage) GetCounterCounted() *expvar.Int

GetCounterCounted returns the metric.

func (*Storage) GetCounterCountedFailed 

func (s *Storage) GetCounterCountedFailed() *expvar.Int

GetCounterCountedFailed returns the metric.

func (*Storage) GetCounterCreated 

func (s *Storage) GetCounterCreated() *expvar.Int

GetCounterCreated returns the metric.

func (*Storage) GetCounterCreatedFailed 

func (s *Storage) GetCounterCreatedFailed() *expvar.Int

GetCounterCreatedFailed returns the metric.

func (*Storage) GetCounterDeleted 

func (s *Storage) GetCounterDeleted() *expvar.Int

GetCounterDeleted returns the metric.

func (*Storage) GetCounterDeletedFailed 

func (s *Storage) GetCounterDeletedFailed() *expvar.Int

GetCounterDeletedFailed returns the metric.

func (*Storage) GetCounterListed 

func (s *Storage) GetCounterListed() *expvar.Int

GetCounterListed returns the metric.

func (*Storage) GetCounterListedFailed 

func (s *Storage) GetCounterListedFailed() *expvar.Int

GetCounterListedFailed returns the metric.

func (*Storage) GetCounterPingFailed 

func (s *Storage) GetCounterPingFailed() *expvar.Int

GetCounterPingFailed returns the metric.

func (*Storage) GetCounterRetrieved 

func (s *Storage) GetCounterRetrieved() *expvar.Int

GetCounterRetrieved returns the metric.

func (*Storage) GetCounterRetrievedFailed 

func (s *Storage) GetCounterRetrievedFailed() *expvar.Int

GetCounterRetrievedFailed returns the metric.

func (*Storage) GetCounterUpdated 

func (s *Storage) GetCounterUpdated() *expvar.Int

GetCounterUpdated returns the metric.

func (*Storage) GetCounterUpdatedFailed 

func (s *Storage) GetCounterUpdatedFailed() *expvar.Int

GetCounterUpdatedFailed returns the metric.

func (*Storage) GetLogger 

func (s *Storage) GetLogger() sypl.ISypl

GetLogger returns the logger.

func (*Storage) GetName 

func (s *Storage) GetName() string

GetName returns the storage name.

func (*Storage) GetType 

func (s *Storage) GetType() string

GetType returns its type.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.
⌂ 换网址