eqmem

package
v1.13.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 23, 2026 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Overview

Package eqmem implements an in-memory entroq that has fine-grained locking and can handle simultaneously stats/task listing and modifications to a large extent.

Example (Journal)
package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/shiblon/entroq"
	"github.com/shiblon/entroq/pkg/backend/eqmem"
)

func main() {
	journalDir, err := os.MkdirTemp("", "eqjournal-")
	if err != nil {
		log.Fatalf("Error opening temp dir for journal: %v", err)
	}
	defer os.RemoveAll(journalDir)

	ctx := context.Background()

	eq, err := entroq.New(ctx, eqmem.Opener(eqmem.WithJournal(journalDir)))
	if err != nil {
		log.Fatalf("Error opening client at dir %q: %v", journalDir, err)
	}
	defer eq.Close()

	resp, err := eq.Modify(ctx,
		entroq.InsertingInto("/queue/of/tasks", entroq.WithValue("hey")),
		entroq.InsertingInto("/queue/of/others", entroq.WithValue("other")),
	)
	if err != nil {
		log.Fatalf("Error adding task: %v", err)
	}
	inserted := resp.InsertedTasks

	// Change the queue for the first insertion.
	if _, err := eq.Modify(ctx, inserted[0].Change(entroq.QueueTo("/queue/of/something"))); err != nil {
		log.Fatalf("Error modifying task: %v", err)
	}

	// Close and reopen, see that everything is still there.
	eq.Close()
	eq = nil

	if eq, err = entroq.New(ctx, eqmem.Opener(eqmem.WithJournal(journalDir))); err != nil {
		log.Fatalf("Error reopening client at dir %q: %v", journalDir, err)
	}
	defer eq.Close()

	empty, err := eq.QueuesEmpty(ctx, entroq.MatchExact("/queue/of/tasks"))
	if err != nil {
		log.Fatalf("Error checking for empty queues: %v", err)
	}
	fmt.Printf("Empty: %v\n", empty)

	ts1, err := eq.Tasks(ctx, "/queue/of/others")
	if err != nil {
		log.Fatalf("Error getting tasks for 'others': %v", err)
	}
	for _, t := range ts1 {
		fmt.Printf("%v: %s\n", t.Queue, t.Value)
	}

	ts2, err := eq.Tasks(ctx, "/queue/of/something")
	if err != nil {
		log.Fatalf("Error getting tasks for 'something': %v", err)
	}
	for _, t := range ts2 {
		fmt.Printf("%v: %s\n", t.Queue, t.Value)
	}

}
Output:
Empty: true
/queue/of/others: "other"
/queue/of/something: "hey"

Index

Examples

Constants

View Source
const DefaultReadinessInterval = 5 * time.Second

DefaultReadinessInterval is how often the backend checks for tasks that become available solely through the passage of time.

Variables

This section is empty.

Functions

func Opener

func Opener(opts ...Option) entroq.BackendOpener

Opener returns a constructor of the in-memory backend.

func TakeSnapshot

func TakeSnapshot(ctx context.Context, journalDir string, cleanup bool) error

TakeSnapshot brings the system up empty, loads a snapshot + journals, then outputs a new snapshot and exits. Cleans up old files after snapshotting if requested. Otherwise they are just moved out of the way.

The snapshot covers every journal file except the live one, the file a running server may still be appending to, which is left in place and replayed after the snapshot on the next start. So a snapshot is safe to take while a server is running, and the directory still needs that last journal file: the snapshot alone does not hold the most recent changes.

Types

type EQMem

type EQMem struct {
	sync.Mutex
	// contains filtered or unexported fields
}

func New

func New(ctx context.Context, opts ...Option) (*EQMem, error)

New returns a new in-memory implementation, ready to be used.

func (*EQMem) Claim

func (m *EQMem) Claim(ctx context.Context, cq *entroq.ClaimQuery) (*entroq.Task, error)

Claim waits for a task to be available to claim.

func (*EQMem) ClaimDocs

func (m *EQMem) ClaimDocs(ctx context.Context, cq *entroq.DocClaim) ([]*entroq.DocSet, error)

ClaimDocs claims every doc set cq names, all or none (see docset.ClaimAll), holding every namespace involved at once, in order.

func (*EQMem) Close

func (m *EQMem) Close() error

Close cleans up this implementation.

func (*EQMem) Docs

func (m *EQMem) Docs(ctx context.Context, rq *entroq.DocQuery) ([]*entroq.Doc, error)

Docs returns a slice of docs in a namespace. If IDs are specified, only those docs are returned (key range is ignored and Limit does not apply). Otherwise, docs are filtered by optional key range and subject to Limit. Results are returned sorted by (key_primary, key_secondary). Each doc carries its set's version and claim.

func (*EQMem) Modify

func (*EQMem) NamespaceStats added in v1.1.0

func (m *EQMem) NamespaceStats(ctx context.Context, qq *entroq.MatchQuery) (map[string]*entroq.NamespaceStat, error)

NamespaceStats returns doc counts per namespace, optionally filtered and limited.

func (*EQMem) QueueStats

func (m *EQMem) QueueStats(ctx context.Context, qq *entroq.QueuesQuery) (map[string]*entroq.QueueStat, error)

QueueStats returns statistics for each queue in the query.

func (*EQMem) Queues

func (m *EQMem) Queues(ctx context.Context, qq *entroq.QueuesQuery) (map[string]int, error)

Queues returns the list of queue and their sizes, based on query contents.

func (*EQMem) Tasks

func (m *EQMem) Tasks(ctx context.Context, tq *entroq.TasksQuery) ([]*entroq.Task, error)

Tasks lists tasks according to the given query. If specific IDs are given, it will block for brief periods to look up corresponding queues for them.

func (*EQMem) Time

func (m *EQMem) Time(_ context.Context) (time.Time, error)

Time returns the current time.

func (*EQMem) TryClaim

func (m *EQMem) TryClaim(ctx context.Context, cq *entroq.ClaimQuery) (*entroq.Task, error)

TryClaim attempts to claim a task from the given queue query. If no task is available, returns nil (not an error).

type Option

type Option func(*EQMem)

Option represents options for creationg of the in-memory implementation.

func WithJournal

func WithJournal(dir string) Option

WithJournal sets up a file-based journal system so that the in-memory implementation can be persisted.

func WithMaxJournalBytes

func WithMaxJournalBytes(max int64) Option

WithMaxJournalBytes sets a maximum on the number of bytes before rotation. Default is wal.DefaultMaxBytes.

func WithMaxJournalItems

func WithMaxJournalItems(max int) Option

WithMaxJournalItems sets a maximum on the number of entries in the journal before rotation. Default is wal.DefaultMaxIndices.

func WithMeterProvider

func WithMeterProvider(mp metric.MeterProvider) Option

WithMeterProvider sets the OTel MeterProvider for claim and modify duration histograms. Defaults to a noop provider.

func WithReadinessInterval added in v1.12.0

func WithReadinessInterval(interval time.Duration) Option

WithReadinessInterval sets how often the backend checks for tasks that became available solely through the passage of time. A non-positive interval disables the readiness loop, leaving claim polling as the fallback.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址