Skip to content
blockberriesPublic

About

A high-performance versioned AVL+ tree for deterministic Merkle state storage. Drop-in replacement for [cosmos/iavl] with faster reads and writes, fewer allocations, and the same ICS-23 proof format.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

AVLBerry

A high-performance versioned AVL+ tree for deterministic Merkle state storage. Drop-in replacement for cosmos/iavl with faster reads and writes, fewer allocations, and the same ICS-23 proof format.

Features

  • Versioned: every SaveVersion() creates an immutable snapshot addressable by version number
  • Merkle proofs: ICS-23 compatible membership and non-membership proofs
  • AVL+ variant: all user data on leaf nodes; inner nodes store only routing keys and hashes
  • O(1) reads: FastNode index bypasses tree traversal for latest-version lookups
  • Zero-allocation hot paths: pre-computed NodeKeys, pooled encoders, stack-allocated readers
  • State sync: streaming Export/Import for snapshot transfer
  • Pluggable storage: bring your own DB implementation, or use the included GoLevelDB/MemDB adapters

Installation

go get github.com/blockberries/avlberry

Quick Start

package main

import (
    "fmt"

    "github.com/blockberries/avlberry"
    "github.com/blockberries/avlberry/db"
)

func main() {
    // Open a persistent database
    ldb, err := db.NewGoLevelDB("myapp", "/tmp/avlberry-example")
    if err != nil {
        panic(err)
    }
    defer ldb.Close()

    // Create a tree with LRU cache of 10,000 nodes
    tree := avlberry.NewMutableTree(ldb, 10000, false, avlberry.NewNopLogger())

    // Load the latest version (no-op for a new database)
    version, err := tree.Load()
    if err != nil {
        panic(err)
    }
    fmt.Println("Loaded version:", version)

    // Write some data
    tree.Set([]byte("alice"), []byte("100"))
    tree.Set([]byte("bob"), []byte("200"))

    // Persist as a new version
    hash, ver, err := tree.SaveVersion()
    if err != nil {
        panic(err)
    }
    fmt.Printf("Saved version %d, root hash: %x\n", ver, hash)

    // Read back
    val, err := tree.Get([]byte("alice"))
    if err != nil {
        panic(err)
    }
    fmt.Println("alice =", string(val))
}

Usage

Creating a Tree

tree := avlberry.NewMutableTree(db, cacheSize, skipFastStorageUpgrade, logger, options...)

Parameters:

  • db: any avlberry.DB implementation (see Database Adapters)
  • cacheSize: LRU cache capacity for tree nodes (0 disables caching)
  • skipFastStorageUpgrade: if true, don't rebuild the FastNode index on load
  • logger: structured logger (avlberry.NewNopLogger() for silence)
  • options: functional options (see below)

Options

Option Default Description
SyncOption(true) false Enable fsync on batch writes
InitialVersionOption(n) 0 Set first version number for new trees
FlushThresholdOption(n) 100000 Batch auto-flush threshold in bytes
AsyncPruningOption(true) false Run version pruning in background goroutine
InMemoryStoreOption(true) false Mirror entire DB in memory (zero-disk reads)
StatOption(stats) nil Enable cache hit/miss statistics

Reading and Writing

// Write (accumulates in working tree)
updated, err := tree.Set([]byte("key"), []byte("value"))
value, removed, err := tree.Remove([]byte("key"))

// Read (checks unsaved changes, then FastNode cache, then tree)
val, err := tree.Get([]byte("key"))
exists, err := tree.Has([]byte("key"))

// Persist working tree as new version
rootHash, version, err := tree.SaveVersion()

// Discard unsaved changes
tree.Rollback()

Versioning

// Load latest version from DB
latestVersion, err := tree.Load()

// Load a specific version
version, err := tree.LoadVersion(42)

// Check if version exists
exists, err := tree.VersionExists(42)

// Delete old versions (pruning)
err := tree.DeleteVersionsTo(40)

// Load and discard all versions after target
err := tree.LoadVersionForOverwriting(42)

Iteration

// Forward iterator over [start, end)
iter, err := tree.Iterator([]byte("a"), []byte("z"), true)
defer iter.Close()
for ; iter.Valid(); iter.Next() {
    fmt.Printf("%s = %s\n", iter.Key(), iter.Value())
}

// Callback-based
tree.Iterate(func(key, value []byte) bool {
    fmt.Printf("%s = %s\n", key, value)
    return false // return true to stop
})

Proofs (ICS-23)

// Generate proof for a key
proof, err := tree.GetProof([]byte("alice"))

// Verify membership
ok, err := tree.VerifyMembership(proof, []byte("alice"))

// Verify non-membership
proof, err := tree.GetNonMembershipProof([]byte("missing"))
ok, err := tree.VerifyNonMembership(proof, []byte("missing"))

// Proof for a specific version
proof, err := tree.GetVersionedProof([]byte("alice"), 42)

State Sync (Export / Import)

// Export: stream all nodes from a saved version
exporter, err := tree.Export()
defer exporter.Close()
for {
    node, err := exporter.Next()
    if err == avlberry.ErrExportDone {
        break
    }
    // send node over network...
}

// Import: reconstruct tree from exported nodes
importer, err := tree.Import(targetVersion)
for _, node := range receivedNodes {
    if err := importer.Add(node); err != nil {
        panic(err)
    }
}
if err := importer.Commit(); err != nil {
    panic(err)
}

ChangeSets

// Apply a batch of changes atomically
cs := &avlberry.ChangeSet{
    Pairs: []avlberry.KVPair{
        {Key: []byte("a"), Value: []byte("1")},
        {Key: []byte("b"), Value: []byte("2")},
        {Delete: true, Key: []byte("old")},
    },
}
version, err := tree.SaveChangeSet(cs)

// Traverse state changes between versions
tree.TraverseStateChanges(1, 10, func(version int64, cs *avlberry.ChangeSet) error {
    fmt.Printf("Version %d: %d changes\n", version, len(cs.Pairs))
    return nil
})

Database Adapters

The db/ subpackage provides production-ready implementations of avlberry.DB:

import "github.com/blockberries/avlberry/db"

// GoLevelDB: persistent storage with bloom filters
ldb, err := db.NewGoLevelDB("name", "/path/to/dir")
ldb, err := db.NewGoLevelDBWithOpts("name", "/path/to/dir", &opt.Options{...})

// MemDB: in-memory (testing, ephemeral state)
mdb := db.NewMemDB()

// PrefixDB: key namespace isolation
pdb := db.NewPrefixDB(ldb, []byte("myapp/"))

// IteratePrefix: convenience for prefix scans
iter, err := db.IteratePrefix(ldb, []byte("users/"))

Custom DB Implementation

Implement the avlberry.DB interface:

type DB interface {
    Get(key []byte) ([]byte, error)
    Has(key []byte) (bool, error)
    Iterator(start, end []byte) (Iterator, error)
    ReverseIterator(start, end []byte) (Iterator, error)
    Close() error
    NewBatch() Batch
    NewBatchWithSize(sizeHint int) Batch
}

The Iterator and Batch interfaces must also be implemented. See interfaces.go for full documentation.

Architecture

See ARCHITECTURE.md for detailed internal documentation covering the node structure, mutation flow, serialization format, caching strategy, and all performance optimizations.

Key design highlights:

  • 12-byte NodeKey (version + nonce) replaces 32-byte hash-based keys
  • FastNode index provides O(1) latest-version reads
  • Stack-based iterators with subtree pruning for range queries
  • WriteThroughDB mirrors the entire database in memory for zero-disk reads
  • Post-order nonce assignment ensures children are persisted before parents
  • Copy-on-write with skip-clone optimization for unsaved nodes

Testing

# Run all tests
go test ./...

# Run with race detector
go test -race ./...

# Run benchmarks
go test -bench=BenchmarkLDB_ -benchmem -count=3 ./...

# Run db/ package tests only
go test ./db/ -v

The test suite includes 540+ unit tests and 87 benchmarks across 51 test files and 7 packages.

Dependencies

Dependency Purpose
github.com/cosmos/ics23/go ICS-23 proof types and verification
github.com/google/btree In-memory B-tree for WriteThroughDB and MemDB
github.com/syndtr/goleveldb GoLevelDB persistent storage adapter

License

See LICENSE for details.

About

A high-performance versioned AVL+ tree for deterministic Merkle state storage. Drop-in replacement for [cosmos/iavl] with faster reads and writes, fewer allocations, and the same ICS-23 proof format.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

⌂ 换网址