Skip to content
unkn0wn-rootPublic

About

Fast(est), sharded Go cache with lock-free reads, self-adaptive SieveTinyLFU, batched async writes and HTTP middleware.

Topics

Resources

Stars

71 stars

Watchers

2 watching

Forks

Latest commit

 

History

308 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Kioshun Logo

Kioshun - In-Memory Cache for Go

"kee-oh-shoon" /kiːoʊʃuːn/

Go Version License CI

Fast, sharded in-memory cache for Go

Index

Installation

go get github.com/unkn0wn-root/kioshun

Quick Start

package main

import (
    "fmt"
    "time"

    "github.com/unkn0wn-root/kioshun"
)

func main() {
    c := kioshun.NewDefault[string, string]()
    defer c.Close()

    c.Set("user:123", "David", kioshun.DefaultExpiration)

    // Set returns after the value is readable.
    c.Set("user:456", "John", kioshun.NoExpiration)

    // SetAsync may queue the write.
    c.SetAsync("user:789", "Paul", 5*time.Minute)

    // Sync waits for all accepted writes.
    c.Sync()

    if value, found := c.Get("user:123"); found {
        fmt.Printf("User: %s\n", value)
    }

}

Configuration

For most caches, start with the built-in defaults:

c := kioshun.NewDefault[string, string]()
defer c.Close()

NewDefault uses DefaultConfig(): MaxSize=10000, DefaultTTL=30m, automatic shard count, CleanupInterval=5m, SieveTinyLFU eviction, stats disabled and the default async write buffer/batch sizes.

When you only need a few changes, start from DefaultConfig() and override the specific fields:

config := kioshun.DefaultConfig()
config.MaxSize = 100000
config.DefaultTTL = time.Hour

c, err := kioshun.New[string, string](config)
if err != nil {
	panic(err)
}

Every Config field, weighted capacity (MaxCost + WithWeigher) and the named cache Manager are covered in configuration.md.

API

c.Set(key, value, ttl time.Duration) error
c.SetAsync(key, value, ttl time.Duration) error
c.SetWithCallback(key, value, ttl, callback func(key, value)) error
c.Get(key) (value, found bool)
c.GetWithTTL(key) (value, ttl time.Duration, found bool)
c.Keys() []K
c.Clear()
c.Sync() error
c.Delete(key) bool
c.Exists(key) bool
c.Size() int64
c.Stats() Stats
c.PolicyStats() PolicyStats
c.Cleanup()
c.Close() error

Set is synchronous and gives immediate 'read-after-write' visibility for the key. SetAsync is optional. It may have committed inline already or it may still be queued. Use Sync when committed visibility is required. A negative TTL other than NoExpiration (-1) means already expired: the call stores nothing and removes the key.

HTTP Middleware

The middleware is a shared HTTP cache. With the default policy, it stores only responses that carry explicit freshness (max-age, s-maxage or Expires) or are marked public.

import "github.com/unkn0wn-root/kioshun/httpcache"

config := httpcache.DefaultConfig()
config.DefaultTTL = 5 * time.Minute // for public responses without max-age or Expires
config.MaxSize = 100000

middleware, err := httpcache.New(config)
if err != nil {
	panic(err)
}
defer middleware.Close()

usersHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Cache-Control", "public, max-age=60")
	w.Write([]byte(`["alice","bob"]`))
})
http.Handle("/api/users", middleware.Wrap(usersHandler))

See http.md for complete documentation.

Benchmarks

Hit ratio and throughput

Kioshun is compared against Ristretto, Otter and Theine by replaying ARC and LIRS request traces (P3, P8, S3, DS1, OLTP, LOOP) through every cache in the same harness.

Hit ratio vs the Belady optimum

Hit ratio vs cache size

Throughput

Hit ratio vs throughput

Results at the 100,000 entry limit:

trace OPT kioshun theine otter ristretto
p3 62.00% 44.94% 38.14% 36.41% 21.22%
p8 77.12% 59.85% 54.10% 56.56% 55.83%
s3 25.42% 13.46% 11.90% 9.06% 9.80%
ds1 5.16% 1.97% 1.51% 1.57% 1.36%
oltp 79.56% 79.29% 76.67% 76.41% 28.30%
loop 99.80% 99.26% 99.79% 99.60% 90.39%

Hit ratio as a percent of that ceiling:

trace kioshun theine otter ristretto
p3 71.3 61.5 58.7 34.2
p8 77.6 70.2 73.3 72.4
s3 49.3 46.8 35.6 38.6
ds1 36.8 29.3 30.4 26.4
oltp 98.8 96.4 96.0 35.6
loop 99.4 100.0 99.8 90.6
geomean 67.9 62.1 59.5 44.8

Microbenchmarks

A separate suite compares Kioshun with Ristretto, BigCache, FreeCache and go-cache using pre-generated workloads with async and strict write modes reported separately:

cd benchmarks
go test -run=TestBenchmarkComparisonGetSetup -count=1 .
go test -bench='BenchmarkCacheComparison' -benchmem -run=^$ -benchtime=1s .
go test -bench=. -benchmem -run=^$ -benchtime=1s -timeout=30m .

About

Fast(est), sharded Go cache with lock-free reads, self-adaptive SieveTinyLFU, batched async writes and HTTP middleware.

Topics

Resources

Stars

71 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors

Languages

⌂ 换网址