gcsemu

package
v1.58.9 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 12, 2026 License: MPL-2.0, MIT Imports: 33 Imported by: 0

Documentation

Overview

Package gcsemu implements a Google Cloud Storage emulator for development.

Index

Constants

View Source
const MaxResults = 1000

MaxResults is the most entries a single list page holds, and the number returned when the caller asks for no limit. Cloud Storage documents both roles for the same number: "The service will use this parameter or 1,000 items, whichever is smaller", with a default of 1000. A larger request is capped rather than rejected.

Variables

This section is empty.

Functions

func BucketMeta

func BucketMeta(baseUrl HttpBaseUrl, bucket string) *storage.Bucket

BucketMeta returns a default bucket metadata for the given name and base url.

func BucketUrl

func BucketUrl(baseUrl HttpBaseUrl, bucket string) string

BucketUrl returns the URL for a bucket.

func DrainRequestHandler

func DrainRequestHandler(h http.HandlerFunc) http.HandlerFunc

DrainRequestHandler wraps the given handler to drain the incoming request body on exit.

func EscapeGlobLiteral added in v1.58.0

func EscapeGlobLiteral(s string) string

EscapeGlobLiteral escapes every character compileGlob reads as pattern syntax, so that a literal object key (or key prefix) can be spliced into a glob pattern.

Object keys are opaque strings and routinely contain glob metacharacters, so a caller scoping a search to the folder "a[1]/" has to escape that prefix before concatenating a pattern onto it. Unescaped, the prefix is read as a pattern in its own right and the search silently matches nothing.

func GzipRequestHandler

func GzipRequestHandler(h http.HandlerFunc) http.HandlerFunc

GzipRequestHandler wraps the given handler to automatically decompress gzipped content.

func InitMetaWithUrls

func InitMetaWithUrls(baseUrl HttpBaseUrl, meta *storage.Object, bucket string, filename string, size uint64)

InitMetaWithUrls "bakes" metadata with intrinsic values, including computed links.

func InitScrubbedMeta

func InitScrubbedMeta(meta *storage.Object, filename string)

InitScrubbedMeta "bakes" metadata with intrinsic values and removes fields that are intrinsic / computed.

func NewClient

func NewClient(ctx context.Context) (*storage.Client, error)

NewClient returns either a real *storage.Cient, or else a *storage.Client that routes to a local emulator if a `GCS_EMULATOR_HOST` environment variable is configured.

func NewFileStore

func NewFileStore(gcsDir string) *filestore

NewFileStore returns a new Store that writes to the given directory.

func NewMemStore

func NewMemStore() *memstore

NewMemStore returns a Store that operates purely in memory.

func NewTestClientWithHost

func NewTestClientWithHost(ctx context.Context, hostUrl string) (*storage.Client, error)

NewTestClientWithHost returns a new Google storage client that connects to the given host:port address.

func ObjectUrl

func ObjectUrl(baseUrl HttpBaseUrl, bucket string, filepath string) string

ObjectUrl returns the URL for a file.

func ScrubMeta

func ScrubMeta(meta *storage.Object)

ScrubMeta removes fields that are intrinsic / computed for minimal storage.

Types

type GcsEmu

type GcsEmu struct {
	// contains filtered or unexported fields
}

GcsEmu is a Google Cloud Storage emulator for development.

func NewGcsEmu

func NewGcsEmu(opts Options) *GcsEmu

NewGcsEmu creates a new Google Cloud Storage emulator.

func (*GcsEmu) BatchHandler

func (g *GcsEmu) BatchHandler(w http.ResponseWriter, r *http.Request)

BatchHandler handles emulated GCS http requests for "storage.googleapis.com/batch/storage/v1".

func (*GcsEmu) Handler

func (g *GcsEmu) Handler(w http.ResponseWriter, r *http.Request)

Handler handles emulated GCS http requests for "storage.googleapis.com".

func (*GcsEmu) InitBucket

func (g *GcsEmu) InitBucket(bucketName string) error

InitBucket creates the given bucket directly.

func (*GcsEmu) ListObjects added in v1.58.0

func (g *GcsEmu) ListObjects(ctx context.Context, baseUrl HttpBaseUrl, bucket string, opts ListOptions) (*storage.Objects, error)

ListObjects lists the objects in the given bucket.

If the bucket does not exist it returns an error satisfying os.IsNotExist.

func (*GcsEmu) Register

func (g *GcsEmu) Register(mux *http.ServeMux)

Register the emulator's HTTP handlers on the given mux.

type GcsParams

type GcsParams struct {
	Bucket   string
	Object   string
	IsPublic bool
}

GcsParams represent a parsed GCS url.

func ParseGcsUrl

func ParseGcsUrl(u *url.URL) (*GcsParams, bool)

ParseGcsUrl parses a GCS url.

type HttpBaseUrl

type HttpBaseUrl string

HttpBaseUrl represents the emulator base URL, including trailing slash; e.g. https://www.googleapis.com/

type ListOptions added in v1.58.0

type ListOptions struct {
	// Prefix restricts the results to objects whose name begins with Prefix.
	Prefix string

	// Delimiter, when non-empty, rolls objects whose name contains Delimiter
	// after Prefix up into the returned Prefixes instead of returning them as objects.
	Delimiter string

	// Cursor is an exclusive lower bound on the object names to return.
	// It is the object name encoded in the caller's page token; see gcsutil.DecodePageToken.
	Cursor string

	// MatchGlob, when non-empty, restricts the results to objects whose full name
	// matches the given glob pattern. See compileGlob for the supported syntax.
	MatchGlob string

	// MaxResults caps the combined number of objects and rolled-up prefixes returned,
	// counting each prefix once however many objects it stands for.
	// If <= 0 or above MaxResults, MaxResults is used.
	MaxResults int
}

ListOptions are the options for listing the objects in a bucket.

type LocalStore added in v1.58.2

type LocalStore interface {
	// LocalPath returns the path holding the given object, or the bucket's own
	// directory when filename is empty.
	LocalPath(bucket string, filename string) (string, error)
}

LocalStore is implemented by stores that keep objects on the local filesystem. A store that doesn't implement it has no paths to give.

type Options

type Options struct {
	// A storage layer to use; if nil, defaults to in-mem storage.
	Store Store

	// If true, log verbosely.
	Verbose bool

	// Optional log function. `err` will be `nil` for informational/debug messages.
	Log func(err error, fmt string, args ...interface{})
}

Options configure the emulator.

type Server

type Server struct {
	Addr string
	*httptest.Server
	*GcsEmu
}

Server is an in-memory Cloud Storage emulator; it is unauthenticated, and only a rough approximation.

func NewServer

func NewServer(laddr string, opts Options) (*Server, error)

NewServer creates a new Server with the given options. The Server will be listening for HTTP connections, without TLS, on the provided address. The resolved address is named by the Addr field. An address with a port of 0 will bind to an open port on the system.

For running a full in-process setup (e.g. unit tests), initialize os.Setenv("GCS_EMULATOR_HOST", srv.Addr) so that subsequent calls to NewClient() will return an in-process targeted storage client.

type Store

type Store interface {
	// CreateBucket creates a bucket; no error if the bucket already exists.
	CreateBucket(bucket string) error

	// Get returns a bucket's metadata.
	GetBucketMeta(baseUrl HttpBaseUrl, bucket string) (*storage.Bucket, error)

	// Get returns a file's contents and metadata.
	Get(url HttpBaseUrl, bucket string, filename string) (*storage.Object, []byte, error)

	// GetMeta returns a file's metadata.
	GetMeta(url HttpBaseUrl, bucket string, filename string) (*storage.Object, error)

	// Add creates the specified file.
	Add(bucket string, filename string, contents []byte, meta *storage.Object) error

	// UpdateMeta updates the given file's metadata.
	UpdateMeta(bucket string, filename string, meta *storage.Object, metagen int64) error

	// Copy copies the file
	Copy(srcBucket string, srcFile string, dstBucket string, dstFile string) (bool, error)

	// Delete deletes the file.
	Delete(bucket string, filename string) error

	// ReadMeta reads the GCS metadata for a file, when you already have file info.
	ReadMeta(url HttpBaseUrl, bucket string, filename string, fInfo os.FileInfo) (*storage.Object, error)

	// Walks the given bucket.
	Walk(ctx context.Context, bucket string, cb func(ctx context.Context, filename string, fInfo os.FileInfo) error) error
}

Store is an interface to either on-disk or in-mem storage

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址