lfstestserver

package module
v0.5.0-dev5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 18, 2025 License: MIT Imports: 34 Imported by: 0

README

LFS Test Server

LFS Test Server is an example server that implements the Git LFS API. It is intended to be used for testing the Git LFS client and is not in a production ready state.

LFS Test Server is written in Go, with pre-compiled binaries available for Mac, Windows, Linux, and FreeBSD.

See CONTRIBUTING.md for info on working on LFS Test Server and sending patches.

Installing

Use the Go installer:

  $ go install github.com/git-lfs/lfs-test-server@latest

Building

To build from source, use the Go tools:

  $ go get github.com/git-lfs/lfs-test-server

Running

Running the binary will start an LFS server on localhost:8080 by default. There are few things that can be configured via environment variables:

LFS_LISTEN      # The address:port the server listens on, default: "tcp://:8080"
LFS_HOST        # The host used when the server generates URLs, default: "localhost:8080"
LFS_METADB      # The database file the server uses to store meta information, default: "lfs.db"
LFS_CONTENTPATH # The path where LFS files are store, default: "lfs-content"
LFS_ADMINUSER   # An administrator username, default: not set
LFS_ADMINPASS   # An administrator password, default: not set
LFS_CERT        # Certificate file for tls
LFS_KEY         # tls key
LFS_SCHEME      # set to 'https' to override default http
LFS_USETUS      # set to 'true' to enable tusd (tus.io) resumable upload server; tusd must be on PATH, installed separately
LFS_TUSHOST     # The host used to start the tusd upload server, default "localhost:1080"

If the LFS_ADMINUSER and LFS_ADMINPASS variables are set, a rudimentary admin interface can be accessed via http://$LFS_HOST/mgmt. Here you can add and remove users, which must be done before you can use the server with the client. If either of these variables are not set (which is the default), the administrative interface is disabled.

To use the LFS test server with the Git LFS client, configure it in the repository's .lfsconfig:

  [lfs]
    url = "http://localhost:8080/"

HTTPS:

NOTE: If using https with a self signed cert also disable cert checking in the client repo.

  [lfs]
    url = "https://localhost:8080/"

  [http]
    sslverify = false

An example usage:

Generate a key pair

openssl req -x509 -sha256 -nodes -days 2100 -newkey rsa:2048 -keyout mine.key -out mine.crt

Make yourself a run script

#!/bin/bash

set -eu
set -o pipefail


LFS_LISTEN="tcp://:9999"
LFS_HOST="127.0.0.1:9999"
LFS_CONTENTPATH="content"
LFS_ADMINUSER="<cool admin user name>"
LFS_ADMINPASS="<better admin password>"
LFS_CERT="mine.crt"
LFS_KEY="mine.key"
LFS_SCHEME="https"

export LFS_LISTEN LFS_HOST LFS_CONTENTPATH LFS_ADMINUSER LFS_ADMINPASS LFS_CERT LFS_KEY LFS_SCHEME

./lfs-test-server

Build the server

go build

Run

bash run.sh

Check the managment page

browser: https://localhost:9999/mgmt

Debugging

lfs-test-server supports a basic cmd to lookup OID's via the cmdline to help in debugging, eg. investigating client problems with a particular OID and it's properties. In this mode lfs-test-server expects the same configuration as when running in daemon mode, but will just executing the requested cmd and then exit.

This is especially helpful in server environments where it's not always possible to get to the web interface easily or where it's just too slow because of DB size.

lfs-test-server cmd <OID>

Outputs the full OID record

Example

% . /etc/default/lfs-instancefoo    # to source server config
% ./lfs-test-server cmd 7c9414fe21ad7b45ffb6e72da86f9a9e13dbb2971365ae7bcb8cc7fbbba7419c
&{Oid:7c9414fe21ad7b45ffb6e72da86f9a9e13dbb2971365ae7bcb8cc7fbbba7419c Size:3334144 Existing:false}

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var Config = &Configuration{}

Config is the global app configuration

Functions

func ContentMatcher 

func ContentMatcher(r *http.Request, m *mux.RouteMatch) bool

ContentMatcher provides a mux.MatcherFunc that only allows requests that contain an Accept header with the contentMediaType

func MetaMatcher 

func MetaMatcher(r *http.Request, m *mux.RouteMatch) bool

MetaMatcher provides a mux.MatcherFunc that only allows requests that contain an Accept header with the metaMediaType

Types

type App 

type App struct {
	Middleware mux.MiddlewareFunc
	// contains filtered or unexported fields
}

App links a Router, ContentStore, and MetaStore to provide the LFS server.

func NewApp 

func NewApp(config *Configuration, content *ContentStore, meta *MetaStore) *App

NewApp creates a new App using the ContentStore and MetaStore provided

func (*App) BatchHandler 

func (a *App) BatchHandler(w http.ResponseWriter, r *http.Request)

BatchHandler provides the batch api

func (*App) CreateLockHandler 

func (a *App) CreateLockHandler(w http.ResponseWriter, r *http.Request)

func (*App) DeleteLockHandler 

func (a *App) DeleteLockHandler(w http.ResponseWriter, r *http.Request)

func (*App) GetContentHandler 

func (a *App) GetContentHandler(w http.ResponseWriter, r *http.Request)

GetContentHandler gets the content from the content store

func (*App) GetMetaHandler 

func (a *App) GetMetaHandler(w http.ResponseWriter, r *http.Request)

GetMetaHandler retrieves metadata about the object

func (*App) LocksHandler 

func (a *App) LocksHandler(w http.ResponseWriter, r *http.Request)

func (*App) LocksVerifyHandler 

func (a *App) LocksVerifyHandler(w http.ResponseWriter, r *http.Request)

func (*App) PostHandler 

func (a *App) PostHandler(w http.ResponseWriter, r *http.Request)

PostHandler instructs the client how to upload data

func (*App) PutHandler 

func (a *App) PutHandler(w http.ResponseWriter, r *http.Request)

PutHandler receives data from the client and puts it into the content store

func (*App) Represent 

func (a *App) Represent(rv *RequestVars, meta *MetaObject, download, upload, useTus bool) *Representation

Represent takes a RequestVars and Meta and turns it into a Representation suitable for json encoding

func (*App) Serve 

func (a *App) Serve(l net.Listener) error

Serve calls http.Serve with the provided Listener and the app's router

func (*App) ServeHTTP 

func (a *App) ServeHTTP(w http.ResponseWriter, r *http.Request)

func (*App) VerifyHandler 

func (a *App) VerifyHandler(w http.ResponseWriter, r *http.Request)

type BatchResponse 

type BatchResponse struct {
	Transfer string            `json:"transfer,omitempty"`
	Objects  []*Representation `json:"objects"`
}

type BatchVars 

type BatchVars struct {
	Transfers []string       `json:"transfers,omitempty"`
	Operation string         `json:"operation"`
	Objects   []*RequestVars `json:"objects"`
}

type Configuration 

type Configuration struct {
	Listen      string `config:"tcp://:8080"`
	Host        string `config:"localhost:8080"`
	ExtOrigin   string `config:""` // consider lfs-test-server may behind a reverse proxy
	MetaDB      string `config:"lfs.db"`
	ContentPath string `config:"lfs-content"`
	AdminUser   string `config:""`
	AdminPass   string `config:""`
	Cert        string `config:""`
	Key         string `config:""`
	Scheme      string `config:"http"`
	Public      string `config:"public"`
	UseTus      string `config:"false"`
	TusHost     string `config:"localhost:1080"`
}

Configuration holds application configuration. Values will be pulled from environment variables, prefixed by keyPrefix. Default values can be added via tags.

func (*Configuration) IsHTTPS 

func (c *Configuration) IsHTTPS() bool

func (*Configuration) IsPublic 

func (c *Configuration) IsPublic() bool

func (*Configuration) IsUsingTus 

func (c *Configuration) IsUsingTus() bool

type ContentStore 

type ContentStore struct {
	// contains filtered or unexported fields
}

ContentStore provides a simple file system based storage.

func NewContentStore 

func NewContentStore(base string) (*ContentStore, error)

NewContentStore creates a ContentStore at the base directory.

func (*ContentStore) Exists 

func (s *ContentStore) Exists(meta *MetaObject) bool

Exists returns true if the object exists in the content store.

func (*ContentStore) Get 

func (s *ContentStore) Get(meta *MetaObject, fromByte int64) (io.ReadCloser, error)

Get takes a Meta object and retreives the content from the store, returning it as an io.ReaderCloser. If fromByte > 0, the reader starts from that byte

func (*ContentStore) Put 

func (s *ContentStore) Put(meta *MetaObject, r io.Reader) error

Put takes a Meta object and an io.Reader and writes the content to the store.

type KVLogger 

type KVLogger struct {
	// contains filtered or unexported fields
}

KVLogger provides a logger that logs data in key/value pairs.

func NewKVLogger 

func NewKVLogger(out io.Writer) *KVLogger

NewKVLogger creates a KVLogger that writes to `out`.

func (*KVLogger) Fatal 

func (l *KVLogger) Fatal(data kv)

Fatal is equivalent to Log() follwed by a call to os.Exit(1)

func (*KVLogger) Log 

func (l *KVLogger) Log(data kv)

Log logs the key/value pairs to the logger's output.

type Lock 

type Lock struct {
	Id       string    `json:"id"`
	Path     string    `json:"path"`
	Owner    User      `json:"owner"`
	LockedAt time.Time `json:"locked_at"`
}

type LockList 

type LockList struct {
	Locks      []Lock `json:"locks"`
	NextCursor string `json:"next_cursor,omitempty"`
	Message    string `json:"message,omitempty"`
}

type LockRequest 

type LockRequest struct {
	Path string `json:"path"`
}

type LockResponse 

type LockResponse struct {
	Lock    *Lock  `json:"lock"`
	Message string `json:"message,omitempty"`
}

type LocksByCreatedAt 

type LocksByCreatedAt []Lock

func (LocksByCreatedAt) Len 

func (c LocksByCreatedAt) Len() int

func (LocksByCreatedAt) Less 

func (c LocksByCreatedAt) Less(i, j int) bool

func (LocksByCreatedAt) Swap 

func (c LocksByCreatedAt) Swap(i, j int)

type MetaObject 

type MetaObject struct {
	Oid      string `json:"oid"`
	Size     int64  `json:"size"`
	Existing bool
}

MetaObject is object metadata as seen by the object and metadata stores.

type MetaStore 

type MetaStore struct {
	// contains filtered or unexported fields
}

MetaStore implements a metadata storage. It stores user credentials and Meta information for objects. The storage is handled by boltdb.

func NewMetaStore 

func NewMetaStore(config *Configuration, dbFile string) (*MetaStore, error)

NewMetaStore creates a new MetaStore using the boltdb database at dbFile.

func (*MetaStore) AddLocks 

func (s *MetaStore) AddLocks(repo string, l ...Lock) error

AddLocks write locks to the store for the repo.

func (*MetaStore) AddUser 

func (s *MetaStore) AddUser(user, pass string) error

AddUser adds user credentials to the meta store.

func (*MetaStore) AllLocks 

func (s *MetaStore) AllLocks() ([]Lock, error)

AllLocks return all locks in the store, lock path is prepended with repo

func (*MetaStore) Authenticate 

func (s *MetaStore) Authenticate(user, password string) (string, bool)

Authenticate authorizes user with password and returns the user name

func (*MetaStore) Close 

func (s *MetaStore) Close()

Close closes the underlying boltdb.

func (*MetaStore) Delete 

func (s *MetaStore) Delete(v *RequestVars) error

Delete removes the meta information from RequestVars to the store.

func (*MetaStore) DeleteLock 

func (s *MetaStore) DeleteLock(repo, user, id string, force bool) (*Lock, error)

DeleteLock removes lock for the repo by id from the store

func (*MetaStore) DeleteUser 

func (s *MetaStore) DeleteUser(user string) error

DeleteUser removes user credentials from the meta store.

func (*MetaStore) FilteredLocks 

func (s *MetaStore) FilteredLocks(repo, path, cursor, limit string) (locks []Lock, next string, err error)

FilteredLocks return filtered locks for the repo

func (*MetaStore) Get 

func (s *MetaStore) Get(v *RequestVars) (*MetaObject, error)

Get retrieves the Meta information for an object given information in RequestVars

func (*MetaStore) Locks 

func (s *MetaStore) Locks(repo string) ([]Lock, error)

Locks retrieves locks for the repo from the store

func (*MetaStore) Objects 

func (s *MetaStore) Objects() ([]*MetaObject, error)

Objects returns all MetaObjects in the meta store

func (*MetaStore) Put 

func (s *MetaStore) Put(v *RequestVars) (*MetaObject, error)

Put writes meta information from RequestVars to the store.

func (*MetaStore) UnsafeGet 

func (s *MetaStore) UnsafeGet(v *RequestVars) (*MetaObject, error)

Get retrieves the Meta information for an object given information in RequestVars DO NOT CHECK authentication, as it is supposed to have been done before

func (*MetaStore) Users 

func (s *MetaStore) Users() ([]*MetaUser, error)

Users returns all MetaUsers in the meta store

type MetaUser 

type MetaUser struct {
	Name string
}

MetaUser encapsulates information about a meta store user

type ObjectError 

type ObjectError struct {
	Code    int    `json:"code"`
	Message string `json:"message"`
}

type Representation 

type Representation struct {
	Oid     string           `json:"oid"`
	Size    int64            `json:"size"`
	Actions map[string]*link `json:"actions"`
	Error   *ObjectError     `json:"error,omitempty"`
}

Representation is object medata as seen by clients of the lfs server.

type RequestVars 

type RequestVars struct {
	Oid           string
	Size          int64
	User          string
	Password      string
	Repo          string
	Authorization string
	// contains filtered or unexported fields
}

RequestVars contain variables from the HTTP request. Variables from routing, json body decoding, and some headers are stored. 

func (v *RequestVars) DownloadLink() string

DownloadLink builds a URL to download the object. 

func (v *RequestVars) UploadLink(useTus bool) string

UploadLink builds a URL to upload the object. 

func (v *RequestVars) VerifyLink() string

type TrackingListener 

type TrackingListener struct {
	net.Listener
	// contains filtered or unexported fields
}

TrackingListener tracks incoming connections so that application shutdown can wait until all in progress connections are finished before exiting.

func NewTrackingListener 

func NewTrackingListener(addr string) (*TrackingListener, error)

NewTrackingListener creates a new TrackingListener, listening on the supplied address.

func (*TrackingListener) Accept 

func (l *TrackingListener) Accept() (net.Conn, error)

Accept wraps the underlying net.Listener's Accept(), keeping track of all connections accepted.

func (*TrackingListener) WaitForChildren 

func (l *TrackingListener) WaitForChildren()

WaitForChildren is called during shutdown. It will return once all the existing connections have finished.

type TusServer 

type TusServer struct {
	// contains filtered or unexported fields
}

func (*TusServer) Create 

func (t *TusServer) Create(oid string, size int64) (string, error)

Create a new upload URL for the given object Required to call CREATE on the tus API before uploading but not part of LFS API

func (*TusServer) Finish 

func (t *TusServer) Finish(oid string, store *ContentStore) error

Move the finished uploaded data from TUS to the content store (called by verify)

func (*TusServer) Start 

func (t *TusServer) Start()

Start launches the tus server & stores uploads in the given contentPath

func (*TusServer) Stop 

func (t *TusServer) Stop()

type UnlockRequest 

type UnlockRequest struct {
	Force bool `json:"force"`
}

type UnlockResponse 

type UnlockResponse struct {
	Lock    *Lock  `json:"lock"`
	Message string `json:"message,omitempty"`
}

type User 

type User struct {
	Name string `json:"name"`
}

type VerifiableLockList 

type VerifiableLockList struct {
	Ours       []Lock `json:"ours"`
	Theirs     []Lock `json:"theirs"`
	NextCursor string `json:"next_cursor,omitempty"`
	Message    string `json:"message,omitempty"`
}

type VerifiableLockRequest 

type VerifiableLockRequest struct {
	Cursor string `json:"cursor,omitempty"`
	Limit  int    `json:"limit,omitempty"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.
⌂ 换网址