fs

package
v0.0.0-...-8f2684f Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 26, 2014 License: MIT Imports: 16 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// Config file
	ConfigFile *goconfig.ConfigFile
	// Home directory
	HomeDir = configHome()
	// Config file path
	ConfigPath = path.Join(HomeDir, configFileName)
	// Global config
	Config = &ConfigInfo{}
)

Global

View Source
var (
	Stats = NewStats()
)

Globals

Functions

func CalculateModifyWindow

func CalculateModifyWindow(fs ...Fs)

Work out modify window for fses passed in - sets Config.ModifyWindow

This is the largest modify window of all the fses in use, and the user configured value

func Check

func Check(fdst, fsrc Fs) error

Checks the files in fsrc and fdst according to Size and MD5SUM

func CheckMd5sums

func CheckMd5sums(src, dst Object) (bool, error)

Check the two files to see if the MD5sums are the same

May return an error which will already have been logged

If an error is returned it will return false

func Choose

func Choose(what string, defaults, help []string, newOk bool) string

Choose one of the defaults or type a new string if newOk is set

func ChooseOption

func ChooseOption(o *Option) string

Choose an option

func ChooseRemote

func ChooseRemote() string

ChooseRemote chooses a remote name

func Command

func Command(commands []string) byte

Command - choose one

func Confirm

func Confirm() bool

Asks the user for Yes or No and returns true or false

func Copier

func Copier(in ObjectPairChan, fdst Fs, wg *sync.WaitGroup)

Read FsObjects on in and copy them

func Copy

func Copy(f Fs, dst, src Object)

Copy src object to dst or f if nil

If dst is nil then the object must not exist already. If you do call Copy() with dst nil on a pre-existing file then some filing systems (eg Drive) may duplicate the file.

func Debug

func Debug(o interface{}, text string, args ...interface{})

Write debuging output for this Object or Fs

func DeleteFiles

func DeleteFiles(to_be_deleted ObjectsChan)

Delete all the files passed in the channel

func DeleteRemote

func DeleteRemote(name string)

Delete a remote

func EditConfig

func EditConfig()

Edit the config file interactively

func EditRemote

func EditRemote(name string)

Edit a remote

func Equal

func Equal(src, dst Object) bool

Checks to see if the src and dst objects are equal by looking at size, mtime and MD5SUM

If the src and dst size are different then it is considered to be not equal.

If the size is the same and the mtime is the same then it is considered to be equal. This is the heuristic rsync uses when not using --checksum.

If the size is the same and and mtime is different or unreadable and the MD5SUM is the same then the file is considered to be equal. In this case the mtime on the dst is updated.

Otherwise the file is considered to be not equal including if there were errors reading info.

func List

func List(f Fs) error

List the Fs to stdout

Lists in parallel which may get them out of order

func ListDir

func ListDir(f Fs) error

List the directories/buckets/containers in the Fs to stdout

func LoadConfig

func LoadConfig()

Loads the config file

func Log

func Log(o interface{}, text string, args ...interface{})

Write log output for this Object or Fs

func Mkdir

func Mkdir(f Fs) error

Makes a destination directory or container

func NewRemote

func NewRemote(name string)

Make a new remote

func OkRemote

func OkRemote(name string) bool

Print the contents of the remote and ask if it is OK

func OutputLog

func OutputLog(o interface{}, text string, args ...interface{})

Outputs log for object

func PairChecker

func PairChecker(in ObjectPairChan, out ObjectPairChan, wg *sync.WaitGroup)

Read FsObjects~s on in send to out if they need uploading

FIXME potentially doing lots of MD5SUMS at once

func Purge

func Purge(f Fs) error

Removes a container and all of its contents

FIXME doesn't delete local directories

func ReadLine

func ReadLine() string

Read some input

func Register

func Register(info *FsInfo)

Register a filesystem

Fs modules should use this in an init() function

func RemoteConfig

func RemoteConfig(name string)

Runs the config helper for the remote if needed

func Rmdir

func Rmdir(f Fs) error

Removes a container but not if not empty

func SaveConfig

func SaveConfig()

Save configuration file.

func ShowRemote

func ShowRemote(name string)

Show the contents of the remote

func ShowRemotes

func ShowRemotes()

Show an overview of the config file

func Sync

func Sync(fdst, fsrc Fs, Delete bool) error

Syncs fsrc into fdst

If Delete is true then it deletes any files in fdst that aren't in fsrc

Types

type Account

type Account struct {
	// contains filtered or unexported fields
}

Account limits and accounts for one transfer

func NewAccount

func NewAccount(in io.ReadCloser) *Account

NewAccount makes a Account reader

func (*Account) Close

func (file *Account) Close() error

Close the object

func (*Account) Read

func (file *Account) Read(p []byte) (n int, err error)

Read bytes from the object - see io.Reader

type ConfigInfo

type ConfigInfo struct {
	Verbose      bool
	Quiet        bool
	DryRun       bool
	ModifyWindow time.Duration
	Checkers     int
	Transfers    int
}

Filesystem config options

type Dir

type Dir struct {
	Name  string    // name of the directory
	When  time.Time // modification or creation time - IsZero for unknown
	Bytes int64     // size of directory and contents -1 for unknown
	Count int64     // number of objects -1 for unknown
}

A structure of directory/container/bucket lists

type DirChan

type DirChan chan *Dir

A channel of Dir objects

type Fs

type Fs interface {
	// String returns a description of the FS
	String() string

	// List the Fs into a channel
	List() ObjectsChan

	// List the Fs directories/buckets/containers into a channel
	ListDir() DirChan

	// Find the Object at remote.  Returns nil if can't be found
	NewFsObject(remote string) Object

	// Put in to the remote path with the modTime given of the given size
	//
	// May create the object even if it returns an error - if so
	// will return the object and the error, otherwise will return
	// nil and the error
	Put(in io.Reader, remote string, modTime time.Time, size int64) (Object, error)

	// Make the directory (container, bucket)
	Mkdir() error

	// Remove the directory (container, bucket) if empty
	Rmdir() error

	// Precision of the ModTimes in this Fs
	Precision() time.Duration
}

A Filesystem, describes the local filesystem and the remote object store

func NewFs

func NewFs(path string) (Fs, error)

NewFs makes a new Fs object from the path

The path is of the form service://path

Services are looked up in the config file

func NewLimited

func NewLimited(fs Fs, objects ...Object) Fs

NewLimited maks a limited Fs limited to the objects passed in

type FsInfo

type FsInfo struct {
	// Name of this fs
	Name string
	// Create a new file system.  If root refers to an existing
	// object, then it should return a Fs which only returns that
	// object.
	NewFs func(name string, root string) (Fs, error)
	// Function to call to help with config
	Config func(string)
	// Options for the Fs configuration
	Options []Option
}

Filesystem info

func Find

func Find(name string) (*FsInfo, error)

Finds a FsInfo object for the name passed in

Services are looked up in the config file

type Limited

type Limited struct {
	// contains filtered or unexported fields
}

This defines a Limited Fs which can only return the Objects passed in from the Fs passed in

func (*Limited) List

func (f *Limited) List() ObjectsChan

List the Fs into a channel

func (*Limited) ListDir

func (f *Limited) ListDir() DirChan

List the Fs directories/buckets/containers into a channel

func (*Limited) Mkdir

func (f *Limited) Mkdir() error

Make the directory (container, bucket)

func (*Limited) NewFsObject

func (f *Limited) NewFsObject(remote string) Object

Find the Object at remote. Returns nil if can't be found

func (*Limited) Precision

func (f *Limited) Precision() time.Duration

Precision of the ModTimes in this Fs

func (*Limited) Put

func (f *Limited) Put(in io.Reader, remote string, modTime time.Time, size int64) (Object, error)

Put in to the remote path with the modTime given of the given size

May create the object even if it returns an error - if so will return the object and the error, otherwise will return nil and the error

func (*Limited) Rmdir

func (f *Limited) Rmdir() error

Remove the directory (container, bucket) if empty

func (*Limited) String

func (f *Limited) String() string

String returns a description of the FS

type Object

type Object interface {
	// String returns a description of the Object
	String() string

	// Fs returns the Fs that this object is part of
	Fs() Fs

	// Remote returns the remote path
	Remote() string

	// Md5sum returns the md5 checksum of the file
	Md5sum() (string, error)

	// ModTime returns the modification date of the file
	ModTime() time.Time

	// SetModTime sets the metadata on the object to set the modification date
	SetModTime(time.Time)

	// Size returns the size of the file
	Size() int64

	// Open opens the file for read.  Call Close() on the returned io.ReadCloser
	Open() (io.ReadCloser, error)

	// Update in to the object with the modTime given of the given size
	Update(in io.Reader, modTime time.Time, size int64) error

	// Storable says whether this object can be stored
	Storable() bool

	// Removes this object
	Remove() error
}

A filesystem like object which can either be a remote object or a local file/directory

type ObjectPair

type ObjectPair struct {
	// contains filtered or unexported fields
}

A pair of Objects

type ObjectPairChan

type ObjectPairChan chan ObjectPair

A channel of ObjectPair

type Objects

type Objects []Object

A slice of Objects

type ObjectsChan

type ObjectsChan chan Object

A channel of Objects

type Option

type Option struct {
	Name     string
	Help     string
	Optional bool
	Examples []OptionExample
}

An options for a Fs

type OptionExample

type OptionExample struct {
	Value string
	Help  string
}

An example for an option

type Purger

type Purger interface {
	// Purge all files in the root and the root directory
	//
	// Implement this if you have a way of deleting all the files
	// quicker than just running Remove() on the result of List()
	Purge() error
}

Optional interfaces

type StatsInfo

type StatsInfo struct {
	// contains filtered or unexported fields
}

Stats limits and accounts all transfers

func NewStats

func NewStats() *StatsInfo

NewStats cretates an initialised StatsInfo

func (*StatsInfo) Bytes

func (s *StatsInfo) Bytes(bytes int64)

Bytes updates the stats for bytes bytes

func (*StatsInfo) Checking

func (s *StatsInfo) Checking(o Object)

Checking adds a check into the stats

func (*StatsInfo) DoneChecking

func (s *StatsInfo) DoneChecking(o Object)

DoneChecking removes a check from the stats

func (*StatsInfo) DoneTransferring

func (s *StatsInfo) DoneTransferring(o Object)

DoneTransferring removes a transfer from the stats

func (*StatsInfo) Error

func (s *StatsInfo) Error()

Error adds a single error into the stats

func (*StatsInfo) Errored

func (s *StatsInfo) Errored() bool

Errored returns whether there have been any errors

func (*StatsInfo) Errors

func (s *StatsInfo) Errors(errors int64)

Errors updates the stats for errors

func (*StatsInfo) GetErrors

func (s *StatsInfo) GetErrors() int64

GetErrors reads the number of errors

func (*StatsInfo) Log

func (s *StatsInfo) Log()

Log outputs the StatsInfo to the log

func (*StatsInfo) String

func (s *StatsInfo) String() string

String convert the StatsInfo to a string for printing

func (*StatsInfo) Transferring

func (s *StatsInfo) Transferring(o Object)

Transferring adds a transfer into the stats

type StringSet

type StringSet map[string]bool

Stringset holds some strings

func (StringSet) String

func (ss StringSet) String() string

String returns all the strings in the StringSet joined by comma

func (StringSet) Strings

func (ss StringSet) Strings() []string

Strings returns all the strings in the StringSet

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址