filesystem

package
v1.0.0-rc.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 12, 2026 License: MIT Imports: 30 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// MaxIgnoreLength is the maximum size, in bytes, of a pattern list.
	MaxIgnoreLength = 32 * 1024
	// MaxIgnorePatterns is the maximum number of non-blank, non-comment lines in a list.
	MaxIgnorePatterns = 256
	// MaxIgnorePatternWildcards is the maximum number of unescaped "*" characters in a pattern.
	MaxIgnorePatternWildcards = 16
)

Limits applied to the gitignore-style pattern lists used by an Archive, whether they come from the Panel with a backup request or from a server's .pelicanignore file.

Variables

View Source 
var ErrNoSpaceAvailable = errors.Sentinel("archive: no space available on disk")

Functions

func FileModeFromUnix 

func FileModeFromUnix(m uint64) os.FileMode

FileModeFromUnix converts a Unix mode such as 0o4755, as sent by the Panel, to an os.FileMode. Go keeps setuid, setgid and sticky in other bits than Unix does, so a plain os.FileMode(0o4755) loses them. Bits above 0o7777 are ignored, as chmod(2) does.

func IsErrorCode 

func IsErrorCode(err error, code ErrorCode) bool

IsErrorCode checks if "err" is a filesystem Error type. If so, it will then drop in and check that the error code is the same as the provided ErrorCode passed in "code".

func IsLinkError 

func IsLinkError(err error) bool

func IsPathError 

func IsPathError(err error) bool

func ValidateIgnore 

func ValidateIgnore(s string) error

ValidateIgnore checks a pattern list against the limits above. The returned error describes the first limit exceeded and is safe to display to the user.

Types

type Archive 

type Archive struct {
	// contains filtered or unexported fields
}

func NewArchive 

func NewArchive(r *os.Root, dir string, opts ...ArchiveOption) (*Archive, error)

NewArchive returns a new archive instance that can be used for generating an archive of files and folders within the provided os.Root. The "dir" value is a child directory within the `os.Root` instance, and the paths stored in the archive are relative to it. Ex. os.Root = '/var/lib/pelican/volumes/{server-uuid}' and dir = "worlds"

func (*Archive) Create 

func (a *Archive) Create(ctx context.Context, f *os.File) error

Create creates an archive at dst with all the files defined in the included Files array.

THIS IS UNSAFE TO USE IF `dst` IS PROVIDED BY A USER! ONLY USE THIS WITH CONTROLLED PATHS!

func (*Archive) Progress 

func (a *Archive) Progress() *progress.Progress

func (*Archive) Stream 

func (a *Archive) Stream(ctx context.Context, w io.Writer) error

Stream walks the given root directory and generates an archive from the provided files.

type ArchiveOption 

type ArchiveOption func(a *Archive) error

func WithIgnored 

func WithIgnored(patterns string) ArchiveOption

WithIgnored excludes every file matching the given gitignore-style pattern list from the archive. The list is validated against the limits defined in ignore.go, and an error is returned if it exceeds them.

func WithMatching 

func WithMatching(files []string) ArchiveOption

WithMatching limits the archive to the given files and directories (and everything within those directories). Paths are relative to the archive's base directory.

func WithProgress 

func WithProgress(p *progress.Progress) ArchiveOption

type Error 

type Error struct {
	// contains filtered or unexported fields
}

func (*Error) Code 

func (e *Error) Code() ErrorCode

Code returns the ErrorCode for this specific error instance.

func (*Error) Error 

func (e *Error) Error() string

Returns a human-readable error string to identify the Error by.

func (*Error) Resolved 

func (e *Error) Resolved() string

Resolved returns the path, or for ErrCodeArchiveIsOutput the extension, the error is about.

func (*Error) Unwrap 

func (e *Error) Unwrap() error

Unwrap returns the underlying cause of this filesystem error. In some causes there may not be a cause present, in which case nil will be returned.

type ErrorCode 

type ErrorCode string
const (
	ErrCodeIsDirectory    ErrorCode = "E_ISDIR"
	ErrCodeDiskSpace      ErrorCode = "E_NODISK"
	ErrCodeUnknownArchive ErrorCode = "E_UNKNFMT"
	ErrCodeDenylistFile   ErrorCode = "E_DENYLIST"
	ErrCodeUnknownError   ErrorCode = "E_UNKNOWN"
	ErrNotExist           ErrorCode = "E_NOTEXIST"
	ErrCodeNotRegular     ErrorCode = "E_NOTREGULAR"
	// ErrCodeArchiveIsOutput is returned when decompressing a single-file archive
	// would write over the archive itself. resolved holds the extension the
	// archive should be renamed to end in.
	ErrCodeArchiveIsOutput ErrorCode = "E_ARCHIVEISOUTPUT"
)

type File 

type File interface {
	io.Reader
	io.ReaderAt
	io.Writer
	io.WriterAt
	io.Seeker
	io.Closer
	io.ReaderFrom
	Name() string
	Stat() (os.FileInfo, error)
	Truncate(size int64) error
	Sync() error
}

File is the handle returned by Filesystem.Touch. Writes made through it are accounted against the disk quota of the Filesystem that opened it.

type Filesystem 

type Filesystem struct {
	// contains filtered or unexported fields
}

func New 

func New(path string, size int64, denylist []string) (*Filesystem, error)

New creates a new Filesystem instance for a given server.

func (*Filesystem) CachedUsage 

func (fs *Filesystem) CachedUsage() int64

Returns the cached value for the amount of disk space used by the filesystem. Do not rely on this function for critical logical checks. It should only be used in areas where the actual disk usage does not need to be perfect, e.g. API responses for server resource usage.

func (*Filesystem) Chmod 

func (fs *Filesystem) Chmod(p string, mode os.FileMode) error

Chmod changes the mode of the given file or directory. os.Root follows a symlink at p, so the mode of a denylisted file cannot be changed through a link to it. Changing it by its own name is left to the caller to allow or refuse, as before.

func (*Filesystem) Chown 

func (fs *Filesystem) Chown(p string) error

Chown recursively iterates over a file or directory and sets the owner on all the underlying files. Iterate over all the files and directories. If it is a file go ahead and perform the chown operation. Otherwise dig deeper into the directory until we've run out of directories to dig into.

The walk happens through the os.Root instance and never follows symlinks, so it cannot change the owner of anything outside the server's data directory.

func (*Filesystem) Chtimes 

func (fs *Filesystem) Chtimes(p string, atime, mtime time.Time) error

Chtimes changes the access and modification times of the given file.

func (*Filesystem) Close 

func (fs *Filesystem) Close() error

Close closes the underlying os.Root instance for the server, and any folder that was open before the server's folder was replaced.

func (*Filesystem) CompressFiles 

func (fs *Filesystem) CompressFiles(ctx context.Context, dir string, name string, paths []string, extension string) (os.FileInfo, string, error)

CompressFiles compresses all the files matching the given paths in the specified directory. This function also supports passing nested paths to only compress certain files and folders when working in a larger directory. This effectively creates a local backup, but rather than ignoring specific files and folders, it takes an allowlist of files and folders.

All paths are relative to the dir that is passed in as the first argument, and the compressed file will be placed at that location named `archive-{date}.tar.gz`.

func (*Filesystem) Copy 

func (fs *Filesystem) Copy(p string) error

Copies a given file to the same location and appends a suffix to the file to indicate that it has been copied.

func (*Filesystem) CreateDirectory 

func (fs *Filesystem) CreateDirectory(name string, p string) error

CreateDirectory creates a new directory ("name") at a specified path ("p") for the server.

p and name are joined before being cleaned, as main does, so a name such as "../name" under "/" stays inside the server, while "e/../../name" still fails.

func (*Filesystem) DecompressFile 

func (fs *Filesystem) DecompressFile(ctx context.Context, dir string, file string) error

DecompressFile will decompress a file in a given directory by using the archiver tool to infer the file type and go from there. This will walk over all the files within the given archive and ensure that there is not a zip-slip attack being attempted by validating that the final path is within the server data directory.

func (*Filesystem) Delete 

func (fs *Filesystem) Delete(p string) error

Delete removes a file or folder from the system. Prevents the user from accidentally (or maliciously) removing their root server data directory.

func (*Filesystem) DirectorySize 

func (fs *Filesystem) DirectorySize(dir string) (int64, error)

DirectorySize determines the directory size of a given location. Returns the size in bytes. This can be a fairly taxing operation on locations with tons of files, so it is recommended that you cache the output.

The walk happens through the os.Root instance, so neither the directory being measured nor anything inside it can resolve to a location outside the server's data directory. Symlinks are never followed.

func (*Filesystem) DiskUsage 

func (fs *Filesystem) DiskUsage(allowStaleValue bool) (int64, error)

Internal helper function to allow other parts of the codebase to check the total used disk space as needed without overly taxing the system. This will prioritize the value from the cache to avoid excessive IO usage. We will only walk the filesystem and determine the size of the directory if there is no longer a cached value.

If "allowStaleValue" is set to true, a stale value MAY be returned to the caller if there is an expired cache value AND there is currently another lookup in progress. If there is no cached value but no other lookup is in progress, a fresh disk space response will be returned to the caller.

This is primarily to avoid a bunch of I/O operations from piling up on the server, especially on servers with a large amount of files.

func (*Filesystem) ExtractStreamUnsafe 

func (fs *Filesystem) ExtractStreamUnsafe(ctx context.Context, dir string, r io.Reader) error

ExtractStreamUnsafe .

func (*Filesystem) File 

func (fs *Filesystem) File(p string) (*os.File, Stat, error)

File returns a reader for a file instance as well as the stat information. Only regular files can be opened; directories return an ErrCodeIsDirectory error and any other type of file (named pipes, sockets, devices) returns an ErrCodeNotRegular error without the file ever being opened. Opening a named pipe for reading blocks until a writer appears, which would hang the caller.

func (*Filesystem) HasSpaceAvailable 

func (fs *Filesystem) HasSpaceAvailable(allowStaleValue bool) bool

HasSpaceAvailable checks if the directory a file is trying to be added to has enough space available for the file to be written to. Because determining the amount of space being used by a server is a taxing operation, we will load it all up into a cache and pull from that as long as the key is not expired.

This operation will potentially be blocked unless allowStaleValue is set to true. See the documentation on DiskUsage for how this affects the call.

func (*Filesystem) HasSpaceErr 

func (fs *Filesystem) HasSpaceErr(allowStaleValue bool) error

The same concept as HasSpaceAvailable however this will return an error if there is no space, rather than a boolean value.

func (*Filesystem) HasSpaceFor 

func (fs *Filesystem) HasSpaceFor(size int64) error

Helper function to determine if a server has space available for a file of a given size. If space is available, no error will be returned, otherwise an ErrNotEnoughSpace error will be raised.

func (*Filesystem) IsIgnored 

func (fs *Filesystem) IsIgnored(paths ...string) error

Checks if the given file or path is in the server's file denylist. If so, an Error is returned, otherwise nil is returned.

The path is checked as provided, after being cleaned, and after being cleaned as if it started at the server root. That way neither "foo/../config.yml" nor "../config.yml" can be used to get around an anchored pattern like "/config.yml"; callers do not all clean a leading "/" off the same way before using the path.

The path is also checked after resolving any symlinks in it, since os.Root follows symlinks that stay inside the server directory. Without that a link such as "notes -> config.yml" would give access to a denylisted file.

func (*Filesystem) IsIgnoredNoFollow 

func (fs *Filesystem) IsIgnoredNoFollow(paths ...string) error

IsIgnoredNoFollow is IsIgnored for operations that act on a directory entry itself rather than on what it points to: deleting and renaming. Symlinks in the parent directories are followed, since os.Root follows them too, but a symlink at the end of the path is not, so a link to a denylisted file can still be deleted or renamed without the file it points to being touched.

func (*Filesystem) ListDirectory 

func (fs *Filesystem) ListDirectory(p string) ([]Stat, error)

ListDirectory lists the contents of a given directory and returns stat information about each file and folder within it. If you only need to know the contents of the directory and do not need mimetype information, call Filesystem.ReadDir directly instead.

func (*Filesystem) MaxDisk 

func (fs *Filesystem) MaxDisk() int64

MaxDisk returns the maximum amount of disk space that this Filesystem instance is allowed to use.

func (*Filesystem) Mkdir 

func (fs *Filesystem) Mkdir(p string, mode os.FileMode) error

Mkdir creates a single directory, owned by the server user.

func (*Filesystem) NewArchive 

func (fs *Filesystem) NewArchive(dir string, opts ...ArchiveOption) (*Archive, error)

NewArchive returns a new archive of the given directory within the server's data directory.

func (*Filesystem) Path 

func (fs *Filesystem) Path() string

Path returns the root path for the Filesystem instance.

func (*Filesystem) ReadDir 

func (fs *Filesystem) ReadDir(p string) ([]fs2.DirEntry, error)

ReadDir returns all the contents of the given directory.

func (*Filesystem) Rename 

func (fs *Filesystem) Rename(from string, to string) error

Rename moves (or renames) a file or directory. 

func (fs *Filesystem) RestoreSymlink(name, target string) error

RestoreSymlink creates name as a symbolic link to target, creating any missing parent directories and replacing an existing file or link at that path. It is used when restoring archives that contain symbolic links. The target is stored as-is: every path Wings opens is resolved inside the server's os.Root, so a link pointing outside of it cannot be followed by Wings.

The link is created and chowned under a temporary name and then renamed into place, so an existing entry is only replaced once the new link is ready. A directory is never replaced.

func (*Filesystem) SafeDeleteRecursively 

func (fs *Filesystem) SafeDeleteRecursively(p string) error

SafeDeleteRecursively deletes a file or directory while respecting the denylist. Denylisted files and directories are left in place, as is every directory that still contains one of them.

func (*Filesystem) SetDiskLimit 

func (fs *Filesystem) SetDiskLimit(i int64)

SetDiskLimit sets the disk space limit for this Filesystem instance.

func (*Filesystem) SpaceAvailableForDecompression 

func (fs *Filesystem) SpaceAvailableForDecompression(ctx context.Context, dir string, file string) error

SpaceAvailableForDecompression returns an ErrCodeDiskSpace error when the files in the archive do not fit in the space the server has left, before anything is written, so a request that cannot fit fails without leaving half an archive behind. This is the check main makes.

The sizes come from the archive's own headers, which can be wrong (a zip can claim any size). That is fine: every write during the decompression still goes through a quota-tracked handle, which stops at the limit whatever the headers said. Single-file compression (.gz, .xz and so on) is not checked here because its size is not known without decompressing it; the per-write limit covers it.

func (*Filesystem) Stat 

func (fs *Filesystem) Stat(p string) (Stat, error)

Stat stats a file or folder and returns the base stat object from go along with the MIME data that can be used for editing files.

func (*Filesystem) Stat2 

func (fs *Filesystem) Stat2(p string) (os.FileInfo, error)
func (fs *Filesystem) Symlink(source, target string) error

Symlink creates a symbolic link at "target" pointing to "source". "target" is relative to the root of the server. A relative "source" is relative to the directory the link is created in, as with symlink(2); an absolute "source" is relative to the root of the server. The link is written as a path relative to the directory it is created in so that it resolves to the same file both on the host and inside the server container. Sources that do not exist, or that are outside the root of the server, are refused.

os.Root.Symlink allows for the creation of a symlink that targets a file outside the root directory. This isn't the end of the world because every read and write in this package goes through os.Root, which refuses to follow it, and within a container it would just point to something in the container filesystem.

func (*Filesystem) Touch 

func (fs *Filesystem) Touch(p string, flag int, mode os.FileMode) (File, error)

Touch acts by creating the given file and path on the disk if it is not present already. If it is present, the file is opened using the defaults which will truncate the contents. The opened file is then returned to the caller.

Writes made through the returned handle are accounted against the disk quota of the server, and will fail once the server runs out of space.

func (*Filesystem) TruncateRootDirectory 

func (fs *Filesystem) TruncateRootDirectory() error

TruncateRootDirectory removes _all_ files and directories from a server's data directory and resets the used disk space to zero.

func (*Filesystem) Write 

func (fs *Filesystem) Write(p string, r io.Reader, newSize int64, mode os.FileMode) error

Write writes a file to the disk.

func (*Filesystem) Writefile deprecated 

func (fs *Filesystem) Writefile(p string, r io.Reader) error

Writefile writes a file to the system. If the file does not already exist one will be created. This will also properly recalculate the disk space used by the server when writing new files or modifying existing ones.

The size of the data is not known ahead of time, so it is written through a quota-tracked handle which fails as soon as the server runs out of space.

Deprecated: prefer the use of Filesystem.Write, which checks that the file fits within the disk limit before writing anything.

type SpaceCheckingOpts 

type SpaceCheckingOpts struct {
	AllowStaleResponse bool
}

type Stat 

type Stat struct {
	os.FileInfo
	Mimetype string
}

func (*Stat) CTime 

func (s *Stat) CTime() time.Time

CTime returns the time that the file/folder was created.

TODO: remove. Ctim is not actually ever been correct and doesn't actually return the creation time.

func (*Stat) MarshalJSON 

func (s *Stat) MarshalJSON() ([]byte, error)

type TarProgress 

type TarProgress struct {
	*tar.Writer
	// contains filtered or unexported fields
}

TarProgress .

func NewTarProgress 

func NewTarProgress(w *tar.Writer, p *progress.Progress) *TarProgress

NewTarProgress .

func (*TarProgress) Write 

func (p *TarProgress) Write(v []byte) (int, error)

Write .

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.
⌂ 换网址