Documentation
¶
Overview ¶
Package backup creates and restores server archives and ships them to storage targets (local/NFS/CIFS mount, SFTP, SMB, S3, GCS, Azure). Credentials are encrypted at rest by the caller; this package never logs them.
Index ¶
- Constants
- Variables
- func Archive(dataDir string, include []string, w io.Writer) (skipped []string, err error)
- func PBSBackupArgs(cfg Config, backupID, srcMount string, excludes []string) []string
- func PBSBackupID(slug, shortID string) string
- func PBSConnectionCheckArgs(cfg Config) []string
- func PBSEnv(cfg Config, passFile string) []string
- func PBSExcludes(dataDir string, include []string) ([]string, error)
- func PBSForgetArgs(cfg Config, snapshot string) []string
- func PBSGroup(backupID string) string
- func PBSPruneArgs(cfg Config, backupID string, keepN, keepDays int) (args []string, ok bool)
- func PBSRepository(cfg Config) string
- func PBSRestoreArgs(cfg Config, snapshot, dstMount string) []string
- func PBSSnapshotListArgs(cfg Config, backupID string) []string
- func PBSSnapshotName(backupID string, at time.Time) string
- func RcloneSetup(ctx context.Context, run RcloneRunner, cfg Config) ([]byte, error)
- func Restore(r io.Reader, destDir string) error
- func Verify(r io.Reader) (entries int, total int64, err error)
- type Config
- type Object
- type PBSSnapshot
- type RcloneRunner
- type Target
Constants ¶
const PBSArchive = "data"
PBSArchive is the single pxar archive every snapshot contains. One archive rather than one per included path, because backup.include lists FILES as well as directories (Minecraft includes server.properties) and a pxar source must be a directory. Inclusion is expressed as exclusions instead — see PBSExcludes.
const PBSDefaultPort = 8007
PBSDefaultPort is the port PBS listens on.
const PBSType = "pbs"
PBSType is the backup_targets.type value for this destination.
const RcloneConfPath = "/ygg/rclone.conf"
RcloneConfPath is where the config file sits inside the container.
const RcloneType = "rclone"
RcloneType is the target type for every service reached through rclone.
const UploadScratchDir = ".ygg_uploads"
UploadScratchDir is the folder inside a server's data directory where a chunked upload accumulates before its final rename. It is scratch space, never content: excluded from archives here, and swept by the API package that writes it. One name, one definition — two copies of it is how an archive quietly starts carrying gigabytes of nothing again.
Variables ¶
var RcloneProviders = map[string]string{
"jottacloud": "jottacloud",
"dropbox": "dropbox",
"box": "box",
"onedrive": "onedrive",
"drive": "drive",
"pcloud": "pcloud",
"mega": "mega",
"webdav": "webdav",
"custom": "",
}
RcloneProviders are the services offered in the panel, with the rclone backend each one is. "custom" is a whole config section pasted as is.
Functions ¶
func Archive ¶
Archive writes a gzip-compressed tar of dataDir to w. When include is non-empty, only those top-level paths (files or directories) are archived — matching a gameskill's backup.include. Paths are stored relative to dataDir.
A file that can't be read (permission denied, or it vanished mid-walk) is recorded in skipped and the archive continues — one unreadable file must not fail the whole backup. The caller should surface skipped so a partial archive isn't mistaken for a complete one. err is reserved for fatal, archive-level failures (a broken tar/gzip stream or a closed sink).
func PBSBackupArgs ¶ added in v0.3.6
PBSBackupArgs builds the backup invocation. srcMount is where the caller has bind-mounted the server's data directory inside the container.
func PBSBackupID ¶ added in v0.3.6
PBSBackupID converts a panel server name plus its short id into a legal PBS backup-id. Each panel server gets its own backup group, which is what makes the incremental upload work: PBS dedupes a snapshot against the previous snapshot in the SAME group, so a stable id per server is the whole trick.
func PBSConnectionCheckArgs ¶ added in v0.3.6
PBSConnectionCheckArgs is the connection test: list the backup groups in the datastore. One call exercises DNS, TLS, the certificate fingerprint, the auth-id, the token secret AND the datastore name, so a green result means a backup will reach the same place.
It lists rather than asking for `status`, and that is measured rather than assumed: `status` needs privileges beyond the datastore and fails with a bare "permission check failed" for a token that can back up perfectly well. `list` needs Datastore.Audit|Datastore.Backup — exactly what a backup token has — so the test passes for every credential that can actually do the job, and fails for every one that cannot.
func PBSEnv ¶ added in v0.3.6
PBSEnv is the environment for a client invocation.
The token secret is passed by FILE, not by value. An -e on the container would be readable from `docker inspect` for as long as Docker keeps the container's JSON on disk, which is longer than the backup and longer than anyone thinks. passFile is a 0600 file the caller creates and removes.
HOME is set because the client writes tickets and known-fingerprints under $HOME/.config; with no HOME it fails in a way that reads like an auth error.
func PBSExcludes ¶ added in v0.3.6
PBSExcludes turns a gameskill's backup.include into the exclusions that leave exactly those paths behind.
pxar takes one source directory and a list of things to leave out, so "back up only these" has to be expressed as "leave out everything else". The list of "everything else" is read from the directory at backup time, which is honest about one edge: a top-level entry created between this read and the walk is included rather than excluded. For a backup that is the safe direction to err, and it beats the alternative of quietly ignoring backup.include — which is what a naive PBS implementation does, and which would silently back up a Minecraft server's entire 40 GB cache alongside its 200 MB world.
An empty include list, or one containing ".", means the whole directory.
func PBSForgetArgs ¶ added in v0.3.6
PBSForgetArgs removes one snapshot. Note that the space is not reclaimed until the PBS server runs garbage collection — chunks are shared, so nothing else could be safe. Callers must not report freed bytes from this.
func PBSPruneArgs ¶ added in v0.3.6
PBSPruneArgs applies retention server-side.
The panel's policy is keep_n ("keep the newest N") and keep_days ("keep anything newer than D days"). PBS has --keep-last, which is exactly keep_n, and --keep-daily, which is "keep the newest snapshot of each of the last D days" — close to keep_days but not identical: with two backups on one day it keeps one of them, and it counts days that HAVE a backup rather than calendar days. That difference is worth stating in the UI, and is why prune is only asked for the rules that were actually set.
Returning ok=false means no policy is configured. Running prune with no --keep flag at all would delete every snapshot in the group, so this must never degrade into "just call prune".
func PBSRepository ¶ added in v0.3.6
PBSRepository builds the repository string the client expects:
[[auth-id@]server[:port]:]datastore
The auth-id itself contains "@" and, for an API token, "!". That is fine here and would not be fine in a shell — which is why every caller passes this as one argv element and never interpolates it into a command line.
An IPv6 literal has to be bracketed or the colons are read as the port separator; a hostname that already looks bracketed is left alone.
func PBSRestoreArgs ¶ added in v0.3.6
PBSRestoreArgs restores a snapshot's pxar archive into dstMount.
func PBSSnapshotListArgs ¶ added in v0.3.6
PBSSnapshotListArgs lists one server's snapshots.
func PBSSnapshotName ¶ added in v0.3.6
PBSSnapshotName is the identifier `restore` and `forget` take.
func RcloneSetup ¶ added in v0.3.33
RcloneSetup turns what an admin entered into a working rclone.conf with one remote named "ygg". For the OAuth services that means running rclone's own configuration, which for Jottacloud exchanges the single-use personal login token for a refreshable one; questions it asks on the way (which drive, which device) are answered with rclone's default.
func Verify ¶
Restore extracts a gzip-tar archive from r into destDir, guarding against path-traversal entries. Verify streams a backup archive and confirms it is a well-formed gzip+tar that decompresses in full — a cheap integrity check (a truncated or corrupt archive fails at the gzip CRC or a short tar entry) that reads everything but writes nothing to disk. Returns the file count and total uncompressed size.
Types ¶
type Config ¶
type Config struct {
Type string `json:"type"` // local | sftp | smb | pbs | s3 | gcs | azure
Path string `json:"path"` // base directory / remote path / share subpath
Host string `json:"host,omitempty"`
Port int `json:"port,omitempty"`
Username string `json:"username,omitempty"` // for pbs: the auth-id, e.g. ygg@pbs!panel
Password string `json:"password,omitempty"` // for pbs: the API token secret
// Object storage. Host is the endpoint (empty = the provider's own), Path is
// a folder inside the bucket. Username/Password are the access key ID and
// secret for s3, the account name and access key (or SAS token) for azure;
// for gcs, Password holds the service account's JSON key.
Bucket string `json:"bucket,omitempty"` // s3/gcs bucket, azure container
Region string `json:"region,omitempty"` // s3 only; for rclone/webdav, the server flavour
// rclone: which service (a key of RcloneProviders), and the rclone.conf the
// panel built at setup. The conf is written by the panel only — never taken
// from a request, never sent to the browser.
Provider string `json:"provider,omitempty"`
RcloneConf string `json:"rclone_conf,omitempty"`
// Proxmox Backup Server only.
Datastore string `json:"datastore,omitempty"`
Namespace string `json:"namespace,omitempty"` // optional; empty = the datastore root
Fingerprint string `json:"fingerprint,omitempty"` // server cert sha256, for self-signed installs
}
Config is the decrypted configuration for opening a target.
It is stored as encrypted JSON, so adding a field is backward compatible: a target written before the field existed decodes with it empty.
type Object ¶
type Object struct {
Name string `json:"name"`
Size int64 `json:"size"`
ModTime time.Time `json:"mod_time"`
}
Object describes a stored archive on a target.
type PBSSnapshot ¶ added in v0.3.6
type PBSSnapshot struct {
Name string `json:"name"` // host/<id>/<rfc3339>, what restore takes
Time time.Time `json:"time"`
Size int64 `json:"size"`
Protected bool `json:"protected"`
}
PBSSnapshot is one restorable point in time.
func ParsePBSSnapshots ¶ added in v0.3.6
func ParsePBSSnapshots(out []byte) ([]PBSSnapshot, error)
ParsePBSSnapshots reads `snapshot list --output-format json`, newest first.
The client prints progress to stderr and JSON to stdout, but a mixed stream is easy to end up with, so this tolerates leading noise by seeking to the first "[". Anything that is not an object with a backup-time is skipped rather than failing the whole listing: one unreadable entry must not hide the snapshots an operator is trying to restore from.
type RcloneRunner ¶ added in v0.3.33
type RcloneRunner interface {
RunRclone(ctx context.Context, conf []byte, argv []string, stdin io.Reader, stdout io.Writer) (newConf []byte, stderr string, err error)
}
RcloneRunner runs one rclone invocation. conf is copied in before it starts; the config as rclone left it is returned, also on failure.
type Target ¶
type Target interface {
Put(ctx context.Context, name string, r io.Reader) (int64, error)
Get(ctx context.Context, name string) (io.ReadCloser, error)
List(ctx context.Context) ([]Object, error)
Delete(ctx context.Context, name string) error
Close() error
}
Target is a storage backend for archives.
func OpenRclone ¶ added in v0.3.33
OpenRclone opens an rclone target. onConf is called with the config whenever a run changed it (a refreshed token), so it can be stored. mu serialises runs against the same login; the caller keeps one per target.