Documentation
¶
Overview ¶
Package backup implements the destination side of databox's backup & restore system (§17).
A backup destination is anything that can hold a set of named files: the Dest interface below. Two production implementations exist — S3-compatible object storage (client-side SigV4 signing over plain net/http, path-style URLs, custom endpoints supported) and SFTP (password auth, host key trusted on first use with the fingerprint logged). A third implementation, dirDest, writes to a local directory and exists for tests and local development only.
The backup engine itself runs inside the server process (pkg/server/backup.go); this package deliberately knows nothing about keys, shards, or manifests — it only moves bytes to and from a destination.
Credential handling ¶
This package itself persists nothing: Credentials arrive at Open and live in the constructed Dest. Per §17, the SERVER additionally holds them AES-256-GCM-encrypted in the system keyspace for the job's lifetime (key derived from cluster secret material — see pkg/server/backup.go, sealCreds) and purges them on completion or cancellation, so any node can resume a job after a coordinator crash without the operator re-supplying secrets. Credentials are never logged; persisted URLs are redacted.
dirdest.go implements Dest against a local directory (file:// URLs).
This destination exists for tests and local development: the backup integration tests use it to exercise the full engine without an S3 or SFTP server, and it is handy for quick manual experiments. A real backup should leave the machine — a node backing up to its own disk protects against nothing the disk can do to you.
s3dest.go implements Dest against S3-compatible object storage using the minimal SigV4 signer in sigv4.go — plain net/http, path-style URLs (http(s)://endpoint/bucket/key), custom endpoints supported so MinIO, Ceph RGW, and the databox S3 gateway itself all work as destinations.
sftpdest.go implements Dest over SFTP (github.com/pkg/sftp on golang.org/x/crypto/ssh). v1 supports password authentication only.
Host key policy: trust-on-first-use per process. The first connection to a host records the presented key and logs its SHA-256 fingerprint so the operator can verify it out of band; a later connection to the same host presenting a different key is refused. This mirrors the console's certificate TOFU model (§6.3) at the SSH layer.
sigv4.go is a minimal AWS Signature Version 4 request signer — just enough to talk to S3-compatible storage without pulling in an AWS SDK (the dependency policy forbids one). The verification counterpart lives in pkg/service/s3; the s3 gateway's tests use this signer to produce requests, so the two implementations cross-check each other.
Wire format (AWS "Authenticating Requests" documentation):
Authorization: AWS4-HMAC-SHA256 Credential=<key>/<date>/<region>/<svc>/aws4_request,
SignedHeaders=<h1;h2;...>, Signature=<hex hmac>
The string-to-sign is built from a canonical request:
METHOD \n uri-encoded-path \n canonical-query \n canonical-headers \n signed-header-names \n payload-hash
Index ¶
Constants ¶
const EmptyPayloadHash = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
EmptyPayloadHash is sha256("") — the payload hash for bodyless requests.
const UnsignedPayload = "UNSIGNED-PAYLOAD"
UnsignedPayload is the x-amz-content-sha256 sentinel for streaming bodies whose hash is not computed up front (allowed over TLS).
Variables ¶
This section is empty.
Functions ¶
func SignRequestV4 ¶
func SignRequestV4(req *http.Request, keyID, secret, region, service, payloadHash string, now time.Time)
SignRequestV4 signs req in place with AWS SigV4. It sets x-amz-date and x-amz-content-sha256 (from payloadHash), then computes the Authorization header. Headers signed: host, x-amz-*, and — when already present on the request — range, content-type and content-md5 (the set S3 SDKs sign).
payloadHash is the hex sha256 of the body, EmptyPayloadHash for no body, or UnsignedPayload to skip body hashing (used for streamed uploads).
Types ¶
type Credentials ¶
type Credentials struct {
AccessKey string // S3 access key ID
SecretKey string // S3 secret access key
S3Endpoint string // S3 endpoint override, e.g. "https://minio.local:9000"; empty = AWS
Region string // S3 signing region; empty = "us-east-1"
SFTPPassword string // SFTP password (password auth only in v1)
}
Credentials carries the destination secrets supplied at job-issue time. See the package comment for how they are held at rest (encrypted, by the server — never in cleartext, never logged).
type Dest ¶
type Dest interface {
// Put stores the full contents of r at path, overwriting any
// existing file. Implementations must not report success until the
// data is durably handed to the destination.
Put(path string, r io.Reader) error
// Get opens the file at path for reading. A missing file returns an
// error satisfying errors.Is(err, fs.ErrNotExist) where the
// implementation can tell.
Get(path string) (io.ReadCloser, error)
// List returns the relative paths of every file under prefix
// (prefix "" lists everything). Order is unspecified.
List(prefix string) ([]string, error)
}
Dest is a backup destination: a flat namespace of files addressed by slash-separated relative paths ("kv-000001.jsonl", "blobs/<sha256>", "manifest.json").
func Open ¶
Open parses a destination URL and constructs the matching Dest.
Supported URL forms:
s3://bucket/prefix (endpoint/keys/region from Credentials) sftp://user@host[:port]/path file:///absolute/path (local directory — tests and dev only)
The second return value is a redacted form of the URL, safe to persist in the job record and to log (any userinfo password is masked).
Source Files