Documentation
¶
Overview ¶
Package sftp implements provider.Store over SSH's file transfer subsystem (M18, DESIGN.md §9).
It is the mirror image of the Drive provider, and every difference follows from one fact: SFTP is natively path-addressed. The seam is already path-addressed (DESIGN.md §2.5), so each Store method is one protocol call and there is no identity to invent — which is why nothing here resembles gdrive's index.go, and why internal/pathindex stays Drive-private. A path index over a path-addressed store would be a cache keyed by its own value.
Three consequences of that shape, all of them stated once in the M17–M21 preamble and all of them visible in this package:
- There is no change feed, so this store implements provider.Enumerator and not provider.ChangeSource, and the M7b sweep is the whole inbound path rather than a safety net under a feed. -sweep-interval is therefore the poll interval, and its 24h default is wrong here (see docs/user/sftp.md).
- A write at an offset is the protocol's native operation, so provider. RangePutter is real — this is the first backend in the tree where M6's range-write path runs against a server rather than a fake.
- No server-computed digest is available, so provider.ContentHasher is NOT implemented and M6's unchanged-content gate declines. See the note above RemoteFile assembly in stat.
Index ¶
- Constants
- func Factory(ctx context.Context, p provider.Params) (provider.Store, error)
- type Config
- type Duration
- type Store
- func (s *Store) Close() error
- func (s *Store) Create(ctx context.Context, p string, r io.Reader) (provider.RemoteFile, error)
- func (s *Store) Enumerate(ctx context.Context, cursor string) ([]provider.RemoteFile, string, error)
- func (s *Store) Get(ctx context.Context, p string) (io.ReadCloser, error)
- func (s *Store) GetRange(ctx context.Context, p string, off, length int64) (io.ReadCloser, error)
- func (s *Store) Mkdir(ctx context.Context, p string) (provider.RemoteFile, error)
- func (s *Store) Move(ctx context.Context, oldPath, newPath string) (provider.RemoteFile, error)
- func (s *Store) Put(ctx context.Context, p string, r io.Reader) (provider.RemoteFile, error)
- func (s *Store) PutRange(ctx context.Context, p string, src io.ReaderAt, size int64, ...) (provider.RemoteFile, error)
- func (s *Store) Remove(ctx context.Context, p string) error
- func (s *Store) Stat(ctx context.Context, p string) (provider.RemoteFile, bool, error)
- func (s *Store) Upload(ctx context.Context, p string, r io.Reader, opts provider.UploadOptions) (provider.Uploaded, error)
Constants ¶
const Kind = "sftp"
Kind is the name this backend is registered and configured under.
Variables ¶
This section is empty.
Functions ¶
func Factory ¶
Factory opens an SFTP store from an undecoded config (M8). Register it under the name a config file should use:
reg := provider.NewRegistry() _ = reg.Register(sftp.Kind, sftp.Factory)
Registration is by hand rather than by init(), for the reasons the Registry documents: no process-global mutable state, and a dependency that is named where it is wired rather than hidden in an import for side effect.
Types ¶
type Config ¶
type Config struct {
// Host is the server, without a port.
Host string `toml:"host"`
// Port defaults to 22.
Port int `toml:"port"`
// User is the SSH account to log in as. Required: guessing it from the local
// username would make the same config file mean different things on different
// machines, and an fstab mount runs as whoever mounted it.
User string `toml:"user"`
// Key is a PEM private key file. It must not be passphrase-protected — see
// authMethods for why there is no setting to supply one.
Key string `toml:"key"`
// Certificate pairs a signed certificate with Key.
Certificate string `toml:"certificate"`
// Agent uses $SSH_AUTH_SOCK. It is a pointer so that an explicit `agent =
// false` is distinguishable from an absent key (M8 rule 6): unset means "use
// the agent if there is one", while false means "do not", and collapsing the
// two would make a deliberate opt-out silently do nothing.
Agent *bool `toml:"agent"`
// KnownHosts is the OpenSSH known_hosts file to verify the server against.
// Defaults to ~/.ssh/known_hosts. There is no way to disable verification.
KnownHosts string `toml:"known-hosts"`
// Root is the remote directory the mount root maps to. Empty means the login
// directory; a relative path resolves against it.
Root string `toml:"root"`
// Timeout bounds connecting, not transferring. Zero means defaultTimeout.
Timeout Duration `toml:"connect-timeout"`
// Concurrency is how many read/write packets may be in flight for a single
// file — what turns one transfer from latency-bound into bandwidth-bound over
// a long link. Zero takes the library's (conservative) default.
Concurrency int `toml:"concurrent-requests"`
}
Config is what an SFTP provider needs to open. The toml tags are the keys a `[account.NAME]` table in drivel's config file uses (M8) — spelled explicitly rather than left to the decoder's case-insensitive field matching, which would spell KnownHosts as "knownhosts".
type Duration ¶
Duration is a time.Duration that decodes from a TOML string ("30s"), because TOML has no duration type and a bare integer would silently mean nanoseconds.
func (*Duration) UnmarshalText ¶
UnmarshalText satisfies the TOML decoder.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is the SFTP provider.
It is safe for concurrent use: the SFTP client itself is (one session multiplexes many requests), and the mutex here guards only the cached connection pointer, never an operation.
func Open ¶
Open connects nothing and returns a Store that logs to lg (nil => the default logger). Everything checkable offline is checked here; the network is not touched until the first operation.
Not dialling at open is deliberate and follows §2.1. A mount must come up whether or not the server is reachable — an fstab mount races the network at boot, and drivel's whole premise is that filesystem operations do not block on it. What that would normally cost is a late, obscure failure for a misconfiguration; checkKnownHost buys most of it back by proving offline that this host could ever be trusted.
func (*Store) Create ¶
Create implements provider.Creator: Put, refusing an occupied path.
Where the session offers hardlink@openssh.com the refusal is atomic: the temporary is linked to the destination, and link(2) fails rather than replace what is there, so two clients creating one path cannot both win. Without the extension — or on a server filesystem that refuses hard links — it is a check and then a rename, which two clients can both pass. Either way an occupied path is reported before anything is uploaded, so the common refusal costs one round trip and no transfer.
func (*Store) Enumerate ¶
func (s *Store) Enumerate(ctx context.Context, cursor string) ([]provider.RemoteFile, string, error)
Enumerate lists the tree under the mount root (provider.Enumerator, M7b).
On this backend the sweep is not a periodic safety net beneath a change feed — it is the *entire* inbound path, because SFTP has no notification of any kind (see the ChangeSource note in sftp.go). Everything a remote edit does reaches the mount through here, so -sweep-interval is the poll interval and its 24h default is wrong for every SFTP mount.
Two rules about failure, and they pull in opposite directions on purpose.
A directory that has disappeared between being queued and being listed is skipped: it is genuinely gone, its children with it, and failing on it would abandon a sweep every time anything was deleted while one ran.
Every *other* listing failure abandons the sweep. That looks harsh for, say, one unreadable subdirectory, and it is the safe direction: a sweep that quietly omits a subtree still reports itself complete, and M7b's delete pass reads "synced before, not observed now" as a deletion. Skipping an unreadable directory would therefore propose deleting everything under it. Abandoning the pass instead is M7b's own rule — refuse a pass you cannot justify — applied one level down.
func (*Store) Get ¶
Get opens p for reading. The returned reader holds a remote file handle and must be closed.
func (*Store) GetRange ¶
GetRange reads a byte range of p (provider.RangeGetter, M5).
SSH_FXP_READ takes an offset, so this is one seek and a bounded read rather than a download with a discarded prefix — which the provider guide singles out as worse than not implementing the interface at all, because the hydrator would then choose the ranged path believing it cheap.
length <= 0 means "to end of object".
func (*Store) Move ¶
Move renames oldPath to newPath, creating the destination's parents.
A source the server does not have becomes provider.ErrNotExist, which the engine turns into "upload the destination as fresh content" — the right answer for a rename of a file that was never pushed.
func (*Store) Put ¶
Put creates or replaces the file at p, creating any missing parents.
The content lands on a temporary name in the destination directory and is renamed into place once it is complete. That costs one extra round trip and buys the property that matters most on this backend: the path never holds a half-written file. A whole-file Put is the fallback for every M6 gate, so it runs constantly, and an interrupted one writing in place would leave the remote truncated — with no server-side digest (see the ContentHasher note above), nothing downstream would ever notice. The engine records its echo from what this returns, so a failed Put records nothing and the next sweep re-pushes.
The temporary's name is UNIQUE PER CALL, and that is a correctness property rather than tidiness. It used to be a function of the destination path alone, so two Puts of one path shared one server-side file: they interleaved at their own offsets, and the loser of the race went on writing through its open handle into the file the winner had already renamed into place — a destination holding a mixture of two uploads, with an echo recorded saying it holds one of them. Two writers is not a rare shape. Within one mount the reconciler pushes through Engine.Push while the worker pool may be pushing the same path (MC-40 found this), and across mounts the server-side name is a shared namespace, so two clients pushing one path collide however carefully either one is ordered.
What it costs is self-cleaning. A stranded temporary used to be overwritten by the next Put of the same path and renamed away with it; a unique one stays until something removes it. Enumerate still skips the prefix, so it is never materialised locally and never inferred as a deletion (M18 rule 7) — it is invisible rather than harmful — but nothing reclaims it, and a periodic sweep of old temporaries is the open follow-up.
func (*Store) PutRange ¶
func (s *Store) PutRange(ctx context.Context, p string, src io.ReaderAt, size int64, extents []ranges.Range) (provider.RemoteFile, error)
PutRange replaces byte extents of p in place (provider.RangePutter, M6).
This is the milestone's headline. SSH_FXP_WRITE takes an offset, so the seam's contract — "replace exactly these extents, neither create nor resize" — maps onto the protocol directly, and M6's range-write path finally runs against a real server instead of a fake. Drive has no partial write at all, so before this backend every byte of gate 2 was exercised by test doubles.
The contract is re-checked here rather than trusted. The engine already verifies that the object exists at exactly size and that the remote still matches the §4 echo before calling — the second of those is what stops extents being spliced into a file someone else has edited, producing a hybrid that existed nowhere — but the seam says the implementation must re-check too, and the reason is sharp: a mismatch means our idea of the file's length is stale, and every offset computed from it lands in the wrong place. Failing is cheap. The engine logs it and re-does the write as a whole-file Put.
func (*Store) Remove ¶
Remove deletes p, recursively for a directory.
**There is no recoverable form on this backend and that is not a default that can be changed.** gdrive answers the seam's "prefer the recoverable form" by trashing, so a reconcile-inferred deletion the -max-deletes cap failed to catch is still recoverable for 30 days. SFTP has no trash, and inventing one — a hidden directory the deleted files are moved into — would sit inside the mount root, where the sweep would enumerate it and pull every deleted file back down. So on this provider -max-deletes is not the guard *in front of* a safety net; it is the only guard there is. docs/user/sftp.md says so in those words.
A path that is already gone succeeds. The engine can reach here twice for one removal (a delete observed by the mount and again inferred by a sweep), and a failure the second time would be a retry loop over an outcome already achieved.
func (*Store) Upload ¶
func (s *Store) Upload(ctx context.Context, p string, r io.Reader, opts provider.UploadOptions) (provider.Uploaded, error)
Upload implements provider.Uploader: Put and Create, carrying the local modification time and checking what is about to be replaced.
The time is set on the temporary, before it is renamed into place, so the object never exists under the path with any other time — the property the interface asks for, since a sweep listing it in between would read a version the engine has no record of. Setting it on a temporary has one cost, which is the stale-temporary reclaim in Enumerate: a file last modified more than a day ago lands in a temporary that looks abandoned for the round trip before its rename, and a sweep meeting it then removes it. The rename fails and the engine's retry uploads it again. SFTP's mtime is whole seconds.
Replaces is checked by a Stat of the destination immediately before the rename, which is the latest moment SFTP allows: there is no conditional rename. A write landing between that Stat and the rename is still replaced unseen — but that window is one round trip, where the engine's own check before the upload left the whole transfer open.