sftp

package
v0.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 2, 2026 License: MIT Imports: 21 Imported by: 0

README

sftp files provider

What it is

A real SFTP server — an SSH transport plus the SFTP subsystem — that accepts uploads from any client on its own port and keeps them in the shared virtual filesystem instead of anywhere on disk. Every upload, mkdir, rename and delete becomes a files.* event tagged with whatever credentials the client presented, so the Files tab shows the tree as it is now and the log shows how it got that way.

SFTP is an SSH subsystem, not FTP with TLS bolted on, so this provider is two layers: x/crypto/ssh runs the handshake and answers a subsystem request for sftp, and pkg/sftp's RequestServer serves the file operations through a files.Session. Nothing reaches the host filesystem — the one file this provider owns on disk is its SSH host key.

What it's for

Reach for this one when whatever is under test already carries an SSH client — a deployment script that scps a build artifact, a batch job that logs in with sftp and puts a file, a language library built on golang.org/x/crypto/ssh or libssh. It is the natural fit for anything that would otherwise need a real OpenSSH server: no daemon to configure, no sshd_config, no host system user to create, and no password to manage unless you want one pinned (see Authentication below).

The one thing worth knowing before reaching for it: by default this provider accepts any credentials, including none at all, via SSH's none authentication method. That is what makes sftp any@host work with no password prompt straight out of a cold start — but it also means credentials are only recorded (Event.Meta.auth) when a client actually offers them, which only happens automatically once you pin username or password and force the client to authenticate for real.

How to test it for real

Booted with TOMMY_NO_UPDATE_CHECK=1 go run . files --sftp-port 2222 (or tommy serve with the provider enabled), a real OpenSSH client drives the whole round trip with no password prompt:

echo 'it works' > ./local.txt
sftp -P 2222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR -b - any@localhost <<'EOF'
put ./local.txt /upload/local.txt
get /upload/local.txt ./downloaded.txt
ls -l /upload
EOF
diff ./local.txt ./downloaded.txt && echo "round-trip ok"

scp (OpenSSH 9 speaks SFTP under the hood) works the same way:

scp -P 2222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR ./local.txt any@localhost:/scp.txt

And the same files read back over HTTP:

curl -s http://localhost:8811/api/v1/files/tree
curl -s http://localhost:8811/api/v1/files/content/upload/local.txt

Verified against a live instance on ports 12222 (sftp) / 18921 (ui) so as not to collide with anything else running at the time: put/get and scp both round-tripped byte-for-byte, mkdir and login both went through with no password prompt (confirming the default NoClientAuth behaviour above), and the HTTP API served the same bytes back. The uploaded file showed up in the same tree from every provider — an SFTP put and an FTP STOR against this plugin land in one shared filesystem, not two.

Try it

echo 'it works' > ./local.txt
sftp -P 2222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR -b - any@localhost <<'EOF'
mkdir /upload
put ./local.txt /upload/local.txt
ls -l /upload
EOF
scp -P 2222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR ./local.txt any@localhost:/local.txt
curl -s http://localhost:8811/api/v1/files/tree
curl -s http://localhost:8811/api/v1/files/content/upload/local.txt

All three are Snippets() entries rendered against the live SnippetCtx, so tommy providers files/sftp prints them with the port this instance actually bound — the 2222 above is only the fallback for when nothing is running. The tests execute the first two with the real OpenSSH client.

No password is needed: the server accepts the SSH none authentication method, which is what keeps a copy-pasted command free of a prompt. Host-key checking is turned off in the snippets on purpose — the key is a throwaway identity for a local fake — but it is a stable throwaway, see below.

The host key persists

The one trap in this provider. An SSH client remembers the key of every host it has talked to. A server that generates a fresh one on each boot makes every second connection fail with REMOTE HOST IDENTIFICATION HAS CHANGED, and the person debugging it blames tommy rather than their known_hosts file.

So on first run an ed25519 key is generated and written to host_key_path with mode 0600 (creating the directory 0700 if needed, and using O_EXCL so two tommys racing on one path agree on a single key), and every later run reads it back. The fingerprint is logged at startup, generated or not:

INFO sftp listening addr=127.0.0.1:2222 host_key=/home/you/.config/tommy/sftp_host_ed25519
     host_key_generated=false fingerprint=SHA256:0Yq…

A key file that exists but cannot be parsed is a startup error naming the file, never a silent regeneration: replacing a key the clients already trust is exactly the outcome that must not happen by accident.

TestHostKeyIsStableAcrossRestarts boots the provider twice against one path and asserts the client sees the identical key both times.

Configuration

[plugins.files.providers.sftp]
enabled       = true
port          = 2222        # 0 binds an ephemeral port
bind          = "127.0.0.1"
host_key_path = "~/.config/tommy/sftp_host_ed25519"   # generated on first run

# username = "app"          # optional: pin the credentials, making auth required
# password = "s3cret"
# authorized_keys = "~/.ssh/authorized_keys"          # optional public-key allowlist

server_version    = "SSH-2.0-tommy"
handshake_timeout = 30      # seconds from accept to authenticated
idle_timeout      = 600     # seconds with no traffic; a live transfer refreshes it
max_connections   = 64
max_auth_tries    = 6

An absent port means 2222; port = 0 means "bind an ephemeral port", which is what the test harness uses. host_key_path defaults to <user config dir>/tommy/sftp_host_ed25519 and understands a leading ~.

Authentication

Anyone gets in by default — with a password, with a key, or with nothing at all — and what they presented lands in Event.Meta.auth, recorded rather than judged (provider rule 1). A client that offers nothing is authenticated with the none method, which is what a fake wants: an application pointed at tommy for the first time should not fail on credentials it has not been given yet.

The consequence worth knowing: a client only sends credentials when the server asks for them, and by default this one does not ask. Pin username or password and it does — authentication becomes mandatory and checked, and the presented password is recorded next to the upload, which is how you exercise your application's error path and see what it actually sent.

Set authorized_keys and public-key authentication is checked against that file instead; a password will not walk past the allowlist unless credentials are pinned as well.

How the handlers map onto the VFS

pkg/sftp SFTP methods files.Session
Fileread Get Open → *files.File, an io.ReaderAt streaming out of the blob store
Filewrite / OpenFile Put, Open OpenFile → an io.WriterAt; committed atomically on close
Filecmd Mkdir Rmdir Remove Rename Setstat Mkdir, Remove, Rename, Chtimes/Truncate
Filelist List Stat Lstat List, Stat → Node.FileInfo()

No path is interpreted here. Every path goes to the session, which resolves it through VFS.Resolve — the single security gate — and there is no host filesystem underneath to escape to. ../../../etc/passwd is clamped at the root the way a chroot does it and lands inside the tree; a NUL byte or a control character is refused outright.

Writes buffer and commit in one step, so a listing never sees a half-written file, and a transfer that dies mid-flight is aborted rather than committed — neither a truncated file nor an event. The commit runs with a context that outlives the request, because pkg/sftp cancels a request's context as it closes the handle, which is exactly when the upload lands.

Deliberately answered with SSH_FX_OP_UNSUPPORTED:

  • Symlink, Link, Readlink — the VFS does not model links, and inventing one would be worse than saying so. Lstat still works: with no symlinks it is the same thing as Stat.

Deliberately accepted and dropped:

  • chmod and chown — there is no mode and no uid in the VFS, and failing here would break every client that preserves a mode after uploading.

Setstat never appends an event: a timestamp or mode fixup is metadata, not a change to what the filesystem holds, and one event per setstat would drown the uploads in the log. A size is applied (the VFS can represent it), and an fsetstat that arrives before an upload is closed — which is how sftp put -p preserves timestamps — is applied to the open handle instead of failing on a file that does not exist yet.

Shell and exec are refused, cleanly

This is an SFTP endpoint, not a shell. A session channel is accepted and only a subsystem request naming sftp is honored; exec, shell and any other subsystem get an explanation on stderr, a refusal, and an exit status, so a client prints an error instead of hanging. A non-session channel is rejected with UnknownChannelType. None of it takes the connection or the listener down, which TestShellAndExecAreRefusedCleanly asserts by running SFTP over the same connection afterwards.

What lands where

the file the shared files.VFS, bytes in core/blob
Event.Type files.upload, files.mkdir, files.rename, files.delete
Event.Provider sftp
Event.Raw.Transport ssh, with the peer address and the operation in the body
Event.Meta.auth method, user, password or key fingerprint, and accepted
Event.Meta.client_version the client's SSH identification string

Failing safely

Every bound is the VFS's — path depth, name and path length, entries per directory, total nodes and file size — plus this provider's own: max_connections (a connection over the limit is closed immediately rather than queued), handshake_timeout (a client that connects and says nothing cannot hold a slot), idle_timeout (refreshed by every read and write, so a slow upload is never cut off) and max_auth_tries.

Files

  • provider.go — the plugin.ListenerProvider: configuration, snippets, the listener lifecycle.
  • ssh.go — the SSH transport: authentication callbacks, channels, the subsystem dispatch, the idle-deadline connection.
  • handlers.go — the four pkg/sftp handlers over a files.Session.
  • hostkey.go — generating, persisting and loading the host key.

Automated tests

go test ./plugins/files/providers/sftp/...
go test -race ./plugins/files/providers/sftp/...

The tests drive a real pkg/sftp client over a real socket against a listener booted through core/testutil on an ephemeral port: byte-exact upload and download, nested mkdir, listings, Rename, Remove, Stat, a tree walk, append and O_EXCL opens, the hostile-path table, default and pinned credentials, the authorized_keys allowlist, refused exec/shell/subsystem requests, concurrent sessions under -race, host-key persistence across a restart, and the two OpenSSH snippets executed as written.

Documentation

Overview

Package sftp is tommy's SFTP listener: a files provider that owns a real SSH port and writes everything uploaded to it into the plugin's shared virtual filesystem.

SFTP is an SSH subsystem rather than FTP with TLS bolted on, so this provider runs two layers: an x/crypto/ssh transport that performs the handshake, records whatever credentials were presented and honors a request for the "sftp" subsystem, and a pkg/sftp RequestServer whose four handlers are backed by a files.Session. Nothing is ever written to the host filesystem - the one file this provider owns on disk is its SSH host key, which must survive a restart or every client fails with a changed-host-key error.

Index

Constants

View Source
const (
	DefaultBind = "127.0.0.1"
	// DefaultHostKeyName is the file name under the user config directory.
	DefaultHostKeyName = "sftp_host_ed25519"
	// DefaultServerVersion is the identification string sent before the
	// handshake. It must start with "SSH-2.0-" to be a legal SSH banner.
	DefaultServerVersion = "SSH-2.0-tommy"
	// DefaultHandshakeTimeout bounds the time between accepting a connection
	// and completing authentication, so a client that connects and says
	// nothing cannot hold a slot forever.
	DefaultHandshakeTimeout = 30 * time.Second
	// DefaultIdleTimeout drops a connection that has neither read nor written
	// for this long. Uploads refresh it as they go, so a slow transfer is
	// never cut off.
	DefaultIdleTimeout = 10 * time.Minute
	// DefaultMaxConnections caps concurrent SSH connections.
	DefaultMaxConnections = 64
	// DefaultMaxAuthTries caps authentication attempts per connection.
	DefaultMaxAuthTries = 6
)

Defaults for the rest of the listener.

View Source
const DefaultPort = 2222

DefaultPort is the port the listener binds when the configuration says nothing. 2222 is the unprivileged SSH convention, and it stays clear of a real sshd on 22.

View Source
const ProviderName = "sftp"

ProviderName is the provider's name, which is also how it is addressed in configuration: [plugins.files.providers.sftp].

Variables

This section is empty.

Functions

func DefaultHostKeyPath

func DefaultHostKeyPath() string

DefaultHostKeyPath is where the key lives when the configuration says nothing: alongside whatever else tommy keeps in the user's config directory. It falls back to the temporary directory only when there is no config directory at all (no HOME), because a key that cannot be written anywhere is a key that changes on every restart - the exact failure this file exists to prevent.

func ExpandPath

func ExpandPath(p string) string

ExpandPath resolves a leading ~ against the user's home directory, which is how the path is written in tommy.toml. Anything else is returned untouched, including a relative path: the process working directory is a legitimate place to keep a throwaway key.

Types

type Config

type Config struct {
	Bind string
	Port int

	// HostKeyPath is where the ed25519 host key is kept. It is generated on
	// first use and read back on every later run: an SSH server that changes
	// its identity between restarts breaks every client that remembered it.
	HostKeyPath string

	// AuthorizedKeysPath, when set, is an OpenSSH authorized_keys file that
	// public-key authentication is checked against. Unset means any key is
	// accepted, and only recorded.
	AuthorizedKeysPath string

	// Username and Password pin the credentials the listener accepts. They are
	// empty by default, which means anyone gets in - with a password, with a
	// key, or with no credentials at all - and what they presented is recorded.
	// Set either one and a password becomes mandatory and is checked, which is
	// how you exercise your application's error path.
	Username string
	Password string

	ServerVersion    string
	HandshakeTimeout time.Duration
	IdleTimeout      time.Duration
	MaxConnections   int
	MaxAuthTries     int
}

Config is the [plugins.files.providers.sftp] section.

Every key is optional. port = 0 means "bind an ephemeral port", which is what the test harness uses; an absent port means DefaultPort.

func LoadConfig

func LoadConfig(pc plugin.ProviderConfig) Config

LoadConfig reads the provider section, applying every default.

func (Config) ListenAddr

func (c Config) ListenAddr() string

ListenAddr is the address Listen binds.

func (Config) RequiresAuth

func (c Config) RequiresAuth() bool

RequiresAuth reports whether the configuration pins credentials. When it does not, anything a client offers is accepted and recorded but never demanded.

type HostKey

type HostKey struct {
	Signer ssh.Signer
	// Path is the file the key lives in.
	Path string
	// Created is true when this run generated the key rather than loading it.
	Created bool
}

HostKey is a loaded SSH host key together with where it came from.

The whole point of this type is the Created flag and the Fingerprint: an SSH client remembers the key of every host it has ever talked to, so a server that generates a fresh one on each boot makes every second connection fail with the alarming REMOTE HOST IDENTIFICATION HAS CHANGED banner - and the person debugging it blames tommy, not their known_hosts file. The key is therefore written to disk on first use and read back forever after, and the fingerprint is logged at startup so it can be compared against what the client reports.

func LoadOrCreateHostKey

func LoadOrCreateHostKey(path string) (HostKey, error)

LoadOrCreateHostKey returns the host key at path, generating and persisting a new ed25519 key the first time.

The file is created with 0600 through O_EXCL, so two tommys racing on the same path end up agreeing on one key rather than overwriting each other, and the key is never left world-readable. A missing parent directory is created with 0700.

A file that exists but cannot be parsed is an error naming the path rather than a silent regeneration: replacing a key the clients already trust is exactly the outcome that must never happen by accident.

func (HostKey) AuthorizedKey

func (k HostKey) AuthorizedKey() string

AuthorizedKey is the key's public half in authorized_keys form, which is what goes into a known_hosts entry.

func (HostKey) Fingerprint

func (k HostKey) Fingerprint() string

Fingerprint is the SHA256 fingerprint OpenSSH prints, in the same form, so a line from the log can be compared with what the client showed.

type Provider

type Provider struct {
	// contains filtered or unexported fields
}

Provider is the SFTP listener.

func New

func New() *Provider

New returns the SFTP provider. Everything it needs - the port, the host key path, the shared filesystem - arrives later, so one value is safe to construct before anything is running.

func (*Provider) Addr

func (p *Provider) Addr(timeout time.Duration) (string, error)

Addr blocks until the listener is bound and returns its address, which is how a test finds the ephemeral port it asked for. The channel is created in New, so Addr and Listen never race.

func (*Provider) BindVFS

func (p *Provider) BindVFS(v *files.VFS)

BindVFS implements files.VFSBinder: the plugin hands every provider the one tree they all write into, so a file uploaded over SFTP is listed over FTP and downloadable from the Files tab.

func (*Provider) Description

func (p *Provider) Description() string

Description implements plugin.Provider.

func (*Provider) Endpoints

func (p *Provider) Endpoints() []plugin.Endpoint

Endpoints implements plugin.Provider. The provider speaks SSH on its own port and mounts no HTTP routes.

func (*Provider) HostKey

func (p *Provider) HostKey() HostKey

HostKey reports the key the running listener presents, once it has bound. It is empty before that; Addr is the way to wait.

func (*Provider) Listen

func (p *Provider) Listen(ctx context.Context, d plugin.Deps) error

Listen implements plugin.ListenerProvider. It loads or generates the host key, binds the port from d.Config, serves SSH until ctx is done, and returns nil on a clean shutdown.

func (*Provider) ListenPort

func (p *Provider) ListenPort(pc plugin.ProviderConfig) plugin.ListenPort

ListenPort implements plugin.PortProvider: where this listener would bind under pc, resolved without binding anything. It is the same value Listen resolves, so `tommy providers` and the running listener cannot disagree.

func (*Provider) Name

func (p *Provider) Name() string

Name implements plugin.Provider.

func (*Provider) Plugin

func (p *Provider) Plugin() string

Plugin implements plugin.Provider.

func (*Provider) RegisterIngress

func (p *Provider) RegisterIngress(mux plugin.Mux, d plugin.Deps)

RegisterIngress implements plugin.Provider. Nothing is mounted: this provider speaks SFTP on a port of its own.

func (*Provider) Snippets

func (p *Provider) Snippets() []plugin.Snippet

Snippets implements plugin.Provider.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址