rfs

package
v0.7.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 7, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package rfs provides a unified filesystem abstraction for runq. LocalFS wraps os.* for same-machine operation; SSHFS wraps sftp + ssh.Session for remote clusters. The HPC backend receives an FS at construction and uses it for all file I/O and command execution, making local vs. remote transparent.

Design decisions:

  • No cache layer: reads go straight through. Incremental-read state (metric offsets, log offsets) is maintained in-memory by the caller.
  • Exec runs a command on the target machine. For SSHFS this means an SSH session; for LocalFS it means os/exec. The caller is responsible for quoting and escaping.
  • CopyToLocal transfers a file from the FS to the local machine. For LocalFS this is a plain file copy; for SSHFS it is an sftp download.

Index

Constants

View Source
const (
	ForwardUp           = "up"           // session established, socket bound
	ForwardReconnecting = "reconnecting" // between sessions (includes pre-first-dial)
	ForwardClosed       = "closed"       // Close() called — no further attempts
)

Forward state labels (ForwardStatus.State).

View Source
const (
	// HostKeyStrict is the default: unknown or changed keys refuse the
	// connection (typed errors naming the fix).
	HostKeyStrict = ""
	// HostKeyAcceptNew auto-trusts and records UNKNOWN hosts (no prompt);
	// a CHANGED key still hard-fails.
	HostKeyAcceptNew = "accept-new"
)

Host key policy values consumed by sshConn/RemoteForward.

Variables

View Source
var ErrExecInterrupted = errors.New("remote command interrupted: outcome unknown")

ErrExecInterrupted marks an Exec whose command STARTED on the remote side but whose verdict was lost (connection dropped mid-flight, session torn down). This is neither a clean transport failure (the command may have run to completion) nor a scheduler verdict (there is no exit code). Callers with side-effecting commands (job submission) must treat it as "outcome unknown" — never as retry-safe (RQ-74).

Functions

func AcceptNewHostKeyCallback

func AcceptNewHostKeyCallback() (ssh.HostKeyCallback, error)

AcceptNewHostKeyCallback is the ssh_config `StrictHostKeyChecking accept-new` passthrough (RQ-74): an UNKNOWN host is trusted and recorded without a prompt — exactly what the user's own ssh would do under that setting, so runq is never more ceremonious than ssh. A MISMATCH still hard-fails: OpenSSH's `no` would tolerate even that, but silently accepting a CHANGED key disables the MITM protection entirely — runq caps the passthrough at accept-new semantics on purpose.

func AppendKnownHost

func AppendKnownHost(hostname string, key ssh.PublicKey) error

AppendKnownHost appends one accepted host key to ~/.ssh/known_hosts in the standard format.

func EffectiveSSH added in v0.7.0

func EffectiveSSH(host string, port int, user, jump string) (effUser, effHost string, effPort int, ok bool)

EffectiveSSH asks OpenSSH (ssh -G) which user, host name and port a connection resolves to after ~/.ssh/config: an alias and its HostName, or an omitted user and its default, are the same endpoint. ok is false when ssh cannot answer (the caller keeps the configured values).

func ResolveAuthMethods

func ResolveAuthMethods(keyPath string, agentSocket ...string) ([]ssh.AuthMethod, error)

ResolveAuthMethods builds the auth chain (RQ-45): the configured key file and the ssh-agent's identities, offered through ONE publickey method — golang.org/x/crypto/ssh tries each method name once, so a second PublicKeys method after an agent that holds no matching key was never attempted ("attempted methods [none publickey]") even though plain ssh logged in with the same file. Signing via the agent stays in the agent; an encrypted key file without an agent is a clear, actionable error, not a parse failure.

func StrictHostKeyCallback

func StrictHostKeyCallback() (ssh.HostKeyCallback, error)

StrictHostKeyCallback verifies against ~/.ssh/known_hosts and maps the library's KeyError into the two typed errors above.

func TOFUHostKeyCallback

func TOFUHostKeyCallback(confirm func(host, fingerprint string) bool) (ssh.HostKeyCallback, error)

TOFUHostKeyCallback wraps strict verification with a consent prompt for the UNKNOWN case only (mismatch is never promptable — that one is a security event, not an onboarding step). On consent the key is appended to known_hosts, making all future strict connections succeed.

func VerifySSHInteractive added in v0.7.0

func VerifySSHInteractive(ctx context.Context, cfg SSHConfig) error

VerifySSHInteractive delegates first-use trust and authentication to the user's terminal. It never stores credentials or bypasses host verification.

Types

type ContactObserved added in v0.7.2

type ContactObserved interface {
	SetContactObserver(observe func(generation uint64, err error))
}

ContactObserved is a filesystem that reports its own transport outcomes.

generation names the connection an outcome speaks for and grows with each new connection. Callbacks run concurrently and outside the manager's locks, so one can arrive after a newer generation's: an observer keeps the newest generation it has applied and discards anything older.

type CopyOption

type CopyOption func(*copyOpts)

CopyOption configures CopyToLocal behavior.

type FS

type FS interface {

	// Stat returns file metadata, like os.Stat.
	Stat(path string) (fs.FileInfo, error)

	// ReadFile reads the named file in full, like os.ReadFile.
	ReadFile(path string) ([]byte, error)

	// Open opens a seekable file handle for incremental reads.
	Open(path string) (File, error)

	// ReadDir reads a directory, like os.ReadDir.
	ReadDir(path string) ([]fs.DirEntry, error)

	// WriteFile writes data to the named file, like os.WriteFile.
	WriteFile(path string, data []byte, perm os.FileMode) error

	// MkdirAll creates a directory tree, like os.MkdirAll.
	MkdirAll(path string, perm os.FileMode) error

	// Exec runs a command on the target machine, returning stdout, stderr,
	// the exit code, and any transport-level error. A non-zero exit code is
	// NOT returned as an error — the caller decides how to interpret it.
	Exec(ctx context.Context, cmd string, args ...string) (stdout, stderr []byte, exitCode int, err error)

	// ExecStream runs a command and returns a stream of its combined output.
	// Used for `runq logs -f` style live tailing over SSH.
	ExecStream(ctx context.Context, cmd string, args ...string) (io.ReadCloser, error)

	// CopyToLocal copies a file from this FS to a local path. For LocalFS
	// this is a file copy; for SSHFS it is an sftp download. The target
	// directory must already exist.
	CopyToLocal(remotePath, localPath string, opts ...CopyOption) error
}

FS is the filesystem abstraction injected into the HPC backend. It covers file reads/writes, directory operations, command execution, and file transfer. Implementations: LocalFS (same machine) and SSHFS (remote).

func WithContext added in v0.6.0

func WithContext(fsys FS, ctx context.Context) FS

type File

type File interface {
	io.Reader
	io.Seeker
	io.Closer
	Stat() (fs.FileInfo, error)
}

File is a seekable, closeable file handle returned by FS.Open.

type ForwardStatus

type ForwardStatus struct {
	SocketPath string `json:"socket_path,omitempty"`
	Owner      string `json:"owner,omitempty"`
	State      string `json:"state"` // up | reconnecting | closed
	Since      int64  `json:"since"` // when the current state was entered
	// LastOnline is the last moment the forward was known to be serving:
	// set on establish and refreshed after successful keepalive. 0 = never
	// online ("no contact yet" ≠ "went down" — doctor renders them apart).
	LastOnline int64 `json:"last_online,omitempty"`
	// Attempts counts consecutive failed sessions since the last up.
	Attempts      int    `json:"attempts,omitempty"`
	LastError     string `json:"last_error,omitempty"`
	Phase         string `json:"phase,omitempty"`
	Reason        string `json:"reason,omitempty"`
	Hint          string `json:"hint,omitempty"`
	NextRetryAt   int64  `json:"next_retry_at,omitempty"`
	LastAttemptAt int64  `json:"last_attempt_at,omitempty"`
	Transport     string `json:"transport,omitempty"`
	DebugCommand  string `json:"debug_command,omitempty"`
	SSHAuthSock   string `json:"ssh_auth_sock,omitempty"`
}

ForwardStatus is one forward's observable state (RQ-74): what /health exposes and doctor renders ("↻ reconnecting · attempt 3 · last online 12m ago"). All timestamps unix seconds.

type HostKeyMismatchError

type HostKeyMismatchError struct {
	Host        string
	Fingerprint string
}

HostKeyMismatchError: the host's key CHANGED. Deliberately alarming — this is either a server reinstall, a login-node pool member with its own key, or an interception. Never promptable and never auto-accepted (even under accept-new): the way out is a copy-paste command, not a click (RQ-74 — the error carries its own fix).

func (*HostKeyMismatchError) Error

func (e *HostKeyMismatchError) Error() string

type LocalFS

type LocalFS struct{}

LocalFS implements FS using the local filesystem and os/exec. It is the default FS for same-machine (login-node) HPC operation.

func NewLocalFS

func NewLocalFS() *LocalFS

NewLocalFS returns a LocalFS instance.

func (*LocalFS) CopyToLocal

func (l *LocalFS) CopyToLocal(remotePath, localPath string, opts ...CopyOption) error

func (*LocalFS) Exec

func (l *LocalFS) Exec(ctx context.Context, cmd string, args ...string) (stdout, stderr []byte, exitCode int, err error)

func (*LocalFS) ExecStream

func (l *LocalFS) ExecStream(ctx context.Context, cmd string, args ...string) (io.ReadCloser, error)

func (*LocalFS) MkdirAll

func (l *LocalFS) MkdirAll(path string, perm os.FileMode) error

func (*LocalFS) Open

func (l *LocalFS) Open(path string) (File, error)

func (*LocalFS) ReadDir

func (l *LocalFS) ReadDir(path string) ([]fs.DirEntry, error)

func (*LocalFS) ReadFile

func (l *LocalFS) ReadFile(path string) ([]byte, error)

func (*LocalFS) Remove added in v0.6.0

func (l *LocalFS) Remove(path string) error

Remove is an optional FS extension used by project sync to delete stale allow-listed files without exposing a broad recursive-delete primitive.

func (*LocalFS) Rename

func (l *LocalFS) Rename(oldPath, newPath string) error

Rename is the optional atomic-replace extension (asserted by callers that write tmp-then-rename, e.g. the pyramid builder).

func (*LocalFS) Stat

func (l *LocalFS) Stat(path string) (fs.FileInfo, error)

func (*LocalFS) WriteFile

func (l *LocalFS) WriteFile(path string, data []byte, perm os.FileMode) error

type OpenSSHConfig added in v0.7.0

type OpenSSHConfig struct {
	Binary string
	Args   []string
	Host   string
}

type RemoteForward

type RemoteForward struct {
	// contains filtered or unexported fields
}

RemoteForward supervises one remote socket forward. Create with NewRemoteForward, drive with Run (blocking; run it in a goroutine), stop with Close.

func NewRemoteForward

func NewRemoteForward(cfg RemoteForwardConfig) *RemoteForward

NewRemoteForward creates a RemoteForward. No network traffic until Run.

func (*RemoteForward) Close

func (f *RemoteForward) Close() error

Close stops the supervisor and tears down the current connection (which unblocks Serve). Safe to call multiple times.

func (*RemoteForward) Run

func (f *RemoteForward) Run(parent context.Context)

Run supervises the forward until ctx is cancelled or Close is called: dial → bind remote socket → Serve, then backoff and repeat. Failures are logged, never fatal — the HPC lane works with or without the forward.

func (*RemoteForward) Status

func (f *RemoteForward) Status() ForwardStatus

Status returns a snapshot of the forward's observable state.

type RemoteForwardConfig

type RemoteForwardConfig struct {
	// SSH is the connection config. IdleTimeout is ignored — a forward
	// listener is persistent by definition (see package comment above).
	SSH SSHConfig

	// SocketPath is the remote unix socket path. Relative paths resolve
	// against the remote user's home (sftp session start dir).
	// Default: ".runq/runq.sock".
	SocketPath string
	Owner      string

	// Serve consumes the remote listener and blocks until the listener
	// dies (the daemon passes http.Serve with the dashboard mux). It is
	// called once per established forward; a non-nil return is logged and
	// triggers a reconnect cycle.
	Serve func(net.Listener) error

	// OnEstablished, when set, is called once per established forward
	// session, right before Serve — the moment the remote socket is bound
	// and listening. The daemon uses it to record target reachability
	// (RQ-74): a forward coming up is transport-level proof of contact, so
	// doctor's "no contact yet" clears without waiting for the next
	// scheduled sensor pass. Must be fast and non-blocking.
	OnEstablished func()

	Logger *slog.Logger
}

RemoteForwardConfig configures a RemoteForward.

type SSHConfig

type SSHConfig struct {
	// OpenSSH is set for user targets. Native SSH remains for injected transports.
	OpenSSH *OpenSSHConfig
	Host    string // original OpenSSH alias; native transports use host[:port]
	User    string
	// AuthMethods are tried in order. Convention (RQ-45): ssh-agent first
	// (the daemon never touches key plaintext), key file second. Build
	// with ResolveAuthMethods.
	AuthMethods []ssh.AuthMethod
	// HostKeyCallback: nil = known_hosts verification per HostKeyPolicy
	// (the daemon default). `runq connect` passes a TOFU callback with a
	// human on the other end. InsecureIgnoreHostKey never appears in this
	// codebase.
	HostKeyCallback ssh.HostKeyCallback

	// HostKeyPolicy applies when HostKeyCallback is nil: HostKeyStrict
	// (zero value) or HostKeyAcceptNew — the ssh_config
	// `StrictHostKeyChecking accept-new` passthrough (RQ-74). Resolve with
	// ResolveHostKeyPolicy against the target's ssh alias.
	HostKeyPolicy string

	// IdleTimeout controls how long a connection survives with no in-flight
	// operations before sshConn tears it down. Zero means never idle-close.
	// Callers that want the "look like a normal SSH user" behavior MUST set
	// this (the daemon does; see backend.NewSSHBackend).
	IdleTimeout time.Duration

	// MaxSessions caps the number of concurrent SSH sessions (Exec calls).
	// sftp runs on its own subsystem and does NOT consume a session slot.
	// Default: 3.
	MaxSessions int

	// OperationTimeout bounds connection setup and each file operation.
	OperationTimeout time.Duration
}

SSHConfig holds the parameters needed to establish an SSH connection. Populated from the target config (see internal/config.TargetConfig).

func ResolveTargetSSH added in v0.6.0

func ResolveTargetSSH(host string, port int, user, key, jump string) (SSHConfig, error)

ResolveTargetSSH retains the alias and explicit overrides. OpenSSH resolves configuration and authentication on every reconnect; resolving the alias in advance would lose Host/Match, ProxyJump and host-key store semantics.

func (SSHConfig) DebugCommand added in v0.7.0

func (c SSHConfig) DebugCommand() string

DebugCommand reproduces fresh noninteractive authentication using the alias and overrides, bypassing existing control masters. No credentials are printed.

type SSHFS

type SSHFS struct {
	// contains filtered or unexported fields
}

SSHFS implements FS over an SSH connection. File operations go through sftp; command execution opens SSH sessions. The underlying *sshConn is lazy — no network traffic until the first call — and self-healing: dead connections are detected and redialed transparently (see sshConn.getConn).

Thread safety: safe for concurrent use. sshConn serializes dial/reconnect; sftp.Client is documented as safe for concurrent use; SSH sessions are bounded by the semaphore in sshConn.

Every operation is bracketed by beginOp/endOpErr so the idle watcher never tears down a connection with work in flight; for handle-returning calls (Open, ExecStream) the bracket extends until the handle is closed.

func NewSSHFS

func NewSSHFS(cfg SSHConfig) *SSHFS

NewSSHFS creates an SSHFS. The connection is lazy — no dial happens here.

func (*SSHFS) Close

func (s *SSHFS) Close() error

Close tears down the SSH connection. The SSHFS should not be used after Close.

func (*SSHFS) CopyToLocal

func (s *SSHFS) CopyToLocal(remotePath, localPath string, opts ...CopyOption) error

CopyToLocal downloads a remote file via sftp. The busy-marker covers the whole transfer — large checkpoint downloads are never cut by idle teardown.

func (*SSHFS) DialUnix added in v0.6.0

func (s *SSHFS) DialUnix(ctx context.Context, path string) (net.Conn, error)

DialUnix opens a remote socket through SSH. Used to prove that a reverse forward actually reaches the expected controller/target before connect succeeds.

func (*SSHFS) Exec

func (s *SSHFS) Exec(parent context.Context, cmd string, args ...string) (stdout, stderr []byte, exitCode int, err error)

Exec runs a remote command via an SSH session and returns stdout, stderr, and the exit code. A non-zero exit is NOT a Go error; transport failures are. ctx ending closes this session only (the remote command gets EOF/HUP); other work on the connection is not disturbed.

func (*SSHFS) ExecStream

func (s *SSHFS) ExecStream(ctx context.Context, cmd string, args ...string) (io.ReadCloser, error)

ExecStream runs a remote command and returns a streaming reader of its combined output. The session, semaphore slot, and busy-marker are released when the command exits (or ctx is cancelled); the pipe's read side reports the command's exit status via CloseWithError.

func (*SSHFS) Home

func (s *SSHFS) Home() (string, error)

func (*SSHFS) MkdirAll

func (s *SSHFS) MkdirAll(path string, perm os.FileMode) error

func (*SSHFS) Open

func (s *SSHFS) Open(path string) (File, error)

func (*SSHFS) ReadDir

func (s *SSHFS) ReadDir(path string) ([]fs.DirEntry, error)

func (*SSHFS) ReadFile

func (s *SSHFS) ReadFile(path string) ([]byte, error)

func (*SSHFS) Remove added in v0.6.0

func (s *SSHFS) Remove(path string) error

func (*SSHFS) RemoveContext added in v0.6.0

func (s *SSHFS) RemoveContext(ctx context.Context, path string) error

RemoveContext allows bounded cleanup with a context independent of a failed transfer while retaining the manager's connection/setup time bounds.

func (*SSHFS) Rename

func (s *SSHFS) Rename(oldPath, newPath string) error

Only atomic replacement or a server's normal rename is permitted. A failed replacement must never delete the existing executable/configuration.

func (*SSHFS) SetContactObserver added in v0.7.0

func (s *SSHFS) SetContactObserver(observe func(generation uint64, err error))

SetContactObserver reports transport outcomes, including file browsing. Context-scoped views share the observer. The callback runs outside conn.mu.

func (*SSHFS) Stat

func (s *SSHFS) Stat(path string) (fs.FileInfo, error)

func (*SSHFS) WithContext added in v0.6.0

func (s *SSHFS) WithContext(ctx context.Context) *SSHFS

WithContext scopes file operations without changing the shared connection.

func (*SSHFS) WriteFile

func (s *SSHFS) WriteFile(path string, data []byte, perm os.FileMode) error

func (*SSHFS) WriteFileContext added in v0.6.0

func (s *SSHFS) WriteFileContext(parent context.Context, path string, data []byte, perm os.FileMode) error

WriteFileContext uses an explicit transfer budget. Connection setup remains independently bounded; a large binary is not limited by a small-file timeout.

type UnknownHostKeyError

type UnknownHostKeyError struct {
	Host        string
	Fingerprint string // SHA256:...
}

UnknownHostKeyError: the host presented a key we have never seen.

func (*UnknownHostKeyError) Error

func (e *UnknownHostKeyError) Error() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址