Skip to content

[RFC] age: define agent protocol capability negotiation #3624

Closed

Description

@yzx9

TL;DR: The age agent currently does not support capability negotiation. We have proposed four options, as detailed in the Proposed options section table. Option 2 is recommended

Summary

In #3508 we agreed to implement Option 1 (raised scanner limit, done in #3529) first, and that Option 2 (separating the command header from the payload, e.g. chunked or length-prefixed transfers) requires protocol versioning to be defined first:

Then for Option 2 we will need to introduce versioning for the Agent protocol (needs to be defined) and only when we have that we can safely implement Option 2.

This issue proposes such a mechanism and lists the alternatives considered, so that the design can be agreed on before implementation.

Problem

The age agent protocol currently has no versioning, no capability discovery, and no written spec. The agent is a long-lived daemon (gopass age agent start), so a client binary will routinely talk to an agent started by an older gopass release. The version skew can be arbitrary in size and age.

The protocol has already changed shape without negotiation in the past:

There is also evidence of drift: Client.Remove() sends a remove command that the agent side has never implemented (dead code since #3218). Nothing constrains the implemented command set to anything declared or documented.

Current protocol surface

Line-oriented text protocol over a unix socket; one command line in, one response line out (OK, OK <payload> or ERR <message>). The client opens a new connection per command (internal/backend/crypto/age/agent/client.go).

Command Purpose
ping liveness probe
status report locked state
identities replace in-memory identities (single line)
decrypt decrypt base64 ciphertext
lock/unlock clear+lock / clear lock flag
set-timeout inactivity auto-lock timeout
quit shut the agent down

Dispatch: internal/backend/crypto/age/agent/agent.go. Unknown commands currently get a single-line ERR unknown command and the connection stays open.

Requirements

Any mechanism must:

  1. Keep old clients working unchanged (bare commands must keep working forever).
  2. Work with zero cooperation from agents already running (they cannot be updated).
  3. Not depend on undocumented error strings for correctness.
  4. Allow forks/third-party implementations to extend the protocol without coordinating version numbers with us.

Proposed options

At a glance:

Criterion hello + protocol version hello + capability set hello + capability set + per-API versions probe by trying
R1: old clients keep working ✅ ✅ ✅ ✅
R2: works with deployed agents ✅ ✅ any ERR to hello ⇒ legacy ✅ versioned forms only sent after hello confirms them ✅
R3: no undocumented error semantics ✅ ✅ ✅ ❌
R4: third-party extension ❌ ✅ namespaced tokens partial — version numbers of existing commands are a shared resource ❌
Expensive 2️⃣ linear order, no parameters 3️⃣ tokens with parameters (maxline=…), can express conditional support 4️⃣ finest granularity; ordering helps only if extensions form a chain 1️⃣ boolean answers only
Maintenance burden 3️⃣ version→feature table in every client + manual bump discipline 2️⃣ immutable-naming discipline (common to all options) 4️⃣ per-command version history 1️⃣ N scattered try/fallback paths

Option 1: hello + protocol version number

> hello gopass/1.19.0
< OK maxline=16777216 version=1.19.0

A single global version answers the wrong question: clients need "can I do X", not "what are you", and translating versions to features requires a client-side mapping table that grows forever and breaks on distro backports and forks. It also assumes features accumulate linearly and gives third parties no way to extend the protocol. (Note that IMAP, ESMTP, TLS and the SSH agent protocol all evolved away from bare version numbers towards capability tokens.)

Option 2 (recommended): hello + capability set

A single stateless, voluntary exchange at the start of a connection:

> hello gopass/1.19.0
< OK ping status identities decrypt lock unlock set-timeout quit maxline=16777216 version=1.19.0
  • Tokens are space-separated on one response line, optionally with =value parameters (boolean capabilities are bare tokens; numeric limits like maxline carry a value).
  • New capabilities are added by adding tokens. Token names are immutable: changed semantics get a new token (e.g. a future chunked-decrypt), old tokens are deprecated, never redefined. In-place semantic changes are only acceptable as fixes for behaviour that was already broken (as in fix(age): send agent identities space-separated on a single line #3509), with a compat note.
  • The client version argument and the version= token are diagnostics only — they are logged, never used for capability decisions.
  • Detecting a pre-hello agent requires no string matching: hello is parameterless from the agent's perspective, so any ERR response means "legacy agent, use conservative behaviour". This works against every agent shipped so far, since unknown commands already yield a single-line ERR and do not close the connection.
  • Forks/third parties can add namespaced tokens (something@example.com), following the SSH agent extension convention (RFC 9987 §5.8).

Because the client opens a new connection per command, doing this exchange at connection start costs the same as an on-demand query, and unlike a cached result it can never go stale across agent restarts within one gopass invocation. It stays voluntary and stateless: agents must not change behaviour based on receiving (or not receiving) hello.

Option 3: hello + capability set + per-API version numbers

> hello gopass/1.19.0
< OK ping=1 status=1 identities=1 decrypt=1 lock=1 unlock=1 set-timeout=1 quit=1 maxline=16777216 version=1.19.0

The finest granularity: each capability token carries a version (e.g. decrypt=2), the client sends each command at the highest mutually supported version (min(client max, agent max)), and the selected version is expressed as a leading argument on the wire (decrypt 2 <ciphertext>).

This works in a hello-first world: the agent advertises decrypt=2 and dispatches on the version argument, and old agents never see versioned forms because they never answered hello (the "too old vs malformed" ambiguity only arises in hello-less probing). The costs: every versioned command carries a version slot and multi-version dispatch forever, each command's version history must be maintained, and the scheme presumes extensions form a nested chain v1 ⊆ v2 ⊆ … — alternative modes such as chunked transfer do not fit a single integer. Version numbers of an existing command are also a shared resource, so extending that command requires coordinating numbers with us.

The recommendation stops at Option 2 because presence checks already compute the same decisions: a client can only ever choose among the modes it knows, ordered by its preference — first(m in myPrefs where agent advertises m). Version numbers add information only when a command's extensions form such a chain, which none does today.

Option 4: probe by trying

Sending a new command and falling back on error works mechanically today, but as the primary mechanism it is too weak: it cannot carry parameters (chunk size limits), it either doubles the payload transmission for large transfers or forces ACK-before-stream discipline, and it depends on error semantics ("unsupported" vs "failed") the protocol never defined — the same ambiguity RFC 9987 later patched with a separate SSH_AGENT_EXTENSION_FAILURE. Instead, the behaviour it relies on should be codified as an invariant (below) that clients may rely on for safety but not for feature decisions.

Open questions

  • Option 2/3: Token grammar: is key[=value], space-separated, single line sufficient?
  • Option 2/3: Should the client version argument be part of hello, or omitted entirely?
  • Self-healing in decrypt.go matches English error strings ("agent is locked", "no identities specified"); stable error codes would be more robust.

Suggested rollout

  1. Implement hello (agent + client) with the token list above; add the invariant tests; remove the dead Client.Remove().
  2. Write the protocol spec.
  3. Follow-up: implement age: agent bufio.Scanner 64 KiB line limit silently drops large decrypt commands #3508 Option 2 (chunked decrypt) gated on a capability token, with the chunk size limit learned from a chunk-max= parameter instead of hardcoded.

Prior art

  • IMAP CAPABILITY (token list; the protocol version itself is a token, IMAP4rev2)
  • ESMTP EHLO (capabilities with parameters, e.g. SIZE 10240000) and FTP FEAT (RFC 2389)
  • SSH agent extensions, RFC 9987 §5.8 (name@domain extension types; the SSH_AGENT_FAILURE vs SSH_AGENT_EXTENSION_FAILURE lesson)
  • gpg-agent Assuan: GETINFO version, OPTION agent-awareness=<ver>, INQUIRE_MAXLEN
  • Redis HELLO (connection-start version negotiation)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      ⌂ 换网址