node

package module
v0.0.0-...-6bfb771 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 13, 2026 License: Apache-2.0 Imports: 46 Imported by: 3

Documentation

Overview

Package node provides the distributed node identity, peer management, and NodeService gRPC server for inter-node communication in a MemQL cluster.

Index

Constants

View Source
const DefaultNodeRevocationCacheTTL = 5 * time.Second

DefaultNodeRevocationCacheTTL is the per-call cache TTL used when NodeRevocationCheck.CacheTTL is zero. 5s is the sweet spot the memql#343 issue notes called out: the operator's revoke click propagates to all nodes within a single cache-window of seconds, and the verifier holds the line via JWKS-only auth in the meantime. Production deploys that want tighter propagation can pass 1s; tests pass 0 (treated as default) or a small duration to exercise the "expired entry" branch.

View Source
const (
	EventBridgeComponentName = common.ComponentName("nodeEventBridge")
)
View Source
const (
	NodeServerComponentName = common.ComponentName("nodeServer")
)
View Source
const (
	OutboxRetentionComponentName = common.ComponentName("meshOutboxRetention")
)

OutboxRetention is the periodic retention sweep for the mesh delivery substrate's coordination tables (memql#1328). The 20260610000000_mesh_delivery_substrate migration documents retention as intent only ("rows are prunable ... past a max-age watermark"); nothing implemented it, so mesh_outbox and mesh_cursor grew without bound -- and traffic rose sharply once per-occurrence EventIDs (memql#1325) and streamed text chunks (memql#1327) started riding the outbox.

What it deletes, every sweepInterval (first run after a short startup delay), mirroring the automation_execution_claims pruning precedent (memql#561 -- a per-node Go ticker on a component run hook):

  • mesh_outbox rows older than the max-age watermark. Chat-reply delivery is a live-stream concern, not an archive; the graph rows themselves remain the source of truth, so age-based deletion is safe. Replay for a reconnecting consumer is bounded to the retention window by construction.
  • mesh_cursor rows whose updated_at is older than the same watermark. Consumer ids are per-pod, so every rollout strands cursor rows that would otherwise accumulate forever.
  • mesh_key_seq is deliberately NOT swept; see pgOutboxStore.SweepOlderThan for the seq-restart hazard that makes deleting allocator rows unsafe. The rows are tiny.

Multi-node safety: deletes are idempotent, so concurrent sweeps across replicas are harmless. The #561 precedent likewise prunes from every replica without a cluster claim, so none is taken here either.

The watermark comes from MEMQL_MESH_OUTBOX_RETENTION (a Go duration string). Default 24h; zero or negative disables the sweep entirely.

View Source
const (
	ParentConnectorComponentName = common.ComponentName("nodeParentConnector")
)
View Source
const (
	PeerManagerComponentName = common.ComponentName("nodePeerManager")
)
View Source
const (
	// RunDeliveryComponentName names the component in the lifecycle.
	RunDeliveryComponentName = common.ComponentName("nodeRunDelivery")
)
View Source
const StreamKind = "stream"

StreamKind is the RoutingKey.Kind used for stream channels. It is distinct from the event kinds (space/session/user) so a stream's per-key seq space and cursor never collide with an event subscription's.

View Source
const (
	TopologyReconcilerComponentName = common.ComponentName("nodeTopologyReconciler")
)
View Source
const (
	WorkerDialerComponentName = common.ComponentName("nodeWorkerDialer")
)

Variables

View Source
var ErrPeerUnreachable = errors.New("peer unreachable")

ErrPeerUnreachable is returned by Connect when a connection configured with SetGiveUpAfter has failed that many attempts in a row without the peer answering. The address it was dialling is presumed gone.

View Source
var ValidNodeTypes = meshNodeTypes()

ValidNodeTypes is the set of MESH node types: the roles that join the peer mesh. Derived from the role table in roles.go (the Mesh column), so the two cannot disagree; identity and edge are roles outside it.

Functions

func CompiledNodeTypeIsTagged

func CompiledNodeTypeIsTagged() bool

CompiledNodeTypeIsTagged reports whether CompiledNodeType() came from a BUILD TAG rather than from the untagged default.

The distinction is the whole precedence rule (memql#5115) and it cannot be recovered from the type alone: an untagged build compiles as bff, so `CompiledNodeType() == NodeTypeBFF` is true both for `go build .` -- which has no opinion and lets MEMQL_NODE_TYPE choose -- and for `go build -tags bff .`, which does and does not.

func DiscoverPeerAddress

func DiscoverPeerAddress(ctx BootstrapContext)

DiscoverPeerAddress queries the DB for a healthy peer to connect to. If the identity already has a parent address (from MEMQL_PARENT_ADDRESS), this is a no-op. Otherwise it reads the cluster topology for any healthy node whose address is non-empty and not this node's own address, and that relays mesh events (not identity), and sets identity.ParentAddress to the first match.

This uses DB-based peer discovery to find mesh peers. The first node in a fresh cluster finds no peers, starts as the mesh root, and waits for others to discover it via their own DB query.

A DISCOVERED parent is a pod address, and a pod is replaced on every rollout. So when discovery supplies the address it also leaves the Identity a way to discover again, which the ParentConnector uses once the parent stops answering (ErrPeerUnreachable) instead of redialling a pod that is gone. On 2026-09-28 a bff redialled the address of an edge pod its rollout had replaced every 30 seconds for as long as it ran. An address from MEMQL_PARENT_ADDRESS is a Service that always routes to a live pod, so it gets no resolver and is redialled as before.

func ForwardedAuthorityFromProto

func ForwardedAuthorityFromProto(p *nodev1.ForwardedAuthority) auth.ForwardedAuthority

ForwardedAuthorityFromProto reads the wire assertion back. A nil message yields the zero value, which auth.VerifyForwardedAuthority refuses: an ABSENT assertion and a malformed one take the same path deliberately, because the contract's premise is that absence is never read as safe.

func ForwardedAuthorityToProto

func ForwardedAuthorityToProto(a auth.ForwardedAuthority, originNodeId, originNodeType string) *nodev1.ForwardedAuthority

ForwardedAuthorityToProto projects the plain-Go assertion onto the wire, stamping the audit-only origin fields from the sending node.

func ForwardsGraphEvent

func ForwardsGraphEvent(topic string) bool

ForwardsGraphEvent reports whether an event on this topic is forwarded to peers under the EFFECTIVE rule set -- the built-ins plus everything registered through RegisterRoutingRule by whichever packages this binary compiled in.

It calls the same evaluateRouting the event bridge calls, deliberately. A gate that re-implemented the matcher, or compared against a string copy of the table, would be asserting about its own copy: the wildcard semantics alone (`v1:planner:*` is an INTRA-segment glob, not a segment wildcard, because a concept id contains no dots) are enough to make a plausible re-implementation disagree with the real one on exactly the rules that matter.

func GraphEventTopic

func GraphEventTopic(verb, conceptId string) string

GraphEventTopic composes the bus topic for a graph row event.

THE ONE PLACE THIS FORMAT IS SPELLED for callers outside the engine's publish path. component/memql/concept_resolver.go builds the same string when it rewrites @trigger(on=x.created); a second hand-rolled fmt.Sprintf in a test or a gate is how a checker comes to assert against a topic nothing publishes -- which passes, forever, while measuring nothing.

verb is one of "created" / "updated" / "deleted". No validation: a caller passing something else gets a topic that matches no rule, which is the honest answer rather than a panic in a routing helper.

func HealthLabel

func HealthLabel(s nodev1.NodeHealthStatus) string

HealthLabel converts a NodeHealthStatus enum value to the lowercase string form stored in the v1:cluster:node concept's `health` enum. The proto enum is the source of truth; callers must go through this function rather than hard-coding strings elsewhere.

func NodeBindingFromContext

func NodeBindingFromContext(ctx context.Context) (nodeId, nodeType string, ok bool)

NodeBindingFromContext returns the (nodeId, nodeType) the verifier extracted from the class="node" JWT for this stream. Used by the Stream() handler to cross-check NodeHello.NodeId / NodeType against the token's claims so a node-class token minted for nodeId=A cannot drive a stream that announces nodeId=B.

Returns ("", "", false) when the binding isn't on the context (single-node dev / no auth interceptor wired). Callers should skip the cross-check in that case.

func NodeClassStreamInterceptor

func NodeClassStreamInterceptor(v *verifier.Verifier, logger *slog.Logger) grpc.StreamServerInterceptor

NodeClassStreamInterceptor returns a gRPC stream interceptor that pins NodeService.Stream to class="node" JWTs minted by the identity service (#105). Every other JWT class (user / pat / worker via the PAT bearer prefix) is rejected with codes.Unauthenticated.

`v` is the per-node verifier (the same one bff/voice/cognition/etc. use for the user-facing gRPC surface). When nil the returned interceptor is a no-op pass-through; single-node dev binaries without a verifier configured run the mesh unauthenticated (there's no inter-node traffic in that case anyway).

The interceptor stamps the (nodeId, nodeType) binding onto the stream's context via withNodeBinding so the Stream() handler can cross-check NodeHello.NodeId / NodeType against them. A token minted for nodeId=A cannot drive a stream that announces nodeId=B.

PATs (verifier.SourcePAT) are rejected -- PATs are issued to human users for CLI use, not to cluster nodes.

func NodeClassStreamInterceptorWithRevocation

func NodeClassStreamInterceptorWithRevocation(v *verifier.Verifier, check *NodeRevocationCheck, logger *slog.Logger) grpc.StreamServerInterceptor

NodeClassStreamInterceptorWithRevocation is NodeClassStreamInterceptor plus the memql#349 revocation gate. When check == nil or check.Resolver == nil, it behaves identically to the base. When wired, every stream open looks up the node-token row (keyed by nodeType + nodeId from the verified JWT claims) and rejects when the row is Active == false.

The lookup goes through a short-TTL in-process cache to keep the stream-open path fast under concurrent peer reconnects. Cache key is (nodeType, nodeId); cache value is "revoked-or-not at time of last lookup". The cache does NOT memoise errors -- a failed resolver call always retries on the next stream open.

func ParseHealthLabel

func ParseHealthLabel(s string) nodev1.NodeHealthStatus

ParseHealthLabel maps the lowercase concept enum form back to the proto enum. Unknown values return CONNECTING (safe transient state).

func ParseWorkerPeers

func ParseWorkerPeers(raw string) ([]WorkerTarget, []WorkerPeerIssue)

ParseWorkerPeers parses the MEMQL_WORKER_PEERS env-var format:

"voice=voice:50059,agent=agent:50055,cognition=cognition:50054"

Whitespace around entries and pairs is tolerated. It returns the usable dial targets plus one WorkerPeerIssue per entry it could not use -- unknown or non-dialable node type, missing "=", empty half, or an exact duplicate of an earlier entry. Empty entries (a stray comma) are formatting, not a mistake, and stay quiet.

func RegisterRoutingRule

func RegisterRoutingRule(rule RoutingRule)

RegisterRoutingRule adds a routing rule from an init() function. Build tags on the caller control which binaries include the registration, so product code (e.g. a pack's integrations package) can declare its own concept patterns without editing the core node package. Order matches registration order; block rules still evaluate first across the combined set (built-in + registered).

func SubscribeStream

func SubscribeStream(
	ctx context.Context,
	substrate DeliverySubstrate,
	streamID string,
	consumerID string,
) (<-chan StreamChunk, func(seq int64) error, error)

SubscribeStream is the consumer-side entry point. The BFF replica that owns a user's WebSocket calls it with the stream id and a stable consumerID (so a reconnect -- even onto a DIFFERENT replica -- replays from the same cursor). It returns an ordered, replayed, deduped channel of StreamChunk and an Ack func to advance the durable cursor. The channel closes when ctx is cancelled OR after a terminal chunk is delivered (whichever comes first).

Replay/catch-up: on (re)connect the substrate replays every chunk with seq > cursor in ascending order before tailing live, so no chunk is lost or reordered across a replica restart. Backpressure: the returned channel is bounded; a slow WS consumer back-pressures the substrate's tail goroutine, which back-pressures (via the durable cursor not advancing) the producer's effective rate.

func SubscribeStreamFrames

func SubscribeStreamFrames(
	ctx context.Context,
	substrate DeliverySubstrate,
	streamID string,
	consumerID string,
) (<-chan StreamFrame, func(seq int64) error, error)

SubscribeStreamFrames is the consumer-side entry point. The node owning the client (the bff replica that holds the user's WebSocket) calls it with the stream id + a stable consumerID. It returns an ordered, replayed, deduped channel of typed StreamFrame and an Ack func to advance the durable cursor. The channel closes after a terminal frame (complete / error / cancel) or on ctx cancellation.

Cross-replica guarantee (inherited from the substrate): on (re)connect the substrate replays every chunk with seq > cursor in ascending order before tailing live, so a stream produced on node A is consumed in order, exactly once, by whichever replica owns the client -- even if that replica restarts mid-stream and a DIFFERENT replica reusing the same logical consumerID takes over. Backpressure: the channel is bounded; a slow client back-pressures the substrate tail, which (via the cursor not advancing) bounds the producer's effective rate.

Types

type AgentBootstrap

type AgentBootstrap struct{}

AgentBootstrap creates dependencies for an Agent node. Agent nodes perform task execution and AI work. They have: Engine + PeerManager + EventBridge + AI providers + tools + NodeServer. They do NOT have: HTTP server for external clients, voice pipeline.

func (*AgentBootstrap) Description

func (*AgentBootstrap) Description() string

func (*AgentBootstrap) NodeDependencies

func (*AgentBootstrap) NodeDependencies(ctx BootstrapContext) ([]common.Dependency, error)

type AiForwardHandler

type AiForwardHandler interface {
	HandleForwardedRequest(ctx context.Context, req *nodev1.AiForwardRequest, send func(*nodev1.NodeServerMessage) error)
	CancelForwardedRequest(ctx context.Context, requestId string)
}

AiForwardHandler is the worker-side entry point for a forwarded AI / voice request. The nodeService invokes it when an inbound NodeClientMessage carries an AiForwardRequest. The handler reads the request, dispatches to the appropriate local AI handler, and uses `send` to deliver each response back to the originating BFF wrapped in an AiForwardResponse.

Left as an interface (rather than concrete type) to keep component/node/ independent of component/grpc/. The gRPC service registers a concrete implementation via SetAiForwardHandler during bootstrap.

type AiForwardResponseSink

type AiForwardResponseSink interface {
	Dispatch(resp *nodev1.AiForwardResponse)
}

AiForwardResponseSink is the BFF-side receiver for responses to forwards originated here. Each inbound AiForwardResponse on a peer connection is routed through this sink (registered via SetAiForwardResponseSink). On the gRPC side an AiForwardRouter satisfies this contract.

type BFFBootstrap

type BFFBootstrap struct{}

BFFBootstrap creates dependencies for a BFF (Backend For Frontend) node. BFF nodes serve domain-specific frontends. They have: Engine + PeerManager + EventBridge + NodeServer. They do NOT have: AI providers, voice pipeline.

func (*BFFBootstrap) Description

func (*BFFBootstrap) Description() string

func (*BFFBootstrap) NodeDependencies

func (*BFFBootstrap) NodeDependencies(ctx BootstrapContext) ([]common.Dependency, error)

type BootstrapContext

type BootstrapContext struct {
	Identity *Identity
	Logger   *slog.Logger
	EventBus *events.Bus
	Engine   *memqlengine.MemQLEngine
	Version  string
	Wiring   *bus.Wiring // Channel-based communication wiring (optional)
}

BootstrapContext provides the shared dependencies that all node bootstraps need.

type CapabilityRouter

type CapabilityRouter struct {
	// contains filtered or unexported fields
}

CapabilityRouter resolves which node can handle a given capability. When a capability is not available locally, it queries peers via the NodeService CapabilityQuery/CapabilityResponse exchange.

func NewCapabilityRouter

func NewCapabilityRouter(identity *Identity, peerManager *PeerManager, logger *slog.Logger) *CapabilityRouter

NewCapabilityRouter creates a CapabilityRouter.

func (*CapabilityRouter) Route

func (r *CapabilityRouter) Route(ctx context.Context, capabilityFQN string) (*RouteResult, error)

Route finds a node that can handle the given capability FQN. It first checks local peers, then sends CapabilityQuery messages to connected peers if no local match is found.

type Deliverable

type Deliverable struct {
	EventID    string         // dedup key, carried on BOTH paths (ADR 4.2)
	Key        RoutingKey     // logical addressing (ADR 4.1)
	Seq        int64          // per-key ordering (ADR 4.3); set by Publish
	Topic      string         // event topic, e.g. graph.node.created....
	Kind       events.Kind    // events.Kind
	Payload    map[string]any // event payload
	OriginNode string         // producer node id; audit only, NOT for addressing
}

Deliverable is one durable, logically-addressed event. EventID is the dedup key carried byte-identical on the durable row AND the mesh fast-path copy. Seq is assigned by Publish (the per-key ordering position); callers leave it zero on the way in.

type DeliverySubstrate

type DeliverySubstrate interface {
	// Publish appends a deliverable to the durable outbox for its routing key
	// and returns the assigned per-key seq, then best-effort emits it on the
	// mesh fast-path. EventID must be stable and unique; a duplicate EventID is
	// an idempotent no-op (ADR 4.2) and returns the already-assigned seq.
	Publish(ctx context.Context, d Deliverable) (seq int64, err error)

	// Subscribe returns an ordered, replayed, deduped stream for one logical
	// key. An EXISTING consumer (one with a cursor row) replays from its
	// cursor (ADR 4.4) before tailing live. A BRAND-NEW consumer starts at
	// the key's current high watermark -- only deliverables produced after
	// Subscribe returns are guaranteed delivered -- unless WithReplayBacklog
	// opts it into replaying the retained backlog from seq 0 (memql#1328).
	// The returned channel is closed when ctx is cancelled. Each delivered
	// Deliverable carries its assigned Seq; the consumer Acks to advance its
	// cursor.
	Subscribe(ctx context.Context, key RoutingKey, consumerID string, opts ...SubscribeOption) (<-chan Deliverable, error)

	// Ack advances the persisted cursor for (key, consumerID) to seq (ADR 4.4).
	// The cursor never moves backwards.
	Ack(ctx context.Context, key RoutingKey, consumerID string, seq int64) error
}

DeliverySubstrate is the durable event-delivery backbone for the mesh (memql#1263, Phase 1 of epic memql#1259). It implements the "outbox + cursor" contract decided in the spike ADR (docs/internal/design/mesh-delivery-substrate-adr.md, memql#1262).

The contract, restated:

  • Logical addressing (ADR 4.1). Every deliverable is addressed by a RoutingKey -- a typed "<kind>:<id>" string (space:/session:/user:) -- never a physical node-id. "Deliver to whoever owns space X" resolves to space:X; the substrate does not know or care which BFF replica holds that key's consumer.
  • At-least-once + idempotent dedup (ADR 4.2). A published deliverable is delivered to every active subscriber of its key at least once, across the durable pull and/or the mesh fast-path, surviving consumer restart and mesh churn. Each deliverable carries a stable EventID; consumers dedup by EventID using the existing time-windowed eventDedup (component/node/dedup.go) so the two paths can never double-deliver.
  • Per-key ordering (ADR 4.3). Deliverables for a single key are delivered in produce order via a per-key monotonic Seq; the cursor advances monotonically. No cross-key ordering is promised (fan-out parallelism).
  • Replay / catch-up on (re)connect (ADR 4.4, amended by memql#1328). On every (re)connect a consumer resumes from its cursor (the highest acked Seq for the key) and replays "seq > cursor" in ascending order before tailing live. A BRAND-NEW consumer (no cursor row) starts at the key's current HIGH WATERMARK by default: its cursor is atomically initialized and persisted at the key's highest allocated seq, so it receives only deliverables produced after it first subscribed. Backlog replay is OPT-IN per subscription (WithReplayBacklog) for consumers that genuinely need first-subscribe history; retention (memql#1328) bounds the replayable window either way.
  • Mesh fast-path coexistence (ADR 4.5). The mesh push (EventBridge) MAY deliver the same deliverable ahead of the durable pull for latency; it carries the same EventID and is deduped against the durable path. The mesh is best-effort and never the source of truth: the CURSOR ADVANCES ON THE DURABLE PATH ONLY, so a dropped durable row can never be skipped by a mesh copy on replay.

This is a new, available capability landing ALONGSIDE the current ad-hoc mesh forwards; nothing is cut over yet. Migrating the chat-reply path (memql#1264), dispatch/client-tool RPC (memql#1265), and token/audio streaming (memql#1266) onto it, and retiring the superseded forwards (memql#1267), are downstream issues.

type DeployControlForwardHandler

type DeployControlForwardHandler interface {
	HandleForwardedRequest(ctx context.Context, req *nodev1.DeployControlForwardRequest, send func(*nodev1.NodeServerMessage) error)
}

DeployControlForwardHandler is the identity-node-side entry point for a forwarded deploy-control call (memql#3380). The nodeService invokes it when an inbound NodeClientMessage carries a DeployControlForwardRequest. Single round-trip semantics: the handler verifies the caller's forwarded authority, dispatches the bridged envelope into the local DeployControlService, and uses `send` to deliver exactly one DeployControlForwardResponse back to the originating node.

Left as an interface so component/node/ stays independent of component/grpc/ and component/deploycontrol/. The identity binary's cluster wiring installs a concrete implementation via SetDeployControlForwardHandler.

type DeployControlForwardResponseSink

type DeployControlForwardResponseSink interface {
	Dispatch(resp *nodev1.DeployControlForwardResponse)
}

DeployControlForwardResponseSink is the originating-node-side receiver for replies to deploy-control forwards started here. Each inbound DeployControlForwardResponse on a peer connection is routed through this sink; the bff-side DeployControlForwardRouter satisfies it.

type EngineExecutor

type EngineExecutor interface {
	Execute(ctx context.Context, query string) error
}

EngineExecutor is the narrow interface the NodeStatusWriter needs on the MemQL engine: the ability to execute a MemQL query/mutation string. The return value is deliberately opaque; status transitions don't consume the result. A concrete implementation wraps memql.MemQLEngine (see bootstrap files) or is stubbed in tests.

type EventBridge

type EventBridge struct {
	*component.Component
	// contains filtered or unexported fields
}

EventBridge connects the local events.Bus to the distributed NodeService mesh. It subscribes to all local events and forwards the ones a routing rule names to its peers; events arriving from peers are published locally ONCE and relayed ONCE (ReceiveForward).

EVERY STREAM CARRIES EVENTS BOTH WAYS (memql#5338). A forward goes to every peer this node holds a stream to, whichever node opened it: a dialed connection, or the push half of a stream the peer opened to this node (inbound_stream.go). Before that epic events went only along OUTBOUND dials, and a node nobody dialed -- the edge, a product bff, mcp -- heard nothing. Design record: docs/superpowers/specs/2026-09-22-mesh-event-delivery-design.md.

func NewEventBridge

func NewEventBridge(identity *Identity, localBus *events.Bus, peerManager *PeerManager, logger *slog.Logger) *EventBridge

NewEventBridge creates an EventBridge that bridges events between the local bus and the mesh, and installs it as peerManager's event sink: every stream this node reads -- accepted, parent, dialed -- delivers into it.

func (*EventBridge) MeshReport

func (eb *EventBridge) MeshReport() MeshReport

MeshReport snapshots this node's delivery report.

func (*EventBridge) Order

func (*EventBridge) Order() int

Order returns the startup order.

func (*EventBridge) PublishHint

func (eb *EventBridge) PublishHint(d Deliverable)

PublishHint emits the deliverable on the mesh as a best-effort latency optimization (the meshFastPath contract, ADR 4.5). It is fire-and-forget: it MUST NOT block the durable path and the substrate's correctness MUST NOT depend on it arriving. The hint is broadcast to every peer; whichever node owns a live subscription for d.Key feeds it to that subscription via HandleFastPath, and every other node discards it.

A hint for a node with no peers (single-node, or a worker before its mesh is up) simply forwards to zero targets -- harmless, the durable pull delivers.

func (*EventBridge) ReceiveForward

func (eb *EventBridge) ReceiveForward(evt *nodev1.EventForward, fromNodeId string)

ReceiveForward is the ONE arrival path for an event a peer sent, whichever direction of whichever stream it came down (memql#5338, D3, D6). It is flooding with duplicate suppression:

  • a FIRST sighting is published on the local bus and relayed;
  • a repeat that travelled FEWER hops than every copy before it is relayed again and not republished, so a copy that won the race the long way round cannot starve the far side of the mesh of hop budget;
  • any other repeat is dropped.

So each node publishes an event once and relays it a bounded number of times -- in practice once -- and there is no loop to prevent: a node that already relayed a copy at least as short does nothing with this one.

func (*EventBridge) SetFastPathSink

func (eb *EventBridge) SetFastPathSink(sink func(Deliverable))

SetFastPathSink wires the local DeliverySubstrate's HandleFastPath as the receiver for inbound mesh fast-path hints (memql#1289). Called from app/cluster.go on every mesh node when the substrate is constructed with this EventBridge as its meshFastPath. With a sink set, an inbound hint EventForward is decoded back to a Deliverable and handed to the substrate, which feeds it into the per-subscription dedup window and wakes a live local subscriber instantly -- the low-latency cross-node path. Without a sink (durable-only), inbound hints are ignored: correctness never depends on them (ADR 4.5 rule 3).

func (*EventBridge) SetWiring

func (eb *EventBridge) SetWiring(w *bus.Wiring)

SetWiring configures the bus wiring for channel-based communication. When set, inbound events from peers are published via the EventPublishCh channel instead of calling localBus.Publish() directly.

type Identity

type Identity struct {
	// ID is a unique identifier for this node instance, generated at startup.
	ID string

	// Type is the node's role in the cluster (from MEMQL_NODE_TYPE env var).
	Type NodeType

	// Version is the service version string.
	Version string

	// Address is the advertised NodeService gRPC address that peers use to
	// connect to this node (from MEMQL_NODE_ADDRESS env var).
	Address string

	// ParentAddress is the NodeService address of the peer that bootstrapped
	// this node. Empty for root nodes and standalone nodes. It is the address
	// the ParentConnector dials FIRST; the connector tracks the one it dials
	// now, which differs once a discovered parent has been re-resolved.
	ParentAddress string

	// Flavor is the domain flavor within the node type. Empty for single-flavor types like cognition.
	// Read from MEMQL_NODE_FLAVOR env var.
	Flavor string

	// Labels contains arbitrary metadata key-value pairs.
	Labels map[string]string

	// BearerToken is the class="node" JWT this binary presents on
	// every outbound NodeService.Stream dial. Read from
	// MEMQL_NODE_TOKEN at startup; empty when no auth is required
	// (single-node dev / clusters that haven't rolled tokens out).
	// The peerConnection wraps its context with this token before
	// opening the stream so the remote NodeServer's class-pin
	// interceptor can verify. See #105.
	//
	// Mutated at runtime by the auth-failure re-mint path (memql#1521):
	// when identity's signing key rotates and a dial is rejected, the
	// reconnect loop re-fetches a freshly-signed token and stores it
	// here so every subsequent dial presents the live token. Guard all
	// access with tokenMu via BearerTokenValue / setBearerToken; direct
	// field reads from a goroutine race with the re-mint write.
	BearerToken string
	// contains filtered or unexported fields
}

Identity holds the runtime identity of this node.

func NewIdentity

func NewIdentity(version string) *Identity

NewIdentity creates a node Identity from environment variables.

For tagged binaries (built with -tags agent/planner/bff/workbench/mcp/ identity/edge) the compiled node type takes precedence over MEMQL_NODE_TYPE, which is reported and ignored when the two disagree. MEMQL_NODE_TYPE selects the type for untagged builds only. See resolveNodeType for why that split is the one the rest of the engine already assumes.

Environment variables:

  • MEMQL_NODE_TYPE: node type (default: bff)
  • MEMQL_NODE_ADDRESS: advertised NodeService address
  • MEMQL_PARENT_ADDRESS: parent peer's NodeService address
  • MEMQL_NODE_ID: explicit node ID (default: generated UUID)
  • MEMQL_NODE_LABELS: comma-separated key=value pairs

func (*Identity) BearerTokenForDial

func (id *Identity) BearerTokenForDial(ctx context.Context, logger *slog.Logger) string

BearerTokenForDial returns the token an outbound dial presents. When this process minted it and it has used nodeTokenRefreshFraction of its lifetime, it is refreshed first -- the one mint that is not an answer to a rejection -- so a dial never presents a token about to lapse. A failed refresh keeps the current token, still valid until its expiry, and is not retried inside nodeRemintMinInterval. An out-of-band token, or one with no known expiry, is returned as it is.

func (*Identity) BearerTokenValue

func (id *Identity) BearerTokenValue() string

BearerTokenValue returns the current node JWT under the token lock. Use this (not the bare field) from any goroutine: the re-mint path (memql#1521) writes BearerToken at runtime, so a direct read races the write.

func (*Identity) CanRemintBearerToken

func (id *Identity) CanRemintBearerToken() bool

CanRemintBearerToken reports whether this node can re-fetch its node token on an auth failure (memql#1521): the self-bootstrap mint path must be configured (MEMQL_NODE_BOOTSTRAP_TOKEN + MEMQL_IDENTITY_VERIFIER_BASE_URL). When the token was provisioned out-of-band via MEMQL_NODE_TOKEN there is nothing to re-mint, so connections leave their reauth hook unset and fall back to the legacy reconnect-with-the-same-token behaviour. The identity node never re-mints (it is the issuer, not a client).

func (*Identity) EnsureBearerToken

func (id *Identity) EnsureBearerToken(ctx context.Context, logger *slog.Logger) error

EnsureBearerToken fills in BearerToken when the operator left MEMQL_NODE_TOKEN empty but opted into self-bootstrap (set MEMQL_NODE_BOOTSTRAP_TOKEN + MEMQL_IDENTITY_VERIFIER_BASE_URL on this binary and the identity service). Calls the identity service's `/node/bootstrap` endpoint with the shared secret to mint a fresh class="node" JWT, then assigns it to id.BearerToken so the peerConnection's outbound dials present it.

No-op when MEMQL_NODE_TOKEN was already set (operator-provisioned tokens win) or when the bootstrap preconditions aren't met (legacy "empty bearer token" behaviour preserved -- some single- node dev paths intentionally run without auth).

Returns a non-nil error only when the operator opted into bootstrap (secret + identity URL both set) AND the mint call failed. The caller can choose whether to block startup (production-grade) or log + proceed; app/cluster.go logs + proceeds so an identity outage during boot doesn't deadlock the whole cluster startup, matching the lenient posture the empty-token branch has carried forward from #105's original design.

memql#338.

func (*Identity) HasParent

func (id *Identity) HasParent() bool

HasParent returns true if this node has a parent peer.

func (*Identity) IsStandalone

func (id *Identity) IsStandalone() bool

IsStandalone is deprecated -- standalone mode no longer exists. Always returns false. All nodes are one of: cognition, agent, planner, bff.

func (*Identity) NodeAddress

func (id *Identity) NodeAddress() string

NodeAddress returns the node's advertised address.

func (*Identity) NodeId

func (id *Identity) NodeId() string

NodeId returns the node's unique identifier.

func (*Identity) RefreshRejectedBearerToken

func (id *Identity) RefreshRejectedBearerToken(ctx context.Context, logger *slog.Logger, rejected string) (string, error)

RefreshRejectedBearerToken answers a peer that refused the token `rejected` as a bad credential (memql#1521): it returns the token the next dial should present, minting one only when a mint can help.

  • Another connection already replaced `rejected`: present the current token. No mint -- N connections refused at once cost one mint.
  • This process minted, or tried to, within nodeRemintMinInterval: refuse with errRemintSuppressed. A token that new was not refused for its key or its age, so a second one would be refused the same way; the caller backs off instead.
  • Otherwise mint, store and return the new token. This is the key-rotation recovery: the stranded token was minted long before the rotation.

Every mint this process makes after boot goes through here or through BearerTokenForDial, both under mintMu.

type LifecycleState

type LifecycleState int

LifecycleState is the node's explicit operational state in the cluster (epic memql#1259 decision #4). It is distinct from common.Lifecycle (which is the component start/stop machinery) and from a peer's observed NodeHealthStatus: LifecycleState is the SELF-asserted intent of THIS node, which is what gets advertised in gossip (NodeHeartbeat.health / PeerInfo.health) so peers can route by it -- only deliver new work to Ready peers and route AROUND Draining/Stopped immediately instead of after a missed-heartbeat timeout.

The legal progression is strictly forward:

Starting -> Ready -> Draining -> Stopped

A node boots in Starting, flips to Ready once it can actually serve, flips to Draining when it begins a graceful shutdown (deploy SIGTERM or operator trigger -- the TRIGGERS land in memql#1269 / #1270; this type only provides the mechanism), and finally Stopped. There is no backward edge: a node never un-drains, and Stopped is terminal. Idempotent self-transitions (X -> X) are allowed and are no-ops.

const (
	// LifecycleStarting is the boot state: dependencies are wiring up and the
	// node is NOT yet ready to serve. Readiness reports not-ready.
	LifecycleStarting LifecycleState = iota

	// LifecycleReady means the node is serving normally. Readiness reports
	// ready (subject to the other registered readiness invariants).
	LifecycleReady

	// LifecycleDraining means the node has begun a graceful shutdown: it is
	// still ALIVE (liveness stays 200 on /livez) and finishing in-flight work,
	// but readiness reports NOT-ready so load balancers / k8s de-route new
	// traffic and peers route around it at once. The drain BEHAVIOUR itself
	// (finish in-flight, flush, close) is memql#1269; this state is only the
	// advertised flag.
	LifecycleDraining

	// LifecycleStopped is terminal: the node has finished draining and is
	// going away. Readiness reports not-ready.
	LifecycleStopped
)

func (LifecycleState) Health

Health maps a lifecycle state to the NodeHealthStatus advertised in gossip (NodeHeartbeat.health / PeerInfo.health). This is the bridge between the self-asserted lifecycle and the wire enum peers already understand, so the gossip contract stays backward-compatible -- existing peers keep reading NodeHealthStatus, they just now learn DRAINING / STOPPED earlier and more explicitly.

  • Starting -> CONNECTING (booting, not yet serving)
  • Ready -> HEALTHY (serving)
  • Draining -> DRAINING (de-routed, finishing in-flight)
  • Stopped -> STOPPED (gone)

func (LifecycleState) String

func (s LifecycleState) String() string

String renders the lifecycle state for logs + the gossip-derived health label. Lowercase to match HealthLabel's existing convention.

type MCPBootstrap

type MCPBootstrap struct{}

MCPBootstrap creates dependencies for an MCP (Model Context Protocol) node -- the protocol head that exposes a deployment's tool surface to external MCP hosts (epic memql#1529). Like the workbench node it is a minimal mesh citizen: Engine + PeerManager + EventBridge + NodeServer (+ ParentConnector when an upstream is configured). It does NOT dial workers or run the AiForward outbound routing the BFF does; it speaks MCP over its own transport (stdio in Phase 0) and reads the engine in-process.

func (*MCPBootstrap) Description

func (*MCPBootstrap) Description() string

func (*MCPBootstrap) NodeDependencies

func (*MCPBootstrap) NodeDependencies(ctx BootstrapContext) ([]common.Dependency, error)
type MeshLink struct {
	// Node is the peer's node id.
	Node string
	// Type is the peer's node type, as it announced itself.
	Type string
	// Via says which way the stream runs: "dialed" (this node opened it),
	// "accepted" (the peer opened it) or "both".
	Via string
}

MeshLink is one peer this node exchanges events with, as its delivery report spells it (memql#5338, D7).

type MeshReport

type MeshReport struct {
	// Receives is false on a node that takes no mesh events by design --
	// identity (meshEventParticipants) -- so a zero `heard` there is the
	// design and not an island.
	Receives bool
	// Since is when this process started counting.
	Since time.Time
	// Links is every peer this node holds a stream to, in either direction.
	Links []MeshLink

	Heard      int64
	Duplicates int64
	Originated int64
	Relayed    int64
	Dropped    int64
	HopLimited int64

	// LastHeardAt is the last first sighting; zero until the first one.
	LastHeardAt time.Time
}

MeshReport is this node's account of what it hears (memql#5338, D7): its links, and what it has done with broadcast events since it started. The node's own v1:cluster:node row carries it, refreshed by the self heartbeat, and MemQL OS draws it in Cluster > Mesh.

func (MeshReport) Wire

func (r MeshReport) Wire() map[string]any

Wire is the report as the `mesh` object on a v1:cluster:node row. Field names are the concept's (dsl/cluster/concepts.memql); lastHeardAt is ABSENT until the first event, because a zero time would read as "heard at the epoch" rather than as "never".

type NodeBootstrap

type NodeBootstrap interface {
	// NodeDependencies returns the node-specific dependencies that should
	// be added to the dependency list. Core dependencies (database, engine,
	// event bus) are created by main.go and passed via BootstrapContext.
	NodeDependencies(ctx BootstrapContext) ([]common.Dependency, error)

	// Description returns a human-readable description of this bootstrap.
	Description() string
}

NodeBootstrap defines the strategy for creating dependencies per node type. Each node type provides its own implementation that only wires the components relevant to that node's purpose.

func BootstrapFor

func BootstrapFor(nodeType NodeType) NodeBootstrap

BootstrapFor returns the appropriate NodeBootstrap for the given node type.

type NodeLifecycle

type NodeLifecycle struct {
	// contains filtered or unexported fields
}

NodeLifecycle is the concurrency-safe holder for a node's LifecycleState. The node is multi-goroutine (heartbeat tickers, the shutdown signal handler, the readiness HTTP handler) so every read/write goes through the mutex.

A NodeLifecycle optionally notifies an observer on every ACTUAL state change (idempotent self-transitions do not fire it) via SetObserver. The node wires this so a flip to Draining immediately re-advertises the new gossip health rather than waiting for the next heartbeat tick.

func NewNodeLifecycle

func NewNodeLifecycle() *NodeLifecycle

NewNodeLifecycle returns a lifecycle starting in LifecycleStarting -- the correct boot state. The node flips it to Ready once it can serve.

func (*NodeLifecycle) Health

Health returns the gossip NodeHealthStatus for the current state. Concurrency-safe convenience for the heartbeat builders.

func (*NodeLifecycle) IsDraining

func (l *NodeLifecycle) IsDraining() bool

IsDraining reports whether the node has entered Draining (or the terminal Stopped). The readiness handler consults this so a draining node reports NOT-ready while still alive on /livez (readiness != liveness).

func (*NodeLifecycle) IsReady

func (l *NodeLifecycle) IsReady() bool

IsReady reports whether the node is in the Ready state (serving). The readiness handler gates on this so a node only reports ready once it has actually flipped to Ready and has not begun draining.

func (*NodeLifecycle) MarkDraining

func (l *NodeLifecycle) MarkDraining() error

MarkDraining transitions to Draining -- the mechanism the graceful-drain (memql#1269) and operator (memql#1270) triggers call. This issue only provides the mechanism; the triggers are downstream.

func (*NodeLifecycle) MarkReady

func (l *NodeLifecycle) MarkReady() error

MarkReady transitions to Ready. Convenience wrapper over Transition.

func (*NodeLifecycle) MarkStopped

func (l *NodeLifecycle) MarkStopped() error

MarkStopped transitions to Stopped (terminal). Convenience wrapper.

func (*NodeLifecycle) SetObserver

func (l *NodeLifecycle) SetObserver(fn func(old, new LifecycleState))

SetObserver installs a callback invoked (without the lock held) on every actual state change. Passing nil clears it. Thread-safe.

func (*NodeLifecycle) State

func (l *NodeLifecycle) State() LifecycleState

State returns the current lifecycle state. Concurrency-safe.

func (*NodeLifecycle) Transition

func (l *NodeLifecycle) Transition(to LifecycleState) error

Transition moves the node to the target state, guarding illegal edges. A legal forward edge (or an idempotent self-edge) returns nil; an illegal edge (e.g. Draining -> Ready, or anything out of Stopped) returns an error and leaves the state unchanged. On an ACTUAL change (not a self-edge) the observer fires after the lock is released.

type NodeRevocationCheck

type NodeRevocationCheck struct {
	Resolver NodeTokenRevocationResolver
	CacheTTL time.Duration
}

NodeRevocationCheck wraps a resolver with the per-call cache the interceptor consults. CacheTTL controls how long an answer is reused before re-asking the resolver; the default of 5s trades a small revocation-propagation lag for protection against DB- saturating fan-out when many nodes restart concurrently.

Resolver == nil disables the whole check (the interceptor falls back to NodeClassStreamInterceptor's no-revocation behaviour).

type NodeRole

type NodeRole struct {
	// Type is the role's name: its build tag, its image suffix and its
	// MEMQL_NODE_TYPE value.
	Type NodeType
	// Description says what the role does, in one sentence.
	Description string
	// Mesh is true for the roles that join the peer mesh (ValidNodeTypes).
	Mesh bool
}

NodeRole is one role a node can play in the cluster.

func RoleFor

func RoleFor(t NodeType) (NodeRole, bool)

RoleFor returns the role named t, and false when t names no role.

func Roles

func Roles() []NodeRole

Roles returns the closed role set, in ENGINE_NODE_TYPES order. The slice is a copy; callers may reorder it.

type NodeServer

type NodeServer struct {
	// contains filtered or unexported fields
}

NodeServer implements the NodeService gRPC server for inter-node communication. It implements common.Dependency.

func NewNodeServer

func NewNodeServer(identity *Identity, peerManager *PeerManager, logger *slog.Logger) *NodeServer

NewNodeServer constructs a NodeService gRPC server.

func (*NodeServer) Address

func (s *NodeServer) Address() string

Address returns the configured listen address.

func (*NodeServer) ComponentName

func (*NodeServer) ComponentName() common.ComponentName

ComponentName identifies the dependency.

func (*NodeServer) IsRunning

func (s *NodeServer) IsRunning() bool

IsRunning reports whether the server lifecycle is active.

func (*NodeServer) Order

func (*NodeServer) Order() int

Order returns the startup priority.

func (*NodeServer) Ready

func (s *NodeServer) Ready() <-chan struct{}

Ready returns a channel that is closed when the server is ready.

func (*NodeServer) SetAiForwardHandler

func (s *NodeServer) SetAiForwardHandler(h AiForwardHandler)

SetAiForwardHandler installs the worker-side AI/voice request handler. Called during bootstrap on Voice / Agent binaries; BFF leaves nil.

func (*NodeServer) SetAiForwardResponseSink

func (s *NodeServer) SetAiForwardResponseSink(sink AiForwardResponseSink)

SetAiForwardResponseSink installs the BFF-side response sink for inbound AiForwardResponse messages received over direct peer-to-peer connections the BFF accepts (not just the outbound parent connection; some topologies have the BFF on the server side of the NodeService handshake). Called during bootstrap on BFF binaries.

func (*NodeServer) SetAuthInterceptor

func (s *NodeServer) SetAuthInterceptor(i grpc.StreamServerInterceptor)

SetAuthInterceptor installs the class="node" JWT enforcement stream interceptor on NodeService.Stream. Call after construction but BEFORE Start (the interceptor is read once at prepareForRun when the gRPC server is created). Passing nil leaves the interceptor unwired for single-node binaries. See NodeClassStreamInterceptor + #105.

func (*NodeServer) SetDeployControlForwardHandler

func (s *NodeServer) SetDeployControlForwardHandler(h DeployControlForwardHandler)

SetDeployControlForwardHandler installs the identity-node-side handler invoked for inbound DeployControlForwardRequest messages (memql#3380). Called during bootstrap on the identity binary; every other node type leaves it nil and answers the forward with a transport error naming the identity node.

func (*NodeServer) SetWorkbenchForwardHandler

func (s *NodeServer) SetWorkbenchForwardHandler(h WorkbenchForwardHandler)

SetWorkbenchForwardHandler installs the workbench-node-side handler invoked for inbound WorkbenchForwardRequest messages. Called during bootstrap on workbench binaries; other node types leave nil.

func (*NodeServer) SetWorkbenchForwardResponseSink

func (s *NodeServer) SetWorkbenchForwardResponseSink(sink WorkbenchForwardResponseSink)

SetWorkbenchForwardResponseSink installs the agent-side response sink for inbound WorkbenchForwardResponse messages received over direct peer-to-peer connections. Called during bootstrap on agent binaries when cluster-mode workbench forwarding is enabled.

func (*NodeServer) SetWorkerForwardHandler

func (s *NodeServer) SetWorkerForwardHandler(h WorkerForwardHandler)

SetWorkerForwardHandler installs the agent-replica-side handler invoked for inbound WorkerForwardRequest messages (memql#4352). Called during bootstrap on agent binaries; every other node type leaves it nil and answers a forward with a transport error naming the absence.

func (*NodeServer) SetWorkerForwardResponseSink

func (s *NodeServer) SetWorkerForwardResponseSink(sink WorkerForwardResponseSink)

SetWorkerForwardResponseSink installs the sink for replies to worker dispatches this node forwarded, arriving over direct peer connections.

func (*NodeServer) Start

func (s *NodeServer) Start(ctx context.Context)

Start boots the NodeService gRPC server.

func (*NodeServer) Stop

func (s *NodeServer) Stop(ctx context.Context)

Stop gracefully shuts down the NodeService gRPC server.

type NodeStatusWriter

type NodeStatusWriter struct {
	// contains filtered or unexported fields
}

NodeStatusWriter bridges PeerManager status transitions to the v1:cluster:node concept store. It is installed as PeerManager's StatusChangeHandler in the bootstrap phase.

On every transition it schedules a MemQL `updateNodeHealth` mutation (see mutations/v1/cluster/updateNodeHealth.memql). That insert shares its `id` with the peer's NodeId so the concept records a time-series of health changes for that node. The engine's existing CDC path emits graph.node.created.<partition>.v1:cluster:node, which the CLI subscribes to for live topology updates.

func NewNodeStatusWriter

func NewNodeStatusWriter(engine EngineExecutor, self *Identity, logger *slog.Logger) *NodeStatusWriter

NewNodeStatusWriter constructs a writer bound to the given engine. Passing a nil engine returns a writer whose Handle is a no-op -- useful in unit tests that exercise PeerManager without a DB.

func (*NodeStatusWriter) Handle

func (w *NodeStatusWriter) Handle(ctx context.Context, peer *nodev1.PeerInfo, old, new nodev1.NodeHealthStatus, lastSeen time.Time)

Handle implements StatusChangeHandler. It is invoked synchronously from PeerManager; we dispatch the actual engine.Execute on a fresh goroutine so we never block the peer-manager tick.

type NodeTokenRevocationResolver

type NodeTokenRevocationResolver interface {
	IsNodeTokenRevoked(ctx context.Context, nodeType, nodeId string) (bool, error)
}

NodeTokenRevocationResolver is the narrow port the interceptor uses to check whether a node token's identity row has been revoked by the operator. Implementations should be fast + cache-friendly -- the function is called on every NodeService.Stream open from every cluster peer.

(nodeType, nodeId) are extracted from the verified JWT's claims; the resolver looks up the v1:identity:identity[node_token] row by that binding (matches memql#347's `identity.Store.LookupNodeTokenIdentityByBinding`).

Returns (true, nil) when the row exists AND Active == false (verifier rejects the stream open with codes.PermissionDenied). Returns (false, nil) when the row is live or doesn't exist (the "operator-CLI mint that pre-dates persistence" case is treated as not-revoked; verifier proceeds). Returns (false, error) when the lookup itself failed; the interceptor logs and refuses the open with codes.Unavailable rather than admitting traffic on a partially-failed revocation check.

type NodeType

type NodeType string

NodeType identifies the role a node plays in the cluster.

const (
	// NodeTypeAgent performs task execution and AI work.
	NodeTypeAgent NodeType = "agent"

	// NodeTypePlanner handles task planning and orchestration.
	NodeTypePlanner NodeType = "planner"

	// NodeTypeBFF serves domain-specific frontends.
	// Default when no build tag is specified.
	NodeTypeBFF NodeType = "bff"

	// NodeTypeWorkbench hosts the sandboxed per-Plan Linux working
	// environment. Receives WorkbenchForwardRequest envelopes on
	// NodeService.Stream and dispatches to the local workbench
	// integration. Agent nodes are the typical client.
	NodeTypeWorkbench NodeType = "workbench"

	// NodeTypeIdentity is the identity / auth service -- the node-token
	// ISSUER, not a mesh worker (so it is intentionally absent from
	// ValidNodeTypes). It runs the `/node/bootstrap` endpoint that every
	// other node calls to mint its class="node" JWT; it must never call
	// that endpoint on itself (see EnsureBearerToken). memql#570.
	NodeTypeIdentity NodeType = "identity"

	// NodeTypeMCP is the MCP (Model Context Protocol) server -- the protocol
	// head that exposes a deployment's tool surface to external MCP hosts
	// (Claude Desktop / Claude Code). A first-class node role selected by the
	// `mcp` build tag (epic memql#1529). It shares the cluster database and
	// joins the mesh like any other role; the tool surface it serves over the
	// wire lands in later phases (#1531+).
	NodeTypeMCP NodeType = "mcp"

	// NodeTypeEdge serves this cluster's hosted web surfaces, resolving a
	// request Host to a v1:platform:site row (epic memql#3700). Like
	// identity it is a real node role that is NOT a mesh worker, so it is
	// deliberately absent from ValidNodeTypes -- nothing dials an edge.
	NodeTypeEdge NodeType = "edge"
)

func CompiledNodeType

func CompiledNodeType() NodeType

CompiledNodeType returns the node type this binary was built for: the one named by its `-tags <type>` build tag, or NodeTypeBFF for an untagged build. Set via the build-tagged compiled_*.go files, one per node type.

func ForwardDecisionFor

func ForwardDecisionFor(topic string) (forward bool, broadcast bool, targetType NodeType)

ForwardDecisionFor reports how an event on topic would be routed under the effective rule set: whether it forwards to peers at all, whether it broadcasts, and which node type it targets when it does not.

Exported for tests that need to exercise a cross-node hop WITHOUT standing up gRPC peers -- notably the automation-run routing coverage, which drives two in-process nodes through a bridge that applies this decision exactly as EventBridge.onLocalEvent does. Without an exported view of the decision such a test would have to hard-code "this topic forwards", which is the assertion it is supposed to be making.

type OutboxRetention

type OutboxRetention struct {
	*component.Component
	// contains filtered or unexported fields
}

OutboxRetention runs the sweep on a component run hook (the #561 pattern).

func NewOutboxRetention

func NewOutboxRetention(dbGetter func() *bun.DB, logger *slog.Logger) *OutboxRetention

NewOutboxRetention builds the production sweeper over the cluster Postgres outbox tables. dbGetter is resolved lazily (the same posture as the substrate store and ClusterExecutionGuard) so construction can precede the database component. Retention is read from MEMQL_MESH_OUTBOX_RETENTION.

func (*OutboxRetention) Retention

func (r *OutboxRetention) Retention() time.Duration

Retention exposes the resolved watermark (tests + health surfaces).

type ParentConnector

type ParentConnector struct {
	*component.Component
	// contains filtered or unexported fields
}

ParentConnector dials the node's parent peer and keeps a single bidirectional NodeService stream open. While the stream is up the connection's heartbeat loop (see connection.go:heartbeatLoop) emits NodeHeartbeat messages on the configured interval; this is what gives the parent's PeerManager the liveness signal it needs to mark a child degraded or offline when it disappears.

ParentConnector is only installed on node types that have a parent address configured. Root nodes and standalone nodes skip it.

func NewParentConnector

func NewParentConnector(identity *Identity, peerMgr *PeerManager, logger *slog.Logger) *ParentConnector

NewParentConnector returns nil when the identity has no parent address, so callers can blindly append the result to the dependency list without branching on node type.

func (*ParentConnector) Order

func (*ParentConnector) Order() int

Order places this after PeerManager so the heartbeat interval is available, but before NodeServer/EventBridge so the stream is up by the time events start flowing.

func (*ParentConnector) SetAiForwardResponseSink

func (pc *ParentConnector) SetAiForwardResponseSink(sink AiForwardResponseSink)

SetAiForwardResponseSink registers a sink for AiForwardResponse messages received over the parent connection. Called during bootstrap on BFF binaries. Workers leave this nil.

func (*ParentConnector) SetDeployControlForwardResponseSink

func (pc *ParentConnector) SetDeployControlForwardResponseSink(sink DeployControlForwardResponseSink)

SetDeployControlForwardResponseSink registers a sink for inbound DeployControlForwardResponse messages received over the parent connection (memql#3380). Called during bff bootstrap; other node types leave it nil.

func (*ParentConnector) SetWorkbenchForwardResponseSink

func (pc *ParentConnector) SetWorkbenchForwardResponseSink(sink WorkbenchForwardResponseSink)

SetWorkbenchForwardResponseSink registers a sink for inbound WorkbenchForwardResponse messages received over the parent connection. Called during agent-node bootstrap when cluster-mode workbench forwarding is enabled.

func (*ParentConnector) SetWorkerForwardResponseSink

func (pc *ParentConnector) SetWorkerForwardResponseSink(sink WorkerForwardResponseSink)

SetWorkerForwardResponseSink registers a sink for inbound WorkerForwardResponse / WorkerForwardStream messages arriving over the parent connection (memql#4352). Set on agent binaries; nil elsewhere.

type PeerEntry

type PeerEntry struct {
	Info       *nodev1.PeerInfo
	LastSeen   time.Time
	Connection *peerConnection // nil if not directly connected
	Monitored  bool
}

PeerEntry represents a known peer in the cluster.

Monitored reports whether this node is the liveness authority for the peer. A PeerManager only fires degraded/offline transitions for monitored peers -- typically peers on the other end of a direct stream (inbound handshake, child's ParentConnector). Peers learned indirectly through PeerIntroduction stay unmonitored so sibling nodes don't false-positive a peer they never receive heartbeats from.

type PeerManager

type PeerManager struct {
	*component.Component
	// contains filtered or unexported fields
}

PeerManager maintains an in-memory peer table and manages the heartbeat lifecycle. It implements common.Dependency.

func NewPeerManager

func NewPeerManager(identity *Identity, logger *slog.Logger) *PeerManager

NewPeerManager creates a PeerManager for the given node identity with default heartbeat timings. Use SetTimings to override.

func (*PeerManager) AllPeers

func (pm *PeerManager) AllPeers() []*PeerEntry

AllPeers returns a snapshot of all current peers.

func (*PeerManager) AttachConnection

func (pm *PeerManager) AttachConnection(nodeId string, conn *peerConnection)

AttachConnection binds an outbound *peerConnection to the PeerEntry for the given nodeId so downstream code (AiForwardRouter, EventBridge, CapabilityRouter) can call Send on it. Callers MUST hold the connection's lifetime until DetachConnection is invoked or the entry is removed; PeerManager does not own the connection.

If no entry exists for nodeId the call is a no-op -- the typical flow is RegisterMonitored(...) followed immediately by AttachConnection, so a missing entry indicates a race that will be corrected on the next register.

func (*PeerManager) ByCapability

func (pm *PeerManager) ByCapability(capabilityFQN string) []*PeerEntry

ByCapability returns peers that advertise the given capability FQN.

func (*PeerManager) ByType

func (pm *PeerManager) ByType(t NodeType) []*PeerEntry

ByType returns all peers of the given node type.

func (*PeerManager) DetachConnection

func (pm *PeerManager) DetachConnection(nodeId string)

DetachConnection clears the *peerConnection on the PeerEntry for nodeId without closing it -- the caller owns the close. Use this before shutting down a connection so subsequent Sends from other components fail fast rather than racing the close.

func (*PeerManager) Get

func (pm *PeerManager) Get(nodeId string) *PeerEntry

Get returns a peer entry by node ID.

func (*PeerManager) HeartbeatInterval

func (pm *PeerManager) HeartbeatInterval() time.Duration

HeartbeatInterval returns the interval between outbound heartbeats. Exposed so peer connections can install their own send tickers using the same cadence the liveness checker expects.

func (*PeerManager) Lifecycle

func (pm *PeerManager) Lifecycle() *NodeLifecycle

Lifecycle returns this node's self-asserted lifecycle state machine (Starting/Ready/Draining/Stopped, epic memql#1259 #4). Callers use it to flip the node Ready at boot, mark it Draining on shutdown (the actual triggers land in memql#1269 / #1270), and to gate readiness. The heartbeat builders read Lifecycle().Health() so the advertised gossip health tracks the lifecycle. Never nil.

func (*PeerManager) Order

func (*PeerManager) Order() int

Order returns the startup order for the PeerManager.

func (*PeerManager) PeerCount

func (pm *PeerManager) PeerCount() int

PeerCount returns the number of registered peers.

func (*PeerManager) PeerInfoList

func (pm *PeerManager) PeerInfoList() []*nodev1.PeerInfo

PeerInfoList returns COPIES of the PeerInfo protos for all current peers.

Copies, because the caller marshals them onto a stream (the NodeWelcome) outside this lock while a heartbeat on another stream can be updating the same peer's Health in place (UpdatePeerHealth). Handing out the table's own pointers was a data race the real-transport gate found the first time it ran several streams at once (mesh_delivery_test.go, under -race).

func (*PeerManager) Register

func (pm *PeerManager) Register(info *nodev1.PeerInfo)

Register adds or updates a peer in the peer table. Equivalent to RegisterPeer(info, false) -- appropriate for fan-out registrations (PeerIntroduction) where this node cannot observe the peer's liveness directly.

func (*PeerManager) RegisterMonitored

func (pm *PeerManager) RegisterMonitored(info *nodev1.PeerInfo)

RegisterMonitored marks the peer as directly-observable: this node's liveness checker will degrade/offline the peer if heartbeats stop. Callers are stream_handler.handleHello (inbound direct peer) and ParentConnector's NodeWelcome handler (outbound to parent).

func (*PeerManager) RegisterPeer

func (pm *PeerManager) RegisterPeer(info *nodev1.PeerInfo, monitored bool)

RegisterPeer adds or updates a peer with an explicit monitored flag.

func (*PeerManager) Remove

func (pm *PeerManager) Remove(nodeId string)

Remove removes a peer from the peer table and fires a STOPPED transition so downstream consumers (the concept store, the CLI) observe the exit.

func (*PeerManager) SelfNodeId

func (pm *PeerManager) SelfNodeId() string

SelfNodeId / SelfNodeType report this node's own identity. The mesh forwarded-auth router stamps them onto every authority assertion as audit provenance, from one place, so no producer call site can forget it (memql#3205). Audit-only: never an input to an authorization decision, so an empty value never changes a verdict.

func (*PeerManager) SelfNodeType

func (pm *PeerManager) SelfNodeType() string

func (*PeerManager) SendCancel

func (pm *PeerManager) SendCancel(nodeId string, msg *nodev1.NodeClientMessage) error

SendCancel queues a CANCEL for a forward this node sent on the named peer's general outbox: the current stream attempt if one is live, the next one if the connection is between attempts.

A CANCEL IS NOT A REQUEST, and the two transports are chosen for opposite reasons. Nobody parks on a cancel, so "queued for the next attempt" is not a hang; and the call it names may outlive the stream it was asked on -- a pull and a probe run under a context the stream ending does not cancel -- so a cancel SendRequest dropped during a reconnect left a download running.

It can overtake its request: the send loop drains this outbox and the attempt's request queue in whichever order a select picks. The receiver remembers a cancel for a request it has not seen and does not run it when it arrives (forward_inflight.go).

It reports an error only when there is no connection to queue on or the outbox refused the message; the caller has nothing to wait for either way.

func (*PeerManager) SendRequest

func (pm *PeerManager) SendRequest(nodeId string, msg *nodev1.NodeClientMessage) (<-chan struct{}, error)

SendRequest hands one REQUEST-shaped envelope -- a forward whose sender parks on the answer -- to the named peer's CURRENT stream attempt, and returns a channel that closes when that attempt ends.

A REQUEST THAT WAS NOT HANDED OVER IS AN ERROR. The general outbox (peerConnection.Send) is the right transport for heartbeats and gossip, which are fire-and-forget: it drops a message when it is full or closed and says so only in a log line, and between attempts it queues for a stream that does not exist yet. A request sent that way was reported as handed off either way, and its sender then waited out its own deadline -- minutes, for an app call -- for an answer that could not come. Refusing here lets the sender report "refused before start" and its route move on.

THE RETURNED CHANNEL IS THE HOLDER-GONE SIGNAL. Every answer to this request comes back on the stream it went out on, so once that attempt ends no answer can arrive; a sender that selects on it fails fast instead of parking.

It is the transport the AI forward already uses (AiForwardRouter.Forward), named here so the worker forwards can use it through the PeerManager without reaching into an unexported connection type.

func (*PeerManager) SetParentConnection

func (pm *PeerManager) SetParentConnection(conn *peerConnection)

SetParentConnection sets the connection to the parent peer.

func (*PeerManager) SetStaleGossipTimeout

func (pm *PeerManager) SetStaleGossipTimeout(d time.Duration)

SetStaleGossipTimeout overrides the window after which an unmonitored, unconnected peer (learned via gossip) is reaped from the peer table. A zero or negative value leaves the current value unchanged. Must be called before Start. Kept separate from SetTimings so the existing 3-arg timing contract (and its callers) stays stable.

func (*PeerManager) SetStatusChangeHandler

func (pm *PeerManager) SetStatusChangeHandler(h StatusChangeHandler)

SetStatusChangeHandler installs a callback invoked on every health transition. Passing nil disables notifications. Thread-safe; may be swapped at runtime.

func (*PeerManager) SetTimings

func (pm *PeerManager) SetTimings(heartbeat, liveness, offline time.Duration)

SetTimings overrides the default heartbeat / liveness / offline durations. Any zero-valued duration leaves its field at the current value. Must be called before Start.

func (*PeerManager) SnapshotByType

func (pm *PeerManager) SnapshotByType(t NodeType) []*PeerEntry

SnapshotByType returns immutable selections for operations that outlive the peer-table lock. The connection remains the selected transport even if the manager detaches or replaces it; its own lifecycle reports disconnection.

func (*PeerManager) TouchPeer

func (pm *PeerManager) TouchPeer(nodeId string)

TouchPeer updates the LastSeen timestamp for a peer and, if the peer was previously marked DEGRADED or CONNECTING, transitions it back to HEALTHY.

func (*PeerManager) UpdatePeerHealth

func (pm *PeerManager) UpdatePeerHealth(nodeId string, reported nodev1.NodeHealthStatus)

UpdatePeerHealth records a peer-reported health status (e.g. DRAINING) and fires a transition if it differs from the current value. Unlike TouchPeer, this does NOT update LastSeen -- pair it with TouchPeer in the heartbeat handler.

type PlannerBootstrap

type PlannerBootstrap struct{}

PlannerBootstrap creates dependencies for a Planner node. Planner nodes handle task planning and orchestration. They have: Engine + PeerManager + EventBridge + NodeServer. They do NOT have: HTTP server, voice pipeline, external client access.

func (*PlannerBootstrap) Description

func (*PlannerBootstrap) Description() string

func (*PlannerBootstrap) NodeDependencies

func (*PlannerBootstrap) NodeDependencies(ctx BootstrapContext) ([]common.Dependency, error)

type RPCHandler

type RPCHandler func(ctx context.Context, req RPCRequest) (map[string]any, error)

RPCHandler processes one inbound request and returns the response payload, or an error. A non-nil error is surfaced to the caller as a typed RPC error reply (the request still gets a reply -- the caller never hangs on a handler failure). method + correlationID are passed through for routing/audit.

type RPCRequest

type RPCRequest struct {
	Method        string         // logical method name (handler dispatch key)
	CorrelationID string         // echoed onto the reply
	ReplyKey      RoutingKey     // where the reply is published (caller's key)
	Payload       map[string]any // request arguments
	OriginNode    string         // producer node id; audit only
}

RPCRequest is the decoded inbound request a Serve handler sees.

type RouteResult

type RouteResult struct {
	// Local is true if this node can handle the capability.
	Local bool

	// NodeId is the ID of the node that can handle the capability.
	NodeId string

	// Address is the gRPC address of the node.
	Address string
}

RouteResult contains the outcome of a capability lookup.

type RoutingExclusion

type RoutingExclusion struct {
	// Pattern is matched against event topics with the same matcher the
	// routing rules use, so `graph.node.*.v1:worker:invocation` excludes
	// all three verbs and a fully-spelled topic excludes exactly one.
	Pattern string

	// Reason is why. REQUIRED, and the gate fails on an empty one: an
	// exclusion without a why is a hole with paperwork, and the next
	// reader cannot tell it from an oversight that someone silenced.
	Reason string
}

RoutingExclusion is one concept-and-verb the mesh deliberately does not carry.

func ExcludedFromForwarding

func ExcludedFromForwarding(topic string) (RoutingExclusion, bool)

ExcludedFromForwarding reports whether a topic is covered by a recorded exclusion, and returns the record.

Uses events.Match rather than string equality so a pattern excluding all three verbs of a concept is one entry rather than three, matching how the forward rules are written.

func RoutingExclusions

func RoutingExclusions() []RoutingExclusion

RoutingExclusions returns the recorded exclusions.

Ordered by how likely a reader is to be looking for them, not alphabetically: the judgment calls first, then the volume denials.

type RoutingKey

type RoutingKey struct {
	Kind string // "space" | "session" | "user"
	ID   string
}

RoutingKey is a logical delivery address (ADR 4.1): a typed "<kind>:<id>" string. It is NEVER a physical node-id. The substrate routes by this key alone, so a producer never has to find the physical replica that owns a key.

func StreamKey

func StreamKey(streamID string) RoutingKey

StreamKey builds the logical routing key for a stream id. The id is the caller's stream/request id (e.g. the chat request_id or transcribe request_id); it is the SAME on the producing worker and the consuming BFF, so "deliver to whoever owns this stream" resolves without naming a node.

func (RoutingKey) String

func (k RoutingKey) String() string

String renders the routing key as the canonical "<kind>:<id>" wire form used as the outbox/cursor column value.

func (RoutingKey) Valid

func (k RoutingKey) Valid() bool

Valid reports whether the routing key is non-empty in both components. The substrate rejects an empty key at Publish/Subscribe time -- an unaddressed deliverable has no logical owner.

type RoutingRule

type RoutingRule struct {
	// Pattern is a glob pattern matched against event topics.
	Pattern string

	// TargetType restricts forwarding to peers of this type.
	// Empty string means broadcast to all connected peers.
	TargetType NodeType

	// Block when true means events matching this pattern should NOT be forwarded.
	Block bool
}

RoutingRule determines how an event should be forwarded to peers.

func RoutingRules

func RoutingRules() []RoutingRule

RoutingRules is the effective routing table: the built-in rules followed by every rule registered through RegisterRoutingRule, in the order evaluateRouting reads them. Exported so the platform graph can record the table (cmd/platformgraph, memql#5727) rather than a copy of it; the slice is the caller's own.

What it holds depends on what the calling binary links: a rule registered from a package's init() is present only when that package is compiled in.

type RunDelivery

type RunDelivery struct {
	*component.Component
	// contains filtered or unexported fields
}

RunDelivery also retains work.run.recovery scheduling hints. They are not graph writes: the consumer must re-read the exact owned stale run before claiming, and the producer observes a real claim rather than treating delivery as progress. This reuses the lifecycle cursor; it transfers no live session.

RunDelivery routes the run-lifecycle graph events (graph.node.created / graph.node.updated for v1:work:run) through the durable DeliverySubstrate (memql#1264, the #1263 outbox + cursor backbone), giving run dispatch a real at-least-once delivery leg (memql#1495) instead of relying on the mesh fast-path alone.

PORTED FROM THE PLAN LIFECYCLE (memql#5053). It was PlanDelivery, and the incident that produced it is the reason it was ported rather than deleted with the concept: the durability claim in eventbridge.go ("the mesh fast-path is purely a latency optimization; the durable substrate is the delivery guarantee") did NOT hold for graph events that ride the broadcast ONLY. When the broadcast was dropped while the consuming node was routeless -- the mesh peer-table decay incident (memql#1388, PR #1494) -- the event never arrived and the work stranded.

The work spine has exactly that shape: v1:work:run events are broadcast (component/node/routing.go), and the RUN DISPATCHER (memql#5054) consumes them off its local bus to claim and execute a run. A dropped broadcast means a compiled run that nobody executes -- which the abandoned sweep then closes with a message about a node going away. So the guarantee moves with the mechanism.

What did NOT move is the recovery backstop: the plan side had the #1389 stranded-plan watchdog polling the database, and the run side has no equivalent. That makes this leg MORE load-bearing here than it was there, not less.

RunDelivery mirrors ChatReplyDelivery exactly, with the parallel addressing inversion (physical "deliver to agent-pod-N" -> logical "deliver to whoever executes runs"):

  • Producer side (every mesh node that emits a run graph event): on a LOCAL, non-remote run graph event, Publish a Deliverable to the substrate keyed by the fixed logical key run:lifecycle. The durable outbox row is the guarantee; the EventBridge mesh broadcast stays as the deduped low-latency fast-path. A run row is written by the node that opened it and by the node executing it, so both run the producer side.
  • Consumer side (executing replicas only): one Subscribes ONCE to run:lifecycle on the substrate and re-publishes each deliverable onto its LOCAL bus -- where the run dispatcher's subscription already consumes it -- then Acks. The durable cursor means a replica whose mesh route was dead when a run event was produced catches up on cursor-pull replay, which is the route gap that would otherwise leave a compiled run unexecuted until the abandoned sweep closed it.

Idempotency vs the mesh copy -- DELIBERATELY DIFFERENT from chat-reply. ChatReplyDelivery SUPPRESSES the inbound mesh copy on the bff because the browser must receive each reply EXACTLY ONCE and the substrate is its single delivery path. Run events have no such constraint, and the reason is the dispatcher's own design: it CLAIMS a run before executing it, against a Postgres-backed guard keyed on the run id (memql#5054). So a run event arriving on BOTH the mesh fast-path and the durable replay is collapsed by that claim; the mesh copy is NOT suppressed. RunDelivery is purely an ADDITIONAL at-least-once path -- it can only ADD a delivery an executing replica would otherwise have missed, never cause a double-execution, because the dedup lives in the consumer rather than at the substrate boundary.

Exactly-once for a single occurrence is still the substrate's per-subscription EventID dedup (the durable-pull copy and any mesh-hint copy of the same EventID collapse to one re-publish); per-key ordering is the substrate's per-key monotonic seq + cursor. The EventID is the run row id suffixed with the bus event's emission timestamp (see publish), so a status transition that re-inserts under the same run id (MemQL is append-only; each write is a new row VERSION under the same logical id) still gets a distinct durable row rather than being swallowed by the outbox's EventID uniqueness -- the same per-occurrence keying the chat-reply presence path needed (memql#1324).

func NewRunDelivery

func NewRunDelivery(identity *Identity, substrate DeliverySubstrate, localBus *events.Bus, runsSteps bool, logger *slog.Logger) *RunDelivery

NewRunDelivery builds the run-delivery router over the given substrate. runsSteps gates the consumer (Subscribe + re-publish) side: replicas that EXECUTE runs (agent) or COMPILE them (planner) consume run lifecycle events off their local bus, so only they subscribe and fan the durable stream back. Every node runs the producer (Publish) side -- a run row is written by whichever node opened it (often the bff) and by whichever one is compiling or executing it. Returns nil when the substrate or local bus is absent (single-node / non-mesh binaries) so the caller can skip wiring it without a nil-guard at every call site.

func (*RunDelivery) Order

func (*RunDelivery) Order() int

Order returns the startup order.

type SelfStatusWriter

type SelfStatusWriter struct {
	*component.Component
	// contains filtered or unexported fields
}

SelfStatusWriter refreshes this process's persisted liveness independently of mesh gossip and health transitions. The slow stale-node sweep needs a real heartbeat even when a healthy process never changes state.

func NewSelfStatusWriter

func NewSelfStatusWriter(self *Identity, lifecycle *NodeLifecycle, engine *memqlengine.MemQLEngine, logger *slog.Logger) *SelfStatusWriter

NewSelfStatusWriter is installed once by every node's app lifecycle wiring.

func (*SelfStatusWriter) Order

func (*SelfStatusWriter) Order() int

func (*SelfStatusWriter) SetMeshReporter

func (w *SelfStatusWriter) SetMeshReporter(fn func() MeshReport)

SetMeshReporter gives the writer this node's delivery report (memql#5338, D7): every refresh then carries it onto the node's own v1:cluster:node row as its `mesh` object, which is what MemQL OS's Cluster > Mesh reads. No new rows and no new writes -- the report rides the heartbeat that already writes the row once a minute.

func (*SelfStatusWriter) SyncLifecycle

func (w *SelfStatusWriter) SyncLifecycle()

SyncLifecycle is called by the existing lifecycle observer. It cancels and joins any periodic write before persisting the new state. app.Run invokes MarkNodeStopped before closing database pools, making this synchronous callback the terminal-write barrier rather than a late dependency stop hook.

type StatusChangeHandler

type StatusChangeHandler func(ctx context.Context, peer *nodev1.PeerInfo, old, new nodev1.NodeHealthStatus, lastSeen time.Time)

StatusChangeHandler is invoked whenever a peer's health status transitions. It is called synchronously from the caller's goroutine (either a liveness tick, a Register, or a TouchPeer) so implementations must return quickly and must not block. A typical implementation schedules an async concept upsert on its own goroutine.

The handler receives a snapshot of the peer info at the transition point. `old` is the previous NodeHealthStatus, `new` is the incoming status. `lastSeen` is the timestamp used to evaluate the transition.

type StreamChunk

type StreamChunk struct {
	StreamID string
	Seq      int64
	Topic    string
	Data     string
	Terminal bool
	Err      string
}

StreamChunk is one delivered piece of a stream as seen by the consumer. Seq is the substrate's per-stream ordering position (strictly increasing, gap-tolerant); the consumer Acks Seq to advance its durable cursor. Terminal marks the last chunk; Err is set when the stream ended abnormally.

type StreamFrame

type StreamFrame struct {
	// StreamID is the logical stream id (chat request id / transcribe request id).
	StreamID string
	// Seq is the substrate's per-stream ordering position. The consumer Acks Seq
	// to advance its durable cursor (so a replica restart replays seq > cursor).
	Seq int64
	// Phase is the lifecycle position (start / delta / complete / error / cancel).
	Phase StreamPhase
	// Topic mirrors the producer's logical topic (e.g. "token", "transcript").
	Topic string
	// Data is the chunk's incremental text on a Delta, the final text on a
	// Complete, and empty on Start / Cancel.
	Data string
	// Meta carries the Start frame's opening metadata and the Complete frame's
	// completion metadata. Nil on Delta frames.
	Meta map[string]any
	// Err is the error string on an Error terminal; empty otherwise.
	Err string
	// Terminal is true on the complete / error / cancel frame.
	Terminal bool
}

StreamFrame is one delivered lifecycle frame as the consumer sees it. It wraps the raw StreamChunk's ordering + data with the decoded lifecycle Phase and the Start/Complete metadata.

type StreamPhase

type StreamPhase string

StreamPhase is the lifecycle position of a delivered StreamFrame. It is carried on the chunk so the consumer can drive a start/delta/complete state machine without inspecting payload shape.

const (
	// StreamPhaseStart is the opening frame of a stream. It carries the
	// stream's opening metadata (provider, format, the turn/request descriptor)
	// in its Meta map and no Data. Exactly one Start per stream, first in seq
	// order.
	StreamPhaseStart StreamPhase = "start"
	// StreamPhaseDelta is an incremental chunk: a token of the reply, or an
	// interim transcript. Zero or more, in seq order, between Start and the
	// terminal.
	StreamPhaseDelta StreamPhase = "delta"
	// StreamPhaseComplete is the normal terminal frame. It carries the final
	// assembled text (token streaming) / final transcript (audio streaming) plus
	// any completion metadata, and is marked terminal so the consumer closes.
	StreamPhaseComplete StreamPhase = "complete"
	// StreamPhaseError is the abnormal terminal frame: the stream ended on a
	// provider/transport error. Carries the error string; terminal.
	StreamPhaseError StreamPhase = "error"
	// StreamPhaseCancel is the caller-initiated terminal frame (the client
	// hung up / the turn was pre-empted). Distinguished from Error so the
	// consumer can surface "cancelled" rather than "failed"; terminal.
	StreamPhaseCancel StreamPhase = "cancel"
)

type StreamPublisher

type StreamPublisher struct {
	// contains filtered or unexported fields
}

StreamPublisher is the worker-side producer. It appends ordered chunks for a single stream to the durable substrate; each Append returns the assigned Seq. Append is a synchronous durable write, which is the backpressure floor: the producer cannot outrun the durable path. A StreamPublisher is single-stream and intended to be driven by one producing goroutine; concurrent Append calls are serialized by the substrate's per-key seq allocator but may interleave in seq order, so a single producer per stream is the contract.

func NewStreamPublisher

func NewStreamPublisher(substrate DeliverySubstrate, streamID, originNode string) *StreamPublisher

NewStreamPublisher builds a producer for streamID over the substrate. originNode is stamped on each row for audit only (never used for addressing).

func (*StreamPublisher) Append

func (p *StreamPublisher) Append(ctx context.Context, topic, data string) (int64, error)

Append publishes one non-terminal chunk and returns its assigned Seq. The returned Seq is the stream-ordering position the consumer will deliver in.

func (*StreamPublisher) Close

func (p *StreamPublisher) Close(ctx context.Context, topic, data string) (int64, error)

Close publishes the terminal chunk that ends the stream. data may be empty (token streams carry the final text in their preceding chunks); pass it when the terminal itself carries a final payload. After Close the consumer closes its output channel.

func (*StreamPublisher) Fail

func (p *StreamPublisher) Fail(ctx context.Context, topic, errMsg string) (int64, error)

Fail publishes a terminal error chunk. The consumer surfaces Err and closes its output channel. It is the abnormal-end counterpart to Close.

func (*StreamPublisher) StreamID

func (p *StreamPublisher) StreamID() string

StreamID returns the stream id this publisher produces to.

type StreamSession

type StreamSession struct {
	// contains filtered or unexported fields
}

StreamSession is the producer-side handle for ONE lifecycle stream, owned by the node producing it (the agent node for token streaming; the voice node for audio streaming). It wraps a StreamPublisher and drives the Start -> Delta* -> terminal lifecycle. Single-producer per stream, exactly like StreamPublisher.

Each method is a synchronous durable Publish, which is the backpressure floor (the producer cannot outrun the durable path); the consumer reads in seq order and Acks, so a slow consumer back-pressures via the bounded channel rather than unbounded buffering.

func NewStreamSession

func NewStreamSession(substrate DeliverySubstrate, streamID, topic, originNode string) *StreamSession

NewStreamSession opens a producer-side lifecycle session for streamID over the substrate. topic is the logical chunk topic stamped on every frame ("token" for the reply token stream, "transcript" for the STT stream). originNode is audit-only (never used for addressing).

func (*StreamSession) Cancel

func (s *StreamSession) Cancel(ctx context.Context) (int64, error)

Cancel publishes the caller-initiated terminal frame (client hung up / turn pre-empted). The consumer surfaces a cancel terminal and closes its channel.

func (*StreamSession) Complete

func (s *StreamSession) Complete(ctx context.Context, finalText string, meta map[string]any) (int64, error)

Complete publishes the normal terminal frame carrying the final assembled text (token streaming) / final transcript (audio streaming) and optional completion metadata. After Complete the consumer closes its channel.

func (*StreamSession) Delta

func (s *StreamSession) Delta(ctx context.Context, data string) (int64, error)

Delta publishes one incremental chunk -- a reply token or an interim transcript. Returns the assigned Seq (the consumer's delivery order). Deltas before Start or after a terminal are rejected so the lifecycle stays well-formed.

func (*StreamSession) DeltaWithMeta

func (s *StreamSession) DeltaWithMeta(ctx context.Context, data string, meta map[string]any) (int64, error)

DeltaWithMeta publishes one incremental chunk carrying per-delta metadata (the audio-transcript path's isFinal / confidence flags). It is the Delta variant for streams whose deltas are not pure text -- the meta rides the frame envelope (JSON encoding when present) and the consumer reads frame.Meta. When meta is empty it is exactly Delta (the cheap compact encoding).

func (*StreamSession) Fail

func (s *StreamSession) Fail(ctx context.Context, errMsg string) (int64, error)

Fail publishes the abnormal terminal frame carrying an error string. The consumer surfaces Err and closes its channel.

func (*StreamSession) Start

func (s *StreamSession) Start(ctx context.Context, meta map[string]any) (int64, error)

Start publishes the opening frame carrying the stream's metadata (the turn / request descriptor: provider, format, sample rate, etc.). It must be called once, before any Delta. meta may be nil.

func (*StreamSession) StreamID

func (s *StreamSession) StreamID() string

StreamID returns the logical stream id this session produces to.

type SubscribeOption

type SubscribeOption func(*subscribeOptions)

SubscribeOption customizes one Subscribe call.

func WithReplayBacklog

func WithReplayBacklog() SubscribeOption

WithReplayBacklog opts this subscription into first-subscribe backlog replay (memql#1328): a BRAND-NEW consumer (no cursor row) starts from seq 0 and replays the retained backlog instead of initializing its cursor at the key's current high watermark. Reserve it for subscriptions that genuinely need first-subscribe history -- per-request stream/RPC keys, where the "backlog" is the head of the very exchange being consumed and the producer legitimately starts before the consumer subscribes. Long-lived keys with per-pod consumer ids (the chat-reply space subscriptions) must NOT use it: every rollout mints brand-new consumers, and replaying a full retention window of history to each fresh pod is exactly the failure mode the high-watermark default removes. An EXISTING cursor row always wins over either mode (the consumer resumes exactly).

type Substrate

type Substrate struct {
	// contains filtered or unexported fields
}

Substrate is the concrete DeliverySubstrate. It owns the durable store, the optional mesh fast-path hint surface, and a registry of live subscriptions.

Dedup scope (ADR 4.2): the substrate reuses eventDedup -- the SAME primitive the mesh uses -- but the dedup WINDOW IS PER-SUBSCRIPTION. A given consumer applies whichever copy of an EventID reaches it first (the mesh fast-path hint or its own durable pull) and drops the other; a DIFFERENT consumer replaying the same key (e.g. a BFF replica taking over) gets its own clean window and re-receives the rows after its cursor. A process-wide dedup would wrongly suppress a legitimate replay to a new owner -- the very catch-up the design exists to guarantee -- so dedup is scoped to the subscription, exactly as the ADR frames it ("consumers dedup by event-id before acting").

The cursor advances on the DURABLE path only (ADR 4.5 rule 4): a mesh hint short-circuits latency and feeds the per-sub dedup window, but the local replay position only moves as durable rows are read, so a dropped durable row can never be skipped by a mesh copy on replay.

func NewSubstrate

func NewSubstrate(store outboxStore, dedupTTL time.Duration, fastPath meshFastPath, logger *slog.Logger) *Substrate

NewSubstrate builds a delivery substrate over the given durable store. dedupTTL sets each subscription's cross-path dedup window (mirror the mesh's eventDedup TTL); fastPath is the optional mesh hint surface (nil = durable-only, which still delivers -- mesh-down is a supported mode per ADR 4.5 rule 3).

func NewSubstrateWithMeshFastPath

func NewSubstrateWithMeshFastPath(dbGetter func() *bun.DB, fastPath meshFastPath, logger *slog.Logger) *Substrate

NewSubstrateWithMeshFastPath builds a production DeliverySubstrate over the cluster Postgres outbox WITH the mesh fast-path wired (memql#1289). The durable outbox is still the delivery guarantee; fastPath (the EventBridge) is the best-effort low-latency hint surface that wakes a cross-node subscriber instantly instead of waiting for the durable-poll floor. The hint carries the same EventID as the durable row, so the per-subscription dedup collapses the two paths to exactly one delivery and the cursor advances on the durable path only (ADR 4.5). This is the wiring entry point app/cluster.go uses on every mesh node; the matching SetFastPathSink(substrate.HandleFastPath) on the same EventBridge feeds inbound hints back into this node's subscriptions.

func NewSubstrateWithStore

func NewSubstrateWithStore(dbGetter func() *bun.DB, logger *slog.Logger) *Substrate

NewSubstrateWithStore builds a production DeliverySubstrate over the cluster Postgres outbox (delivery_store_pg.go), durable-only (no mesh fast-path). This is the wiring entry point used by app/cluster.go for the chat-reply migration (memql#1264): the durable outbox alone is the cross-replica delivery guarantee. dbGetter is resolved lazily so the substrate can be constructed before the database component is up. dedupTTL mirrors the mesh's eventDedup window (defaultDedupTTL) so a per-subscription cross-path dedup behaves consistently with the rest of the mesh.

func (*Substrate) Ack

func (s *Substrate) Ack(ctx context.Context, key RoutingKey, consumerID string, seq int64) error

Ack advances the persisted cursor for (key, consumerID) (ADR 4.4). The store enforces monotonicity, so an out-of-order or stale Ack is harmless.

func (*Substrate) HandleFastPath

func (s *Substrate) HandleFastPath(d Deliverable)

HandleFastPath is the mesh-side entry point for a fast-path copy (ADR 4.5). The mesh EventBridge calls this when it receives an EventForward that carries a substrate deliverable. The hint is routed to every live local subscription of the key: each delivers it to its consumer for LATENCY and records the EventID in its own dedup window so its later durable pull of the same row is dropped (first path wins). The hint NEVER advances a cursor -- only the durable read does (rule 4) -- so a mesh copy can short-circuit latency but never correctness. It is fire-and-forget: a hint for a key with no live local subscriber is simply discarded (the durable pull still delivers).

func (*Substrate) Publish

func (s *Substrate) Publish(ctx context.Context, d Deliverable) (int64, error)

Publish appends to the durable outbox (the guarantee) and then best-effort emits the mesh fast-path hint (the latency optimization). See ADR 4.1-4.3.

func (*Substrate) Subscribe

func (s *Substrate) Subscribe(ctx context.Context, key RoutingKey, consumerID string, opts ...SubscribeOption) (<-chan Deliverable, error)

Subscribe returns an ordered, replayed, deduped stream for one logical key (ADR 4.4, amended by memql#1328). An existing consumer replays from its cursor; a brand-new consumer starts at the key's current high watermark unless WithReplayBacklog is passed. It then tails live, waking on a Publish to this key, a mesh fast-path hint, or the poll interval. The channel closes on ctx cancellation.

The starting position is resolved SYNCHRONOUSLY, before Subscribe returns, so the high-watermark init can never race a producer into losing events: every deliverable produced after Subscribe returns has seq above the pinned watermark and is delivered; deliverables produced before it MAY be skipped (that is the point of the default). Callers that publish a row and then expect it on a subscription they opened FIRST keep that guarantee verbatim.

type SubstrateRPC

type SubstrateRPC struct {
	// contains filtered or unexported fields
}

SubstrateRPC layers request/response RPC onto the durable DeliverySubstrate (memql#1265, Phase 2 of epic memql#1259). The substrate already provides the four delivery guarantees (logical addressing, at-least-once + idempotent dedup, per-key ordering, replay/catch-up); RPC adds exactly the two things the epic's decision 3 names on top of them -- a correlation id and reply-routing BY LOGICAL KEY -- and nothing else. No new wire format, no new broker: a request is a Deliverable published to the callee's logical key and a reply is a Deliverable published to the caller's reply key.

The whole point is that BOTH legs are addressed by logical key, never by a physical node-id (epic decision 2). A caller does not say "send the reply to bff-pod-3"; it says "send the reply to <replyKey>", and whichever replica owns that key -- including a DIFFERENT replica than the one that issued the Call, if the caller restarted or the key moved -- replays the reply from its cursor and matches it back to the pending call by correlation id. This is the star-topology delivery bug removed by construction for the request/response pattern, the same way memql#1263 removed it for the event pattern.

Contract realization (all inherited from the substrate, not reinvented):

  • Logical addressing (ADR 4.1): the request rides calleeKey, the reply rides replyKey; neither is a node-id.
  • At-least-once + idempotent dedup (ADR 4.2): both legs are substrate Publishes carrying a stable EventID, deduped per-subscription so a mesh fast-path copy and the durable pull collapse to one delivery.
  • Per-key ordering (ADR 4.3): inherited; RPC does not rely on cross-call ordering, only on the substrate delivering each reply at least once.
  • Replay/catch-up (ADR 4.4): a reply produced while the caller's reply-key owner is mid-reconnect is replayed from the cursor when the owner returns -- this is what makes a reply survive a caller-side restart.

SubstrateRPC adds ONLY: the correlation id, the reply-key plumbing, and an in-process pending-waiter table keyed by correlation id. At-most-once semantics for the CALLER are achieved by the waiter firing exactly once (the first delivered reply for a correlation id wins; the substrate's per-sub dedup already collapses duplicate copies of the same reply EventID, and the waiter is removed on first match so a late genuinely-distinct replay is dropped as "no waiter").

func NewSubstrateRPC

func NewSubstrateRPC(substrate DeliverySubstrate, selfKey RoutingKey, consumerID string, logger *slog.Logger) *SubstrateRPC

NewSubstrateRPC builds an RPC client/server over the substrate. selfKey is this process's logical reply address; consumerID is the stable logical consumer id used for the reply-key cursor (mirror the caller's stable identity so a restart replays unacked replies). The reply pump is started lazily on the first Call/Serve via ensureStarted, bound to the context passed there, so a never-used RPC client opens no subscription.

func (*SubstrateRPC) Call

func (r *SubstrateRPC) Call(ctx context.Context, calleeKey RoutingKey, method string, payload map[string]any) (map[string]any, error)

Call issues a request to calleeKey and blocks until the matching reply lands on selfKey (matched by correlation id) or ctx is done. The reply is routed by LOGICAL KEY: Call publishes the request carrying selfKey as the reply key, so the callee's reply finds whichever replica owns selfKey -- which may differ from this one if the key moves. Returns the response payload, or an error (ctx cancellation/timeout, a handler error surfaced by the callee, or a publish failure).

At-least-once on the wire, at-most-once for the caller: the substrate may deliver the reply more than once (durable pull + mesh fast-path), but the per-subscription dedup collapses copies of the same EventID and the waiter fires exactly once on the first match, after which it is removed.

func (*SubstrateRPC) Close

func (r *SubstrateRPC) Close()

Close tears down the reply pump. Idempotent. Safe to call from app shutdown.

func (*SubstrateRPC) Serve

func (r *SubstrateRPC) Serve(ctx context.Context, ownKey RoutingKey, handler RPCHandler) error

Serve subscribes to ownKey and dispatches every inbound request to handler, publishing the handler's result (or error) back to the request's reply key with the same correlation id. It runs until ctx is cancelled. ownKey is the logical address callers target with Call; consumerID for the request-side subscription is r.consumerID, so a Serve restart replays unacked requests from its cursor (a request produced while the handler replica was mid-restart is not lost -- ADR 4.4). Blocking; run it in a goroutine.

type TopologyReconciler

type TopologyReconciler struct {
	*component.Component
	// contains filtered or unexported fields
}

TopologyReconciler is the active topology reconciliation loop / fast reaper (epic #1871, issue #1874). It closes the gap left by the lazy heartbeat prune (pruneStaleClusterNodes, 10-min cron / 30-min window): a pod that crashes, is OOM-killed, or is forcibly replaced is never deregistered and lingers in v1:cluster:node for up to 30+ minutes. This loop drives topology freshness to SECONDS by comparing the DB node set against two live signals every few seconds and retiring nodes that are demonstrably gone:

  1. Mesh membership (liveness fast-reap). The live set is this node plus every healthy peer the PeerManager currently knows. Only roles covered by BFF direct discovery qualify; gossip-only siblings do not. A registered, non-stopped v1:cluster:node that is ABSENT from that live set -- continuously, for a short grace window -- is marked health="stopped". This catches the crashed-pod case the gossip StatusChangeHandler misses (it only authors transitions for directly-monitored peers; an unmonitored sibling that dies is merely reaped from the in-memory table, never marked down in the DB). Using the in-memory mesh as the liveness source -- rather than a DB lastSeen freshness heartbeat -- keeps this loop limited to terminal transitions. SelfStatusWriter separately refreshes each process for the slow sweep.

  2. Deployment status (orphan reap). Nodes whose deploymentId belongs to a deployment the deploy driver has marked superseded / failed / rolled_back (supersededDeployments, #1872/#1873) are retired IMMEDIATELY -- no grace -- so a superseded deployment's pods clear from topology within seconds of promotion completing.

Multi-node posture: every BFF replica runs this loop, but a Postgres session-level advisory lock (the CronLeader pattern, #561) elects ONE cluster-wide leader that actually reconciles -- so two replicas never double-retire the same node. Only BFF owns the unrestricted worker fan-out; identity and worker roles have narrower peer views that cannot prove a node globally absent. Failover is automatic: if the leader BFF pod dies its lock connection drops and Postgres releases the lease, and the next poll on a surviving replica re-acquires it.

Relationship to the 30-min prune: this loop OWNS the fast path (offline/ stopped within seconds). pruneStaleClusterNodes stays as the lazy backstop -- it only ever marks nodes the fast path missed, and both write the same idempotent terminal health="stopped" (staleClusterNodes excludes already-stopped rows), so they can never write conflicting statuses for the same node.

func NewTopologyReconciler

func NewTopologyReconciler(
	self *Identity,
	peerMgr *PeerManager,
	engine reconcileEngine,
	dbGetter func() *bun.DB,
	logger *slog.Logger,
) *TopologyReconciler

NewTopologyReconciler builds a reconciler only for BFF, whose peer view owns the unrestricted worker fan-out. Other roles return nil and must not contend for the global lease: missing peers in their narrower graphs are not absent from the cluster. A nil engine, nil peerMgr, or nil dbGetter degrades to a no-op loop (it still starts as a Dependency but reconciles nothing) so non-mesh / DB-less binaries and tests are tolerated. Interval + grace come from the env knobs with sane defaults.

func (*TopologyReconciler) Order

func (*TopologyReconciler) Order() int

Order runs the reconciler late in the startup sequence.

type WorkbenchBootstrap

type WorkbenchBootstrap struct{}

WorkbenchBootstrap creates dependencies for a Workbench node. Workbench nodes host the sandboxed per-Plan Linux working environment that agent nodes call into for headless work. They have: Engine + PeerManager + EventBridge + NodeServer. They do NOT have: HTTP server beyond healthchecks, voice pipeline, external client access, AiForward routing.

func (*WorkbenchBootstrap) Description

func (*WorkbenchBootstrap) Description() string

func (*WorkbenchBootstrap) NodeDependencies

func (*WorkbenchBootstrap) NodeDependencies(ctx BootstrapContext) ([]common.Dependency, error)

type WorkbenchForwardHandler

type WorkbenchForwardHandler interface {
	HandleForwardedRequest(ctx context.Context, req *nodev1.WorkbenchForwardRequest, send func(*nodev1.NodeServerMessage) error)
	CancelForwardedRequest(ctx context.Context, requestId string)
}

WorkbenchForwardHandler is the workbench-node-side entry point for a forwarded workbench dispatch. The nodeService invokes it when an inbound NodeClientMessage carries a WorkbenchForwardRequest. Single round-trip semantics (no streaming): the handler executes the action, builds a response, and uses `send` to deliver exactly one WorkbenchForwardResponse back to the originating agent node.

Left as an interface so component/node/ stays independent of the concrete workbench integration. The workbench node-type build's transport wires a concrete implementation via SetWorkbenchHandler.

type WorkbenchForwardResponseSink

type WorkbenchForwardResponseSink interface {
	Dispatch(resp *nodev1.WorkbenchForwardResponse)
}

WorkbenchForwardResponseSink is the agent-side receiver for responses to workbench dispatches originated here. Each inbound WorkbenchForwardResponse on a peer connection is routed through this sink. The agent-side WorkbenchForwardRouter satisfies this.

type WorkerDialer

type WorkerDialer struct {
	*component.Component
	// contains filtered or unexported fields
}

WorkerDialer opens outbound NodeService streams for AI and worker forwards. BFF nodes dial worker types; agent nodes narrow the set to sibling agents or workbench nodes so requests reach the replica that holds their state.

The dial set is reconciled from two sources:

  1. Static seeds (from MEMQL_WORKER_PEERS). Used for deterministic first-boot when the DB has not yet been populated -- e.g. in a multi-node cluster where workers boot in parallel with the BFF and may not have written their v1:cluster:node row by the time the first frontend request arrives.

  2. DB discovery: the dialer subscribes to cluster:node created/updated events AND polls every 30s, reconciling its connection table against healthy worker rows. New workers are dialed; workers that transition to OFFLINE/STOPPED or disappear from the DB are dropped.

Each active dial owns a *peerConnection (the same type ParentConnector uses). On NodeWelcome we register the peer as monitored in PeerManager and bind the *peerConnection onto its PeerEntry so AiForwardRouter / EventBridge / CapabilityRouter can find it.

func NewWorkerDialer

func NewWorkerDialer(
	identity *Identity,
	peerMgr *PeerManager,
	engine *memqlengine.MemQLEngine,
	eventBus *events.Bus,
	seeds []WorkerTarget,
	logger *slog.Logger,
) *WorkerDialer

NewWorkerDialer constructs a dialer. Returns nil if we would have nothing to do -- no seeds AND no engine/eventBus for DB discovery -- so callers can unconditionally append the result to the dependency list without branching.

peerMgr is required (it's where we publish discovered connections). identity is required for the NodeHello handshake on each dialed stream.

func (*WorkerDialer) ActiveAddresses

func (wd *WorkerDialer) ActiveAddresses() []string

ActiveAddresses returns the addresses this dialer currently has an outbound stream to. Primarily useful in tests and for operator diagnostics.

func (*WorkerDialer) IncludeDialTypes

func (wd *WorkerDialer) IncludeDialTypes(types ...NodeType)

IncludeDialTypes adds routes to a shared, narrowed dialer without dropping its existing routes. An unrestricted dialer already includes every type. Like SetDialTypes, call this during wiring, before Start.

func (*WorkerDialer) Order

func (*WorkerDialer) Order() int

Order returns the startup priority for the dialer.

func (*WorkerDialer) SetAiForwardResponseSink

func (wd *WorkerDialer) SetAiForwardResponseSink(sink AiForwardResponseSink)

SetAiForwardResponseSink installs the sink that receives AiForwardResponse messages arriving on our managed connections. Called during bootstrap on BFF binaries; workers leave this nil. Thread-safe; may be called at any time.

func (*WorkerDialer) SetDeployControlForwardResponseSink

func (wd *WorkerDialer) SetDeployControlForwardResponseSink(sink DeployControlForwardResponseSink)

SetDeployControlForwardResponseSink installs the sink that receives DeployControlForwardResponse messages arriving on our managed connections (memql#3380). Called during bff bootstrap -- the bff dials the identity node for the deploy surface. Thread-safe; may be called at any time.

func (*WorkerDialer) SetDialTypes

func (wd *WorkerDialer) SetDialTypes(types ...NodeType)

SetDialTypes narrows the dialer to a specific set of worker types. An empty or nil slice restores the default (dial all worker types). Intended for workers that need a limited peer mesh -- e.g. cognition only needs outbound streams to agent peers. Must be called before Start() for the filter to apply to the initial reconcile.

func (*WorkerDialer) SetWorkbenchForwardResponseSink

func (wd *WorkerDialer) SetWorkbenchForwardResponseSink(sink WorkbenchForwardResponseSink)

SetWorkbenchForwardResponseSink installs the sink that receives WorkbenchForwardResponse messages arriving on our managed connections. Called during agent-node bootstrap when cluster-mode workbench forwarding is enabled; other node types leave nil. Thread-safe; may be called at any time.

func (*WorkerDialer) SetWorkerForwardResponseSink

func (wd *WorkerDialer) SetWorkerForwardResponseSink(sink WorkerForwardResponseSink)

SetWorkerForwardResponseSink installs the sink that receives WorkerForwardResponse / WorkerForwardStream messages arriving on our managed connections (memql#4352). Called during agent-node bootstrap; other node types leave it nil. Thread-safe; may be called at any time.

type WorkerForwardHandler

type WorkerForwardHandler interface {
	HandleForwardedRequest(ctx context.Context, req *nodev1.WorkerForwardRequest, send func(*nodev1.NodeServerMessage) error)
	CancelForwardedRequest(ctx context.Context, requestId string)
	// HandleForwardedModelCall is the same hop for a ModelCall (epic
	// memql#4676). It lives on this interface rather than a parallel one
	// because it is the same subsystem answering about the same machines
	// through the same install point -- a second interface would be a
	// second thing for a node's cluster wiring to forget to install, and
	// forgetting it presents as "that model is offline" on exactly half
	// the requests.
	HandleForwardedModelCall(ctx context.Context, req *nodev1.ModelForwardRequest, send func(*nodev1.NodeServerMessage) error)
	CancelForwardedModelCall(ctx context.Context, requestId string)
	// HandleForwardedModelPull is the same hop for a model pull (epic
	// memql#5103). It joins this interface for the reason the ModelCall
	// half did -- one subsystem, one install point -- and the cost of
	// getting that wrong is worse here: a pull is named by a PERSON who
	// pressed a button on one machine's page, so a hop that is not wired
	// does not degrade to another machine, it simply refuses.
	HandleForwardedModelPull(ctx context.Context, req *nodev1.ModelPullForwardRequest, send func(*nodev1.NodeServerMessage) error)
	CancelForwardedModelPull(ctx context.Context, requestId string)
	// HandleForwardedModelProbe is the same hop for a model probe (epic
	// memql#5146). It joins here for the reason the pull did, and the cost of
	// an unwired hop is the same shape: a probe names ONE machine, because a
	// measurement of a different machine answers a question nobody asked, so an
	// unwired hop does not degrade -- it refuses.
	HandleForwardedModelProbe(ctx context.Context, req *nodev1.ModelProbeForwardRequest, send func(*nodev1.NodeServerMessage) error)
	CancelForwardedModelProbe(ctx context.Context, requestId string)
	// HandleForwardedAppCall is the same hop for one app-door call -- a chat,
	// structured or vision turn answered by a signed-in Claude Code or Codex
	// (the planner/app-source design, section 3a). It joins this interface
	// for the ModelCall half's reason: the same subsystem answering about the
	// same machines through the same install point. An unwired hop refuses
	// before start, so the caller's chain moves on to its next source.
	HandleForwardedAppCall(ctx context.Context, req *nodev1.AppCallForwardRequest, send func(*nodev1.NodeServerMessage) error)
	CancelForwardedAppCall(ctx context.Context, requestId string)
	// HandleForwardedAppSignIn tells the machine one Fleet sign-in action. It joins
	// here for the ModelCall half's reason: one install point for one subsystem.
	HandleForwardedAppSignIn(ctx context.Context, req *nodev1.AppSignInForwardRequest, send func(*nodev1.NodeServerMessage) error)
}

WorkerForwardHandler is the agent-replica-side entry point for a dispatch forwarded from ANOTHER agent replica (memql#4352). A cockpit machine's WorkerService stream terminates on exactly one replica; the turn wanting to use it is served wherever the mesh routed the request. This is how the two are joined.

Left as an interface so component/node stays independent of the concrete worker integration, which is build-tagged `agent`. The agent binary's cluster wiring installs a concrete implementation via SetWorkerForwardHandler; every other node type leaves it nil and answers a forward with a transport error rather than dropping it -- the sender is parked on the reply, and a dropped message turns "this node cannot serve that machine" into a hang.

type WorkerForwardResponseSink

type WorkerForwardResponseSink interface {
	Dispatch(resp *nodev1.WorkerForwardResponse)
	DispatchStream(chunk *nodev1.WorkerForwardStream)
	// The ModelCall half of the same hop (epic memql#4676).
	DispatchModel(resp *nodev1.ModelForwardResponse)
	DispatchModelDelta(delta *nodev1.ModelForwardDelta)
	// The model-pull half (epic memql#5103).
	DispatchModelPull(resp *nodev1.ModelPullForwardResponse)
	DispatchModelPullProgress(p *nodev1.ModelPullForwardProgress)
	// The model-probe half (epic memql#5146).
	DispatchModelProbe(resp *nodev1.ModelProbeForwardResponse)
	DispatchModelProbeProgress(p *nodev1.ModelProbeForwardProgress)
	// The app-door call half (the planner/app-source design, section 3a).
	DispatchAppCall(resp *nodev1.AppCallForwardResponse)
	// The app sign-in half: the machine was told, or why not.
	DispatchAppSignIn(resp *nodev1.AppSignInForwardResponse)
}

WorkerForwardResponseSink is the originating replica's receiver for replies to worker dispatches it forwarded. Both the single response and the streamed chunks route through it, keyed by request_id.

type WorkerPeerIssue

type WorkerPeerIssue struct {
	// Entry is the offending text, verbatim (trimmed) as the operator wrote
	// it, so it can be grepped for in the deployment manifest.
	Entry string
	// Reason is an operator-facing explanation of why it was rejected.
	Reason string
}

WorkerPeerIssue is one MEMQL_WORKER_PEERS entry that produced no dial target, paired with why. ParseWorkerPeers returns these instead of logging so it stays pure and trivially testable; the caller (app.workerPeerSeeds) turns each one into a boot-time warning.

memql#3450: rejected entries used to be dropped in silence, which meant a documented configuration could do nothing while looking correct.

type WorkerTarget

type WorkerTarget struct {
	NodeType NodeType
	Address  string
	NodeId   string
}

WorkerTarget is a (type, address) pair the WorkerDialer knows to dial. NodeId may be empty for targets sourced from a static env-var seed (we learn the real NodeId from NodeWelcome on the first handshake) and populated for targets sourced from a v1:cluster:node DB row.

Directories

Path Synopsis
gen module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址