automations

package module
v0.0.0-...-86254be Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 14, 2026 License: Apache-2.0 Imports: 54 Imported by: 3

Documentation

Overview

budget.go

Global automation-execution budget (memql#1142). This is the automation- side analogue of the LLM kill-switch (memql#1145): a HARD ceiling that STOPS a storm, not a log line that merely observes one.

Before this, the only thing standing between a misfiring automation and an unbounded re-fire loop was: (a) storm detection that was LOG-ONLY (>20/min per automation -> a WARN, nothing blocked), and (b) a concurrency limiter that caps how many run AT ONCE but not how many run PER MINUTE. A failing automation (e.g. an emit-failed-card automation that re-fires on its own failure, or a plan-level loop that re-creates a plan each cycle) could therefore execute hundreds of times a minute, and each execution can drive fresh LLM calls / plan churn.

The budget is a process-global, cross-executor ceiling: both the event and schedule executors (built separately in scheduler.go) share ONE budget instance, so the cap is truly global. Three dimensions, all windowed:

  • a TOTAL executions/window ceiling across every automation, and
  • a per-automation executions/window ceiling, and
  • a per-(automation, row) ceiling for graph events,

whichever trips first skips the execution with a clear status. Defaults are generous for real traffic (a healthy cluster runs well under them) and lethal to a storm. Even when the budget is generous, the LLM kill-switch (memql#1145) is the hard money stop behind it -- defense in depth.

Package automations provides an automation engine for MemQL. Automations support multi-step processes with conditional branching, iteration, parallel execution, and data flow between steps.

Index

Constants

View Source
const (
	// WaitKindApproval is an existing kind: a person decides.
	WaitKindApproval = "approval"
	// WaitKindRetry is a run the sweep should re-dispatch unchanged. The
	// symptom said the failure was a blip.
	WaitKindRetry = "retry"
	// WaitKindReplan is a run whose remaining plan is wrong from the failed
	// step on. The planner's sweep re-plans the gap, keeping the prefix.
	WaitKindReplan = "replan"
	// WaitKindRepair is a run whose step did something other than what it
	// promised. The failed step re-runs with the violation as guidance.
	WaitKindRepair = "repair"
)

failureWaitKinds are the `waitingOn.kind` values this path writes. They are constants because the sweep on the other side matches them, and a typo would leave a run waiting for something no sweep looks for.

View Source
const (
	// RerunReasonRerun is a person running one step again, as a new version.
	RerunReasonRerun = "rerun"
	// RerunReasonHeadMove is the re-run a head move starts when the version
	// it made current has no matching downstream: it re-runs the first stale
	// step and everything after it.
	RerunReasonHeadMove = "headMove"
	// RerunReasonBranch is a fork run's first execution: the fork step and
	// what follows run live, the prefix is served from the source by
	// reference.
	RerunReasonBranch = "branch"
	// RerunReasonReplan is a re-planned run's first execution on its new
	// template (memql#5664): the request names the template's first step the
	// run never reached, the completed prefix is served from the run's own
	// rows, and the new steps run, each as its first version. It is served
	// exactly as a re-run is -- under a claim of its own, so the lease of the
	// execution that failed does not strand it -- and asks for nothing but
	// where to start.
	RerunReasonReplan = "replan"
)

Re-run reasons, as run.rerun.reason carries them.

View Source
const (
	// TopicRunRequest carries an operator-initiated run from the node that
	// received the request to the node type that should execute it.
	//
	// The "automationrun." prefix is NOT cosmetic. defaultRoutingRules blocks
	// "automation.#" from crossing the mesh -- automation lifecycle chatter is
	// deliberately node-local -- and these topics must cross it. They are a
	// separate tree with their own forward rule for that reason; renaming them
	// under "automation." would re-block them, silently, with a green suite.
	TopicRunRequest = "automationrun.request"

	// TopicRunTrace carries one trace frame back from the executing node to
	// the node that received the request.
	TopicRunTrace = "automationrun.trace"
)
View Source
const (
	RunCodeOK                = 0
	RunCodeInvalidArgument   = 3
	RunCodeDeadlineExceeded  = 4
	RunCodeNotFound          = 5
	RunCodePermissionDenied  = 7
	RunCodeResourceExhausted = 8
	RunCodeFailedPrecond     = 9
	RunCodeInternal          = 13
	RunCodeUnavailable       = 14
)

Canonical gRPC status codes (google.rpc.Code numbering), duplicated here as plain ints so this package -- which is its own Go module and has no reason to depend on grpc -- can report a code the stream handler passes through verbatim. The stream surface carries failures IN the reply rather than as a returned error, because an error out of a multiplexed stream handler tears the whole stream down (the convention deploy_control_handlers.go set).

View Source
const (
	ModeSingle   = "single"
	ModeQueued   = "queued"
	ModeRestart  = "restart"
	ModeParallel = "parallel"
)

The four modes.

View Source
const (
	// TriggeredBySchedule is a cron firing.
	TriggeredBySchedule = "schedule"
	// TriggeredByEventPrefix opens an event firing's triggeredBy,
	// "event:<topic>". It is used inside a SQL LIKE pattern, so it must never
	// hold %, _, a quote or a backslash (TestTheSystemRunPolicySelectsTheSchedulersOwnFirings).
	TriggeredByEventPrefix = "event:"
)

The triggeredBy spellings of the core scheduler's own firings, written onto every run row its journal opens. Named once because a reader outside this package selects runs by them: integrations/work's operational retention retires finished, unowned, goalless runs fired by either (MEMQL_WORK_SYSTEM_RUN_RETENTION_DAYS), and a run carrying another spelling is not a system run to it.

View Source
const CancelPollInterval = 2 * time.Second

CancelPollInterval bounds how often a run asks whether it has been cancelled. Two seconds is under human reaction time for a stop button and far above the rate at which steps complete, so it costs a handful of reads on any run long enough for anyone to want to stop it.

View Source
const (
	ClusterGuardComponentName = common.ComponentName("automationClusterGuard")
)

ClusterExecutionGuard makes EVENT-triggered automations exactly-once across the cluster, and makes any double-fire observable (#561).

When a MemQL node-type runs >=2 replicas, the same triggering event can reach more than one replica (the EventBridge dedup is per-process). Each replica calls Claim(automation, dedupKey) before executing; the claim inserts a row keyed by (automation_name, dedup_key) whose PRIMARY KEY lets exactly one replica win -- the others skip. dedupKey is the automation's InitialChainHead (a deterministic fingerprint of the triggering event), so the same event yields the same key on every replica.

OBSERVABILITY -- the whole point of the safety net:

  • every cross-replica duplicate that gets PREVENTED is WARN-logged with the automation + key + node, and counted (DuplicatesPrevented). A rising count proves events ARE reaching multiple replicas AND that we are correctly collapsing them to one execution.
  • a claim that can't reach the DB executes UNGUARDED (fail-open: never drop legitimate work) but is WARN-logged + counted (ClaimErrors) so the double-fire WINDOW is never silent.
  • the claim rows themselves (claimed_by + claimed_at) are an audit trail: a true double-fire is impossible at the DB (the PK rejects the second insert), so any suspected double-fire is investigable from the table.
  • a periodic summary line logs the running counts.

Started as a Dependency: a background loop prunes expired claim rows and emits the summary.

View Source
const ComponentName = common.ComponentName("automations")

ComponentName identifies this package in logs and environment variable lookups.

View Source
const (
	CronLeaderComponentName = common.ComponentName("automationCronLeader")
)

CronLeader elects a single cluster-wide owner of SCHEDULED (cron) automations via a Postgres session-level advisory lock (znasllc-io/memql#561).

Why: every MemQL node runs the automation scheduler, and a cron entry fires independently on each one -- so the daily purges + the */5min feedback sweep run once PER NODE TYPE today, and would run once per REPLICA too. Event-triggered automations are unaffected (the mesh routes each event to one pod), so only the schedule path needs gating.

Mechanism: each node holds one dedicated DB connection (resolved via the injected dbGetter -- wired to the DIRECT, non-pooled endpoint in app/, since the lock is held for the leader's lifetime and a transaction-mode pooler would recycle the backend out from under it, memql#1930) and calls pg_try_advisory_lock(key) on it. Exactly one node acquires the lock and becomes the leader; the scheduler only executes cron firings on the leader (see Scheduler.LeaderGate). The lock is session-scoped, so if the leader pod dies its connection drops and Postgres releases the lock automatically -- another node's poll then takes over (failover). When no DB is reachable the node is simply not leader (fail-closed: better to skip a maintenance cron than double-run a non-idempotent one).

View Source
const DefaultBreakerThreshold = 5

DefaultBreakerThreshold is the consecutive-failure count at which an authored automation's breaker trips. Small on purpose: an authored automation that fails several times in a row is misbehaving, and pausing it early limits blast radius (wasted runs, log noise, downstream side effects).

Variables

View Source
var (
	// ErrRerunStepInvalid refuses a re-run whose target is not one of the
	// automation's own top-level steps. It wraps work.ErrNestedStep or
	// work.ErrStepNotInRun, which say which.
	ErrRerunStepInvalid = errors.New("the re-run names a step this run cannot re-run")
	// ErrForkPrefixUnavailable refuses a run whose prefix cannot be served by
	// reference: the run holding a step's version was not loaded, never
	// finished the step, or has moved on to another version of it since.
	// Executing the step instead would break the one promise a branch makes.
	ErrForkPrefixUnavailable = errors.New("a step before the re-run cannot be served from the run that holds its version")
)
View Source
var (
	// ErrRunNotFound is returned when no v1:work:run row carries the id.
	ErrRunNotFound = errors.New("work run not found")
	// ErrRunJournalInvalid is returned when the journal cannot be resumed
	// from: no run, no automation name, or no unfinished step.
	ErrRunJournalInvalid = errors.New("work run journal is invalid")
	// ErrAutomationChanged is returned when the automation's definition
	// fingerprint moved since the run started.
	ErrAutomationChanged = errors.New("automation definition changed since the run started")
	// ErrNonRetryableStep is returned when the resume point has an external
	// effect and AllowSideEffects was not set.
	ErrNonRetryableStep = errors.New("step has an effect or no proven replay-safe classification")
	// ErrResumeArgsContract is returned when the run's stored variables do
	// not satisfy the automation's args contract. It is decided before any
	// step runs and a second attempt binds the same variables to the same
	// contract, so the dispatcher fails the run with it (memql#5664) rather
	// than leaving it at `running` for the abandoned sweep to misname.
	ErrResumeArgsContract = errors.New("resume args contract violation")
	// ErrResumeResultMissing means a completed producer's bound value was
	// not retained. Repeating its effect is not a way to reconstruct evidence.
	ErrResumeResultMissing = errors.New("completed step result is unavailable for resume")
)
View Source
var ErrJournalRequired = errors.New("required work journal unavailable")

ErrJournalRequired means execution stopped at an unconfirmed journal boundary. The last effect may have happened; callers must reconcile it before replaying. It is not an ordinary step error that retry or an error handler can safely ignore.

Functions

func AdvanceChain

func AdvanceChain(prevHead, contentId string) string

AdvanceChain computes the next chain head after a step.

func AuthorContext

func AuthorContext(ctx context.Context, ownerUserId string) context.Context

AuthorContext returns a ctx carrying the AUTHOR's authz envelope: an AccessContext + JWT-style claims stamped with the author's userId and a non-privileged writer role. Per-row authz reads this to confine the run to what the author may touch. Exported so the app's RunAutomationFunc wiring and tests can build the same envelope.

Deliberately NOT the system actor (role:system) the core scheduler injects: authored automations must not get the engine-wide system bypass.

func BuildBeforeWriteHooks

func BuildBeforeWriteHooks(engine *memql.MemQLEngine, autos []*Automation, registry StepExecutorRegistry, logger *slog.Logger) map[string][]memql.BeforeWriteHook

BuildBeforeWriteHooks builds the same hooks used at boot and by the corpus runner.

func ChainTrackingEnabled

func ChainTrackingEnabled(exec *AutomationExecution) bool

ChainTrackingEnabled checks if an execution has chain tracking data.

func CheckArgs

func CheckArgs(automation *Automation, payload map[string]any) error

CheckArgs reports whether payload satisfies automation's declared args contract: the bind a run's resume makes against its stored variables before any step runs (ResumeFrom), asked of a template before it is installed on a run (memql#5664) -- a re-planned draft declaring an argument the run never had would otherwise be installed and refuse every resume. nil for an automation that declares no args.

func ComputeBranchFingerprint

func ComputeBranchFingerprint(parentStepId, branchName string, result *StepResult) string

ComputeBranchFingerprint creates a ContentId for a parallel branch.

func ComputeInitialChainHead

func ComputeInitialChainHead(automationName, triggeredBy string, triggeringEvent map[string]any, inputFingerprint string) string

ComputeInitialChainHead creates the starting chain state for an execution.

Feeds execution dedup and the cross-replica cluster guard, so it must be a pure function of the automation identity plus real inputs. See eventFingerprintData above for the wall-clock rule and what it costs to break it.

func ContextWithAutomationParent

func ContextWithAutomationParent(ctx context.Context, parentRunID string) context.Context

ContextWithAutomationParent records the synchronous caller's journal link. It grants no execution authority; resumed children must prove both directions.

func ContextWithStepObserver

func ContextWithStepObserver(ctx context.Context, obs StepObserver) context.Context

ContextWithStepObserver returns a context that carries obs, so every step recorded by an Execute* call descending from it is reported. A nil obs returns ctx unchanged.

func FingerprintInput

func FingerprintInput(input any) string

FingerprintInput creates a deterministic hash of a run's input (exec.Input).

An automation's run carries no input -- only a logic run records its arguments there -- so an automation's chain does not reach this. The wall-clock rule that governs every key in this file lives on eventFingerprintData above, next to the key that IS live -- an earlier revision of this change put it here, on the strength of "whoever builds the next key will start at the surviving helper", which was wrong: a reader of running code never lands here.

func FlattenLoopGraph

func FlattenLoopGraph(g *LoopGraph) ([]FlatAutomation, []FlatTrigger)

FlattenLoopGraph flattens LoopGraphRows(g), in the rows' order: automations by name, each one's outgoing edges by target, concept and topic.

func FormatValue

func FormatValue(v any) string

FormatValue renders a value as text: a string is itself, a scalar its canonical spelling, anything else its JSON.

func InstallBeforeWriteHooks

func InstallBeforeWriteHooks(engine *memql.MemQLEngine, autos []*Automation, registry StepExecutorRegistry, logger *slog.Logger)

InstallBeforeWriteHooks registers the compiled bodies on the writing engine. The normal sequence runner executes them with no work journal or lifecycle events. Adjustment attribution goes to the log store; no second row exists.

func IsStepRetryable

func IsStepRetryable(stepType StepType) bool

IsStepRetryable reports the types whose semantics alone prove they have no effect. Functions and composers additionally need stepRetryable's body walk.

func LintLoopTree

func LintLoopTree(logger *slog.Logger, root fs.FS) ([]memql.LintDiagnostic, error)

LintLoopTree applies the strict automation check to a mounted product bundle and the embedded tree. It restores the global mount and concept state.

func LoopGraphRows

func LoopGraphRows(g *LoopGraph) []map[string]any

LoopGraphRows is g as one row per automation, in the graph's own order (name, then origin). Every key a row has no value for is ABSENT rather than empty -- a scheduled automation has no trigger, and "" would read as a trigger nobody can describe -- except the three lists, which are always present, because a client reading `.length` must not guard for absence on the automations that write nothing, which are most of them.

A row:

name, stratum, template                 always
trigger, triggerKind, triggerConcept    an event-triggered automation
schedule                                a scheduled one
triggerFilter                           its @filter's source
writes, publishes, opaque               concept ids, topics, and the calls
                                        the graph cannot see into
edgesOut                                [{to, concept, topic, via, decided, reason}]
loop, mode                              {maxDepth, until}, {kind, max}
cycle                                   {members, permittedBy, path, permitted}
problem                                 the refusal the load would make
origin                                  the file it was loaded from

func NewLogger

func NewLogger() *slog.Logger

NewLogger creates a logger for the automations component using common.NewLogger. This ensures proper color support and component-level enable/disable via environment variables.

func NewOfflineEngine

func NewOfflineEngine(logger *slog.Logger, registry concept.Registry) (*memql.MemQLEngine, error)

NewOfflineEngine builds the function registry used by loop analysis without connecting a database or starting event subscribers.

func NotifyStepObserver

func NotifyStepObserver(ctx context.Context, result *StepResult)

NotifyStepObserver reports one step result to the context's observer, if any.

Exported because the observer is part of the LocalAutomationRunner contract (run_relay.go): anything standing in for the scheduler -- another dispatch implementation, or a test double in another package -- has to be able to drive the trace, and a contract only the executor can satisfy is not a contract. The executor's own call sites go through the unexported alias below.

func PrepareExpressions

func PrepareExpressions(a *Automation) error

PrepareExpressions parses every expression of an automation once and caches the nodes (see the file comment for where). Every path that turns compiled JSON into an executable Automation calls it -- the tree loader and the LogicRunner's statement logic -- and the executor prepares an automation built in Go before its first run (ensurePrepared).

An error names the step and the position: a parse error, or an expression whose static cost estimate exceeds tiers.MaxStaticCost.

PrepareExpressions has no concept registry, so it cannot check a trigger filter against its concept's declared field types; the loader, which has one, calls prepareExpressions with it.

func PrepareRerun

func PrepareRerun(run *RunJournal, sources []*RunJournal, automation *Automation) (*RunJournal, *ResumeOptions, error)

PrepareRerun builds what a run carrying a re-run request resumes from, and the options to resume it with.

The journal is the run's own -- its id, its variables, its head, its request -- with every step before the target that the head names by reference rehydrated from the run holding that version (sources, by id). A reference is served only if that run's newest row of the step is the version the head names and finished: a run that has since moved on no longer holds the result the head promises, and serving whatever it holds now would put an answer under the branch that its upstream never computed. A branch's every prefix step must resolve this way or the request is refused (ErrForkPrefixUnavailable) -- executing it instead is exactly what a branch promises not to do.

The resume point is rerunResumePoint's. The options allow side effects: the resume point of a re-run is a step the person asked to run again, which is a decision to run its effect again under a new idempotency key.

A goal run's variables came from the goal's caller, so its journal is marked caller-supplied whatever the row says (ExecuteAdopted's rule, memql#2888): a branch's row is written by the act, and an unmarked copy would promote the template's steps to internal origin on caller-chosen arguments.

func RerunSources

func RerunSources(run *RunJournal, automation *Automation) []string

RerunSources lists the runs a re-run's prefix is served from: every run the head names for a step before the target other than the run itself, and, for a branch step the head does not name, the run it was forked from. The dispatcher loads each (LoadRunJournal) and passes them to PrepareRerun.

func RunCancelRequested

func RunCancelRequested(ctx context.Context, exec journalExecutor, runId string) (asked bool, by string, err error)

RunCancelRequested is cancelRequested's read for a step that runs WHOLE where no step boundary can see it: an app session on somebody's machine (integrations/agent/worker), which stops on the answer the run's next boundary would give rather than on a second reading of the row. exec is the engine the read goes through, as the journal's own is.

IT RETURNS THE READ'S ERROR, and failing open is the caller's: a session watcher logs it and asks again at its next tick. A blank run id, a nil engine and a run with no row all answer false with no error.

func RunStatementBody

func RunStatementBody(ctx context.Context, key string, steps []*Step, stepCtx *StepContext, ev *Evaluator) (bool, any, error)

RunStatementBody runs a nested statement list -- a loop iteration's body, a parallel branch -- through the sequence runner of the automation it belongs to, over ev (a child frame the caller opened). key names the list within the step that holds it: `<step id>/<n>` for a `for`'s nth item, the block's id for a branch. It returns whether a return ended the list and with what value.

func StepDeterministicFingerprint

func StepDeterministicFingerprint(step *Step, result *StepResult) string

StepDeterministicFingerprint computes a content-addressed ID from the deterministic aspects of a step execution.

func ToMinimalStepResults

func ToMinimalStepResults(steps map[string]*StepResult) map[string]*MinimalStepResult

ToMinimalStepResults converts a map of full StepResults to MinimalStepResults. This reduces checkpoint size by omitting large payloads while preserving the data needed for evaluator rehydration.

func ToSlice

func ToSlice(v any) ([]any, error)

ToSlice reads a value as a list: a list is itself, and a query result is its bundle's nodes.

func TriggerRow

func TriggerRow(event *events.Event) memql.ExprRow

TriggerRow is the row a trigger filter's parameter binds: the triggering event read as a stored row.

A graph CDC event (graph.node.<action>.<concept>, executor_mutation.go) carries `id` / `nodeId`, `concept`, `nodeType`, `actor`, `createdAt` (on .created), the stored payload FLATTENED onto the top level -- so a payload key could shadow any of those -- and the full stored payload again under `payload`. So the row is built from the parts that cannot be shadowed where that is possible: its payload is the `payload` object, its id the `nodeId` alias, its concept the topic's concept segment, its creator the actor the bus stamped in the event metadata, and its createdAt the payload's `createdAt`, else the event's own time (an .updated event carries none).

Any other event has no row: its whole payload becomes the row's payload, with no intrinsics, so `row.<field>` still reads what the event carried.

func UnwrapStepResult

func UnwrapStepResult(v any) any

UnwrapStepResult peels the Bundle-wrapped engine envelope off a stored step result so what a later statement reads is the FLAT value the construct returned.

A `logic` or `query` call stores its raw engine result (a *memql.ExecuteResult) as the step Result. An object-literal `return { ... }` or a scalar return lands its value in the envelope's flat output payload; left on the envelope, `decide.x` resolved to nil at real-DB runtime, which blocked the decide->persist pattern (the two cognition logics in #2235; the engine-level root cause in #2271).

ONLY a flat-output result is unwrapped. A Bundle-backed result (a query's rows) is returned UNCHANGED: the run reads it as rows. Non-envelope values (a scalar bool from `return true`, a plain map) are also returned unchanged.

func V1Text

func V1Text(v any) string

V1Text is a v1 value as the text a string-typed position takes (a topic, a URL, an id): absent is "", and everything else is FormatValue's rendering.

func ValidateRunJournal

func ValidateRunJournal(j *RunJournal, automation *Automation, idEngine *id.Engine) error

ValidateRunJournal is the resume precondition: a run, a resume point, and an automation that has not changed underneath it.

func VerifyExecutionChain

func VerifyExecutionChain(exec *AutomationExecution) error

VerifyExecutionChain validates that an execution's chain is internally consistent, including nested child chains in forEach and parallel steps. Returns nil if the chain is valid, or an error describing the break point.

func WorkStepId

func WorkStepId(runId, stepKey string) string

WorkStepId is the v1:work:step row id the journal writes for one step key of one run: the run's short id and the sanitized key (journal.go's workStepId, which states why the id is composed rather than hashed).

EXPORTED FOR THE APP-SESSION DELEGATE (epic memql#5414). A step handed to an app arrives there carrying its run id and its step KEY, and the delegate stamps the subrun it opens onto the step ROW. Writing to the key named no row at all, so every delegated step silently kept no childRunId. The delegate derives the row id through this function rather than restating the formula, because a second copy is the one that stops agreeing the day the journal's id changes -- and nothing would report it, the stamp would just miss again.

Types

type ActionStepConfig

type ActionStepConfig struct {
	// Ref is the pinned-by-default action reference (id@version, or a bare id
	// to float to the latest version).
	Ref string `json:"ref"`
	// Args bind the action's parameters at invocation time.
	Args map[string]any `json:"args,omitempty"`
	// Surface is an optional explicit surface binding the resolver honors
	// before policy/availability.
	Surface string `json:"surface,omitempty"`
}

ActionStepConfig configures an action-library replay step (#1758, epic #1734).

type ArgsField

type ArgsField struct {
	// Name is the field name (e.g., "subject", "email")
	Name string `json:"name"`

	// Type is the expected type: "string", "number", "int"/"integer",
	// "bool"/"boolean", "object", "array", "any".
	Type string `json:"type"`

	// Optional indicates if the field can be missing (false means required)
	Optional bool `json:"optional,omitempty"`

	// Enum restricts the value to a fixed set (@enum on the args field).
	Enum []any `json:"enum,omitempty"`

	// MaxLength caps a string field's rune count (@maxLength). Zero =
	// unbounded. Enforced on string-typed fields only.
	MaxLength int `json:"maxLength,omitempty"`

	// Pattern is a regex a string field must match (@pattern). Empty = no
	// constraint. Enforced on string-typed fields only.
	Pattern string `json:"pattern,omitempty"`

	// Secret marks this args field as carrying a value the automation's
	// TRIGGER CONCEPT annotates `@secret`, so a rejected value on it is
	// replaced with redactedArgValue in the validation message instead of
	// being quoted into it -- and therefore out of the WARN log that message
	// lands in (memql#3183, epic memql#3111).
	//
	// NOT a compiler output and deliberately not `json:"secret"`-tagged for
	// input: the automation compiler emits no such field, and the flag is
	// derived, never authored. markSecretArgsFields stamps it at LOAD time
	// from the concept registry (see args_secret.go). The `json:"-"` keeps a
	// caller-supplied automation bundle from setting or CLEARING it by
	// hand -- an authoring-sandbox bundle unmarshals through the same
	// Automation struct, and `"secret": false` in submitted JSON must not be
	// able to un-redact a field the loader stamped.
	Secret bool `json:"-"`

	// Items defines the element schema for array-typed fields.
	Items *ArgsField `json:"items,omitempty"`

	// Nested contains nested field assertions for object types
	Nested []*ArgsField `json:"nested,omitempty"`
}

ArgsField defines a single field's type assertion.

type ArgsSchema

type ArgsSchema struct {
	// Fields declares the expected input fields and their types.
	Fields []*ArgsField `json:"fields,omitempty"`

	// AdditionalProperties controls whether payload keys the schema does not
	// declare are rejected. nil (the default) is tolerant-reader: unknown
	// payload fields are simply not bound (ADR Decision 2).
	AdditionalProperties *bool `json:"additionalProperties,omitempty"`
}

ArgsSchema is an automation's declared input contract -- the `args { }` block lowered by the compiler (event-payload-binding ADR Decision 1). It mirrors the field grammar logic/query already use.

type AuthoredBreaker

type AuthoredBreaker struct {
	// contains filtered or unexported fields
}

AuthoredBreaker holds one circuit breaker per (owner, automation). It is thread-safe; runs from different automations touch independent keys, so a storm of failures in one automation never contends with or trips another.

func NewAuthoredBreaker

func NewAuthoredBreaker(threshold int, onTrip BreakerOnTrip) *AuthoredBreaker

NewAuthoredBreaker builds a breaker. threshold <= 0 uses DefaultBreakerThreshold. onTrip may be nil (the breaker still tracks state and reports IsOpen, just with no pause side-effect).

func (*AuthoredBreaker) Allow

func (b *AuthoredBreaker) Allow(owner, name string) bool

Allow reports whether a firing for (owner, name) should run. A tripped (open) breaker returns false, so even if a stale subscription fires the run is suppressed -- belt-and-suspenders alongside the scheduler pausing it.

func (*AuthoredBreaker) IsOpen

func (b *AuthoredBreaker) IsOpen(owner, name string) bool

IsOpen reports whether (owner, name)'s breaker is tripped.

func (*AuthoredBreaker) RecordFailure

func (b *AuthoredBreaker) RecordFailure(subject BreakerSubject, errMsg string) bool

RecordFailure increments the consecutive-failure count for the subject's (owner, automation) and trips the breaker when it reaches the threshold. Returns true when THIS call is the one that opened the breaker (so the caller logs/surfaces it once). The onTrip callback fires once, outside the lock, on the tripping call, naming the subject's bundle and version.

func (*AuthoredBreaker) RecordSuccess

func (b *AuthoredBreaker) RecordSuccess(owner, name string)

RecordSuccess resets the consecutive-failure count for (owner, name). A success on a still-closed breaker clears any partial failure streak; it does NOT re-close an already-open breaker (that needs an explicit Reset, which re-activation performs) so a flapping automation can't auto-recover into a fault loop.

func (*AuthoredBreaker) Reset

func (b *AuthoredBreaker) Reset(owner, name string)

Reset clears the breaker state for (owner, name) -- a fresh start. The scheduler calls this on (re-)activation so a fixed automation runs again.

type AuthoredOwnerGate

type AuthoredOwnerGate struct {
	// contains filtered or unexported fields
}

AuthoredOwnerGate decides whether THIS node owns a given owner's scheduled authored automations, so an owner's schedule fires once cluster-wide (on its assigned node) rather than once per replica.

func NewAuthoredOwnerGate

func NewAuthoredOwnerGate(selfNodeId string, membership MembershipFunc) *AuthoredOwnerGate

NewAuthoredOwnerGate builds the gate. selfNodeId is this node's stable id (e.g. node.Identity.NodeId()); membership returns the live node set. A nil membership (or one that returns <=1 node) makes this node own every owner -- the single-node default.

func (*AuthoredOwnerGate) AssignedNode

func (g *AuthoredOwnerGate) AssignedNode(ownerUserId string) string

AssignedNode returns the node id that owns ownerUserId's authored automations under the current membership view. Exported for management surfaces ("which node runs owner X's automations"). Returns this node's id when membership is single-node.

func (*AuthoredOwnerGate) OwnsOwner

func (g *AuthoredOwnerGate) OwnsOwner(ownerUserId string) bool

OwnsOwner reports whether this node is the assigned runner for ownerUserId's scheduled authored automations. It is the func wired into the authored scheduler's owner gate. Owner with empty id is never owned (defensive -- an unowned firing should not run anywhere rather than everywhere).

type AuthoredRunnerOptions

type AuthoredRunnerOptions struct {
	Logger       *slog.Logger
	Engine       *memql.MemQLEngine
	EventBus     *events.Bus
	StepRegistry StepExecutorRegistry
	// ClusterGuard claims each EVENT firing per (automation, event) in the
	// shared database, so one event runs an authored automation once however
	// many replicas observe it as local. nil = no cross-replica claim (a single
	// node; tests); the per-process claim still applies. Typically the
	// *ClusterExecutionGuard the core scheduler's event executor claims through.
	ClusterGuard executionClaimer
}

AuthoredRunnerOptions configures the runner the authored scheduler fires through.

type AuthoredScheduler

type AuthoredScheduler struct {
	// contains filtered or unexported fields
}

AuthoredScheduler subscribes activated authored automations to their triggers and runs them under the author's authz envelope. It is thread-safe and mutable after startup (activations arrive at runtime).

func NewAuthoredScheduler

func NewAuthoredScheduler(opts AuthoredSchedulerOptions) (*AuthoredScheduler, error)

NewAuthoredScheduler builds the authored scheduler. Loader, EventBus and Run are required. A nil logger is tolerated.

func (*AuthoredScheduler) Activate

func (s *AuthoredScheduler) Activate(construct *memql.AuthoredConstruct) error

Activate compiles the authored construct's automation source and wires its trigger (event subscription or cron entry). The construct MUST be an `automation` kind carrying its `.memql` source on construct.Source; its ownerUserId is the authz envelope the runs execute under. Re-activating the same (owner, name) replaces the prior subscription in place (a version bump); an Activate that is refused leaves the prior subscription wired.

func (*AuthoredScheduler) ActiveCount

func (s *AuthoredScheduler) ActiveCount() int

ActiveCount returns how many authored automations are currently wired.

func (*AuthoredScheduler) Deactivate

func (s *AuthoredScheduler) Deactivate(owner, name string)

Deactivate tears down a live authored automation's trigger (event unsubscribe or cron-entry removal). A no-op when nothing is registered for (owner, name), so it doubles as the idempotent pre-step of Activate.

func (*AuthoredScheduler) IsActive

func (s *AuthoredScheduler) IsActive(owner, name string) bool

IsActive reports whether (owner, name) is currently wired.

func (*AuthoredScheduler) SetGlobalGate

func (s *AuthoredScheduler) SetGlobalGate(gate func() bool)

SetGlobalGate installs (or replaces) the cluster-wide global kill switch after construction -- the app wires it once the engine is live so the gate can read v1:identity:clusterSettings.authoredAutomationsEnabled. Safe to call against a running scheduler.

func (*AuthoredScheduler) SetOwnerGate

func (s *AuthoredScheduler) SetOwnerGate(gate func(ownerUserId string) bool)

SetOwnerGate installs (or replaces) the cluster owner gate after construction. The app wires it during the cluster phase once the peer mesh is available so an owner's SCHEDULED authored automations fire on exactly one node cluster-wide. nil restores the single-node default (this node owns every owner). Safe to call against a running scheduler.

func (*AuthoredScheduler) Stop

func (s *AuthoredScheduler) Stop()

Stop tears down every subscription + the cron scheduler. After Stop the scheduler is unusable.

type AuthoredSchedulerOptions

type AuthoredSchedulerOptions struct {
	BeforeWriteEngine   *memql.MemQLEngine
	BeforeWriteRegistry StepExecutorRegistry
	Logger              *slog.Logger
	Loader              *Loader
	EventBus            *events.Bus
	// Run executes a compiled authored automation. Required.
	Run RunAutomationFunc
	// Breaker, when set, fault-isolates each authored automation: a run that
	// errors increments that automation's breaker, and on trip the scheduler
	// auto-pauses ONLY that automation (the others, and core, keep running).
	// nil = no breaker (every run fires regardless of prior faults). The app
	// wires a breaker whose onTrip pauses the automation on every replica;
	// tests supply their own.
	Breaker *AuthoredBreaker
	// OwnerGate, when set, leader-gates owner-scoped SCHEDULED authored
	// automations in a cluster: a cron firing for owner O runs only on the
	// node OwnerGate(O) returns true for, so it fires once cluster-wide rather
	// than once per replica whose cron ticked. EVENT triggers never consult it:
	// they fire on the node where their event happened (fireOnEvent). nil =
	// ungated (every node runs every owner's schedules -- the single-node /
	// dev default). Typically AuthoredOwnerGate.OwnsOwner.
	OwnerGate func(ownerUserId string) bool
	// GlobalGate, when set, is the cluster-wide GLOBAL KILL SWITCH for authored
	// automations (epic memql#954, issue #961): it returns false when the
	// v1:identity:clusterSettings.authoredAutomationsEnabled flag is off, and the
	// scheduler then suppresses EVERY firing for EVERY owner -- the governance
	// hard stop. Checked before owner-gating + the breaker so a globally-halted
	// cluster runs nothing. nil = always enabled (no global switch wired).
	GlobalGate func() bool
}

AuthoredSchedulerOptions configures the authored scheduler.

type Automation

type Automation struct {
	BeforeWrite *BeforeWriteConfig `json:"beforeWrite,omitempty"`
	// Trusted reports that this automation's SOURCE came from the registered
	// DSL tree rather than from a caller (memql#2800).
	//
	// It gates whether the automation's steps may reach @serverOnly
	// constructs. That trust is a property of WHERE THE BODY CAME FROM, not
	// of the dispatch call: an earlier fix stamped internal origin
	// unconditionally in executeStep, reasoning that "executeStep is
	// reachable only from automation execution". True, and not sufficient --
	// automation execution includes automations whose body the CALLER
	// SUPPLIED. RunBundleDryRun compiles submitted source and drives the real
	// Executor, so MCP run_inline_automation and the planner's LLM-emitted
	// bundle could each wrap a @serverOnly read in a step and have it run
	// with internal origin.
	//
	// FALSE IS THE ZERO VALUE, deliberately: an automation built by any path
	// that has not explicitly declared its source trusted is treated as
	// caller-supplied. Only the unified tree loader sets it.
	Trusted bool `json:"-"`

	// Source is the layer of the DSL tree the loader read this automation from
	// (dsl.SourceOf): the engine's own tree, a compiled pack, or a runtime
	// domain -- which is what a package staged through Deployables is. Set by
	// the tree loader from the tree that served the file, never from the
	// automation's source, and copied by Snapshot.
	//
	// It decides one thing: only SourceEngine is granted the cluster's
	// maintenance principal (maintenanceOrigin, executor.go). The zero value is
	// SourceUnknown, so an automation built by any other path -- compiled from
	// a caller's source, built in Go, decoded from JSON -- is not the engine's.
	Source memqldsl.Source `json:"-"`

	// Name is the unique identifier for this automation (camelCase).
	Name string `json:"name"`

	// Description provides human-readable context for the automation.
	Description string `json:"description,omitempty"`

	// Schedule is an optional cron expression for scheduled execution.
	// If omitted, the automation must be triggered manually or by events.
	Schedule      string `json:"schedule,omitempty"`
	ScheduleNode  string `json:"scheduleNode,omitempty"`
	ScheduleLease string `json:"scheduleLease,omitempty"`

	// Trigger defines event-based triggers for this automation.
	// When an event matching the pattern is published, this automation runs.
	Trigger *TriggerConfig `json:"trigger,omitempty"`

	// Preconditions are first-class deterministic checks (Epic 4 /
	// memql#2139) evaluated -- in order -- at the START of the run, after the
	// trigger fires but BEFORE any statement runs. Each precondition is a
	// boolean expression over the same deterministic evaluator that decides a
	// statement's condition and a trigger filter (no LLM). A precondition
	// that evaluates false is a MISS: it aborts the run cleanly (no
	// statement runs) and emits the structured
	// healing.precondition.missed signal the self-healing repair loop
	// (E4.4) subscribes to. A miss is BOTH the clean repair trigger and
	// the cross-machine portability mechanism -- a literal asserted by a
	// precondition that does not hold on this machine is a precondition
	// that misses here.
	Preconditions []*Precondition `json:"preconditions,omitempty"`

	// Steps are the ordered list of operations to execute.
	Steps []*Step `json:"steps"`

	// Args is the automation's declared input contract -- the `args { }`
	// block (event-payload-binding ADR Decision 1, memql#2363). When
	// non-nil, the trigger/invoke-by-reference path binds event.payload into
	// these fields and validates them (fire-time validation, Decision 2)
	// BEFORE any step runs; the validated map is exposed to the evaluator as
	// `args`. Nil = untyped event.payload access (backward compatible with
	// every automation that has not yet declared a contract).
	Args *ArgsSchema `json:"args,omitempty"`

	// Enabled controls whether this automation is active. Defaults to true.
	Enabled *bool `json:"enabled,omitempty"`

	// MCPPromoted is true when the automation carries @mcp: it is promoted into
	// its own first-class MCP tool rather than only being reachable through the
	// generic run_automation dispatcher (epic memql#1529 Phase 4 #1534).
	MCPPromoted bool `json:"mcpPromoted,omitempty"`

	// Template is true when the automation carries @template: it is invoked
	// by a v1:work:run that named it, and is never fired by the graph
	// (memql#5048).
	//
	// It is the THIRD way an automation can be reachable. validateTriggerWiring
	// refuses an automation with no event trigger and no schedule, because one
	// that silently never runs is indistinguishable from one that works until
	// the day you need it -- and that reasoning is unchanged. A template is
	// reachable; it is just reachable by being NAMED rather than by being
	// subscribed. The same gate therefore refuses a template that ALSO carries
	// a trigger, so the two classes cannot blur into one.
	Template bool `json:"template,omitempty"`

	// JournalRequired stops execution when an intent, receipt or run write
	// cannot be confirmed. It does not promise exactly-once external effects.
	JournalRequired bool `json:"journalRequired,omitempty"`
	// JournalOnWrite holds a background event journal until a graph write,
	// failure or human wait. Successful no-ops create no work rows.
	JournalOnWrite bool `json:"journalOnWrite,omitempty"`

	// UsesDeployerFleet runs each `builtin ai` call on the fleet of the person who
	// staged this automation's package's DSL (component/memql/deployer_fleet.go).
	UsesDeployerFleet bool `json:"usesDeployerFleet,omitempty"`
	// Domain is the DSL domain this automation was loaded from, read from its path
	// at load and never from its source. Not part of the definition's fingerprint.
	Domain string `json:"-"`

	// Loop is @loop(maxDepth=N, until=row => P): this automation closes a
	// deliberate cycle the load would otherwise refuse, and bounds it (D18,
	// epic memql#5380). Nil when the annotation is absent.
	Loop *LoopConfig `json:"loop,omitempty"`

	// Mode is @mode(<kind>[, max=N]): how concurrent fires of this
	// automation behave in one process (D19). Nil runs every fire in
	// parallel, unbounded -- the behaviour before the annotation existed.
	Mode *ModeConfig `json:"mode,omitempty"`

	// Reads are the payload fields the dedup key keeps for this automation
	// (loop_reads.go): the fields it can read, when everything it reaches
	// reads nothing else. Nil keeps the whole payload but its version clock.
	Reads []string `json:"-"`

	// JournalTriggerReads are the payload keys an event firing of this
	// automation records on its run's triggerEvent (journal_trigger.go): what
	// its statements and the readers of a recorded trigger can read. Nil
	// records the whole payload, which is also the answer for an automation
	// built in Go rather than loaded.
	JournalTriggerReads []string `json:"-"`

	// Origin tracks where this automation was loaded from (file path).
	Origin string `json:"-"`
	// contains filtered or unexported fields
}

Automation represents a complete automation definition.

func (*Automation) DefinitionFingerprint

func (a *Automation) DefinitionFingerprint(engine *id.Engine) string

DefinitionFingerprint computes a content-addressed hash of the automation definition. Used to detect if the automation changed between checkpoint creation and resume.

func (*Automation) IsEnabled

func (a *Automation) IsEnabled() bool

IsEnabled returns whether the automation should run.

func (*Automation) IsEventTriggered

func (a *Automation) IsEventTriggered() bool

IsEventTriggered returns true if this automation has an event trigger.

func (*Automation) IsScheduled

func (a *Automation) IsScheduled() bool

IsScheduled returns true if this automation has a cron schedule.

func (*Automation) IsTemplate

func (a *Automation) IsTemplate() bool

IsTemplate reports whether this automation is invoked by a run rather than fired by the graph.

type AutomationExecution

type AutomationExecution struct {
	// SourceTrusted mirrors Automation.Trusted onto the run, so the step
	// dispatch can consult it without re-plumbing the automation record
	// through every frame (memql#2800). False is the zero value; see
	// Automation.Trusted for why that direction is deliberate.
	SourceTrusted bool `json:"-"`

	// CallerSuppliedPayload records that this run's TRIGGER PAYLOAD came from a
	// caller, independently of where the automation's body came from.
	//
	// Kept as its own field rather than re-derived from
	// (automation.Trusted && !SourceTrusted) because it has to survive into a
	// CHECKPOINT and back out through resume, and a derived value silently
	// becomes wrong the moment either input is reconstructed differently --
	// which is exactly how memql#2888's first fix was bypassable.
	CallerSuppliedPayload bool `json:"-"`

	// Actor is who this run acted as, recorded by the executor where it
	// installs the run's actor. The zero value means no actor was installed:
	// the run was refused or skipped before it started.
	Actor RunActor `json:"-"`

	// ID is a unique identifier for this execution.
	ID string `json:"id"`

	// AutomationName identifies which automation was run.
	AutomationName string `json:"automationName"`

	// Status is "running", "completed", "failed", "cancelled", "waiting".
	Status string `json:"status"`

	// Input contains the result of the input query.
	Input any `json:"input,omitempty"`

	// Steps contains results for each executed step.
	Steps map[string]*StepResult `json:"steps"`

	// Returned and Output are what a statement body's `return` ended the run
	// with (epic memql#5370): Returned is false when the body ran off its end,
	// and Output is the value (nil for a bare return).
	Returned bool `json:"returned,omitempty"`
	Output   any  `json:"output,omitempty"`

	// Error contains the automation-level error if any.
	Error string `json:"error,omitempty"`

	// FailedStepId, FailedStepType, FailedStepAttempt and FailedStepRetries
	// are what the failure path's symptom table reads (epic memql#5127). They
	// are recorded at the point of failure rather than derived afterwards
	// because the two things the table most needs -- how many attempts were
	// spent and how many were allowed -- exist only inside the executor's
	// retry loop, and "was this tried until it ran out" is the difference
	// between a blip worth retrying and a stall worth escalating.
	FailedStepId      string `json:"-"`
	FailedStepType    string `json:"-"`
	FailedStepAttempt int    `json:"-"`
	FailedStepRetries int    `json:"-"`

	// PreconditionMissed records that a first-class precondition did not hold
	// on this machine, which emitPreconditionMiss already published as an
	// event. It is carried on the execution as well because the symptom table
	// reads a VALUE, and re-deriving "was this a precondition miss" from the
	// error text would be a parser of our own output (epic memql#5127).
	PreconditionMissed bool `json:"-"`

	// ErrorValue is the same failure as a VALUE, kept for the journal
	// (epic memql#5096). It is `json:"-"` and never serialized: a checkpoint
	// carries the string, and an error value that survived a round trip
	// through JSON would be a different error wearing the same words.
	//
	// It exists because one decision needs the error's TYPE rather than its
	// text -- whether a failed run parks on a shut inference door or fails --
	// and the structured door report is on the error the router raised.
	// Reading that report back out of the message would be a parser of our
	// own output.
	ErrorValue error `json:"-"`

	// StartedAt is when the execution began.
	StartedAt time.Time `json:"startedAt"`

	// CompletedAt is when the execution finished.
	CompletedAt time.Time `json:"completedAt"`

	// Duration is the total execution time.
	Duration time.Duration `json:"duration"`

	// TriggeredBy indicates what started this execution.
	// "schedule", "manual", "event:{topic}" (TriggeredBySchedule,
	// TriggeredByEventPrefix)
	TriggeredBy string `json:"triggeredBy,omitempty"`

	// StepOrder preserves the declared execution sequence for verification.
	StepOrder []string `json:"stepOrder,omitempty"`

	// ChainHead is the final content-addressed state after all steps.
	ChainHead string `json:"chainHead,omitempty"`

	// InitialChainHead captures the starting state (automation + trigger context).
	InitialChainHead string `json:"initialChainHead,omitempty"`

	// InputFingerprint is the deterministic hash of the input query result.
	InputFingerprint string `json:"inputFingerprint,omitempty"`
	// contains filtered or unexported fields
}

AutomationExecution represents a complete automation run.

func ExecuteInScope

func ExecuteInScope(ctx context.Context, a *Automation, args map[string]any, opts ExecutorOptions) (*AutomationExecution, error)

ExecuteInScope interprets a callable template within a native operation that has already been admitted and owns its effects. Unlike firing an automation, this neither opens nor closes a work run, injects a system actor, replaces a step's model-call association, nor applies event admission a second time. The caller MUST supply a restricted registry and retain authorization, cancellation, effect journaling and recovery. This is a Go runtime seam, never an alternative wire entry point for arbitrary submitted DSL.

func NewExecution

func NewExecution(automationName, triggeredBy string) *AutomationExecution

NewExecution creates a new automation execution.

func (*AutomationExecution) AddStepResult

func (e *AutomationExecution) AddStepResult(result *StepResult)

AddStepResult records a step's result.

func (*AutomationExecution) Cancel

func (e *AutomationExecution) Cancel()

Cancel marks the execution as cancelled.

func (*AutomationExecution) Complete

func (e *AutomationExecution) Complete()

Complete marks the execution as successfully completed.

func (*AutomationExecution) Fail

func (e *AutomationExecution) Fail(err error)

Fail marks the execution as failed.

func (*AutomationExecution) RecordFailedStep

func (e *AutomationExecution) RecordFailedStep(step *Step, attempt int)

RecordFailedStep notes which step failed and how hard it was tried. It is called at the point of failure, beside Fail, because the attempt count and the retry budget exist only inside the executor's retry loop -- and re-deriving them afterwards from the error text would be a parser of our own output, which is the mistake ErrorValue exists to avoid.

type AutomationStepConfig

type AutomationStepConfig struct {
	// Name is the automation identifier to invoke.
	Name string `json:"name"`

	// Async runs the automation without waiting for completion.
	// When true, the step returns immediately after triggering.
	Async bool `json:"async,omitempty"`

	// Args are the arguments passed to the sub-automation. They
	// become the sub-automation's input envelope so the
	// sub-automation's `event.X` references resolve against them
	// (i.e. a procedurally-called automation sees the call args
	// the same way an event-triggered one sees the event payload).
	// Optional; absent for invocations that just pass the parent's
	// envelope through unchanged.
	Args map[string]any `json:"args,omitempty"`
}

AutomationStepConfig configures invoking another automation.

type AutomationTrigger

type AutomationTrigger interface {
	// TriggerAutomation starts an automation by name.
	// Returns the execution result and any error.
	TriggerAutomation(ctx context.Context, name string) (*AutomationExecution, error)

	// TriggerAutomationWithArgs starts an automation by name and
	// passes the supplied args as the sub-automation's input
	// envelope. The args appear under `event.X` from the
	// sub-automation's perspective, the same way an event-triggered
	// invocation would see the event payload. Used by the
	// automation-within-automation step kind for procedural
	// composition.
	TriggerAutomationWithArgs(ctx context.Context, name string, args map[string]any) (*AutomationExecution, error)
}

AutomationTrigger allows steps to invoke other automations.

type BeforeWriteConfig

type BeforeWriteConfig struct {
	On      string `json:"on"`
	Concept string `json:"concept"`
	Filter  string `json:"filter,omitempty"`
	// contains filtered or unexported fields
}

type BlockStepConfig

type BlockStepConfig struct {
	Steps []*Step `json:"steps"`
}

BlockStepConfig configures a block step: its own list of steps.

type BreakerOnTrip

type BreakerOnTrip func(info BreakerTripInfo)

BreakerOnTrip is invoked exactly once when an automation's breaker opens. The scheduler wires it to pause the offending automation and surface the fault. It must not block long (it runs inline on the failing run's goroutine) and must not itself fault the breaker -- it is the isolation boundary.

type BreakerSubject

type BreakerSubject struct {
	Owner      string
	Automation string
	BundleId   string
	Version    int
}

BreakerSubject is the run a failure is recorded against: the (owner, automation) the breaker keys on, and the bundle and version it was armed from, which a trip hands on in BreakerTripInfo.

type BreakerTripInfo

type BreakerTripInfo struct {
	Owner      string
	Automation string
	// BundleId and Version name the construct the tripping run was armed
	// from (memql#6022). The pause a trip leads to runs later, off the failing
	// run's goroutine, and a fix may be armed under the same name by then;
	// only these tell the version that failed from the one that replaced it.
	BundleId  string
	Version   int
	Failures  int
	LastError string
	TrippedAt time.Time
}

BreakerTripInfo describes a breaker opening, handed to the onTrip callback and recorded for the management surface.

type CanonicalIdResolver

type CanonicalIdResolver func(ctx context.Context, value, conceptType string) (string, error)

CanonicalIdResolver normalizes an id-shaped value to canonical form for the named concept (`<partition>:<conceptType>:<bareSlug>`). Wired by the engine boot path so automation steps can call canonicalId(value, "<conceptType>") and get the same answer as mutations / queries. When unset, callers should fall back to the raw value (degraded but safe).

type ClassifySymptomInput

type ClassifySymptomInput struct {
	StepKey      string
	StepType     string
	ErrorCode    string
	ErrorMessage string
	Attempt      int
	Trace        []map[string]any
}

ClassifySymptomInput is what the one model call is given. It mirrors the classifySymptom prompt's declared field list.

type ClusterExecutionGuard

type ClusterExecutionGuard struct {
	*component.Component
	// contains filtered or unexported fields
}

ClusterExecutionGuard implements executionClaimer.

func NewClusterExecutionGuard

func NewClusterExecutionGuard(dbGetter func() *bun.DB, logger Logger) *ClusterExecutionGuard

NewClusterExecutionGuard builds the guard. dbGetter is resolved lazily.

func (*ClusterExecutionGuard) Claim

func (g *ClusterExecutionGuard) Claim(ctx context.Context, automationName, dedupKey string) bool

Claim returns true when THIS node should execute the automation (it won the claim, or the guard is degraded and fails open) and false when another replica already claimed it (a prevented cross-replica duplicate). The claim row is permanent within the prune retention: once won, no peer can re-take it until the retention sweep deletes it. This is what event-triggered automations and planner plan-execution need -- exactly-once, never re-run.

func (*ClusterExecutionGuard) ClaimErrors

func (g *ClusterExecutionGuard) ClaimErrors() int64

func (*ClusterExecutionGuard) ClaimWithTTL

func (g *ClusterExecutionGuard) ClaimWithTTL(ctx context.Context, automationName, dedupKey string, ttl time.Duration) bool

ClaimWithTTL is Claim with an optional attempt-scoped lease. It is the opt-in variant (memql#2548): the default Claim path (ttl <= 0) is unchanged, so automation/planner callers keep exactly-once-within-retention semantics.

With ttl > 0 an expired claim is RE-WINNABLE: a peer takes it over (a fresh insert and a takeover of a stale row both count as "won"), while a claim still within its ttl is honoured (the peer loses, exactly as before). This bounds the outbound-delivery claim/stamp wedge: a replica that dies after claiming an attempt but before stamping a terminal status leaves the row pending with a persisted claim; without a lease every peer re-claims-and-loses that key until the 1h retention prune elapses. The stored expiry also prevents cleanup, or a caller with a shorter TTL, from shortening a live claim. Rows written before expiry support use their age.

The ttl must exceed the longest time a LIVE claimant holds the claim (one delivery attempt) or a slow-but-alive node is stolen from, double-running the attempt; the outbound worker sets it well above its per-request HTTP timeout. Delivery is already at-least-once, so a rare re-take under an extreme stall re-delivers rather than corrupts.

func (*ClusterExecutionGuard) Claimed

func (g *ClusterExecutionGuard) Claimed() int64

func (*ClusterExecutionGuard) DuplicatesPrevented

func (g *ClusterExecutionGuard) DuplicatesPrevented() int64

DuplicatesPrevented / ClaimErrors / Claimed expose the running counts for tests + health surfaces.

func (*ClusterExecutionGuard) ReleaseClaim

func (g *ClusterExecutionGuard) ReleaseClaim(ctx context.Context, automationName, dedupKey string) bool

ReleaseClaim gives back a claim THIS node took, so a later delivery of the same event can claim it again. It is for work that was claimed and then could not begin -- nothing was attempted -- and so must remain possible; work that began keeps its claim for the retention window, which is the point of one. Only the claimant's own row is removed. It reports whether a claim was removed; a claim that cannot be reached is left, and logged, and the work stays claimed (the safe side: a duplicate is worse than a delay).

func (*ClusterExecutionGuard) StrictClaimer

func (g *ClusterExecutionGuard) StrictClaimer() StrictClusterClaimer

StrictClaimer supplies the fail-closed policy used for work run execution, where admitting two replicas can duplicate a file or another side effect.

type CronLeader

type CronLeader struct {
	*component.Component
	// contains filtered or unexported fields
}

CronLeader is a Dependency; it acquires/holds the lease over its lifetime.

func NewCronLeader

func NewCronLeader(dbGetter func() *bun.DB, logger *slog.Logger) *CronLeader

NewCronLeader builds the leader. dbGetter is resolved lazily (the DB may not be Ready at construction). A nil logger is tolerated.

func NewScopedCronLeader

func NewScopedCronLeader(scope string, dbGetter func() *bun.DB, logger *slog.Logger) *CronLeader

NewScopedCronLeader elects one runner independently of the general cron lease. The composition layer decides which nodes may compete for a scope. Replicas using the same scope share a lease; distinct scopes do not block one another. It uses the same direct connection and failover contract.

func (*CronLeader) IsLeader

func (cl *CronLeader) IsLeader() bool

IsLeader reports whether this node currently owns the scheduled-automation lease. Wire it as Scheduler.LeaderGate.

func (*CronLeader) ScopedGate

func (cl *CronLeader) ScopedGate(scope string) func() bool

ScopedGate registers an independent advisory lock on this leader's ONE session. Adding schedules must not consume another lifetime connection: enough scopes otherwise exhaust the bounded direct pool before Ready closes. Call during composition; each scope retains its existing lock key and gate.

type ErrorStrategy

type ErrorStrategy string

ErrorStrategy defines how to handle step failures.

const (
	// ErrorStrategyStop halts the automation on error (default).
	ErrorStrategyStop ErrorStrategy = "stop"

	// ErrorStrategyContinue continues to the next step on error.
	ErrorStrategyContinue ErrorStrategy = "continue"

	// ErrorStrategyRetry retries the step (with optional count).
	ErrorStrategyRetry ErrorStrategy = "retry"
)

type Evaluator

type Evaluator struct {
	// contains filtered or unexported fields
}

Evaluator is an automation run's state (see the file comment).

func NewEvaluator

func NewEvaluator() *Evaluator

NewEvaluator creates an empty run state.

func (*Evaluator) Bind

func (e *Evaluator) Bind(name string, v any)

Bind binds a name in this Evaluator's current frame -- a loop variable.

func (*Evaluator) CanonicalIdResolver

func (e *Evaluator) CanonicalIdResolver() CanonicalIdResolver

CanonicalIdResolver returns the wired resolver (may be nil).

func (*Evaluator) ChildFrame

func (e *Evaluator) ChildFrame() *Evaluator

ChildFrame is childFrame for the steps package: a clone whose names live in a new frame inside this one's.

func (*Evaluator) Clone

func (e *Evaluator) Clone() *Evaluator

Clone creates a copy of the run state for a nested context (a loop iteration, a parallel branch).

Every resolver travels with the copy, the canonicalId one included. It used to be left behind, so a canonicalId() inside a loop or a parallel branch silently fell back to the raw value while the same call one level up canonicalised it (memql#5367).

func (*Evaluator) EvalV1

func (e *Evaluator) EvalV1(ctx context.Context, n ast.ExpressionNode) (any, error)

EvalV1 evaluates one parsed v1 expression over the run.

func (*Evaluator) EvalV1Condition

func (e *Evaluator) EvalV1Condition(ctx context.Context, n ast.ExpressionNode) (bool, error)

EvalV1Condition evaluates one parsed v1 condition over the run: a boolean, with absent read as false, a stored row field of another type not true, and anything else refused (condition_not_boolean).

func (*Evaluator) ExprOptions

func (e *Evaluator) ExprOptions() memql.EvalOptions

ExprOptions is the memql.EvalOptions of an evaluation over this run: the run clock, the variable resolvers and the canonicalId resolver the run was seeded with. Predicates stays nil until the engine exposes spec and trait bodies as v1 lambdas; a predicate application then fails as an unknown function rather than meaning something by accident.

func (*Evaluator) InStatementBody

func (e *Evaluator) InStatementBody() bool

InStatementBody reports whether the Evaluator runs a statement body.

func (*Evaluator) ResolveV1Map

func (e *Evaluator) ResolveV1Map(ctx context.Context, m map[string]any) (map[string]any, error)

ResolveV1Map resolves a v1 value map (see ResolveV1Value). Nil stays nil.

func (*Evaluator) ResolveV1Value

func (e *Evaluator) ResolveV1Value(ctx context.Context, v any) (any, error)

ResolveV1Value resolves a v1 value: an *ExprLeaf evaluates, a map or list resolves each entry, and anything else is a literal and is returned as it is. CONTAINER RULE (rule 30's last paragraph, memql#3627): an entry whose expression evaluates to the Absent sentinel contributes nothing -- its key or element is omitted -- while an explicit nil is kept. A TOP-LEVEL absent value is returned as memql.Absent for the caller to decide about.

func (*Evaluator) RunScope

func (e *Evaluator) RunScope() *RunScope

RunScope returns the run's state as a memql.ExprScope.

func (*Evaluator) SetCanonicalIdResolver

func (e *Evaluator) SetCanonicalIdResolver(r CanonicalIdResolver)

SetCanonicalIdResolver wires the engine's id-canonicalization helper into the run. Required for canonicalId() to work inside automation step expressions; without it, callers fall back to the raw value (which means automation-derived ids may diverge from mutation-derived ids when callers pass bare slugs).

func (*Evaluator) SetCustom

func (e *Evaluator) SetCustom(name string, value any)

SetCustom seeds a root.

func (*Evaluator) SetLogger

func (e *Evaluator) SetLogger(logger *slog.Logger)

SetLogger sets the logger for warning messages.

func (*Evaluator) SetSecretResolver

func (e *Evaluator) SetSecretResolver(resolver VariableResolver)

SetSecretResolver sets the resolver behind secret("NAME") (v1:platform:partitionSecret with fallback to v1:platform:globalSecret). Returns decrypted plaintext.

func (*Evaluator) SetSystemSecretResolver

func (e *Evaluator) SetSystemSecretResolver(resolver VariableResolver)

SetSystemSecretResolver sets the resolver behind systemSecret("NAME") (v1:platform:globalSecret). Returns decrypted plaintext.

func (*Evaluator) SetSystemVariableResolver

func (e *Evaluator) SetSystemVariableResolver(resolver VariableResolver)

SetSystemVariableResolver sets the resolver behind systemVar("NAME") (v1:platform:globalVariable).

func (*Evaluator) SetVariableResolver

func (e *Evaluator) SetVariableResolver(resolver VariableResolver)

SetVariableResolver sets the resolver behind var("NAME").

func (*Evaluator) StepCondition

func (e *Evaluator) StepCondition(ctx context.Context, step *Step) (bool, error)

StepCondition decides whether a step runs: its condition, parsed at load, through EvalCondition over the run. A step without a condition runs, and a condition its automation never prepared is refused -- there is nothing else to read its text with.

type EventStepConfig

type EventStepConfig struct {
	// Topic is the event topic: a string literal or an expression leaf.
	Topic string `json:"topic"`

	// Kind is the event kind. Defaults to "message".
	Kind string `json:"kind,omitempty"`

	// Payload is the event payload: each leaf a literal or an expression
	// leaf.
	Payload map[string]any `json:"payload,omitempty"`
	// contains filtered or unexported fields
}

EventStepConfig configures an event publication step.

func (EventStepConfig) MarshalJSON

func (c EventStepConfig) MarshalJSON() ([]byte, error)

MarshalJSON writes the config with its lifted leaves.

func (*EventStepConfig) UnmarshalJSON

func (c *EventStepConfig) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes the config, lifting a non-string `topic`.

type Executor

type Executor struct {
	// contains filtered or unexported fields
}

Executor orchestrates automation execution.

func NewExecutor

func NewExecutor(opts ExecutorOptions) *Executor

NewExecutor creates a new automation executor.

func (*Executor) Close

func (e *Executor) Close()

Close releases executor resources (e.g., deduplication cleanup goroutines). Safe to call multiple times.

func (*Executor) Execute

func (e *Executor) Execute(ctx context.Context, automation *Automation, triggeredBy string) (*AutomationExecution, error)

Execute runs an automation and returns the execution result.

func (*Executor) ExecuteAdopted

func (e *Executor) ExecuteAdopted(ctx context.Context, automation *Automation, adopt RunAdoption) (*AutomationExecution, error)

ExecuteAdopted runs an automation as an EXISTING run.

It is the whole of what memql#5054 found missing: something that takes a run in `running` carrying an automationName and actually executes it.

The execution's id IS adopt.RunId, so the journal's step rows, its heartbeats and its terminal close all land on that run. The run row is advanced rather than inserted -- see workJournal.adoptRun.

The caller is responsible for the CLAIM. This function will happily run the same run twice if called twice; refusing here would need a database read this package deliberately does not do, and the claim the work integration already holds (ClusterExecutionGuard, Postgres-backed) is both stronger and the one every other cross-replica dispatch in the tree uses.

func (*Executor) ExecuteWithClientEvent

func (e *Executor) ExecuteWithClientEvent(ctx context.Context, automation *Automation, triggeredBy string, triggeringEvent *events.Event) (*AutomationExecution, error)

ExecuteWithClientEvent runs an automation whose TRIGGER PAYLOAD came from a caller, and therefore never grants internal origin however trusted the automation's source is (memql#2888).

The #2800 rule -- "trust rides on the automation's SOURCE, not on which function does the dispatching" -- is right and is not sufficient. Its earlier phrasing carried the half that explains why:

a client can certainly cause an automation to fire, but it cannot choose
which constructs the authored body invokes.

On the MCP run_automation path the client still cannot choose the CONSTRUCT, but it chooses the ARGUMENT -- and for a @serverOnly construct the argument IS the authorization decision, because test/dslconformance/conformance_test.go treats @serverOnly as the bucket that EXEMPTS a construct from carrying any caller-scope filter. Origin is the only gate, so an unchecked argument is an unchecked query.

Concretely, before this existed: run_automation("killSwitchSuspendsRunningPlans", {node:{id: <any user>}}) reached runningPlansForUser (@serverOnly) with an attacker-chosen userId and then transitioned every plan it returned through updatePlanStatus, which stamps `id: args.planId` with no owner predicate -- a cross-user write, not just a read leak. @filter does not help: the scheduler evaluates it on the event-bus path, not here, and the caller supplies the payload it would test.

So the rule is: internal origin requires BOTH a trusted source AND a trigger payload the caller did not supply.

func (*Executor) ExecuteWithEvent

func (e *Executor) ExecuteWithEvent(ctx context.Context, automation *Automation, triggeredBy string, triggeringEvent *events.Event) (*AutomationExecution, error)

ExecuteWithEvent runs an automation with an optional triggering event, from a SERVER-ORIGINATED trigger: a graph event, a cron tick, a sub-automation. The automation's source decides whether its steps reach the engine with internal origin.

func (*Executor) ResumeFrom

func (e *Executor) ResumeFrom(
	ctx context.Context,
	journal *RunJournal,
	automation *Automation,
	opts *ResumeOptions,
) (*AutomationExecution, error)

ResumeFrom resumes execution from the work journal. It rehydrates the evaluator with the completed step results the journal holds, then continues execution from the specified step (or the unfinished one if not specified), on the SAME run id.

func (*Executor) SetSymptomClassifier

func (e *Executor) SetSymptomClassifier(c SymptomClassifier)

SetSymptomClassifier installs the classifier. Called once from app/ on a node that can reach a model. It lands on the journal, which is the only thing on this path that outlives one execution.

type ExecutorOptions

type ExecutorOptions struct {
	Logger            *slog.Logger
	Engine            *memql.MemQLEngine
	EventBus          *events.Bus
	StepRegistry      StepExecutorRegistry
	AutomationTrigger AutomationTrigger

	// SandboxRun marks this executor as driving a Gate-2 behavioural dry-run.
	//
	// A sandboxed run writes NO journal (memql#2932). A preview has nothing
	// to resume, and the record escaped the sandbox: the journal writer goes
	// straight to the engine, never through the interception layer, so a
	// "dry-run" left durable, RESUMABLE rows in the live graph.
	//
	// Since memql#2943 the sandbox registry refuses every step type it has not
	// classified, so the journal was the last write that escaped, and it is the
	// one that matters for #2890 / #2908, because it is the only escaping write
	// that is a RESUMABLE TOKEN.
	//
	// The failing preview was the one that wrote: the journal closes the run
	// at `failed` on step failure, so the refusal a preview exists to REPORT
	// was what produced the resumable row. NewExecutor holds no journal at all
	// when this is set, which is the mechanical form of that rule.
	SandboxRun bool

	// TerminalFailures is for a bounded native analysis capability whose scope
	// cannot expand through classification, a budget approval or replanning.
	// Journal writes remain enabled; only failure recovery is disabled.
	TerminalFailures bool

	// CancelPollInterval bounds how often a run re-reads its own
	// `cancelRequested` flag (memql#5066, cancel.go). Zero takes the default.
	//
	// It is an option only so a test can drive more than one boundary check
	// inside a run that finishes in microseconds. It is deliberately NOT
	// exposed per automation: a knob there would let one template opt out of
	// a control that exists to bound damage.
	CancelPollInterval time.Duration

	// ChainTrackingEnabled enables content-addressed chain tracking for executions.
	// When enabled, each step produces a deterministic fingerprint that chains to
	// the previous state, enabling replay verification.
	// This is separate from deduplication - chain tracking provides verification
	// without necessarily blocking duplicate executions.
	ChainTrackingEnabled bool

	// DedupEnabled enables execution deduplication based on initial chain head.
	// When true, executions with the same automation + trigger context + input
	// will be skipped within the DedupTTL window.
	// Should only be enabled for event-triggered automations where idempotency matters.
	// Scheduled and manual runs should NOT use dedup (same schedule = same fingerprint).
	DedupEnabled bool

	// DedupTTL configures how long to remember executions for deduplication.
	// Defaults to 10 minutes if DedupEnabled is true and this is zero.
	DedupTTL time.Duration

	// MaxConcurrentExecutions limits how many automations can run simultaneously.
	// Prevents database connection exhaustion during event storms.
	// Defaults to 10 if not set (0 means use default, not unlimited).
	MaxConcurrentExecutions int

	// ClusterGuard extends dedup across replicas (#561): after the per-process
	// dedup passes, the executor claims the (automation, dedup-key) in the DB
	// so an event reaching multiple replicas executes once cluster-wide. nil =
	// single-replica behaviour. Only set on the EVENT executor.
	ClusterGuard executionClaimer
}

ExecutorOptions configures the automation executor.

type ExprLeaf

type ExprLeaf struct {
	Src  string
	Node ast.ExpressionNode
}

ExprLeaf is a value-map leaf that is an expression: `{"$expr": src}` in the compiled JSON, parsed once at load. It marshals back to the same shape, so a step that is re-serialised (a journal row, a test fixture) round-trips.

func (*ExprLeaf) MarshalJSON

func (l *ExprLeaf) MarshalJSON() ([]byte, error)

MarshalJSON writes the compiled form.

type FieldWriteConfig

type FieldWriteConfig struct {
	Field string `json:"field"`
	Value any    `json:"value"`
}

type FlatAutomation

type FlatAutomation struct {
	Name string
	// File is the file the automation was loaded from, relative to the DSL
	// root (cluster/automations.memql), or the loader's origin when it did not
	// come from the tree.
	File string
	// Writes is the concept ids the automation writes, in the graph's order.
	Writes []string
	// Attrs carries stratum and, when present, template, trigger,
	// triggerKind, triggerConcept, schedule, filter, writes, publishes, loop,
	// mode, cycle, cyclePermitted and problem. Absent rather than empty.
	Attrs map[string]string
}

FlatAutomation is one automation of the static loop graph with every fact a string.

type FlatTrigger

type FlatTrigger struct {
	From, To string
	// Attrs carries concept, topic, via, decided and reason, absent when empty.
	Attrs map[string]string
}

FlatTrigger is one edge of the static loop graph: From's writes or publishes can fire To.

type ForEachStepConfig

type ForEachStepConfig struct {
	// Source is an expression naming the collection.
	// Example: `input`, `classify`
	Source string `json:"source"`

	// Filter is an optional condition to filter items.
	// Example: `item.classification == "hot"`
	// The current item is available as "item".
	Filter string `json:"filter,omitempty"`

	// As defines the variable name for the current item.
	// Defaults to "item". Available under that name in nested steps.
	As string `json:"as,omitempty"`

	// Do contains the steps to execute for each item.
	Do []*Step `json:"do"`

	// Concurrency limits parallel execution. 0 or 1 means sequential.
	Concurrency int `json:"concurrency,omitempty"`
}

ForEachStepConfig configures iteration over a collection.

type FunctionSource

type FunctionSource interface {
	Registry() work.Registry
}

FunctionSource is the construct call graph the loop analysis walks.

func NewFunctionSource

func NewFunctionSource(fns *memql.FunctionRegistry) FunctionSource

NewFunctionSource builds the call graph once from every function fns holds, keyed by every name a call can use: the qualified `<namespace>.<name>` and, where it is unambiguous, the bare name (LookupIndex, as the engine resolves a construct reference out of a compiled body). A nil fns is an empty graph, in which every call is unresolved.

It knows no concepts, so it cannot see the two fields the engine rewrites after a template renders -- a written id in an outgoing @relationship field, and a @scrubPii mutation's PII fields; the loader's check reads concepts too (newFunctionSource).

type FunctionStepConfig

type FunctionStepConfig struct {
	// Name is the function identifier (without parentheses).
	Name string `json:"name"`
	// Args are optional function arguments passed at invocation time.
	Args map[string]any `json:"args,omitempty"`
	// Kind is the construct kind the statement wrote before the call --
	// query, mutation, logic or builtin (epic memql#5370). It decides the
	// shape of the value the call binds (statementValue). Empty for a step
	// built in Go without one.
	Kind string `json:"kind,omitempty"`
}

FunctionStepConfig configures a MemQL function invocation.

type GraphAutomation

type GraphAutomation struct {
	Name, Origin, Trigger, Schedule, Filter string
	Template                                bool
	Stratum                                 int
	Writes, Publishes, Opaque               []string
	BeforeWrite                             *BeforeWriteConfig
	Loop                                    *LoopConfig
	Mode                                    *ModeConfig
	Cycle                                   int // index into Cycles, -1 when none
}

GraphAutomation is one automation of the graph.

type GraphCoverage

type GraphCoverage struct {
	Automations, Resolved, Unresolved, Opaque int
}

GraphCoverage counts what the graph could see: the construct and sub-automation calls it resolved, those it could not, and the opaque ones (builtins, actions, webhooks) it lists without following.

type GraphCycle

type GraphCycle struct {
	Members, PermittedBy, Path []string
}

GraphCycle is one strongly connected component that holds a cycle.

type GraphEdge

type GraphEdge struct {
	From, To, Concept, Topic string
	Via                      []string
	Decided                  bool
	Reason                   string
}

GraphEdge is one automation's write or publish firing another.

type Loader

type Loader struct {
	// contains filtered or unexported fields
}

Loader loads automation definitions from the unified DSL tree.

func NewLoader

func NewLoader(opts LoaderOptions) *Loader

NewLoader creates a new automation loader using embedded files. If opts.Logger is nil, a new logger is created using core.NewLogger with the component's configured color and level from environment variables.

func (*Loader) CompileSource

func (l *Loader) CompileSource(source, origin string) (*Automation, error)

CompileSource compiles a SINGLE automation from an in-memory .memql source string, against the loader's registry, WITHOUT touching any filesystem or live engine state. It runs the exact same parse -> concept-resolution -> trigger-normalization -> compile -> validate pipeline the bootstrap loader uses for on-disk automation files, so a caller (e.g. the authoring sandbox's Gate 1) gets identical parse/bind diagnostics. `origin` is a synthetic path used only for error context and version derivation; it never has to exist on disk.

The registry is consulted read-only for `use`-import concept resolution; nothing is registered into it. Pass an isolated registry clone to bind candidate constructs in isolation.

func (*Loader) LoadAll

func (l *Loader) LoadAll() ([]*Automation, error)

LoadAll loads all automation definitions from the new domain-first DSL tree at dsl.Tree(). Each `<domain>/automations.memql` file contributes a list of automations (extracted via slice-based parsing).

This used to run two further fs.WalkDir passes over an `l.fsys` field, under a comment saying they were "retained for tests that inject a custom FS via the fsys field". No such test existed -- verified across all history (`git log --all -S ".fsys ="`) -- and LoaderOptions exposed no seam to inject through, so it was never an intentional injection point and both passes always walked the empty `fstest.MapFS{}` NewLoader hardcoded.

(Precisely: an IN-PACKAGE test could have assigned `loader.fsys` directly, since every test here is `package automations`. So the seam was reachable in principle; it was simply never used. An earlier draft of this comment said "and none could", which was wrong.)

Removed in memql#2858, along with the `.json` automation loader they reached, which had no live source. If a custom-FS seam is ever actually wanted, add an FS field to LoaderOptions and a test that uses it -- that is a feature decision, not cleanup.

func (*Loader) LoadByName

func (l *Loader) LoadByName(name string) (*Automation, error)

LoadByName loads a runnable automation by its canonical name.

Canonical-name rule (memql#1663). run_automation runs AUTOMATIONS -- the `automation <name>` construct is the runnable entry point (it carries the trigger + the step chain). Resolution is deterministic and tries, in order:

  1. Exact automation name (e.g. registerNode / deregisterNode / bootstrapCluster / pruneStaleClusterNodes / expireDelegations). This is the canonical, documented name and the only previously-supported form.
  2. A logic's name, resolving to the automation whose statement calls it. This makes a logic reachable by the name an author naturally reaches for (the QA pain in #1663: `revokeExpiredDelegations` resolves to the `expireDelegations` automation that calls it).

If no automation matches but the name corresponds to a logic construct that no automation wraps, a clear "not a runnable entry point" error is returned (it is internal-only) -- which beats the old misleading "not found".

Both the live and the dry-run run_automation paths resolve through this one method, so they cannot diverge.

func (*Loader) LoadFromTree

func (l *Loader) LoadFromTree(tree fs.FS) ([]*Automation, error)

LoadFromTree is LoadFromUnifiedTree's walk over any tree laid out as dsl.Tree() is, through every rule the boot walk applies. Boot hands it dsl.Tree(); the edition-2026 migration's tests hand it the tree as the rewrites carry it (migrated_tree_load_test.go, the steps package's automation corpus).

func (*Loader) LoadFromUnifiedTree

func (l *Loader) LoadFromUnifiedTree() ([]*Automation, error)

LoadFromUnifiedTree walks dsl.Tree() looking for `<domain>/automations.memql` files (dsl.LoaderReadsConstruct), extracts every `automation NAME { ... }` block, and compiles each via the existing compileMemQL pipeline.

Slice extraction mirrors the pattern used in component/memql/function_slices.go: regex-match the header, walk balanced braces to the closer, walk backwards from the header to gather the @-attribute + comment preamble.

Returns the list of compiled automations. EVERY load problem, across all three phases -- read, extract, compile -- is collected and, unless the MEMQL_DSL_ALLOW_SKIPS break-glass is set, returned as an error, so the node refuses to boot half-loaded rather than running silently without the automation (memql#2830).

Directories whose name starts with `_` or `.` are skipped as soft-disabled, matching every other DSL walker.

func (*Loader) Snapshot

func (l *Loader) Snapshot(source *Automation) (*Automation, error)

Snapshot prepares an owned copy of a loaded definition. Scope hosts check and execute this copy so a loader reload cannot replace preflighted work. Derived trust and secret annotations are copied, never accepted from JSON.

func (*Loader) StaticGraph

func (l *Loader) StaticGraph() (*LoopGraph, error)

StaticGraph loads every automation of the tree and builds the static loop graph over them: what the OS and the architecture model draw. It refuses when the loader has no function registry, because a graph built without one has no edges and would read as a tree with no cycles.

func (*Loader) Validate

func (l *Loader) Validate(automation *Automation) error

Validate performs full validation on an automation.

type LoaderOptions

type LoaderOptions struct {
	Logger   *slog.Logger
	Registry memoryNodes.Registry
	// Functions is the engine's function registry, from which the static
	// loop check reads what each automation writes (memql#5381). Without it
	// the check does not run, and the load logs that it did not.
	Functions *memql.FunctionRegistry
}

LoaderOptions configures the automation loader.

type LocalAutomationRunner

type LocalAutomationRunner interface {
	// LookupAutomation returns the automation REGISTERED under name, or nil.
	LookupAutomation(name string) *Automation

	// TriggerAutomationWithClientEvent dispatches a run whose trigger payload
	// came from a caller. See the method on Scheduler for why the
	// caller-supplied distinction is a security boundary and not a naming one.
	TriggerAutomationWithClientEvent(ctx context.Context, name string, event *events.Event) (*AutomationExecution, error)
}

LocalAutomationRunner is the seam onto the deployed automation registry and its dispatch. *Scheduler satisfies it; a test can substitute a fake so a routing test is about routing rather than about the automation engine.

type Logger

type Logger interface {
	Debug(msg string, args ...any)
	Info(msg string, args ...any)
	Warn(msg string, args ...any)
}

Logger is the minimal logging surface the guard needs (satisfied by *slog.Logger). Kept as an interface so tests can inject a recorder.

type LogicRunner

type LogicRunner struct {
	// contains filtered or unexported fields
}

LogicRunner implements memql.LogicRunner: it runs a logic's statement body, compiled at load, on the sequence runner an automation's statements run on (logic_statements.go).

A logic call does not fire automation lifecycle events, burn a concurrency slot, or take part in storm detection or dedup: those are properties of automations, not of a function call. Its statements journal as the file comment of logic_statements.go describes.

Wire at app bootstrap via `engine.SetLogicRunner(automations.NewLogicRunner(...))` once the step registry is built. When no runner is wired the engine's executeLogicFunctionCall surfaces an actionable error.

func NewLogicRunner

func NewLogicRunner(engine *memql.MemQLEngine, registry StepExecutorRegistry, logger *slog.Logger) *LogicRunner

NewLogicRunner constructs a LogicRunner. The step registry and engine must be the same instances the automation scheduler / executor use -- per-step caching, integration plug-ins, and AI providers all hang off the engine, and the step executors look them up via stepCtx.Engine.

func (*LogicRunner) PrepareLogicBody

func (r *LogicRunner) PrepareLogicBody(name string, body []map[string]any) (*Automation, error)

PrepareLogicBody creates owned statements for preflight and subsequent RunPreparedLogicBody execution. It does not retain the loader's mutable body or share the ordinary dispatch cache.

func (*LogicRunner) RunLogicBody

func (r *LogicRunner) RunLogicBody(ctx context.Context, fnName string, body []map[string]any, args map[string]any) (any, error)

RunLogicBody implements memql.LogicRunner (see the file comment).

func (*LogicRunner) RunPreparedLogicBody

func (r *LogicRunner) RunPreparedLogicBody(ctx context.Context, a *Automation, args map[string]any) (any, error)

RunPreparedLogicBody executes the exact statements already admitted by a scoped host. The host owns the prepared definition and never mutates it.

func (*LogicRunner) WithoutJournal

func (r *LogicRunner) WithoutJournal() *LogicRunner

WithoutJournal returns a runner whose logic journals nothing, however it is called: the dry-run sandbox's (component/automations/steps), whose preview must leave no run behind, whatever the logic writes.

type LoopConfig

type LoopConfig struct {
	MaxDepth    int             `json:"maxDepth"`
	Until       string          `json:"until"`
	UntilLambda *ast.LambdaExpr `json:"-"`
}

LoopConfig is @loop(maxDepth=N, until=row => P) (D18; D-H of the loop protection plan). The automation closes a deliberate cycle, stops when P holds on its triggering row -- which its @filter must say, by holding P's negation -- and runs at most MaxDepth times in one causal chain.

type LoopGraph

type LoopGraph struct {
	Automations []GraphAutomation
	Edges       []GraphEdge
	Cycles      []GraphCycle
	Problems    []LoopProblem
	Coverage    GraphCoverage
	// contains filtered or unexported fields
}

LoopGraph is the static graph over a set of automations.

func BuildLoopGraph

func BuildLoopGraph(automations []*Automation, src FunctionSource, depthCap int) *LoopGraph

BuildLoopGraph builds the static graph over automations, with the call graph src reads (a nil src is an empty one, in which every construct call is unresolved and the coverage says so). depthCap, when positive, is the chain depth cap a @loop's maxDepth must not exceed; the load's own prepare step owns that check, and a caller holding the cap passes it so an automation built another way is held to it too.

The graph is deterministic: automations sort by (name, origin), and every list the graph holds is sorted, so the same input in any order builds the same graph -- it is printed in a refusal and drawn in the OS.

type LoopProblem

type LoopProblem struct {
	Automation, Origin, Code, Message string
}

LoopProblem is one refusal the graph makes.

type LoopRefusal

type LoopRefusal struct {
	Reason     string       // metrics.LoopStopDepth | metrics.LoopStopLoopBound
	Automation string       // the automation that would have run
	Depth      int          // the depth the refused run would have had
	Cap        int          // the bound it exceeded
	Cause      events.Cause // the refused run's cause, chain included
}

LoopRefusal is a run the loop protection stopped before it started (D19, D-C): loop_depth_exceeded, carrying the chain.

func (*LoopRefusal) Error

func (r *LoopRefusal) Error() string

Error is the refusal in one line: the automation, the depth and the bound it passed, and the chain that led here, ending with the rule id the corpus reads -- "loop_depth_exceeded: <automation> would run at depth 17, past the cap of 16; chain: a (run-1) -> b (run-2) -> ... [loop_depth_exceeded]". It is the errorMessage the refused run's row carries.

func (*LoopRefusal) PriorRuns

func (r *LoopRefusal) PriorRuns() []events.Link

PriorRuns is the chain that led to the refusal, oldest first: the refused run's own chain without the refused run, which never ran. It is the chain the refused run's row records and the one its message names.

type MembershipFunc

type MembershipFunc func() []string

MembershipFunc returns the current set of live node ids in the cluster (including this node). The authored owner gate reads it on every scheduled firing so a membership change rebalances owner assignments without a restart. An empty or nil result means "single node" -- this node owns everything.

type MinimalStepResult

type MinimalStepResult struct {
	StepId    string         `json:"stepId"`
	Status    string         `json:"status"`
	Result    any            `json:"result,omitempty"`
	Error     string         `json:"error,omitempty"`
	Metadata  map[string]any `json:"metadata,omitempty"`
	ContentId string         `json:"contentId,omitempty"`
	// Value is StepResult.Bound: the value a statement bound its name to,
	// which a resumed statement body rebinds the name to.
	Value         any  `json:"value,omitempty"`
	ValueRecorded bool `json:"valueRecorded,omitempty"`
}

MinimalStepResult stores only the fields the evaluator needs to be rehydrated on resume. It is what a v1:work:step row's `result` carries, and keeping it minimal is why a run's journal does not grow with the size of every query result the run read.

type ModeConfig

type ModeConfig struct {
	Kind string `json:"kind"`
	Max  int    `json:"max,omitempty"`
}

ModeConfig is @mode(<kind>[, max=N]) (D19; D-G of the loop protection plan). Max applies to queued (the most fires that may wait) and parallel (the most runs at once); 0 takes the default.

type ModeRefusal

type ModeRefusal struct {
	Automation, Mode, InFlightRunId string
	Max                             int
}

ModeRefusal describes a fire refused by the process-local concurrency mode.

func (*ModeRefusal) Error

func (r *ModeRefusal) Error() string

type ParallelStepConfig

type ParallelStepConfig struct {
	// Branches are the steps to execute concurrently.
	Branches []*Step `json:"branches"`

	// Wait strategy: "all" (default), "any", or "none".
	// "all" - wait for all branches to complete
	// "any" - continue when first branch completes
	// "none" - fire and forget
	Wait string `json:"wait,omitempty"`

	// FailFast stops all branches when one fails.
	FailFast bool `json:"failFast,omitempty"`
}

ParallelStepConfig configures concurrent step execution.

type Precondition

type Precondition struct {
	// ID is a stable identifier for this precondition within its
	// automation (e.g. "engineImageDigestPinned"). It keys the miss
	// signal so the repair loop (E4.4) and the typed-patch model (E4.3)
	// can target the exact construct to heal. Required.
	ID string `json:"id"`

	// Check is the deterministic boolean expression that must hold for
	// the automation to proceed. Evaluated false => the precondition
	// MISSES. Required. Example: `args.imageDigest != ""`.
	Check string `json:"check"`

	// Literal optionally names the machine-specific literal this
	// precondition asserts (a path, an id, an endpoint). It is the
	// portability hint the repair loop reads when proposing a
	// relativize-literal / rebind-param patch (E4.3): the literal that
	// does not hold here is what must be relativized to travel. Optional;
	// when empty the repair loop infers the literal from Check.
	Literal string `json:"literal,omitempty"`

	// Description is human-readable context surfaced in the miss signal
	// and in the cockpit healed-pack UI (E4.6). Optional.
	Description string `json:"description,omitempty"`
	// contains filtered or unexported fields
}

Precondition is a first-class deterministic check attached to an automation (Epic 4 / memql#2139). It is the unit of self-healing portability: a precondition asserts that some condition holds before the automation's steps run, and a miss is the clean repair trigger.

A precondition is intentionally minimal and DETERMINISTIC -- it never calls an LLM. The harness evaluates Check the way it evaluates a statement's condition and a trigger filter (EvalCondition over the run), over the roots the run binds before its first statement: args, actor, event, config, partition and now. The deploy spine stays authored/deterministic: preconditions guard it but are never themselves LLM-healed.

func EvaluatePreconditions

func EvaluatePreconditions(preconditions []*Precondition, eval *Evaluator) (*Precondition, bool)

EvaluatePreconditions runs each precondition's deterministic check against the supplied evaluator (the same one the steps use, already seeded with the run's event, input, config and var context). It returns the FIRST precondition that misses (evaluates false or errors), or nil when all hold. A check that errors is treated as a miss -- the conservative choice for a self-healing trigger: an unevaluable literal on this machine is exactly the cross-machine drift a precondition exists to catch. The boolean second return is true when a miss occurred.

type RerunSpec

type RerunSpec struct {
	// RequestId identifies the request. The dispatcher claims the run under
	// it, so a re-run is never refused by the lease of the execution before
	// it.
	RequestId string
	// RequestedAt distinguishes the intake's fresh heartbeat from one written
	// by an executor. An interrupted prior version may still have running steps.
	RequestedAt time.Time
	// Reason is RerunReasonRerun, RerunReasonHeadMove or RerunReasonBranch.
	Reason string
	// StepKey is the top-level step the request targets: the step re-run, the
	// first stale step of a head move, or the fork step of a branch.
	StepKey string
	// Override is what the person changed for the targeted step (D20); nil
	// when they changed nothing about the call.
	Override *common.StepOverride
	// Snapshot is the workspace the targeted session step starts against.
	Snapshot *common.WorkspaceSnapshot
	// Workspace is a fresh workspace for every step of this execution; empty
	// keeps the default.
	Workspace string
	// RequestedBy is the person who asked. The targeted version names them
	// as its author when the override carries a prompt or inputs.
	RequestedBy string
	// OverrideRecord is the override exactly as the request stored it. The
	// targeted version's `override` field records it verbatim, so a reader
	// sees what the person asked for rather than this package's reading of
	// it. Nil records one rendered from Override.
	OverrideRecord map[string]any
	// Versions is the version each re-executed step runs as, when the act
	// that planned the request wrote it (rerun.versions, from
	// work.PlanRerun over EVERY recorded version). It is a FLOOR: the step
	// rows this package reads are the collapsed newest row-versions, and
	// after a head move re-asserted an earlier version the newest row no
	// longer carries the highest version number -- so without the plan's
	// numbers a re-run could reuse one. Nil falls back to the newest row.
	Versions map[string]int
}

RerunSpec is a re-run request as the executor serves it: decoded from run.rerun, or built by a caller that drives ResumeFrom itself.

type ResumeOptions

type ResumeOptions struct {

	// FromStep overrides the resume point (defaults to the failed step).
	FromStep string

	// AllowSideEffects explicitly permits repeating effects or unclassified
	// operations. It is not deduplication and must not be set by automatic
	// recovery merely because an operation lost its receipt.
	AllowSideEffects bool

	// Rerun serves a re-run request (epic memql#5414, task memql#5415): the
	// run need not have a failed step, every step from the resume point on
	// runs as a new version, the steps before it are served from what they
	// recorded, and the override reaches the targeted step alone. FromStep
	// empty resumes where the request stands (rerunResumePoint);
	// PrepareRerun builds the journal and the options together.
	Rerun *RerunSpec

	// Overrides are the overrides a REPLAY applies, by top-level step key:
	// the replayed run's StepOverrides. Each reaches its own step alone, as a
	// re-run's does (withRunContext). A re-run's own request wins for the
	// step it targets.
	Overrides map[string]*common.StepOverride
	// contains filtered or unexported fields
}

ResumeOptions configures resume behavior.

type ReturnStepConfig

type ReturnStepConfig struct {
	Value string `json:"value,omitempty"`
}

ReturnStepConfig configures a return step: the canonical v1 source of the returned value, empty for a bare `return`.

type RunAccepted

type RunAccepted struct {
	RunId                 string
	Automation            string
	RanDeployedDefinition bool
	DefinitionNote        string
	TriggerKind           string
	TriggerTopic          string
	RequestedOnNodeId     string
	RequestedOnNodeType   string
	TargetNodeType        string
}

RunAccepted is the first frame of every run.

type RunActor

type RunActor struct {
	// UserId is the acting actor's user id.
	UserId string
	// Role is its role.
	Role string
	// Inherited is true when the identity came with the context the run was
	// started under -- a caller's, an author's -- and false when the executor
	// gave the automation its own actor.
	Inherited bool
	// MaintenancePrincipal is true when the run held its own automation's
	// maintenance principal (auth.IsMaintenancePrincipalOf).
	MaintenancePrincipal bool
}

RunActor says who a run acted as. The executor records it from the context it has just installed the run's actor on (installRunActor), so anything that reports on a run states its actor from the run itself and does not derive it a second time.

type RunAdoption

type RunAdoption struct {
	// RunId is the v1:work:run row this execution IS. The execution's id
	// becomes this, so every step row, heartbeat and terminal status the
	// journal writes lands on the run the caller is already watching.
	RunId string

	// TriggeredBy is recorded on the run. Compile-dispatched runs pass
	// "compiled"; the sweep's backstop passes "swept". Empty defaults to
	// "adopted" rather than being left blank, because an empty triggeredBy
	// is indistinguishable from a row written before the field existed.
	TriggeredBy string

	// Variables are the bound args the compiled template runs with. They
	// are seeded as the execution's input so step args resolve against
	// them, and are NOT re-written to the row -- compile already recorded
	// them, and a second writer of one field is a field with two versions
	// of the truth.
	Variables map[string]any

	// Journal restores the original context when recovering a scheduler run.
	// Goal runs retain the caller-origin boundary for compiled variables.
	Journal *RunJournal

	// Overrides are the overrides a REPLAY applies, by top-level step key
	// (epic memql#5414): the replayed run's StepOverrides. The model calls a
	// replay serves were journaled with each step's override applied, and
	// the request hash covers what the override added, so a step replayed
	// without its override asks a different question and diverges. Each
	// reaches its own step alone (withRunContext).
	Overrides map[string]*common.StepOverride
}

RunAdoption names the existing run an execution should take over.

type RunAutomationFunc

type RunAutomationFunc func(ctx context.Context, automation *Automation, triggeringEvent *events.Event) error

RunAutomationFunc executes one compiled authored automation under the already-author-scoped ctx, with the triggering event (nil for a scheduled firing). The app wires this to the live Executor; tests supply a recorder. It returns an error so the per-automation circuit breaker (increment 3) can observe faults.

func NewAuthoredRunner

func NewAuthoredRunner(opts AuthoredRunnerOptions) RunAutomationFunc

NewAuthoredRunner builds the RunAutomationFunc the app hands the authored scheduler: event firings through the claiming executor, scheduled firings through the plain one.

type RunComplete

type RunComplete struct {
	RunId              string
	Status             string
	DurationMs         int64
	StepCount          int32
	Error              string
	ErrorCode          int32
	ErrorMessage       string
	ExecutedOnNodeId   string
	ExecutedOnNodeType string
}

RunComplete terminates a trace. Exactly one is emitted per run, including for a run that was refused before it started -- a caller can therefore always park on it.

type RunJournal

type RunJournal struct {
	Spine                 map[string]any
	ExecutionAuthority    map[string]any
	HeartbeatAt           time.Time
	HasRunningStep        bool
	GoalId                string
	Status                string
	OwnerUserId           string
	Variables             map[string]any
	Mode                  string
	ReplayPolicy          string
	ForkedFromRunId       string
	ForkAtStepKey         string
	WaitingOn             map[string]any
	HumanResumeId         string
	ParentRunId           string
	CancelRequested       bool
	RunId                 string
	AutomationName        string
	TemplateVersion       string
	TemplateConstructId   string
	TemplateFingerprint   string
	TriggeredBy           string
	Input                 any
	InputFingerprint      string
	TriggerEvent          map[string]any
	CallerSuppliedPayload bool
	ChainHead             string
	InitialChainHead      string
	StepOrder             []string
	// Steps holds the completed (done) steps by key, in the MinimalStepResult
	// shape the evaluator is rehydrated from.
	Steps map[string]*MinimalStepResult
	// FailedStep is the key of the step at `failed` or `running` with no
	// receipt -- the default resume point.
	FailedStep string
	// StepStates is every step's latest row, by key: its status and the
	// attempt it recorded. A statement body resumes from these (epic
	// memql#5370): a `failed` statement that carries `on error continue` was
	// continued past, and stays so.
	StepStates map[string]StepState
	// MaxAttempt is the highest version each top-level step's newest row
	// records, by key: its attempt or its version, whichever is higher. A
	// step executed again runs one past it (epic memql#5414). It is read off
	// the collapsed newest rows, so after a head move re-asserted an earlier
	// version it is that version's number -- which is why a re-run takes the
	// plan's numbers as a floor (RerunSpec.Versions).
	MaxAttempt map[string]int
	// Head is the run's stored head (epic memql#5414, design D18): the
	// current version of every top-level step, an entry naming another run
	// when that version lives there. Nil on a run written before it existed.
	Head work.Head
	// StaleSteps are the steps whose current version was computed against an
	// upstream that is no longer current: written by a person's act, and
	// taken off one by one as a re-run gives each a finished new version.
	StaleSteps []string
	// Rerun is the pending re-run request, decoded from run.rerun; nil when
	// absent or cleared to {}.
	Rerun *RerunSpec
	// StepOverrides is the override each top-level step's newest row
	// records, by key: the override its head version ran with (the newest
	// row IS the head, a head move re-asserting the version it makes
	// current). Only steps somebody overrode appear. A replay of this run
	// applies them (ResumeOptions.Overrides, RunAdoption.Overrides), because
	// the model calls its journal holds were made with them.
	StepOverrides map[string]*common.StepOverride
	// Routing is the owner's routing choice for every model call of the run
	// (run.routing, the Ask route picker's), read off the row so the agent
	// that executes the run needs nothing from the node that took the turn.
	// The zero value routes by the rules.
	Routing common.RouteChoice
	// OutcomeRecovery retains a completed App's reporting-only failure. Its
	// human answer resumes the same generation, unlike a fresh effect retry.
	OutcomeRecovery bool
}

RunJournal is what resume needs from the rows: the run's envelope and the completed steps' trimmed results.

func LoadRunJournal

func LoadRunJournal(ctx context.Context, exec journalExecutor, runId string) (*RunJournal, error)

LoadRunJournal reads one run and its steps, under the journal's own synthetic cluster actor. The caller decides WHO may resume before calling this; the actor here exists because the rows are the deployment's and the reads carry `actor.isClusterOwner==true`.

type RunRelay

type RunRelay struct {
	// contains filtered or unexported fields
}

RunRelay owns the invoke path on one node: it runs locally-targeted runs, relays remotely-targeted ones over the mesh, and -- on the receiving side -- executes runs other nodes relayed to it.

func NewRunRelay

func NewRunRelay(opts RunRelayOptions) (*RunRelay, error)

NewRunRelay builds a relay. It does not subscribe: call Start.

func (*RunRelay) NodeId

func (r *RunRelay) NodeId() string

NodeId reports the node this relay runs on.

func (*RunRelay) NodeType

func (r *RunRelay) NodeType() string

NodeType reports the node type this relay runs on.

func (*RunRelay) Run

func (r *RunRelay) Run(ctx context.Context, req RunRequest, sink RunSink)

Run executes one operator-initiated run and reports every frame to sink. It returns only when the run is finished, refused or timed out, and it ALWAYS emits exactly one Complete frame before returning.

func (*RunRelay) Start

func (r *RunRelay) Start()

Start subscribes the two relay topics. Safe to call on a relay with no event bus (single-node / no-mesh binaries): the local path never touches the bus, so the relay is fully functional without it.

func (*RunRelay) Stop

func (r *RunRelay) Stop()

Stop releases the relay's subscriptions.

type RunRelayOptions

type RunRelayOptions struct {
	Logger *slog.Logger

	// Runner is the local dispatch seam. Required.
	Runner LocalAutomationRunner

	// EventBus carries the cross-node request/trace hop. Nil disables remote
	// targeting: a run naming a different node type is then refused with
	// UNAVAILABLE rather than hanging, which is the honest answer for a
	// binary with no mesh.
	EventBus *events.Bus

	// NodeId / NodeType identify THIS node. They are reported on every frame
	// so the UI (and the cluster-e2e assertion) can see which node took the
	// request and which one ran the automation.
	NodeId   string
	NodeType string

	// Claimer, when set, makes a relayed run exactly-once across the replicas
	// of the target node type: the request broadcasts to every peer, and the
	// claim decides which replica of the matching type actually runs it.
	// Typically the same *ClusterExecutionGuard the scheduler uses. Nil =
	// single-replica behaviour (every matching node would run it).
	Claimer executionClaimer

	// MaxConcurrentRuns caps operator-initiated runs in flight on this node.
	// Zero takes defaultMaxConcurrentRuns.
	MaxConcurrentRuns int

	// DefaultTimeout / MaxTimeout bound a run. Zero takes the defaults.
	DefaultTimeout time.Duration
	MaxTimeout     time.Duration
}

RunRelayOptions configures a RunRelay.

type RunRequest

type RunRequest struct {
	// Automation is the automation's DSL name. Required.
	Automation string

	// Payload is the synthesized trigger event's payload.
	Payload map[string]any

	// Concept optionally names the concept the payload row belongs to, used
	// to make a wildcard trigger pattern concrete.
	Concept string

	// Topic optionally overrides the synthetic event's topic outright.
	Topic string

	// TargetNodeType routes the run to a node type. Empty = this node.
	TargetNodeType string

	// Timeout bounds the run from the caller's point of view. Zero takes the
	// default; the relay also caps it.
	Timeout time.Duration

	// IncludeStepOutput attaches each step's result payload to its trace
	// entry.
	IncludeStepOutput bool
}

RunRequest is one operator-initiated automation run.

type RunScope

type RunScope struct {
	// contains filtered or unexported fields
}

RunScope is a run's state as a memql.ExprScope. It is cheap to build: one per evaluation.

func (*RunScope) Lookup

func (s *RunScope) Lookup(name string) (any, bool)

Lookup resolves one bare name (see the file comment for the order).

type RunSink

type RunSink interface {
	Accepted(RunAccepted)
	Step(RunStep)
	Complete(RunComplete)
}

RunSink receives a run's frames in order. Implementations are called from the goroutine driving the run and must not block for long.

type RunStep

type RunStep struct {
	RunId      string
	Sequence   int32
	StepId     string
	Status     string
	DurationMs int64
	Output     any
	Error      string
}

RunStep is one step's outcome.

type Scheduler

type Scheduler struct {
	// contains filtered or unexported fields
}

Scheduler manages scheduled automation execution.

func NewScheduler

func NewScheduler(opts SchedulerOptions) (*Scheduler, error)

NewScheduler creates a new automation scheduler. If opts.Logger is nil, a new logger is created using common.NewLogger with the component's configured color and level from environment variables.

func (*Scheduler) AutomationGraphRows

func (s *Scheduler) AutomationGraphRows() ([]map[string]any, error)

AutomationGraphRows is the static loop graph over the automations this scheduler registered, one row per automation (LoopGraphRows), satisfying memql.AutomationGraphSource.

BUILT ONCE PER REGISTERED SET. The set is keyed by (name, definition) -- the pointer, not only the name, so an automation re-registered under the same name is never served its predecessor's edges -- and today it is filled once, at the scheduler's start, so the graph is walked once per boot however often the page is read.

The rows are SHARED between callers and must not be mutated; the engine marshals each one and lets go of it.

func (*Scheduler) CatalogAutomations

func (s *Scheduler) CatalogAutomations() []memql.AutomationCatalogEntry

CatalogAutomations reports every loaded automation to the engine's construct catalog (memql#3749), satisfying memql.AutomationCataloger.

It reports what this SCHEDULER holds, not what the tree declares. That is the point of the seam: an automation the loader skipped is not in the map, and a file walk would have listed it as something the cluster runs.

func (*Scheduler) ComponentName

func (s *Scheduler) ComponentName() common.ComponentName

ComponentName returns the component identifier.

func (*Scheduler) EnterAutomationInvocation

func (s *Scheduler) EnterAutomationInvocation(ctx context.Context, _ string) (context.Context, error)

func (*Scheduler) GetAutomations

func (s *Scheduler) GetAutomations() []*Automation

GetAutomations returns all loaded automations.

func (*Scheduler) IsRunning

func (s *Scheduler) IsRunning() bool

IsRunning reports whether the scheduler is active.

func (*Scheduler) LookupAutomation

func (s *Scheduler) LookupAutomation(name string) *Automation

LookupAutomation returns the automation the scheduler REGISTERED under name, or nil. It is the deployed definition -- the same object the bus subscription and the cron entry hold -- which is exactly what the run path (memql#3310) must report on, since session-define does not cover automations and there is therefore never a caller-supplied body to prefer.

Distinct from Loader.LoadByName, which re-reads the tree and can resolve a name the scheduler did not register (a disabled automation, a wrapped logic construct). Callers that want "what would fire if this event happened" want this one.

func (*Scheduler) Order

func (s *Scheduler) Order() int

Order returns the startup order (after engine).

func (*Scheduler) Ready

func (s *Scheduler) Ready() <-chan struct{}

Ready returns a channel that is closed when the scheduler is ready.

func (*Scheduler) ResolveAutomation

func (s *Scheduler) ResolveAutomation(_ context.Context, name string) (*Automation, error)

ResolveAutomation returns only the deployed definition used for invocation.

func (*Scheduler) ResumeAutomation

func (s *Scheduler) ResumeAutomation(ctx context.Context, executionId string, fromStep string) (*AutomationExecution, error)

ResumeAutomation resumes a failed automation from a checkpoint. It loads the checkpoint by executionId, validates it, and resumes execution from the specified step (or the failed step if fromStep is empty).

func (*Scheduler) SetWiring

func (s *Scheduler) SetWiring(w *bus.Wiring)

SetWiring configures the bus wiring for channel-based communication. When set, the scheduler can route engine requests through the bus instead of calling engine methods directly.

func (*Scheduler) Start

func (s *Scheduler) Start(ctx context.Context)

Start begins the scheduler.

func (*Scheduler) Stop

func (s *Scheduler) Stop(ctx context.Context)

Stop halts the scheduler.

func (*Scheduler) TriggerAutomation

func (s *Scheduler) TriggerAutomation(ctx context.Context, name string) (*AutomationExecution, error)

TriggerAutomation manually triggers an automation by name.

func (*Scheduler) TriggerAutomationWithArgs

func (s *Scheduler) TriggerAutomationWithArgs(ctx context.Context, name string, args map[string]any) (*AutomationExecution, error)

TriggerAutomationWithArgs runs a sub-automation by name with a synthetic event whose payload carries the supplied args. The sub-automation's evaluator sees the args under `event.X` (because `event` translates to `ctx.input` and the executor stamps the event payload as the input envelope).

Used by the automation-within-automation step kind: when a parent automation has `step welcome { automation seedX { ... } }`, the AutomationExecutor calls this method so the sub-automation receives the call's args as its input.

func (*Scheduler) TriggerAutomationWithClientEvent

func (s *Scheduler) TriggerAutomationWithClientEvent(ctx context.Context, name string, event *events.Event) (*AutomationExecution, error)

TriggerAutomationWithClientEvent runs a named automation with a synthetic event whose PAYLOAD CAME FROM A CALLER, and is the entry point the stream's run path (memql#3310) uses.

The distinction from TriggerAutomationWithEvent is a security boundary, not a naming preference. Trust rides on where the automation's BODY came from (memql#2800) AND on whether the caller supplied the trigger payload (memql#2888): a trusted body driven by a caller-chosen argument still must not reach a @serverOnly construct, because for a @serverOnly construct the argument IS the authorization decision. ExecuteWithClientEvent stamps client origin for exactly that reason, and persists the fact onto any checkpoint the run mints so a later resume cannot promote it back to internal origin.

The MCP run_automation path made the same choice for the same reason. Any new caller-driven invoke path belongs here rather than on TriggerAutomationWithEvent -- if you find yourself reaching for that one from a request handler, this comment is the reason not to.

Uses the EVENT executor, so a run is subject to the same per-process dedup and cross-replica claim a real trigger is. That is deliberate: an operator firing the same run twice in a few seconds is far more likely to be a double-click than an intentional second execution, and the dedup window makes the invoke path behave like the trigger path it is standing in for.

func (*Scheduler) TriggerAutomationWithEvent

func (s *Scheduler) TriggerAutomationWithEvent(ctx context.Context, name string, event *events.Event) (*AutomationExecution, error)

TriggerAutomationWithEvent triggers an automation with a synthetic event. This allows HTTP endpoints to invoke event-triggered automations (like bootstrapUser) with the same event data structure as if the event had been published via the event bus.

type SchedulerOptions

type SchedulerOptions struct {
	Logger       *slog.Logger
	Loader       *Loader
	Engine       *memql.MemQLEngine
	EventBus     *events.Bus
	StepRegistry StepExecutorRegistry

	// LeaderGate, when set, restricts SCHEDULED (cron) automation execution
	// to the one cluster-wide leader node (#561). nil = every node runs
	// crons (pre-#561 / single-node default). Typically CronLeader.IsLeader.
	LeaderGate func() bool

	// ScheduleGate optionally selects leadership per automation. When set it
	// replaces LeaderGate for scheduled firings only; manual and event runs are
	// unchanged. The application supplies placement and lease policy.
	ScheduleGate func(string) bool

	// ClusterGuard, when set, makes EVENT-triggered automations exactly-once
	// across replicas (#561) -- the event executor claims each (automation,
	// event) in the DB so only one replica runs it. nil = single-replica
	// behaviour. Typically a *ClusterExecutionGuard.
	ClusterGuard executionClaimer
}

SchedulerOptions configures the automation scheduler.

type Step

type Step struct {
	FieldWrite *FieldWriteConfig `json:"fieldWrite,omitempty"`
	// ID identifies this step within its list: the key its result and its
	// journal record carry. A statement's value is read by the name it binds
	// (Binds), never by its id.
	ID string `json:"id"`

	// Name is a human-readable description of the step.
	Name string `json:"name,omitempty"`

	// Type determines which executor handles this step.
	Type StepType `json:"type"`

	// OnError defines error handling strategy. Defaults to "stop".
	OnError ErrorStrategy `json:"onError,omitempty"`

	// RetryCount specifies how many times to retry on failure (when onError is "retry").
	RetryCount int `json:"retryCount,omitempty"`

	// Condition is an optional expression that must be true for the step to run.
	// Example: `classify.count() > 0`
	Condition string `json:"condition,omitempty"`

	// Event configuration (type: "event")
	Event *EventStepConfig `json:"event,omitempty"`

	// Function configuration (type: "function")
	Function *FunctionStepConfig `json:"function,omitempty"`

	// Action configuration (type: "action") -- action-library replay (#1758).
	Action *ActionStepConfig `json:"action,omitempty"`

	// Automation configuration (type: "automation")
	Automation *AutomationStepConfig `json:"automation,omitempty"`

	// ForEach configuration (type: "forEach")
	ForEach *ForEachStepConfig `json:"forEach,omitempty"`

	// Parallel configuration (type: "parallel")
	Parallel *ParallelStepConfig `json:"parallel,omitempty"`

	// Binds is the name a step binds its value under, when its statement
	// has one (`x := ...`). It is separate from ID: the id names the step in
	// the run record and is unique within its list, and two sibling branches
	// may bind the same name.
	Binds string `json:"binds,omitempty"`

	// Returns marks a call step written `return <call>`: the sequence ends
	// after it, with the call's value as the returned value.
	Returns bool `json:"returns,omitempty"`

	// Expression is an expression step's canonical v1 source (type
	// "expression").
	Expression string `json:"expression,omitempty"`

	// Return configures a return step (type "return").
	Return *ReturnStepConfig `json:"return,omitempty"`

	// Block configures a block step (type "block").
	Block *BlockStepConfig `json:"block,omitempty"`

	// Exprs holds this step's expressions parsed once at load
	// (PrepareExpressions): the step executors evaluate them with EvalExpr,
	// and read every string leaf of its value maps as a literal. Nil for a
	// step that was never prepared, which the executors refuse.
	Exprs *StepExprs `json:"-"`
}

Step represents a single operation in an automation.

type StepContext

type StepContext struct {
	// ResumeChild exists only for a proven suspended automation call. It never
	// authorizes an ordinary invocation or a different child/argument set.
	ResumeChildStepID string
	ResumeChild       func(context.Context, string, string, map[string]any) (*AutomationExecution, error)

	Logger            *slog.Logger
	Engine            *memql.MemQLEngine
	EventBus          *events.Bus
	Evaluator         *Evaluator
	Execution         *AutomationExecution
	AutomationTrigger AutomationTrigger
	TriggeringEvent   *events.Event // The event that triggered this automation (if any)

	// PreviousChainHead is the chain state before this step executes. The
	// chain advances over the top-level statements only: a statement in a
	// `for` body or a parallel branch carries the chain position of the
	// statement that holds it.
	PreviousChainHead string

	// ChainTrackingEnabled indicates whether chain tracking is active.
	// Propagated from ExecutorOptions to child contexts.
	ChainTrackingEnabled bool
}

StepContext holds the context for step execution.

type StepExecutorRegistry

type StepExecutorRegistry interface {
	Execute(ctx context.Context, step *Step, stepCtx *StepContext) (*StepResult, error)
}

StepExecutorRegistry provides step executors.

type StepExprs

type StepExprs struct {
	// Condition gates the step.
	Condition ast.ExpressionNode
	// Source is a forEach step's source.
	Source ast.ExpressionNode
	// Filter is a forEach step's per-item filter.
	Filter ast.ExpressionNode
	// Topic is an event step's.
	Topic ast.ExpressionNode
	// Value is an expression step's expression, or a return step's value
	// (nil for a bare return).
	Value ast.ExpressionNode
}

StepExprs is one step's expressions, parsed at load. A nil field is a position the step does not use.

type StepObserver

type StepObserver func(*StepResult)

StepObserver is called once per step as the step's result is recorded, with the executor's own *StepResult -- the same value that lands in AutomationExecution.Steps and feeds the next step's condition evaluation.

It is called SYNCHRONOUSLY on the executing goroutine, in step order, and must therefore not block: a slow observer slows the automation. The run relay's observer does a non-blocking send onto a buffered channel and drops on overflow, which is the right shape for anything wired here.

The *StepResult must not be mutated. It is live: later steps read it back through the evaluator as $steps.<id>.

type StepResult

type StepResult struct {
	// StepId identifies which step produced this result.
	StepId string `json:"stepId"`

	// Status is "success", "failed", "skipped", "waiting".
	Status string `json:"status"`

	// Result contains the step's output data.
	Result any `json:"result,omitempty"`

	// Error contains any error message.
	Error string `json:"error,omitempty"`

	// Duration is how long the step took.
	Duration time.Duration `json:"duration"`

	// StartedAt is when the step began.
	StartedAt time.Time `json:"startedAt"`

	// CompletedAt is when the step finished.
	CompletedAt time.Time `json:"completedAt"`

	// Children contains results from nested steps (forEach, parallel, switch).
	Children []*StepResult `json:"children,omitempty"`

	// Metadata contains additional step-specific information.
	Metadata map[string]any `json:"metadata,omitempty"`

	// ContentId is the deterministic fingerprint of this step's execution.
	ContentId string `json:"contentId,omitempty"`

	// PreviousChainHead links to the chain state before this step executed.
	PreviousChainHead string `json:"previousChainHead,omitempty"`

	// ChildFingerprints contains ordered fingerprints for ForEach/Parallel children.
	ChildFingerprints []string `json:"childFingerprints,omitempty"`

	// Bound is the value a statement bound its name to, or returned, as its
	// consumers read it (epic memql#5370): what the journal records so a
	// resumed statement body can rebind the name without re-running the
	// statement. Nil outside statement bodies, for a statement that binds
	// nothing, and for a query's rows past maxJournaledRows (resume re-reads
	// those).
	Bound any `json:"-"`
	// BoundRecorded distinguishes a saved nil from a value omitted for size.
	BoundRecorded bool `json:"-"`
}

StepResult contains the outcome of executing a step.

type StepState

type StepState struct {
	Status     string
	ChildRunId string
	Attempt    int
	// Version is the version the row belongs to (epic memql#5414); 0 on a
	// row written before the field existed, whose attempt is its version.
	Version int
}

StepState is one step's latest journal row: its status and its attempt.

type StepType

type StepType string

StepType defines the kind of step operation.

const (
	// StepTypeEvent publishes an event to the event bus: `publish`.
	StepTypeEvent StepType = "event"

	// StepTypeFunction calls a query, a mutation, a logic or a builtin.
	StepTypeFunction StepType = "function"

	// StepTypeAction replays an action-library action by reference (#1758).
	StepTypeAction StepType = "action"

	// StepTypeForEach runs its list once per item: `for`.
	StepTypeForEach StepType = "forEach"

	// StepTypeParallel runs its branches at once: `parallel`.
	StepTypeParallel StepType = "parallel"

	// StepTypeAutomation invokes another automation.
	StepTypeAutomation StepType = "automation"

	// StepTypeExpression evaluates one expression in process: `x :=
	// <expression>`.
	StepTypeExpression StepType = "expression"

	// StepTypeReturn ends the sequence it is in with a value: `return
	// <expression>`.
	StepTypeReturn StepType = "return"

	// StepTypeBlock runs a list of steps in order in a scope of its own: a
	// parallel statement's branch.
	StepTypeBlock StepType = "block"
)

The step kinds are the ones a statement compiles to (epic memql#5370, component/language/compiler/body_compile.go).

const StepTypeFieldWrite StepType = "fieldWrite"

type StrictClusterClaimer

type StrictClusterClaimer struct {
	// contains filtered or unexported fields
}

StrictClusterClaimer admits work only after PostgreSQL confirms its claim. It shares the guard's claims and counters without using the scheduler's allowance for unguarded execution during a storage outage.

func (StrictClusterClaimer) ClaimWithTTL

func (c StrictClusterClaimer) ClaimWithTTL(ctx context.Context, name, key string, ttl time.Duration) bool

func (StrictClusterClaimer) HasLiveClaim

func (c StrictClusterClaimer) HasLiveClaim(ctx context.Context, name string, keys []string, ttl time.Duration) (bool, error)

HasLiveClaim applies the same conflict test as ClaimWithTTL to exact aliases of an existing claim. It does not acquire, renew, release or rename a claim. Once-only callers honor any existing row; recovery honors its stored lease, or the ordinary takeover interval for a row created before leases existed.

func (StrictClusterClaimer) HasRecentClaim

func (c StrictClusterClaimer) HasRecentClaim(ctx context.Context, name string, keys []string, maxAge time.Duration) (bool, error)

HasRecentClaim observes real arbitration, without acquiring or refreshing a lease. Recovery can briefly preserve a run claimed on a peer even before its first heartbeat; an old claim is never counted as progress indefinitely.

func (StrictClusterClaimer) Release

func (c StrictClusterClaimer) Release(ctx context.Context, name, key string) bool

Release gives back a claim this node took, so the same (name, key) can be claimed again. See ClusterExecutionGuard.ReleaseClaim.

type SymptomClassifier

type SymptomClassifier interface {
	// Level is the level the implementation resolves the call at. It is on
	// the interface so a test can assert it is `fast` without reaching into
	// the implementation: the cheapest tier is a property of the design, not
	// an implementation detail.
	Level() airoute.Level
	Classify(ctx context.Context, in ClassifySymptomInput) (work.Symptom, work.Evidence, error)
}

SymptomClassifier is the ONE model call this path may make. It is reached only when component/work's deterministic table has no opinion.

The interface exists rather than a direct engine call because component/automations' journal holds a query executor and nothing else, and because a test must be able to COUNT the calls: "a rules-classified failure makes zero provider calls" is only checkable if the seam can be counted.

type TriggerConfig

type TriggerConfig struct {
	// Event is the topic pattern to subscribe to.
	// Supports glob patterns: "graph.node.*", "automation.#", etc.
	Event string `json:"event"`

	// Filter is an optional condition that must be true for the trigger to
	// fire: a one-parameter lambda over the triggering row.
	// Example: `row => row.status == "completed"`
	Filter string `json:"filter,omitempty"`

	// FilterLambda is Filter parsed once (PrepareExpressions): the
	// `row => <condition>` lambda whose parameter binds the triggering row.
	FilterLambda *ast.LambdaExpr `json:"-"`
}

TriggerConfig defines event-based triggers for an automation.

type VariableResolver

type VariableResolver func(ctx context.Context, name string) (string, error)

VariableResolver resolves a variable name to its value: var("NAME"), systemVar("NAME"), secret("NAME"), systemSecret("NAME").

Directories

Path Synopsis
Package steps provides step executors for the automation engine.
Package steps provides step executors for the automation engine.
Package workflowhost binds a native execution scope to installed MemQL templates.
Package workflowhost binds a native execution scope to installed MemQL templates.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址