proving

package
v0.31.39 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 13, 2026 License: Apache-2.0 Imports: 43 Imported by: 0

Documentation

Overview

Package proving runs the proving corpus and turns it into figures.

The four pure sub-packages -- figure, scenario, scorecard, capability -- plus world and cassette carry everything that can be decided over values. This package is the half that touches the engine: it drives the real automation executor against a real Postgres so the journal, the resume and the durability claims are measured rather than modelled.

The split is a BUILD-GRAPH fact, not a promise: those packages import nothing outside the standard library and each other, and TestProvingPureSubpackagesImportNothingBeyondStdlib reads `go list -deps` to say so. That is what a nested Go module would have bought, at none of its twelve gates' cost.

Index

Constants

View Source
const (
	LiftCreated   = "created"
	LiftRelifted  = "relifted"
	LiftUnchanged = "unchanged"
)

The lift outcomes the driver branches on (integrations/procedure.LiftOutcome).

View Source
const SyntheticModelId = "synthetic"

SyntheticModelId marks a cassette whose responses are PLACEHOLDERS rather than a recording of a real model.

This is the honesty seam of the whole CI tier, so it is worth being blunt about what a synthetic cassette is and is not.

A cassette's job is to make a run deterministic without a provider. Its CONTENT -- the words a model said -- affects no structural measurement: how many model exchanges a plan needs, how many steps a resume re-executes, and whether an effect duplicated are properties of the PLAN and the PLATFORM, not of the model's prose. So a placeholder response measures those honestly.

What it CANNOT measure is anything model-dependent: tokens, dollars, whether the model's answer was any good. Design P1 already forbids the CI tier from publishing those, and Figures() emits `notMeasurableOnReplay` for every one of them. The sentinel is the belt to that braces: a figure derived from a synthetic cassette carries "synthetic" as its model id, so a number that escaped into a model-dependent column would say so on the page.

`memql-bench record` against a real provider replaces a synthetic cassette with a real recording, and nothing else changes.

Variables

This section is empty.

Functions

func CompileCallsOnCatalogHit

func CompileCallsOnCatalogHit(statement string, inputKeys []string) (int, error)

CompileCallsOnCatalogHit measures the installed default Spine, stopping at its first model boundary. No duplicate native routing policy is involved.

func CompileCallsOnCatalogMiss

func CompileCallsOnCatalogMiss(statement string, inputKeys []string) (int, error)

CompileCallsOnCatalogMiss is the negative control: a miss must reach the classifier, otherwise the zero-call hit measurement proves nothing.

func Figures

func Figures(s scenario.Scenario, results map[figure.Arm]ArmResult, overhead *Overhead, prov figure.Provenance) ([]scorecard.Entry, []scorecard.Property, error)

func NeedsCassette

func NeedsCassette(s scenario.Scenario) bool

NeedsCassette reports whether a scenario has any reasoning step, and therefore needs a cassette on each arm.

func SyntheticCassette

func SyntheticCassette(s scenario.Scenario, arm figure.Arm, recordedAt string) cassette.Cassette

SyntheticCassette builds a placeholder cassette covering every reasoning step a scenario can reach on one arm.

func WriteCassette

func WriteCassette(dir string, c cassette.Cassette) (string, error)

WriteCassette writes one cassette to dir under its canonical name.

Types

type AppHandover added in v0.23.6

type AppHandover interface {
	Handover(ctx context.Context, h HandoverOrder) (HandoverResult, error)
}

AppHandover is the app's side of a fallback: a goal a replay could not finish, handed back with what the replay had already done. The fixture app implements it, and the runner's AppFallback seam is adapted onto it.

type AppStep added in v0.23.6

type AppStep struct {
	RunId   string
	StepKey string
	// StepId is the step's ROW -- the id the journal writes for the key
	// (component/automations.WorkStepId). The session is stamped on it as the
	// step's childRunId, the back-pointer the session delegate stamps.
	StepId string
}

AppStep is the step being served.

type AppStepAnswer added in v0.23.6

type AppStepAnswer struct {
	// ChildRunId is the session's recording; SessionId is the session.
	ChildRunId string
	SessionId  string
	// Failed says an action failed and the session stopped there; the step
	// then fails with Error.
	Failed bool
	Error  string
}

AppStepAnswer is what the app's session did for the step.

type AppStepServer added in v0.23.6

type AppStepServer func(ctx context.Context, step AppStep) (AppStepAnswer, error)

AppStepServer is the app answering the step a goal's run hands it: what the fixture app does, called from INSIDE the executor's step with the step's own context. So a re-run's override, which the executor puts on the targeted step's context alone (component/automations withRunContext), reaches the app exactly as it reaches the session delegate in production.

type ApprovalState added in v0.23.6

type ApprovalState struct {
	ApprovalId  string
	ConstructId string
	// Decision is "approved", "rejected", or "" while it waits for a person.
	Decision string
}

ApprovalState is one procedurePromotion approval.

type ArmResult

type ArmResult struct {
	Scenario string
	Arm      figure.Arm

	// Passed is the verifier's answer.
	Passed bool
	// Failures are the verifier's complaints, in its own words.
	Failures []string

	// RecordedResponses is how many cassette turns the run consumed. In the
	// CI tier this is the honest stand-in for "provider calls": a run that
	// consumed none made none.
	RecordedResponses int
	// StepsExecuted is how many step bodies ran, including repeats.
	StepsExecuted int
	// StepsReExecuted counts bodies that ran more than once. On a resumed
	// platform run this must be zero.
	StepsReExecuted int
	// Resumed reports that the run was resumed from its journal.
	Resumed bool
	// SameRunId reports that the resumed run kept the original run's id --
	// the property that makes "what happened to run X" one story rather than
	// two executions sharing a prefix.
	SameRunId bool
	// Effects and Duplicates are the fake world's counters.
	Effects    int
	Duplicates int
	// DuplicateDetail names what repeated, for a failure message.
	DuplicateDetail string
	// WallClock is how long the arm took IN THIS PROCESS. It is recorded and
	// deliberately NOT published as a CI figure -- a shared runner's
	// wall-clock is the runner's, not the product's (design P1).
	WallClock time.Duration
	// RunId is the journal run this arm produced, empty for the baseline arm
	// which keeps no journal by construction.
	RunId string
	// Err is a runner-level failure: the scenario could not be run at all,
	// which is different from a scenario that ran and failed its verifier.
	Err error

	// The procedure lifecycle's counters, all about its ONE measured goal
	// (procedure_lifecycle.go):
	//
	// ReplaysServedWithoutModel is 1 when a TRUSTED procedure served the goal
	// itself with no model and no app call, else 0.
	ReplaysServedWithoutModel int
	// AppCalls is the fixture app's sessions during the goal. The app is the
	// lifecycle's only intelligence, so a session is a model reach.
	AppCalls int
	// DuplicatedAcrossDivergence is the world's duplicates during the goal --
	// whose failure the scenario injects, so that a replay stops part-way and
	// the app takes over.
	DuplicatedAcrossDivergence int
	// contains filtered or unexported fields
}

ArmResult is one arm's run of one scenario: what happened, and everything counted while it happened.

type CompletedReport added in v0.23.6

type CompletedReport struct {
	// Index is the procedure's step index.
	Index          int
	Tool           string
	IdempotencyKey string
	SideEffect     bool
}

CompletedReport is one step a replay ran before it stopped.

type ConstructState added in v0.23.6

type ConstructState struct {
	ConstructId         string
	GoalSignature       string
	Rung                string
	PromotionApprovalId string

	// What the version replays, read only for the constructs read back after
	// the last goal (Constructs), and empty where it was not.
	//
	// RecordedFrom are the recordings the version was generalized from, as
	// its provenance names them (procedure.recordedFrom.runIds): written when
	// the version is created and never while it stays unchanged.
	RecordedFrom []string
	// Steps are the version's steps as the commands a replay runs, a free
	// parameter written {{input}} after the goal input that binds it, and a
	// position no goal input binds written {{?hole}}.
	Steps []string
	// Unreadable says why the version could not be read, when it could not.
	Unreadable string
}

ConstructState is where one learned construct stands.

type FeedbackOrder added in v0.23.6

type FeedbackOrder struct {
	GoalStep
	// Version is the version judged -- the one the person is looking at.
	Version int
	Verdict string
	// Axes are what a dislike found wrong, by name.
	Axes   []string
	Reason string
}

FeedbackOrder is one verdict on one version of a step.

type FixtureApp added in v0.23.6

type FixtureApp struct {
	// contains filtered or unexported fields
}

FixtureApp is the proving world's app. One per arm of one scenario.

func (*FixtureApp) Calls added in v0.23.6

func (a *FixtureApp) Calls() int

Calls is how many sessions the app ran -- each one a model reach.

func (*FixtureApp) Handover added in v0.23.6

Handover is the app's side of a fallback (AppHandover): a goal a replay could not finish, handed back with what the replay had already done.

The guidance names completed steps by the PROCEDURE's step index. The lifecycle's procedure is learned from exactly these actions in this order, so that is also the action's index here -- and the tool is checked as well, so a guidance that named some other step is not honoured by accident. A mismatch makes the app redo the action, which the world then records as a duplicate: the loud direction, not the quiet one.

func (*FixtureApp) Serve added in v0.23.6

func (a *FixtureApp) Serve(ctx context.Context, g appGoal) (appSession, error)

Serve performs a goal from its first action: an ordinary session.

func (*FixtureApp) ServeStep added in v0.23.6

func (a *FixtureApp) ServeStep(ctx context.Context, g appGoal, step AppStep) (appSession, error)

ServeStep performs a goal as the app handed ONE STEP of the goal's run: the step the platform's executor is running, whose context it is given. The session is recorded beneath the goal's run and stamped onto the step's row by the recorder, which is where the session delegate stamps it.

The step's context is read as the delegate reads it (planSession): a re-run names a fresh workspace, and the version it runs as carries the person's override -- the level asked for, and what they disliked about the version it replaces, which the session is told in its prompt in the delegate's own words (component/memql's StepOverrideGuidance). The fixture app does not reason with any of it; it is told, and the driver checks that it was.

type GoalAttempt added in v0.28.2

type GoalAttempt struct {
	ScenarioID string           `json:"scenarioId"`
	Setup      string           `json:"setup"`
	Repetition int              `json:"repetition"`
	State      string           `json:"state"`
	Reason     string           `json:"reason,omitempty"`
	Result     *goaleval.Result `json:"result,omitempty"`
}

type GoalClient added in v0.28.2

GoalClient is the ordinary authenticated work API. The evaluator has no provider client, solver, patch writer, or bypass of the worker's permissions.

type GoalOrder added in v0.23.6

type GoalOrder struct {
	OwnerUserId   string
	Statement     string
	Input         map[string]any
	GoalSignature string
	// ProcedureConstructId is the learned procedure the serve decision chose,
	// "" when the app serves. It decides what the run names as its template,
	// exactly as compile's choice does.
	ProcedureConstructId string
}

GoalOrder is a goal a person asks for.

type GoalReport added in v0.28.2

type GoalReport struct {
	SchemaVersion int             `json:"schemaVersion"`
	Matrix        goaleval.Matrix `json:"matrix"`
	StartedAt     time.Time       `json:"startedAt"`
	FinishedAt    time.Time       `json:"finishedAt"`
	Attempts      []GoalAttempt   `json:"attempts"`
}

type GoalRunner added in v0.28.2

type GoalRunner struct {
	// Optional, explicitly selected candidate; never applied to ordinary work.
	LessonID      string
	LessonReceipt string
	Client        GoalClient
	Files         fs.FS
	CatalogDir    string
	OutputDir     string
	EngineCommit  string
	HarnessCommit string
	PollInterval  time.Duration
}

func (*GoalRunner) Run added in v0.28.2

func (g *GoalRunner) Run(ctx context.Context, catalog goaleval.Catalog, matrix goaleval.Matrix) (GoalReport, error)

Run executes the selected matrix sequentially. Missing injection authority is recorded as not-run; it is never counted as a failed or successful model run.

type GoalStep added in v0.23.6

type GoalStep struct {
	OwnerUserId string
	RunId       string
	StepKey     string
}

GoalStep names one step of one goal's run, and whose it is.

type HandoverOrder added in v0.23.6

type HandoverOrder struct {
	OwnerUserId string
	// GoalRunId is the run the goal is being served in.
	GoalRunId string
	// Completed are the steps the replay ran. The app must not redo them.
	Completed []CompletedReport
	Diagnosis string
	// Prompt is the guidance as the app reads it.
	Prompt string
}

HandoverOrder is a goal handed back to the app.

type HandoverResult added in v0.23.6

type HandoverResult struct {
	// ChildRunId is the session's recording: the repaired run.
	ChildRunId string
	SessionId  string
}

HandoverResult is what the app's session produced.

type LadderMove added in v0.23.6

type LadderMove struct {
	From, To, Reason string
}

LadderMove is one move of the ladder.

type LearnReport added in v0.23.6

type LearnReport struct {
	ConstructId string
	// Lift is what the pass did to the catalog: "created", "relifted",
	// "unchanged", or "" when nothing cleared the floor.
	Lift string
	// Rung is the construct's rung after the pass.
	Rung   string
	Reason string
}

LearnReport is what one learning pass did.

type LifecycleHarness added in v0.23.6

type LifecycleHarness interface {
	// Open writes the ladder's values for the run -- overlay's non-zero
	// fields over the deployment's current row, a zero field keeping the
	// deployment's value -- PROVES the owner reads them back, and returns what
	// the run records through and decides with. A ladder that silently read
	// the defaults would run a lifecycle that never proposes, and pass or fail
	// for a reason nobody wrote down, so a read-back that disagrees is an
	// error rather than a figure.
	Open(ctx context.Context, ownerUserId string, overlay work.LadderPolicy) (*LifecyclePlatform, error)
}

LifecycleHarness opens the learned-procedure half of the platform for one platform-arm run of one lifecycle scenario. cmd/memql-bench installs one over the real engine; the db-free tests install fakes. A Runner with none fails a lifecycle scenario at RUN time, naming what is missing, rather than publishing a figure for a lifecycle nothing drove.

func NewEngineLifecycle added in v0.23.6

func NewEngineLifecycle(e *memqlengine.MemQLEngine, db *bun.DB, gates *sql.DB, logger *slog.Logger) LifecycleHarness

NewEngineLifecycle is the harness cmd/memql-bench installs on the Runner. db is the engine's own database handle: the work spine's version history reads it directly, as it does on a node. gates is the pool the spine's write gates run on (database.NewGatePool over the same database, as a node's Database opens its GateDB): a lifecycle's decisions -- a promotion approved, a step run again -- each take one, and none is taken on db.

type LifecyclePlatform added in v0.23.6

type LifecyclePlatform struct {
	// Recorder writes the fixture app's sessions into the work spine:
	// integrations/work's SessionWriter in the lane.
	Recorder workerservice.SessionRecorder
	Spine    WorkSpine
	// NewLadder builds the runner with its seams bound to THIS run's world and
	// app: the dispatcher and the prober over the world, the app as the
	// fallback. Built after the app, because the app records through Recorder.
	NewLadder func(world *ProcedureWorld, app AppHandover) (ProcedureLadder, error)
	// Close puts back what Open changed -- the ladder's values. Nil when
	// nothing needs putting back.
	Close func(ctx context.Context) error
	// Warnings are what the platform logged at WARN or above while the
	// lifecycle ran. The runner LOGS a write the engine refused and carries
	// on -- a replay must not fail a goal over its own bookkeeping -- so a
	// lifecycle that discarded the log would pass with every step row and
	// receipt refused. Each one is a verifier failure. Nil when there is no
	// log to read.
	Warnings func() []string
}

LifecyclePlatform is what Open returns.

type Overhead

type Overhead struct {
	Ratio  float64
	OK     bool
	Reason string
}

Figures turns one scenario's arm results into published figures.

This function IS design decision P1 made executable: the CI tier publishes only what a replay can honestly measure, and everything else is `unmeasured` with the reason attached. Every `notMeasurableOnReplay` below is a figure this suite COULD print a number for and refuses to, because the number would be a property of the CI runner or a trivial consequence of determinism rather than a fact about the product.

The discipline held for the work spine's two unbuilt seams until memql#4999 built them, and it is worth recording what it bought: a benchmark that reported "zero provider calls" because nothing in the path calls a provider would have told a lie that reads exactly like the headline result. Those figures named the missing code instead, and when the code arrived the reason CHANGED rather than the number appearing -- the CI harness still cannot reach the seam. `seamNotBuilt` has no caller today and stays in the set for the next one. Overhead is the journal-overhead measurement, or the reason there is none.

type ProcedureLadder added in v0.23.6

type ProcedureLadder interface {
	// LearnFromRun mines the corpus a succeeded recording belongs to and
	// lifts what it finds -- what the learnFromSucceededRun automation does,
	// under the recording owner's actor, when a recording run succeeds.
	LearnFromRun(ctx context.Context, ownerUserId, recordingRunId string) (LearnReport, error)
	// ShadowCompare replays every procedure on shadow for the recording's
	// goal beside the app's own actions, in a sandbox, and moves the ladder
	// on the comparison.
	ShadowCompare(ctx context.Context, ownerUserId, recordingRunId string) ([]ReplayReport, error)
	// DecidePromotion applies a DECIDED procedurePromotion approval to the
	// ladder -- what the onProcedurePromotionDecided automation does.
	DecidePromotion(ctx context.Context, ownerUserId, approvalId string) (LadderMove, error)
	// Replay serves one goal from a construct on a serving rung.
	Replay(ctx context.Context, order ReplayOrder) (ReplayReport, error)

	// ProcedureFor is compile's exact tier for one goal signature: the learned
	// construct on the ladder for it, at its CURRENT rung. ok is false when
	// there is none.
	ProcedureFor(ctx context.Context, ownerUserId, goalSignature string) (ConstructState, bool, error)
	// Constructs reads back every learned construct the owner has.
	Constructs(ctx context.Context, ownerUserId string) ([]ConstructState, error)
}

ProcedureLadder is what the lifecycle driver needs from the learned-procedure runner. The first four are the runner's own entry points, by the names the plan gives them (docs/superpowers/plans/2026-09-23-procedure-certification- replay.md, section 1.4); the last two are the reads compile and the verifier make. Every call names the owner, and every call is made AS the owner: the learned constructs, the runs and the approvals are all on the owner's composite tier, and each automation that makes these calls in production borrows the owner the event names (learnFromSucceededRun included) rather than acting as the cluster -- so an adapter that made them as anybody else would measure a path production does not take.

type ProcedureWorld added in v0.23.6

type ProcedureWorld struct {
	// contains filtered or unexported fields
}

ProcedureWorld wraps the scenario's world with the failures the CURRENT goal injects.

type ReplayOrder added in v0.23.6

type ReplayOrder struct {
	OwnerUserId string
	ConstructId string
	// Mode is "canary" or "trusted" -- the rung the serve decision read.
	Mode string
	// GoalRunId is the run of the goal the replay serves; GoalId and
	// Statement are its goal.
	GoalRunId string
	GoalId    string
	Statement string
	// StepKey is the goal run's statement that asked for the replay. It keys
	// the replay run, so a statement executed again finds the replay it
	// already made.
	StepKey string
	// Input is the goal's input, which binds the procedure's free parameters.
	Input map[string]any
}

ReplayOrder serves one goal from a construct.

type ReplayReport added in v0.23.6

type ReplayReport struct {
	ConstructId string
	Mode        string
	// Served: a canary or trusted replay answered the goal itself.
	Served bool
	// Match: a shadow replay matched the app on every step.
	Match        bool
	Diverged     bool
	DivergedStep int
	Diagnosis    string
	// StartRefused: nothing ran -- a precondition did not hold, a parameter
	// could not be bound, or the rung had moved since the serve decision.
	StartRefused bool
	// FellBack: the goal was handed to the app.
	FellBack bool
	// ModelCalls is what the replay ITSELF reached. A fallback's calls are
	// the app's, and are counted by the app.
	ModelCalls int
	// Completed are the steps the replay ran, with their idempotency keys.
	Completed []CompletedReport
	// Proposed: this comparison proposed a promotion, and
	// PromotionApprovalId is the approval raised for it.
	Proposed            bool
	PromotionApprovalId string
	// From and To are the ladder's move; Rung is the rung the replay found
	// the procedure on -- the runner's own reading, which decides what ran.
	From, To    string
	Rung        string
	ReplayRunId string
	// Code is the replay run's error code when it did not serve or match.
	Code string
	// NotCompared (shadow): the comparison was not made where it ran --
	// no workbench, or a sandbox that refused its first step -- so the
	// ladder did not hear of it.
	NotCompared bool
}

ReplayReport is what one replay -- canary, trusted or shadow -- did.

type RerunOrder added in v0.23.6

type RerunOrder struct {
	GoalStep
	// Level is the level the person asks for instead of the step's own; empty
	// keeps the step's.
	Level string
}

RerunOrder is a step run again.

type Runner

type Runner struct {
	Engine    *memqlengine.MemQLEngine
	Cassettes cassette.Set
	Logger    *slog.Logger
	Prov      figure.Provenance
	// Lifecycle opens the learned-procedure half of the platform for a
	// procedure lifecycle scenario's platform arm (procedure_lifecycle.go).
	// Nil fails such a scenario loudly rather than skipping it.
	Lifecycle LifecycleHarness
}

Runner drives the corpus.

func (*Runner) MeasureJournalOverhead

func (r *Runner) MeasureJournalOverhead(ctx context.Context, s scenario.Scenario) (msPerStep float64, ok bool, reason string)

MeasureJournalOverhead answers "what does the journal itself add per step".

The measurement this REPLACES, and why it was wrong

The obvious instrument is the platform arm's wall-clock over the baseline arm's, and it is nonsense. The baseline is an in-process loop that touches no database; the platform opens an executor and writes journal rows to Postgres. The first published figure was a ratio of 34,000, which is not the journal's overhead -- it is the cost of having a database at all, wearing the journal's label.

That is the failure the whole suite exists to prevent, produced by the suite, so it is worth naming precisely: the number was arithmetically correct and measured something other than what its name said.

The instrument that is right

The SAME automation, the SAME steps, the SAME executor and the SAME process, run twice: once with an engine and once without. `newWorkJournal` returns nil for a nil executor, so an executor built with no engine runs every step and writes no journal row -- and every step here is a function step dispatched to the proving registry, which never needs the engine. The only difference between the two runs is the journal, which is what makes the difference the journal's.

The two arms ALTERNATE rather than running in blocks, so a machine that warms up or is disturbed mid-measurement disturbs both equally.

ok is false when the measurement cannot be made honestly -- an unjournaled run that failed, or one too fast to time. The caller publishes `unmeasured` with the returned reason rather than a number.

func (*Runner) Run

func (r *Runner) Run(ctx context.Context, corpus scenario.Corpus) (SuiteResult, error)

Run executes the whole corpus on both arms and assembles the scorecard.

func (*Runner) RunScenario

func (r *Runner) RunScenario(ctx context.Context, s scenario.Scenario, arm figure.Arm) (res ArmResult)

RunScenario runs one scenario on one arm.

func (*Runner) WriteRows

func (r *Runner) WriteRows(ctx context.Context, result SuiteResult) (runId string, problems []string)

WriteRows publishes a suite result as v1:bench:run plus one v1:bench:sample per figure.

A ROW WRITE NEVER FAILS THE RUN. The journal takes the same position for the same reason: the measurement is the product of the run, and losing the published copy of it must not turn a green suite red. Every refusal is returned so the caller can report it, and the caller reports it as a WARNING beside a successful envelope rather than as a failure.

type StepRun added in v0.23.6

type StepRun struct {
	// Version is the version of the step it ran as.
	Version int
	// Status is the run's status as the execution left it -- succeeded,
	// failed, or a wait the failure path parked it on.
	Status string
}

StepRun is how one execution of a goal's step ended.

type StepVersionState added in v0.23.6

type StepVersionState struct {
	StepKey    string
	Version    int
	Status     string
	Current    bool
	ChildRunId string
	// Level is the level the version was asked for, from its override; empty
	// when nobody asked for one.
	Level string
}

StepVersionState is one version of one step, as the platform reads it back.

type SuiteResult

type SuiteResult struct {
	Scorecard scorecard.Scorecard
	// Results are every arm result, for the row writer and for a failure that
	// needs to say what happened.
	Results []ArmResult
	// ControlFailures are negative controls that read the wrong way -- a
	// lower-is-better control that came back zero, or a higher-is-better one
	// that did not. THEY ARE FATAL, and they are kept separate from ordinary
	// verifier failures because they mean something different: not "the
	// platform regressed" but "the instrument is dead, and every green figure
	// it produced means nothing".
	ControlFailures []string
	// VerifierFailures are scenarios whose platform arm did not satisfy its
	// own verifier.
	VerifierFailures []string
}

SuiteResult is one whole run of the corpus.

func (SuiteResult) Blocking

func (r SuiteResult) Blocking() []string

Blocking reports whether the suite's own run failed, before any comparison against a previous scorecard. A verifier failure and a dead instrument both block; a cost movement does not (design P2).

type WorkSpine added in v0.23.6

type WorkSpine interface {
	// OpenGoal opens the goal a person asked for and the run that serves it,
	// AFTER the serve decision -- as compile does -- so the run names what
	// serves it. It carries the goal signature and the goal's input as its
	// variables: what compile writes on a run it decided, and what an app
	// session's recording inherits from the run it was delegated from.
	OpenGoal(ctx context.Context, g GoalOrder) (goalId, runId string, err error)
	// CloseGoal ends the goal's run the way its template's completion would:
	// succeeded when the goal was served, failed with why when it was not.
	CloseGoal(ctx context.Context, ownerUserId, runId string, served bool, why string) error
	// DecideApproval is the person's decision, through the work spine's OWN
	// decide handler: the owner check and the artifact-hash gate included.
	DecideApproval(ctx context.Context, ownerUserId, approvalId, decision string) error
	// PromotionApprovals reads back the owner's procedurePromotion approvals:
	// every one still pending, and each of known -- the ones the lifecycle saw
	// raised -- whatever its decision.
	PromotionApprovals(ctx context.Context, ownerUserId string, known []string) ([]ApprovalState, error)

	// ExecuteAppGoal executes the run of a goal the app serves through the
	// platform's automation executor, as an agent executes a compiled run:
	// the run is ADOPTED, its one step is handed to the app -- serve, called
	// from inside the step with the step's own context -- the session is
	// recorded beneath the run and stamped as the step's childRunId, and the
	// executor journals the step's version and closes the run. That is what
	// gives the goal a step a person can judge and run again.
	ExecuteAppGoal(ctx context.Context, step GoalStep, serve AppStepServer) (StepRun, error)
	// RerunStep is a person running a finished goal run's step again as a new
	// version: the work spine's own rerunStep act as that person, then the
	// request served as an agent serves it -- from the run's rows, the step
	// executed with the person's override on its context alone, the run
	// closed again.
	RerunStep(ctx context.Context, o RerunOrder, serve AppStepServer) (StepRun, error)
	// RecordFeedback is a person's verdict on one version of a run's step,
	// through the work spine's own recordFeedback act as that person. It
	// answers the feedback observation's id.
	RecordFeedback(ctx context.Context, o FeedbackOrder) (string, error)
	// StepVersions reads back every version of every step of one run, as the
	// step timeline reads it: workStepVersions, as the run's owner.
	StepVersions(ctx context.Context, ownerUserId, runId string) ([]StepVersionState, error)
}

WorkSpine is what the driver needs from the work spine.

Directories

Path Synopsis
Package capability implements the capability-script contract for a Go binary.
Package capability implements the capability-script contract for a Go binary.
Package cassette is the CI tier's recorded model responses.
Package cassette is the CI tier's recorded model responses.
Package figure is the proving suite's honesty layer: a measured number, what it was measured on, and the vocabulary for saying that it was not measured at all.
Package figure is the proving suite's honesty layer: a measured number, what it was measured on, and the vocabulary for saying that it was not measured at all.
Package scenario is the proving corpus's format: what a scenario says, and what the loader refuses.
Package scenario is the proving corpus's format: what a scenario says, and what the loader refuses.
Package scorecard is the committed artifact and the page generated from it.
Package scorecard is the committed artifact and the page generated from it.
Package world is the fake external world a proving scenario runs against.
Package world is the fake external world a proving scenario runs against.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址