memoryNodes

package
v0.0.0-...-86254be Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 14, 2026 License: Apache-2.0 Imports: 29 Imported by: 38

Documentation

Overview

Package memoryNodes -- concept_parser.go

Parses a concept.memql file into a *Concept. As of Phase 2 of the language-improvements plan, the grammar parsing happens in the shared component/language/parser; this file holds only the bridge from the shared AST (parser.ConceptDecl) to the memory-nodes Concept type, plus the JSON-Schema builder that emits draft-07 schemas from the parsed shape.

The legacy hand-rolled parser (~500 LOC of bespoke lexing) used to live here; it was retired in Phase 2 once parser.parseConceptDecl landed.

Index

Constants

View Source
const (
	ConceptCognitionParticipant         = "v1:cognition:participant"
	ConceptCognitionParticipantPresence = "v1:cognition:participant:presence"
	ConceptCognitionSession             = "v1:cognition:session"
	ConceptCognitionSpace               = "v1:cognition:space"
	ConceptCognitionSpaceContext        = "v1:cognition:space:context"
	ConceptCognitionTextChunk           = "v1:cognition:text:chunk"
	ConceptCognitionTurnState           = "v1:cognition:turn:state"
	ConceptCognitionUtterance           = "v1:cognition:utterance"
)

Cognition domain concepts (v1:cognition:*)

View Source
const (
	ConceptDataRecord = "v1:data:record"
	ConceptDataPolicy = "v1:data:policy"
	ConceptDataLog    = "v1:data:log"
)

Data domain concepts (v1:data:*)

View Source
const (
	ConceptWorkGoal        = "v1:work:goal"
	ConceptWorkRun         = "v1:work:run"
	ConceptWorkStep        = "v1:work:step"
	ConceptWorkModelCall   = "v1:work:modelCall"
	ConceptWorkApproval    = "v1:work:approval"
	ConceptWorkObservation = "v1:work:observation"
)

Work-spine concepts (v1:work:*) -- the execution model's spine (docs/superpowers/specs/2026-09-05-work-spine-design.md). Constants rather than literals because the journal (component/automations), the routing rules and the resume path all have to agree on the spelling.

View Source
const (
	ConceptPlatformGlobalSecret      = "v1:platform:globalSecret"
	ConceptPlatformGlobalVariable    = "v1:platform:globalVariable"
	ConceptPlatformPartitionSecret   = "v1:platform:partitionSecret"
	ConceptPlatformPartitionVariable = "v1:platform:partitionVariable"
)

Platform domain concepts (v1:platform:*)

View Source
const (
	ConceptIdentityUser        = "v1:identity:user"
	ConceptIdentityIdentity    = "v1:identity:identity"
	ConceptIdentityDelegation  = "v1:identity:delegation"
	ConceptIdentityAuthSession = "v1:identity:authSession"
)

Identity domain concepts (v1:identity:*)

View Source
const (
	ConceptClusterNode       = "v1:cluster:node"
	ConceptClusterNodeType   = "v1:cluster:nodeType"
	ConceptClusterSpawnEvent = "v1:cluster:spawnEvent"
)

Cluster domain concepts (v1:cluster:*)

View Source
const (
	ConceptMemQLAutomationRun  = "v1:memql:automation:run"
	ConceptMemQLAutomationStep = "v1:memql:automation:step"
	ConceptMemQLBackendUser    = "v1:memql:backend:user"
)

MemQL internal concepts (v1:memql:*) -- engine-internal state, not user-facing config. User-facing secrets / variables live under v1:platform:* (see Platform domain above). MemQL runtime concepts -- used in Go code but registered dynamically (no concept directory on disk).

View Source
const (
	NodeTypeObject     = "object"
	NodeTypeCollection = "collection"
	NodeTypeReference  = "reference"
)
View Source
const (
	FieldSourcePayload = "payload"
	FieldSourceTable   = "table"
)
View Source
const (
	MemoryNodesTable       = "MemoryNodes"
	SecretMemoryNodesTable = "SecretMemoryNodes"
)
View Source
const CodeBytesUnder = "bytes_under"

CodeBytesUnder is the stable rule id of a @bytes whose `under` names no place, or a place that is not built from the row's own values.

View Source
const CodeFieldAnnotationType = "field_annotation_type"

CodeFieldAnnotationType is the stable rule id of a @bytes or @secretName on a field that cannot hold what the annotation says it holds.

View Source
const CodePatternInvalid = "pattern_invalid"

CodePatternInvalid is the stable rule id of a `@pattern` whose value is not a regular expression, on a concept field or an args field alike.

View Source
const (
	ComponentName = common.ComponentName("memoryNodesDB")
)
View Source
const (
	ConceptActionsAction = "v1:actions:action"
)

Action-library concepts (v1:actions:*) -- the reusable, replayable action library (epic #1734). `action.intent` is embedded into node_vectors (vectorField='intent') so the planner can cosine-search the library (#1758).

View Source
const (
	ConceptAuthoringConstruct = "v1:authoring:construct"
)

Authoring domain concepts (v1:authoring:*) -- the graph-stored substrate for planner-authored DSL bundles (epic #954). The construct row is the per-construct member of a bundle and the catalog's reuse unit (#957): a promoted construct grows a content vector in node_vectors keyed by its id so similarTo can rank near-matches.

View Source
const ConceptUser = ConceptMemQLBackendUser

Deprecated: Use ConceptMemQLBackendUser instead.

View Source
const ObservedRowsSQL = `(SELECT * FROM "MemoryNodes" UNION ALL SELECT * FROM memory_liveness)`

staged-data: MUST-NOT-GATE -- this shared row source serves both ordinary authorized reads and maintenance. Filtering here would hide staged rows from mutation read-merge and erase fields on the next write. Callers apply their own publication and row-admission rules after selecting the current version. ObservedRowsSQL includes the one current observation of a record. Historical observation samples are deliberately not retained. Business versions and deletion markers remain in MemoryNodes.

Variables

View Source
var (
	// ErrBunNotConfigured is returned when the bun database is not configured.
	ErrBunNotConfigured = errors.New("bun database not configured")

	// ErrStoreUnavailable is returned when the underlying store is not configured.
	ErrStoreUnavailable = ErrBunNotConfigured

	// ErrDuplicateNode is returned when an insert violates a unique constraint.
	// For example, inserting a second active AI participant for the same (partitionId, agentId).
	ErrDuplicateNode = errors.New("duplicate node: record already exists")
)

Functions

func All

func All() map[string]*Concept

All returns a copy of all registered concepts keyed by concept name.

func AllFilesystemConcepts

func AllFilesystemConcepts() []string

AllFilesystemConcepts returns all concept IDs that have corresponding concept directories on disk. Used for startup validation.

func CompactSchemaBatch

func CompactSchemaBatch(ctx context.Context, db *bun.DB) (int, error)

CompactSchemaBatch preserves payload, provenance and version timestamps. The primary-key walk bounds each read even when all earlier rows were already converted. Inline rows inserted after the walk starts are normalized by the trigger, including backdated restores behind the cursor.

func CompilePattern

func CompilePattern(pattern string) (*regexp.Regexp, error)

CompilePattern compiles a `@pattern` value, refusing one that is not a regular expression with a *PatternError.

func ConceptNameFromPath

func ConceptNameFromPath(dir string) (string, error)

ConceptNameFromPath derives the canonical concept identifier from a directory path under concepts/.

func EnableAccountStampingWhenReady

func EnableAccountStampingWhenReady(ctx context.Context, db *bun.DB, nodeID string, allowActivation bool) (bool, error)

EnableAccountStampingWhenReady is the handshake for the switch that lets the engine record the account of a row: fill accountId on a write to a concept declared @belongsTo(account), and assign the rows written before (AccountStampingEnabled and AccountOwnershipRunner in component/memql).

The switch waits for the fleet because the release that stamps the field is also the release that adds it to the concepts. While that release rolls out, nodes of the previous one still serve, their copy of those concepts does not declare the field, and a node refuses to update a row carrying a key its concept does not declare. A node of the previous release never advertises here, so it holds the switch until it has stopped.

It is the reader handshake the observation stores use (liveness.go), on tables of its own (migration 20261112100000_account_ownership), and not a second mechanism. Its own tables because an advertisement says one thing: a node that reads memory_liveness has said nothing about accountId. A release recorded on the node's v1:cluster:node row would not do instead: nodes write each other's rows, and a node of the previous release would refuse a row carrying the new key.

allowActivation is the operator's hold, MEMQL_ACCOUNT_STAMPING_ACTIVATE. A held node still advertises and never turns the switch on. The answer is the cluster's either way: once the switch is on a held node is told so, and what it does about that is its caller's decision (the runner neither stamps nor assigns). Nothing turns the switch off, because rows carry the field from then on.

Regression tests: component/memql/account_stamping_db_test.go.

func EnableLivenessStorageWhenReady

func EnableLivenessStorageWhenReady(ctx context.Context, db *bun.DB, nodeID string) (bool, error)

EnableLivenessStorageWhenReady waits for every non-stopped engine reader. An offline old binary may still serve reads, so it blocks this rollout.

func EnablePipelineRunObservationsWhenReady

func EnablePipelineRunObservationsWhenReady(ctx context.Context, db *bun.DB, nodeID string) (bool, error)

EnablePipelineRunObservationsWhenReady is the third handshake, for the pipeline run's lease renewal. A reader of the first two releases does not list v1:pipelines:run in HasLiveness and must block it; once every non-stopped node advertises here the renewal is an observation on the whole cluster. Like the other two it never turns itself off: observations already stored must stay readable.

It runs only where schema references are enabled (SchemaCompactor.run), so on a cluster that never set MEMQL_DATABASE_SCHEMA_REFERENCES_ACTIVATE every renewal stays a version. Where references are already enabled (production since 2026-10-14) it activates once every replica runs an engine that advertises it: deploying that engine is the decision. Regression tests: TestPipelineRunObservationsWaitForTheirOwnReaderHandshake (component/memql) and TestLeaseRenewalsAreObservationsAcrossTwoReplicas (component/pipelinerun).

func EnableReconcileObservationsWhenReady

func EnableReconcileObservationsWhenReady(ctx context.Context, db *bun.DB, nodeID string) (bool, error)

EnableReconcileObservationsWhenReady has a separate handshake because a reader of the first liveness release does not enumerate these concepts from the observation table. Such a reader must block this second rollout.

func EnableSchemaReferencesWhenReady

func EnableSchemaReferencesWhenReady(ctx context.Context, db *bun.DB, nodeID string, allowActivation bool) (bool, error)

EnableSchemaReferencesWhenReady advertises reader support even while first activation is held. Explicit operator opt-in and every non-retired reader's recent advertisement are both required to enable references. A stopped row alone does not prove an old process has closed its serving streams: the operator must finish that rollout before opting in. Once enabled, false does not downgrade storage; returning true keeps existing conversion resumable.

func HasLiveness

func HasLiveness(concept string) bool

HasLiveness identifies the closed set whose observation-only writes live outside the versioned store. All authorization still runs in the engine.

A concept enters this set together with its own reader handshake (an Enable...WhenReady below and the storage switch it guards), because this list is what a reader knows: a reader without the concept here enumerates its latest keys from MemoryNodes alone (latestScanKeys, latestScanWalk) and reports every observation-only write to a held journal as a business write (executeWrite's observationOnly). Adding a FIELD to a listed concept's observation list in memory_node_store_liveness needs no handshake.

v1:pipelines:run is listed for its lease renewal alone (driverHeartbeatAt, migration 20261108190000_pipeline_run_lease_observations): the lease is the run's liveness, every 30 s, and nothing reads a superseded renewal. No payload lookup or walk index serves the concept (payloadLookups, payloadWalks in component/memql), so listing it changes only which table its latest keys come from, not an index path.

func HydrateSchemas

func HydrateSchemas(ctx context.Context, db bun.IDB, nodes []MemoryNode) error

HydrateSchemas restores the exact schema version referenced by each row. Call at public read and archive boundaries, never substitute the currently loaded concept: older versions can have a different definition or be deletes. Inline schemas remain readable while the background conversion progresses.

func IsReservedPayloadField

func IsReservedPayloadField(name string) bool

func MergeAll

func MergeAll(concepts map[string]*Concept)

MergeAll adds the provided concepts to the existing registry, overwriting entries with matching names. Unlike ReplaceAll it preserves entries not present in the input map.

Used by the unified loader (Pass 2 of the DSL restructure) so it can register new-tree concepts on top of the legacy loader's registrations without clobbering anything. Once Pass 2 retires the legacy loader, callers can switch back to ReplaceAll.

func ReplaceAll

func ReplaceAll(concepts map[string]*Concept)

ReplaceAll swaps the existing registry contents with the provided concepts.

func StripReservedPayloadFields

func StripReservedPayloadFields(payload map[string]any) map[string]any

StripReservedPayloadFields removes reserved fields (id, createdAt, createdBy, etc.) from the payload. This modifies the input map in-place. Use clonePayload first if you need to preserve the original.

func ValidateConceptConstants

func ValidateConceptConstants(registry Registry) error

ValidateConceptConstants checks that every filesystem-backed concept constant has a corresponding entry in the registry. Call after the concepts are loaded to catch drift between constants and the concept tree at startup.

Types

type BytesField

type BytesField struct {
	Name string
	// Tree: @bytes(tree) -- the field names a prefix, and the bytes are every
	// object under it, not one object.
	Tree bool
	// Counted: @bytes(counted) -- an account export counts the bytes in the
	// file's report and never carries them.
	Counted bool
	// Under is the place the row's own bytes live in, the template as
	// written (`library/{ownerUserId}`). Places and Admits read it.
	Under string
	// Concept is the concept that declares the field: the concept a `{id}`
	// placeholder's id must be of.
	Concept string
	// OwnerField is the field the concept's @rowAuthz(owner=...) names, or
	// "" when it names none. Its placeholder's id must be a user's.
	OwnerField string
}

BytesField is one top-level field a concept declares @bytes on.

func (BytesField) Admits

func (f BytesField) Admits(rowID string, payload map[string]any, key string) bool

Admits reports whether key lies STRICTLY inside the row's place: split on "/", every literal segment equal, every placeholder segment the same id as the row's value (sameId, with the concept that placeholder expects), and at least one more non-empty segment after the place. A key with a relative segment anywhere -- after either slash, and once percent-decoded -- is refused, since storage could resolve it outside.

For a `counted` field an admission is for SIZING only: a reader checks Counted before it opens anything, because a counted place may be literal and so admit every row's bytes.

func (BytesField) Places

func (f BytesField) Places(rowID string, payload map[string]any) []string

Places answers the key prefixes the row's own bytes may live under: Under with each placeholder replaced by every spelling of the row's value (spellings), segments joined by "/", no trailing slash, one prefix per combination, deduplicated and sorted. The blob reader compares these prefixes exactly, so the list is every spelling a real key can have and no other -- the same set Admits accepts. A placeholder with no usable value (missing, empty, not a string, with surrounding whitespace, holding a separator, or with a spelling that percent-decodes to a relative step or a separator) gives an empty slice: a row that does not say whose bytes these are owns none.

A `counted` field's places are for SIZING only: a reader checks Counted before it opens anything, because a counted place may be literal and so hold every row's bytes.

type BytesUnderError

type BytesUnderError struct {
	Field string
	Under string
	Msg   string
}

BytesUnderError refuses a @bytes whose `under` does not name the row's own place.

func (*BytesUnderError) Error

func (e *BytesUnderError) Error() string

func (*BytesUnderError) RuleCode

func (e *BytesUnderError) RuleCode() string

RuleCode is the refusal's stable rule id (baseloader.CodedRefusal).

type Composable

type Composable struct {
	// As is what a person and a prompt call this concept when
	// its id is not that word ("invoice" for
	// v1:billing:invoiceRecord). Optional; empty means the
	// concept's own entity name.
	As string `json:"as,omitempty"`
	// Fields is the projection worth composing FROM, in
	// declaration order. Optional; empty means "the shape's
	// own projection decides", which is the honest default
	// for a concept with few fields. It exists because a
	// compose prompt handed forty fields spends its context
	// on ids.
	Fields []string `json:"fields,omitempty"`
	// List names the QUERY that returns the caller's rows of
	// this concept.
	//
	// IT IS HERE BECAUSE MemQL HAS NO GENERIC ROW READ, and
	// that is a feature rather than a gap: every read goes
	// through a declared construct, which is what makes row
	// authz enforceable at all. So a mark saying only "this
	// concept is composable" would name something the
	// Materializer then has nothing to run against -- the
	// person picks a concept and the app has no read.
	//
	// THERE IS DELIBERATELY NO SECOND `readBy` ARGUMENT for a
	// by-id read, and the reason is worth knowing before
	// adding one: a MemQL query takes NAMED arguments, so
	// calling one by id means knowing the name it declares
	// ("invoiceId", "fileId", ...) -- and a mark cannot know
	// that without either carrying the name too or guessing
	// it. Guessing composes a call the engine refuses with a
	// message about an undeclared argument, which reads as the
	// row being gone. So a single row is resolved through
	// THIS query and selected by id, which uses exactly the
	// read the concept declared and needs no knowledge the
	// mark does not carry. The cost is a page read per source;
	// the `query` source kind is the O(1) escape for a concept
	// where that matters.
	//
	// Optional. A concept marked without it is composable but
	// not directly pickable, and the Sources column says so
	// rather than offering a control that cannot work.
	//
	// VALIDATED FOR FORM HERE AND FOR EXISTENCE AT USE. The
	// concept pass runs before the query registry exists, so
	// this loader can only check that the name is an
	// identifier. A name resolving to no query produces an
	// error at the moment somebody picks that concept, naming
	// the query -- and TestComposableListsResolve in
	// integrations/compose resolves every one declared in THIS
	// tree at build time. A product bundle mounted at
	// MEMQL_DSL_PATH gets the use-time error, which is the
	// same coverage split cmd/memqllint has.
	List string `json:"list,omitempty"`
}

Composable is the Materializer's mark: this concept's rows are worth composing a file FROM (epic memql#4977, design D2). Nil when the concept did not declare `@composable(...)`.

AN UNMARKED CONCEPT IS UNMARKED, NOT FORBIDDEN. The mark is a ranking and a hint -- the Sources column lists marked concepts first and the compose prompt is handed the same list -- never a gate. A gate would mean a product whose DSL bundle forgot the annotation ships a Materializer that cannot see its own data, and the product concepts most worth composing from are exactly the ones that arrive at MEMQL_DSL_PATH rather than in this tree.

It rides the wire beside DisplayCard for the same reason DisplayCard does: one derivation, read by the person's picker and by the model's prompt, so the two cannot be looking at different sets.

type Concept

type Concept struct {
	Name          string                     `json:"concept"`
	SchemaId      string                     `json:"schemaId"`
	Schemas       map[string]json.RawMessage `json:"schemas"`
	NodeType      string                     `json:"type"`
	Description   string                     `json:"description,omitempty"`
	Relationships []RelationshipDefinition   `json:"relationships,omitempty"`

	// Version is the explicit version prefix declared via
	// @version("vN"). Empty means the version was derived from
	// the concepts/vN/... directory layout.
	Version string `json:"version,omitempty"`

	// DisplayCard carries the per-concept rendering hints
	// declared via `@displayCard(...)`. Nil when the concept
	// did not declare the annotation. See memql#160.
	DisplayCard *DisplayCard `json:"displayCard,omitempty"`

	// Composable carries the Materializer's mark, declared via
	// `@composable(...)`. Nil when the concept did not declare
	// it -- which is most concepts, and which means "unmarked",
	// never "forbidden". See the Composable type above and
	// epic memql#4977's design record, section D.
	Composable *Composable `json:"composable,omitempty"`

	// Retention is an opt-in physical version-retirement policy.
	Retention *RetentionPolicy `json:"retention,omitempty"`

	// RowAuthz carries the row-authorization tier declared via
	// `@rowAuthz(...)`: who may see this concept's rows, stated
	// once on the concept instead of as an `actor.*` term each
	// author must remember to type into every filter over it.
	// Nil when the concept declared no tier.
	//
	// THE QUERY PATH READS THIS FIELD (Phase 3, memql#3172).
	// component/memql/rowauthz_enforce.go ANDs the tier's
	// predicate into the plan and separately admits each emitted
	// row against it, so nil here and a tier here are different
	// RESULT SETS -- and, since memql#3174, different write
	// outcomes. This is not carried metadata, and the comment
	// here described it as such for months after it stopped being
	// (swept in memql#3987). What watches it is
	// TestRowAuthzEnforcementLandGate, in component/memql.
	//
	// The type is the parser's own, not a local mirror, so the
	// loader and the memqlmigrate codemod cannot drift in what a
	// declaration means.
	RowAuthz *parser.RowAuthzDecl `json:"rowAuthz,omitempty"`

	// ServerWritten requires internal origin at the engine's common write
	// boundary. Read admission remains the separate RowAuthz contract.
	ServerWritten bool `json:"serverWritten,omitempty"`

	// Origin and MirroredTo carry the data-origins declaration
	// (epic memql#4378): WHERE changes to this concept are made,
	// and WHO ELSE holds a copy. Together they derive DataState --
	// mirror, origin, or native -- which is what the registry, both
	// SDKs, the write guard and the portal badge read.
	//
	// They store what the concept DECLARED, not what it means: an
	// absent @origin leaves Origin empty, and empty means MemQL
	// (see parser.OriginDecl.IsMemQLOrigin). Normalising "" to
	// "memql" here would put the annotation's default into 100+
	// registry entries that never wrote it, and the derivation
	// already treats the two as one answer. The WIRE carries the
	// effective value; storage records the declaration.
	//
	// THE WRITE PATH READS THESE. A concept whose DataState is
	// mirror is read-only by construction: component/memql's
	// mirror write guard refuses every write to it that does not
	// come from the connector Origin names -- mutation, tool
	// handler, raw insert and staged write alike. So nil-vs-set
	// here is a different set of ACCEPTED WRITES, not carried
	// metadata.
	Origin     string   `json:"origin,omitempty"`
	MirroredTo []string `json:"mirroredTo,omitempty"`

	// BelongsTo is the concept's `@belongsTo(...)` declaration: which
	// account its rows belong to, or that it is cluster-wide (design
	// record docs/superpowers/specs/2026-11-12-account-row-ownership-design.md).
	//
	// NIL MEANS UNDECLARED, which is not the same statement as
	// cluster-wide. The engine's own tree may not leave it out (a test
	// over the corpus fails the build); a mounted product bundle that
	// does is reported at load, and its rows are treated as cluster-wide
	// until it declares.
	//
	// THE WRITE PATH READS THIS: the engine's account stamp fills
	// `accountId` on a row of a concept whose declaration is
	// account-scoped (component/memql/account_stamp.go). It records the
	// account and opens the row to nobody; sharing a row with an account's
	// members is RowAuthz's `account=` argument.
	BelongsTo *parser.BelongsToDecl `json:"belongsTo,omitempty"`
	// contains filtered or unexported fields
}

Concept provides runtime helpers for interacting with memory nodes described by a concept definition.

func BuildConceptFromDecl

func BuildConceptFromDecl(decl *parser.ConceptDecl, conceptName string) (*Concept, error)

BuildConceptFromDecl builds a *Concept from an already-parsed ConceptDecl + an externally-derived concept name. This is the shared core both ParseConceptMemQL and the unified loader call; the difference is only in how the name was computed.

Used by the unified loader to register concepts found in the new domain-first tree where the file's path no longer encodes the concept name (the name comes from @version + @namespace + declaration name, assembled via languageAst.AssembleConceptIdFromDecl).

func Get

func Get(name string) (*Concept, error)

Get retrieves a concept by name.

func List

func List() []*Concept

List returns a slice copy of all registered concepts.

func ParseConceptMemQL

func ParseConceptMemQL(content []byte, dirPath string) (*Concept, error)

ParseConceptMemQL parses a concept.memql file into a *Concept using the shared language parser. dirPath is used to derive the fully- qualified concept name (e.g. "concepts/v1/identity/user" -> "v1:identity:user").

This is the LEGACY entry point: it expects the path-derived name convention from the original kind-first tree. The unified loader (see component/memql/unified_loader.go) calls BuildConceptFromDecl directly with a name assembled from the structural @version + @namespace attributes.

func (*Concept) AccountEngineManaged

func (c *Concept) AccountEngineManaged() bool

AccountEngineManaged reports whether the ENGINE decides which account a row of this concept belongs to: the concept is account-scoped and did not declare `stated`. It is the one question the WRITERS of the stamp ask -- the write path's fill and the assignment of existing rows -- and it is narrower than AccountScoped on purpose: a `stated` concept's own mutations name the account, under rules that predate the declaration, and the engine neither fills the field there nor refuses a change to it.

func (*Concept) AccountScoped

func (c *Concept) AccountScoped() bool

AccountScoped reports whether this concept's rows belong to accounts: it declared an account form of @belongsTo, of any kind. It is the question the READERS of the stamp ask (the inventory, export): is `accountId` on a row of this concept the row's account at all.

func (*Concept) BytesFields

func (c *Concept) BytesFields() []BytesField

BytesFields returns every top-level field the concept's definition schema marks with x-bytes (emitted from @bytes), sorted by name. Nil when there is none, and for a nil concept.

func (*Concept) ClusterWide

func (c *Concept) ClusterWide() bool

ClusterWide reports whether this concept DECLARED `@belongsTo(cluster)`: its rows are never inside an account. False for an undeclared concept -- see the file comment.

func (*Concept) Create

func (c *Concept) Create(ctx context.Context, store Store, params CreateParams) (Node, error)

Create validates the payload against the definition schema and inserts a new memory node.

func (*Concept) DataState

func (c *Concept) DataState() parser.DataState

DataState returns the derived relationship between MemQL and this concept's data: mirror, origin, or native.

func (*Concept) DeclaredConnectors

func (c *Concept) DeclaredConnectors() []string

DeclaredConnectors returns every connector name the concept names -- its external origin, plus every mirror target -- in a stable order. This is the set the boot check resolves against the connector registry.

func (*Concept) DeclaredFields

func (c *Concept) DeclaredFields() []string

DeclaredFields returns the names of every top-level payload field declared on the concept's definition schema, INCLUDING @internal / @serverSet fields. Backs the spec/shape binding validator (epic #2281): a concept-bound spec may predicate by bare name on any declared field, internal ones included. Names are returned in ascending order for deterministic diagnostics.

func (*Concept) DeclaredNestedFields

func (c *Concept) DeclaredNestedFields(block string) []string

DeclaredNestedFields returns the declared sub-field names of one nested block, or nil when the concept declares no such block (or declares it as a free-form object with no sub-fields).

It exists for the read-modify-write paths that pull a nested object out of the database and write it back whole (memql#3641). Since every nested block now rejects undeclared keys, such a path has to filter the object it read -- a row written before a field was retired still carries it, and the write-back would be refused for a key the CALLER never chose. Filtering against the CONCEPT rather than a hardcoded list is the point: a field declared later is not silently dropped by a stale list somebody forgot to grow.

Reads the same definition schema classifyFields does, so "declared" means one thing across this file.

func (*Concept) DefinitionSchema

func (c *Concept) DefinitionSchema() (json.RawMessage, error)

DefinitionSchema returns the definition schema document.

func (*Concept) DefinitionSchemaId

func (c *Concept) DefinitionSchemaId() (string, error)

DefinitionSchemaId returns the $id of the definition schema.

func (*Concept) Delete

func (c *Concept) Delete(ctx context.Context, store Store, params DeleteParams) (Node, error)

Delete inserts a deletion tombstone for the provided identifier.

func (*Concept) DeletionSchema

func (c *Concept) DeletionSchema() (json.RawMessage, error)

DeletionSchema returns the deletion schema document, if registered.

func (*Concept) DeletionSchemaId

func (c *Concept) DeletionSchemaId() (string, error)

DeletionSchemaId returns the $id of the deletion schema, if registered.

func (*Concept) DeriveContentId

func (c *Concept) DeriveContentId(payload map[string]any) string

DeriveContentId generates a deterministic content-addressed ID from the payload. Uses the same algorithm and salt as Create() for exact ID prediction.

func (*Concept) EffectiveOrigin

func (c *Concept) EffectiveOrigin() string

EffectiveOrigin returns the origin name a client should be shown: the declared connector, or "memql" when MemQL originates the data. Unlike the stored Origin field this is never empty, which is what lets a wire consumer render a badge without re-implementing the default.

func (*Concept) InternalFields

func (c *Concept) InternalFields() []string

InternalFields returns the names of every top-level payload field the concept marks @internal (x-internal) -- server-only fields that are never projected and never accepted from caller args. Ascending order.

func (*Concept) IsMirror

func (c *Concept) IsMirror() bool

IsMirror reports whether this concept is a faithful copy of data owned elsewhere -- the state that makes it read-only by construction.

func (*Concept) OriginDecl

func (c *Concept) OriginDecl() parser.OriginDecl

OriginDecl returns the concept's data-origins declaration. The zero value is meaningful: it is a NATIVE concept, which is what an undeclared concept is.

func (*Concept) PIIFields

func (c *Concept) PIIFields() []string

PIIFields returns the names of every top-level field the concept's definition schema marks with the x-pii custom keyword (emitted from the field's @pii annotation). The engine's hard-delete scrub (memql#1711) consults this list to zero every personally-identifying field generically, so adding a new PII field to a concept needs only the @pii annotation -- the scrub picks it up automatically and cannot drift out of sync with a hand-maintained list. Returns nil when the concept declares no PII fields. Order follows JSON map iteration and is not significant -- the scrub clears every name regardless.

func (*Concept) ProjectableFields

func (c *Concept) ProjectableFields() []string

ProjectableFields returns the names of every top-level payload field a shape's default projection should expose: all declared fields EXCEPT those marked @internal (x-internal). @serverSet fields ARE projectable (a `status` a mutation stamps is routinely rendered), so the default projection includes public + serverSet fields and excludes only the server-only @internal set. Names are returned in ascending order for deterministic shape templates. Backs the C5 empty-shape default projection (memql#2035).

NOTE createdAt / createdBy are NOT examples of this: they are reserved row intrinsics (see reservedPayloadFields), rejected as declared payload fields by ensureReservedFieldsNotDeclared, so they can never be @serverSet at all. They were cited here as examples until memql#2960, and the same error reached the authoring reference from this comment.

func (*Concept) PublicFields

func (c *Concept) PublicFields() []string

PublicFields returns the names of every top-level payload field that is neither @internal nor @serverSet -- the caller-writable, projectable subset. Names are returned in ascending order. Backs the C5 mutation accept-list validation (memql#2035).

func (*Concept) Query

func (c *Concept) Query(ctx context.Context, store Store, params QueryParams) ([]Node, error)

Query retrieves concept records applying default filters (e.g. skipping deletions).

func (*Concept) RedactValidationError

func (c *Concept) RedactValidationError(err error) error

redactSecretValidationError rewrites the leaf messages of a jsonschema validation error so no @secret field's value survives into the string Concept.Create returns (memql#3184).

This is the concept-payload half of @secret enforcement. The concept's own JSON schema is the ONLY place @minimum / @maximum / @format / @minLength are enforced -- a mutation's args block need not mirror them -- so without this the annotation was bypassed entirely on this path.

Three properties matter and each is load-bearing:

  1. The error is a TREE (*jsonschema.ValidationError with Causes), and ValidationError.Error() renders the FIRST LEAF, not the root. A nested violation -- the optional-datetime oneOf puts every datetime failure one level down -- carries the value on a leaf whose InstanceLocation is the only thing that identifies the field. Redaction therefore walks to the leaves and rewrites there.

  2. Only leaves whose keyword is in valueInterpolatingKeywords AND whose InstanceLocation resolves to an x-secret schema node are touched. Every other message is returned byte-for-byte unchanged, which is what TestConceptCreate_NonSecretMessagesAreByteIdentical pins.

  3. NESTED x-secret IS SUPPORTED HERE, deliberately and explicitly. Concept.SecretFields() is top-level only (and stays that way -- see TestSecretFieldsIsTopLevelOnly in component/memql), so it is NOT the accessor used here; instanceLocationIsSecret walks the raw definition schema along the failing instance pointer instead. That is required rather than optional: propertyToJSONSchema recurses for nested objects, array items, map values and discriminated-union variants (concept_parser.go), so @secret is expressible at any depth, and a top-level-only check would silently miss it. x-secret is also treated as INHERITED by everything below it, so a violation at /creds/apiKey is redacted when `creds` itself is @secret.

The returned error is the same *jsonschema.ValidationError with its leaf messages rewritten in place; the caller's %w chain and any errors.As on it keep working. Non-jsonschema errors are returned untouched. RedactValidationError is the exported entry point for callers OUTSIDE this package that run a concept's JSON schema themselves rather than going through Concept.validate -- the DSL-callable `validate` and `preflight` builtins (component/memql/executor_builtin.go) compile the concept's definition schema and call Validate directly, then surface every leaf message in a result payload returned to the caller.

It mutates the ValidationError tree in place and returns the same error, so a caller that goes on to read BasicOutput() sees the redacted messages. Non-jsonschema errors pass through untouched.

func (*Concept) RequiredFields

func (c *Concept) RequiredFields() []string

RequiredFields extracts the list of required field names from the definition schema. Returns an empty slice if no required fields are declared or if the schema cannot be parsed.

func (*Concept) SchemaVariant

func (c *Concept) SchemaVariant(variant string) (json.RawMessage, error)

SchemaVariant returns the raw schema document for the requested variant.

func (*Concept) SchemaVariantId

func (c *Concept) SchemaVariantId(variant string) (string, error)

SchemaVariantId returns the $id associated with the requested schema variant.

func (*Concept) SecretFields

func (c *Concept) SecretFields() []string

SecretFields returns the names of every top-level field the concept's definition schema marks with the x-secret custom keyword (emitted from the field's @secret annotation).

The engine's validation-error redaction (memql#3036) consults this list. Derived generically, exactly as PIIFields is: annotating a new field is the whole change, and there is no hand-maintained list to drift out of sync.

SCOPE. Read this before assuming a value is covered. @secret now redacts on every validation surface that quotes a rejected value, but it is still NOT a secrecy guarantee -- the limits below are as load-bearing as the coverage.

The list is derived from an EXHAUSTIVE enumeration of validators (every jsonschema `.Validate(` call site and every value-quoting rejection message under component/), not from adding one entry at a time. That method is the point: three successive incremental passes each shipped a scope paragraph that walked past a surface the next pass found, and the exhaustive sweep done for memql#3182 turned up two more (the DSL `validate` and `preflight` builtins) that the epic's own four-surface model did not contain. Extend this list by re-running the enumeration, never by appending.

COVERED -- the memql function-args validator (component/memql/function_validator.go), which quotes a rejected argument value at five sites (enum / minimum / maximum / pattern / date-time). memql#3036.

COVERED -- the tool-args validator, MemQLEngine.validateToolArgs (component/memql/tool_execution.go). Compiled from the SAME ArgsSchema that carries the Secret flag, and it runs BEFORE the function-args validator on the agent path, so it is the surface a rejected secret reaches first. Both exits are redacted (memql#3182): the message returned to the MODEL, and the WARN, which now redacts per key from the ArgsSchema instead of serializing the whole args map -- degrading to a values-free argKeys list when the tool's arguments cannot be classified. Coverage resolves through the tool's @handler function, so an explicitly-authored tool dispatching to the same function is covered too; a query/webhook/delegate handler is unclassifiable and logs keys only.

COVERED -- the automation args binder (component/automations/args_binding.go), a second validator mirroring this rule set over EVENT payloads; a graph.node.created event carries the row's fields flattened into its payload (executor_mutation.go). The loader stamps ArgsField.Secret from the trigger topic's concept, so the enum and pattern refusals and the WARN they are written to all print <redacted> -- closing the one path by which a concept row value could reach a STRUCTURED LOG. memql#3183.

COVERED -- concept payload JSON-schema validation (Concept.validate, so Create and Delete alike). This is where @minimum / @maximum / @format declared on the CONCEPT are actually enforced -- they are checked nowhere else, so unlike the function-args validator this needs no automation and no matching argument name. Six jsonschema keywords interpolate the instance value (minimum, maximum, exclusiveMinimum, exclusiveMaximum, multipleOf, format -- derived by reading the library, not guessed); all six redact, and every other message is byte-identical to an unannotated field's. memql#3184.

COVERED -- the DSL-callable `validate` and `preflight` builtins (component/memql/executor_builtin.go), which compile this same concept schema themselves and surface every leaf message in a RESULT PAYLOAD returned to the caller. Worse than an error string: no log escape is needed for the disclosure. Found by the memql#3182 enumeration.

Redaction here resolves secrecy through the SCHEMA at the failing instance location, so it covers @secret at any nesting depth and inherits it onto array elements -- deliberately unlike SecretFields() itself, which is TOP-LEVEL ONLY (below) and is not the accessor those paths use.

NOT COVERED -- matching is BY ARGUMENT NAME, not by write target, on the two args-validator surfaces. markSecretArgsFields stamps an args field whose NAME appears in this list, so a mutation whose insert block writes `apiKey: args.credential` on a @secret `apiKey` field leaves `credential` UNREDACTED. Renaming between arg and field is common style in this corpus. (The concept-payload and builtin surfaces are exempt: they resolve through the schema, not through an argument name.)

NOT COVERED -- query results. A @secret value is returned in full by any query projecting it, to every caller the row is returned to. The annotation is a statement about what a validation message may quote, not about who may read: WHO REACHES THE ROW is the concept's @rowAuthz to decide, so a concept carrying @secret must also declare a tier that names an audience. component/memql's TestEveryConceptWithASealedFieldDeclaresARowTier fails the build on one that does not -- v1:platform:globalSecret and v1:platform:partitionSecret declare the cluster-owner tier, and v1:platform:sourceCredential an owned one. Within that audience the value is still not redacted, and a projection that leaves the field out is the only thing that withholds it.

NOT COVERED -- length, everywhere and deliberately. "value too long (%d runes, max %d)" (function_validator.go, args_binding.go) and the jsonschema minLength / maxLength messages report a rune count for a secret field too. They quote no value, but a length is a disclosure. Left uniform on purpose: redacting it on one surface would make that surface the sole diverging one while withholding nothing the others do not already print.

UNCLASSIFIED -- prompt input-schema validation (component/memql/ai_prompts.go, PromptTemplate.ValidateData). It runs a jsonschema over caller data and its messages interpolate the instance the same way, but the schema is built from the prompt's own field list and is not concept-derived, so there is no x-secret in it to read. Nothing is redacted there; treat it as uncovered rather than as safe.

Returns nil when the concept declares no secret fields. Order follows JSON map iteration and is not significant.

func (*Concept) SecretNameFields

func (c *Concept) SecretNameFields() []string

SecretNameFields returns every top-level field the concept's definition schema marks with x-secret-name (emitted from @secretName), sorted. Nil when there is none, and for a nil concept.

func (*Concept) ServerSetFields

func (c *Concept) ServerSetFields() []string

ServerSetFields returns the names of every top-level payload field the concept marks @serverSet (x-serverSet) -- fields stamped server-side and never accepted from caller args (but projectable). Ascending order.

func (*Concept) SetContentIdSalt

func (c *Concept) SetContentIdSalt(salt string)

SetContentIdSalt configures the server-side salt used for content-addressed ID derivation.

func (*Concept) SharedRows

func (c *Concept) SharedRows() bool

SharedRows reports whether this concept declared `@belongsTo(account, shared)`: beside its accounts' rows it holds rows that belong to no account and are provided by the cluster to every account (a seeded skill beside one a person wrote). On such a concept a row with no account is a statement, not a gap, so nothing may fill it in with a fallback.

func (*Concept) Stays

func (c *Concept) Stays() string

Stays returns the reason this concept's rows never leave the cluster -- one of parser.BelongsToStaysReasons -- or "" when the kind travels, is not account-scoped, or declared nothing. Only an account export asks it.

func (*Concept) Travels

func (c *Concept) Travels() bool

Travels reports whether an account export carries this concept's rows: the kind is account-scoped and declared no `stays`. Narrower than AccountScoped on purpose -- a kind that stays is the account's all the same.

type CreateParams

type CreateParams struct {
	Actor    string
	Payload  map[string]any
	ID       string
	Concept  string
	Clock    func() time.Time
	Metadata map[string]string
}

CreateParams describes the payload persisted when creating a new concept record.

type DeleteParams

type DeleteParams struct {
	Actor  string
	ID     string
	Reason string
	Clock  func() time.Time
}

DeleteParams describes the deletion tombstone persisted for a concept record.

type DisplayCard

type DisplayCard struct {
	// Primary is the payload field that names the row (e.g.
	// "name" for agents, "title" for spaces, "goal" for plans,
	// "inviteeEmail" for invitations). Mandatory when the
	// annotation is present.
	Primary string `json:"primary"`
	// Secondary is contextual: a role, a type discriminator, a
	// kind enum. Optional.
	Secondary string `json:"secondary,omitempty"`
	// Tertiary is extra context (owner display, parent space,
	// short tag). Optional.
	Tertiary string `json:"tertiary,omitempty"`
	// Status is a boolean or short-enum field that drives a
	// colored badge in the row chrome. Optional.
	Status string `json:"status,omitempty"`
}

DisplayCard captures the per-concept rendering hints declared via the `@displayCard(...)` annotation. Concept-agnostic clients (the cockpit's Concepts tab, future generic browsers) project rows through these slot names instead of carrying per-concept rendering code. Nil when the concept didn't declare the annotation -- clients should fall back to a generic "id + intrinsics" rendering.

See memql#160 for the design notes + slot semantics.

type EnvKeys

type EnvKeys struct {
	ContentIdSalt string
}

EnvKeys defines the environment variable suffixes to read.

type EnvLoader

type EnvLoader struct {
	Prefix string
	Keys   EnvKeys
}

EnvLoader describes how to load EnvOptions.

type EnvOptions

type EnvOptions struct {
	ContentIdSalt string
}

EnvOptions captures memory nodes configuration derived from environment variables.

func LoadDefaultEnvOptions

func LoadDefaultEnvOptions() (EnvOptions, error)

LoadDefaultEnvOptions loads EnvOptions using the default prefix.

func LoadEnvOptions

func LoadEnvOptions(loader EnvLoader) (EnvOptions, error)

LoadEnvOptions resolves EnvOptions from environment variables.

type FieldAnnotationTypeError

type FieldAnnotationTypeError struct {
	Annotation string
	Field      string
	Msg        string
}

FieldAnnotationTypeError refuses a @bytes or @secretName the field's type cannot carry.

func (*FieldAnnotationTypeError) Error

func (e *FieldAnnotationTypeError) Error() string

func (*FieldAnnotationTypeError) RuleCode

func (e *FieldAnnotationTypeError) RuleCode() string

RuleCode is the refusal's stable rule id (baseloader.CodedRefusal).

type MemoryNode

type MemoryNode struct {
	bun.BaseModel `bun:"table:MemoryNodes,alias:mn"`
	ID            string          `bun:",pk" json:"id"`
	CreatedAt     time.Time       `bun:"\"createdAt\",pk,type:TIMESTAMPTZ" json:"createdAt"`
	CreatedBy     string          `bun:"\"createdBy\",notnull" json:"createdBy"`
	Concept       string          `bun:",notnull" json:"concept"`
	Type          string          `bun:"type,notnull" json:"type"`
	Schema        json.RawMessage `bun:"type:JSONB,notnull" json:"schema"`
	Payload       json.RawMessage `bun:"type:JSONB,notnull" json:"payload"`
	Metadata      json.RawMessage `bun:"type:JSONB,notnull,default:'{}'" json:"metadata,omitempty"`

	// Provenance is engine-stamped per-version metadata describing
	// where this row came from -- seed name, mutation name,
	// originating automation + trigger event, etc. NOT NULL: the
	// mutation executor rejects writes whose Go context carries no
	// provenance.Provenance value, so every row that ever lands
	// has a real attribution. See component/provenance for the
	// shape + helpers.
	Provenance json.RawMessage `bun:"type:JSONB,notnull" json:"provenance"`
}

type MemoryNodesDatabase

type MemoryNodesDatabase struct {
	*database.TimescaleDBDatabase
}

func NewMemoryNodesDatabase

func NewMemoryNodesDatabase(args ...database.DatabaseArg) (*MemoryNodesDatabase, error)

func (*MemoryNodesDatabase) AssertCriticalSchema

func (db *MemoryNodesDatabase) AssertCriticalSchema(ctx context.Context) error

AssertCriticalSchema returns a non-nil error if any required table is absent (or the database is not connected). It lets GET /readyz prove schema presence WITHOUT database credentials, since the staging DB is firewalled to in-cluster egress and an external deploy gate cannot reach it directly (#657).

func (*MemoryNodesDatabase) BunDB

func (db *MemoryNodesDatabase) BunDB() *bun.DB

BunDB exposes the underlying Bun database handle. It returns nil until the database dependency has started.

func (*MemoryNodesDatabase) CreateMemoryNode

func (db *MemoryNodesDatabase) CreateMemoryNode(ctx context.Context, node *MemoryNode) error

func (*MemoryNodesDatabase) FindMemoryNodes

func (db *MemoryNodesDatabase) FindMemoryNodes(ctx context.Context, params QueryParams) ([]MemoryNode, error)

staged-data: MUST-NOT-GATE -- see ListRecentMemoryNodes above. This one is the clearest case of the three: component/memql's bunStore.QueryMemoryNodes is a near-verbatim copy of this function living one module up, and that copy is where the ruling is recorded, because that is where the staged set is visible.

func (*MemoryNodesDatabase) InsertMemoryNode

func (db *MemoryNodesDatabase) InsertMemoryNode(ctx context.Context, node *MemoryNode) error

InsertMemoryNode satisfies the memorynodes.Store interface by delegating to CreateMemoryNode.

func (*MemoryNodesDatabase) ListRecentMemoryNodes

func (db *MemoryNodesDatabase) ListRecentMemoryNodes(ctx context.Context, createdBy string, limit int) ([]MemoryNode, error)

staged-data: MUST-NOT-GATE -- and the same ruling covers LoadMemoryNode and FindMemoryNodes below (epic memql#3974, task memql#3984).

Gating here would create a SECOND SOURCE OF TRUTH for concept visibility, in the one module that cannot see the first. The staged set lives on the ENGINE (memql#3980 keys it per canonical id in MemQLEngine.promotedAuthored); component/database sits below component/memql in the module graph and cannot import it, so a gate in this file would have to reimplement or re-derive the set. Two derivations of "which concepts are staged" agree on the day they are written and diverge afterwards, and the divergence that publishes rows is invisible from outside -- which is precisely the failure this tier is built to prevent.

The precedent is already here and points the same way: component/memql's bunStore.QueryMemoryNodes is a near-verbatim duplicate of FindMemoryNodes, lives one module up, and is ADJUDICATED there -- MUST-NOT-GATE, because it backs loadPriorPayload's read-merge, with the reasoning recorded on it. Two copies of one read, one ruling, made where the state is visible. A ruling made here would be made without the state and could not be kept in step.

Second, narrower fact, recorded because it changes what a future reader should do rather than what they should think: these three have ZERO in-tree callers. That does NOT make them safe to ignore -- MemoryNodesDatabase is exported and something could wire it as a Store tomorrow, silently inheriting this decision. It makes the gate belong at that wiring, where the engine is in scope, and not here.

func (*MemoryNodesDatabase) LoadMemoryNode

func (db *MemoryNodesDatabase) LoadMemoryNode(ctx context.Context, id string) (*MemoryNode, error)

staged-data: MUST-NOT-GATE -- see ListRecentMemoryNodes above; a gate in this module would be a second, un-synchronisable source of truth for the staged set, which lives on the engine.

func (*MemoryNodesDatabase) QueryMemoryNodes

func (db *MemoryNodesDatabase) QueryMemoryNodes(ctx context.Context, params QueryParams) ([]MemoryNode, error)

QueryMemoryNodes satisfies the memorynodes.Store interface by delegating to FindMemoryNodes.

type MemoryRegistry

type MemoryRegistry struct {
	// contains filtered or unexported fields
}

MemoryRegistry maintains a thread-safe map of concept names to Concept definitions.

Known gap: duplicate canonical ids last-win SILENTLY (memql#3008)

The map is keyed by canonical id, so two declarations that assemble to the same id collapse to one entry and whichever registers last wins. Nothing on the boot path reports it -- there is no duplicate detector for concepts anywhere in loading.

Recorded rather than fixed, deliberately. Boot also loads product bundles delivered at runtime through MEMQL_DSL_PATH, so turning last-wins into a refusal here could stop a deployed product from starting, and that call needs visibility into bundles outside this repo. The LINT closes the half that can be closed safely: component/memql/dslimports reports a same-id collision as a hard error naming both files and the id (see duplicateConceptCollision), so a tree in THIS repo cannot reach boot carrying one. Closing the boot half deserves its own issue and its own blast-radius measurement.

func CloneDefaultRegistry

func CloneDefaultRegistry() *MemoryRegistry

CloneDefaultRegistry returns an ISOLATED *MemoryRegistry seeded from a snapshot of the global default registry. Mutating the clone (via MergeAll / ReplaceAll) never touches the global default, and later registrations into the default never leak into a previously-taken clone -- the two share no underlying map.

This backs the authoring sandbox's candidate-defined-concept path (issue #956): a planner-authored bundle can declare NEW concepts and overlay them onto the clone so later constructs in the same bundle bind against them, all without ever mutating the live engine's concept registry.

Note the *Concept pointers are shared with the default (a shallow copy of the name->pointer map). That is intentional: the clone exists to ADD candidate concepts, not to rewrite existing ones, and Concept values are treated as immutable once registered.

func NewRegistry

func NewRegistry(concepts map[string]*Concept) *MemoryRegistry

NewRegistry returns an ISOLATED *MemoryRegistry holding exactly concepts.

Unlike CloneDefaultRegistry it is seeded from nothing, which is the point: it backs a validation pass that must build a WHOLE tree's concepts from scratch and hand them to MemQLEngine.Init without the global default registry -- the one every live read consults for schemas and row-authz tiers -- being touched at any moment of the pass.

That distinction is load-bearing for component/packages. The pre-existing offline pass (LintUnifiedTree) reaches the same answer by emptying the default registry and reloading it, which is correct in a CLI that owns its process and is a security hole in a serving one: for the width of the pass every concept's declared tier is absent, and an absent tier admits everybody. A pass built on this constructor never has that window.

The map is copied, so a later mutation of the caller's map does not reach the registry.

func (*MemoryRegistry) All

func (r *MemoryRegistry) All() map[string]*Concept

All returns a copy of all registered concepts keyed by concept name.

func (*MemoryRegistry) Clone

func (r *MemoryRegistry) Clone() *MemoryRegistry

Clone returns an isolated copy of this registry. The returned registry has its own concept map seeded from the receiver's current contents; subsequent mutations on either side are independent.

func (*MemoryRegistry) Get

func (r *MemoryRegistry) Get(name string) (*Concept, error)

Get retrieves a concept by name or returns an error when not found.

func (*MemoryRegistry) List

func (r *MemoryRegistry) List() []*Concept

List returns a slice copy of all registered concepts.

func (*MemoryRegistry) MergeAll

func (r *MemoryRegistry) MergeAll(concepts map[string]*Concept)

MergeAll adds the provided concepts to the registry, replacing any existing entries with the same name. Empty inputs are no-ops.

func (*MemoryRegistry) Remove

func (r *MemoryRegistry) Remove(name string) bool

Remove deletes a concept from the registry by canonical id and reports whether an entry was actually removed. A name the registry does not hold is a no-op returning false.

This is the ONLY way a registered concept ever leaves the registry, and it exists for exactly one caller: the authoring demote of a promoted concept with ZERO rows under it (memql#3756). That case has to free the name -- a concept promoted by typo, with nothing written to it, must be cleanly withdrawable or `wodget` is taken forever on that cluster.

Deliberately NOT reachable for a concept the engine loaded from the DSL tree: nothing in the loader calls this, and the demote path that does is gated on the promotion marker, so a sealed core concept can never be unregistered. The registry itself does not know the difference -- the gate lives with the authoring lifecycle that owns it, in component/memql.

Atomic under the registry's own lock, unlike a read-modify-ReplaceAll by the caller: a concurrent promote merging a different concept cannot be lost between the read and the write.

func (*MemoryRegistry) ReplaceAll

func (r *MemoryRegistry) ReplaceAll(concepts map[string]*Concept)

ReplaceAll swaps the existing registry contents with the provided concepts.

type Node

type Node struct {
	ID        string
	Concept   string
	Type      string
	CreatedAt time.Time
	CreatedBy string
	Schema    json.RawMessage
	Payload   json.RawMessage
	Metadata  json.RawMessage
}

Node represents a memory node materialized through a concept accessor.

type PatternError

type PatternError struct {
	Pattern string
	Err     error
}

PatternError is the refusal of a `@pattern` that does not compile. It names the pattern as written, why it does not compile, and what to do; the owning field is named by the caller, which knows which receiver it is.

func (*PatternError) Error

func (e *PatternError) Error() string

Error renders the refusal with its rule id last, in brackets (D24).

func (*PatternError) RuleCode

func (e *PatternError) RuleCode() string

RuleCode is the refusal's stable rule id (baseloader.CodedRefusal).

func (*PatternError) Unwrap

func (e *PatternError) Unwrap() error

Unwrap exposes the regexp package's own error.

type QueryOrder

type QueryOrder int

QueryOrder specifies the order of results when querying memory nodes.

const (
	// QueryOrderCreatedAtDesc specifies descending order by createdAt.
	QueryOrderCreatedAtDesc QueryOrder = iota
	// QueryOrderCreatedAtAsc specifies ascending order by createdAt.
	QueryOrderCreatedAtAsc
)

type QueryParams

type QueryParams struct {
	IDs            []string
	Concept        string
	IncludeDeleted bool
	CreatedBy      string
	Limit          int
	Order          QueryOrder
}

QueryParams configures filters applied when querying concept records.

type Registry

type Registry interface {
	Get(name string) (*Concept, error)
	List() []*Concept
}

Registry exposes read-only access to loaded concept metadata.

func DefaultRegistry

func DefaultRegistry() Registry

DefaultRegistry returns the globally shared Registry instance.

type RelationshipDefinition

type RelationshipDefinition struct {
	Type          string `json:"type"`
	Field         string `json:"field"`
	FieldSource   string `json:"fieldSource,omitempty"`
	TargetConcept string `json:"targetConcept"`
	Direction     string `json:"direction"`
	As            string `json:"as,omitempty"`
}

RelationshipDefinition captures the structure of relationship metadata declared in concept.json files.

Type and As are two INDEPENDENT axes (memql#3652). Type is the closed, engine-defined structural set -- what the engine does with the edge. As is the open, author-defined domain label -- what the edge MEANS -- validated for form only and never for membership, so a new domain verb can never require an engine release.

func (*RelationshipDefinition) UnmarshalJSON

func (r *RelationshipDefinition) UnmarshalJSON(data []byte) error

UnmarshalJSON normalizes relationship definitions authored with either camelCase or snake_case keys.

type RetentionPolicy

type RetentionPolicy struct {
	TTL        time.Duration `json:"ttl"`
	KeepLatest bool          `json:"keepLatest"`
	Archive    string        `json:"archive"`
}

RetentionPolicy is declared on a concept. A nil policy changes nothing. TTL measures stored version time, not a user-controlled payload field.

type SchemaCompactor

type SchemaCompactor struct {
	*component.Component
	// contains filtered or unexported fields
}

SchemaCompactor converts old inline schemas after startup. Each transaction owns the shared cursor row, so a second replica skips the work and a crash rolls back both the batch and its cursor. Serving and liveness never wait on this maintenance loop; every batch has its own five-second deadline.

func NewSchemaCompactor

func NewSchemaCompactor(provider func() *bun.DB, nodeID func() string, logger *slog.Logger) (*SchemaCompactor, error)

func (*SchemaCompactor) Order

func (*SchemaCompactor) Order() int

type SecretConcept

type SecretConcept struct {
	Name        string                     `json:"concept"`
	SchemaId    string                     `json:"schemaId"`
	Schemas     map[string]json.RawMessage `json:"schemas"`
	NodeType    string                     `json:"type"`
	Description string                     `json:"description,omitempty"`
}

SecretConcept mirrors Concept but is intended for secret memory node handling.

type SecretMemoryNode

type SecretMemoryNode struct {
	bun.BaseModel `bun:"table:SecretMemoryNodes,alias:smn"`
	ID            string          `bun:",pk" json:"id"`
	CreatedAt     time.Time       `bun:"\"createdAt\",pk,type:TIMESTAMPTZ" json:"createdAt"`
	CreatedBy     string          `bun:"\"createdBy\",notnull" json:"createdBy"`
	Concept       string          `bun:",notnull" json:"concept"`
	Type          string          `bun:"type,notnull" json:"type"`
	Schema        json.RawMessage `bun:"type:JSONB,notnull" json:"schema"`
	Payload       json.RawMessage `bun:"type:JSONB,notnull" json:"payload"`
	Metadata      json.RawMessage `bun:"type:JSONB,notnull,default:'{}'" json:"metadata,omitempty"`

	// Provenance: same shape + contract as on MemoryNode.
	Provenance json.RawMessage `bun:"type:JSONB,notnull" json:"provenance"`
}

type Store

type Store interface {
	InsertMemoryNode(ctx context.Context, node *MemoryNode) error
	QueryMemoryNodes(ctx context.Context, params QueryParams) ([]MemoryNode, error)
}

Store defines the persistence operations required by concept runtime helpers.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址