Documentation
¶
Overview ¶
Package observe is the runtime side of the architecture framework: a thin instrumentation surface call sites use to emit per-invocation records (count, duration, args, result, error) that the cockpit's drill-down view consumes alongside the static topology model.
Design goals:
Stay out of the way when the level is "off" (the default in production). The fast-path is one map lookup, one comparison, and a stack-allocated zero value -- no allocations, no channels, no syscalls.
Stay Go-native. Built on context.Context, log/slog, and standard channels; no SDK dependencies. The sink is an interface that a TimescaleDB writer plugs into; the default sink logs structured events via slog so something useful still happens when no writer is registered.
Be safe by default. Argument capture redacts anything whose parameter name matches /(?i)pass|token|secret|key|auth/ unless the declaring function explicitly listed it as safe via a //memql:observe redact=... source marker (consulted by the static extractor; this package consumes the resulting attrs only at runtime).
Usage at a call site that should report:
func (h *Handler) Login(ctx context.Context, user, password string) (err error) {
defer observe.Method(ctx, "method:.../auth.(*Handler).Login").
Args(observe.Arg("user", user), observe.Arg("password", password)).
End(&err)
...
}
The fully-qualified name string must match the `id` of the corresponding Method node in generated topology.model.json. Offline tools can join on that key; this runtime does not load the generated graph.
Index ¶
- Constants
- func ClearProfile(fqn string)
- func ProfileSize() int
- func Register(s Sink)
- func SetProfile(fqn string, level Level, expiresAt time.Time)
- func SetTraceExtractor(f TraceIDFromContext)
- func SweepExpiredProfiles(now time.Time) int
- type ArgKV
- type BunDBProvider
- type CodeProfileSubscriber
- func (s *CodeProfileSubscriber) ComponentName() common.ComponentName
- func (s *CodeProfileSubscriber) IsRunning() bool
- func (s *CodeProfileSubscriber) Order() int
- func (s *CodeProfileSubscriber) Ready() <-chan struct{}
- func (s *CodeProfileSubscriber) Start(_ context.Context)
- func (s *CodeProfileSubscriber) Stop(_ context.Context)
- type Invocation
- type Level
- type Record
- type Sink
- type SinkComponent
- type TimescaleSink
- type TimescaleSinkOptions
- type TraceIDFromContext
Constants ¶
const CodeProfileSubscriberComponent = common.ComponentName("observe.codeProfileSubscriber")
CodeProfileSubscriberComponent is the bus-side ComponentName.
const ComponentName = common.ComponentName("observe")
ComponentName is the bus-side identifier for the observability runtime. Used in lifecycle logs and in the dependency ordering math; not surfaced to end users.
const EnvLevelVar = "MEMQL_OBSERVE_LEVEL"
EnvLevelVar is the env var the package reads at init() time to pick the global default. Per-method levels (set live via the codeProfile concept once that lands) override this; until then it's the only knob.
Variables ¶
This section is empty.
Functions ¶
func ClearProfile ¶
func ClearProfile(fqn string)
ClearProfile removes the override for fqn. Equivalent to setting the level to off for callers that want to evict the row rather than leave a tombstone entry.
func ProfileSize ¶
func ProfileSize() int
ProfileSize is intended for tests / metrics. Returns the current number of cached entries (including not-yet-swept expired ones).
func Register ¶
func Register(s Sink)
Register installs a Sink that consumes invocation records. The default sink logs to slog; the TimescaleDB writer registers a real Sink at startup. Passing nil resets to the default.
func SetProfile ¶
SetProfile installs (or updates) a per-FQN level override. Called by the codeProfile CDC subscriber when a row is created or updated; safe to call concurrently. Passing LevelOff is the canonical way to disable an override -- the helper falls through to the default level, which is normally LevelOff too unless the process-wide default has been raised.
expiresAt is honored at lookup time; the helper still consults SweepExpiredProfiles periodically to keep the entries map small. Passing the zero time disables expiry.
func SetTraceExtractor ¶
func SetTraceExtractor(f TraceIDFromContext)
SetTraceExtractor installs the trace-correlation hook.
func SweepExpiredProfiles ¶
SweepExpiredProfiles removes entries whose expiresAt is in the past. The cron-style automation calls this on a fixed cadence (typically every minute). Returns the number of entries removed.
Types ¶
type ArgKV ¶
type ArgKV struct {
Name string
Value any
Redact bool // if true, value is force-redacted regardless of level
}
ArgKV is one captured argument. Captured at the call site via the Arg() / RedactedArg() constructors; the helper decides whether (and how) the value lives in the final Record based on the active Level and the function's redact list.
func Arg ¶
Arg captures a single argument by name. Use this for safe values (ids, modes, public counts). The helper still applies a name- pattern redactor by default, so even Arg("password", x) emits a redacted form unless the function's source marker explicitly listed "password" in `redact=` (which inverts the meaning: that list names the params SAFE to capture).
func RedactedArg ¶
RedactedArg captures a name with a placeholder value. The Record always shows "<redacted>" for these even at LevelVerbose. Useful for arguments the caller knows are sensitive without relying on the name pattern.
type BunDBProvider ¶
BunDBProvider is the narrow seam between the observe runtime and the MemQL database component. The sink doesn't import the database package directly because that's a heavy graph; instead the bootstrap hands it a closure that resolves to the underlying *bun.DB at Start time. Returns nil if the database isn't ready yet (the sink retries lazily).
type CodeProfileSubscriber ¶
type CodeProfileSubscriber struct {
// contains filtered or unexported fields
}
CodeProfileSubscriber bridges v1:observability:codeProfile concept events to the in-process observe.profileCache. The flow:
- A cockpit user (or seed script, or admin tool) inserts a codeProfile row -- the engine writes it through and emits a graph.node.created.v1:observability:codeProfile event (the codeProfile concept is @scope("global"), so events fire under _system regardless of the writer's partition envelope).
- This subscriber matches the pattern, extracts codeReference / level / expiresAt from the event payload, and calls SetProfile so the next observe.Method(fqn) call honors the new level.
- A periodic sweeper (started in Start) drops expired entries so the cache stays small after a "verbose for an hour" window closes.
Lifecycle is a Dependency so the app bootstrap can wire it in the engine phase, where the event bus is already alive.
func NewCodeProfileSubscriber ¶
func NewCodeProfileSubscriber(logger *slog.Logger, bus *events.Bus) *CodeProfileSubscriber
NewCodeProfileSubscriber constructs the bridge. The events.Bus is taken at construction time; the actual Subscribe call runs in Start so a not-yet-ready bus surfaces as a startup error rather than as a constructor panic.
func (*CodeProfileSubscriber) ComponentName ¶
func (s *CodeProfileSubscriber) ComponentName() common.ComponentName
func (*CodeProfileSubscriber) IsRunning ¶
func (s *CodeProfileSubscriber) IsRunning() bool
Standard Dependency surface so the app bootstrap can hold this alongside the rest of the component graph. Order is chosen to match the engine phase's other lightweight subscribers (>5, well after the database is up).
func (*CodeProfileSubscriber) Order ¶
func (s *CodeProfileSubscriber) Order() int
func (*CodeProfileSubscriber) Ready ¶
func (s *CodeProfileSubscriber) Ready() <-chan struct{}
func (*CodeProfileSubscriber) Start ¶
func (s *CodeProfileSubscriber) Start(_ context.Context)
Start subscribes to the codeProfile event pattern and launches the expiry sweeper. Idempotent.
func (*CodeProfileSubscriber) Stop ¶
func (s *CodeProfileSubscriber) Stop(_ context.Context)
Stop closes the subscription and stops the sweeper.
type Invocation ¶
type Invocation struct {
// contains filtered or unexported fields
}
Invocation is the in-flight handle returned by Method() / Func(). Construct once at the top of an instrumented function, optionally attach args / result, and finalize with End(&err) (typically via defer). The struct is small and stack-allocated; no allocations on the off-path.
func Func ¶
func Func(ctx context.Context, fqn string) Invocation
Func is an alias for Method that reads better at the call site for non-receiver functions. It records the same record kind and fqn format ("func:..."); the cockpit renders both under the same drill-down node.
func Method ¶
func Method(ctx context.Context, fqn string) Invocation
Method returns an Invocation handle for a method call. fqn must match the corresponding model.Node.ID (e.g. "method:github.com/.../auth.(*Handler).Login").
Level resolution, in precedence:
- Per-FQN override installed via SetProfile (driven by the v1:observability:codeProfile CDC subscriber).
- Process-wide DefaultLevel (set from MEMQL_OBSERVE_LEVEL at start; mutable via SetDefaultLevel).
When the resolved level is LevelOff, the returned handle is the zero value of Invocation and every subsequent method is a no-op; the helper never touches the heap on the off-path.
func (Invocation) Args ¶
func (i Invocation) Args(kvs ...ArgKV) Invocation
Args attaches the captured arguments to the in-flight invocation. At LevelCount, the call is a fast no-op (the slice never gets walked). At LevelMeta, only the names + types + sizes are kept -- no values. At LevelVerbose, the values are kept unless the name matches the default redact pattern OR the corresponding ArgKV carries Redact: true.
The redact-by-default pattern catches obvious-by-name secrets even on functions that haven't yet been annotated with a //memql:observe redact=... marker; opt-out is by listing the param name in the marker (the extractor stamps the *safe* list onto the Method's `redact_args` attr, which the runtime sink inverts at write time -- see sink.go).
func (Invocation) End ¶
func (i Invocation) End(errp *error)
End finalizes the invocation and emits a Record to the active Sink. errp may be nil (function returns no error); when non-nil and *errp != nil, the error text lands on Record.Error.
Pass-by-pointer to *error lets the canonical
defer observe.Method(ctx, "...").End(&err)
shape work without callers having to capture the error in two places. errp is read at End time, so deferred error wrapping just before return is captured correctly.
func (Invocation) Result ¶
func (i Invocation) Result(v any) Invocation
Result attaches the return value for capture at LevelVerbose. Below LevelVerbose, the call is a no-op so callers can wrap it unconditionally.
type Level ¶
type Level int
Level is the runtime verbosity dial. Values correspond exactly to the observe_level attribute the static extractor stamps on Method / Func nodes from the //memql:observe source marker. Comparison is numeric so a runtime check ("am I at least at meta?") is one integer compare.
func DefaultLevel ¶
func DefaultLevel() Level
DefaultLevel returns the current process-wide default level. Cheap and lock-free in the hot path: read once at start of each Method() call, branch on it, and skip the rest of the helper if the value is LevelOff.
func ParseLevel ¶
ParseLevel turns the textual form (off / count / meta / verbose) into a Level. Unknown / empty strings resolve to LevelOff so a typo in .env never accidentally turns observability on; the caller can detect this case by checking ok.
func SetDefaultLevel ¶
SetDefaultLevel changes the process-wide default. Callers should only use this when responding to a live codeProfile update; tests may also call it to scope a single test. Returns the previous value so the caller can restore it (test cleanup pattern).
type Record ¶
type Record struct {
FQN string // matches model.Node.ID for a Method/Func
Ts time.Time // start time
Duration time.Duration // end - Ts
Error string // non-empty when the caller's *error was non-nil
Args map[string]any // captured per active Level; redacted values appear as "<redacted>"
Result any // captured at LevelVerbose only
TraceID string // from SetTraceExtractor
SpanID string // from SetTraceExtractor
Level Level // the level at which this record was captured (audit)
}
Record is one invocation row written by a Sink. Field set mirrors the v1:observability:invocation concept on the MemQL side and the code_invocation hypertable on the Postgres side, so a Sink can serialize directly without an intermediate type.
One name diverges on purpose: Args lands in the `args` column of code_invocation but is spelled `arguments` on the concept, because `args` is a reserved engine namespace and cannot be a MemQL payload property (memql#3613). The concept is a declaration of this table's shape, not its storage -- nothing maps concept field to column, so the SQL column name is unchanged.
type Sink ¶
Sink consumes finalized Records. The package ships a default slog-backed sink so importing observe with no further wiring still produces useful output (a debug log line per call). Production deployments register a buffered TimescaleDB writer via Register(); the helper itself is sink-agnostic.
Implementations MUST be non-blocking; the helper calls Write synchronously from the instrumented function's deferred path and a slow sink directly tax the application's request latency. Buffered sinks should drop on full rather than block, so the instrumentation never becomes a head-of-line waiter.
type SinkComponent ¶
type SinkComponent struct {
// contains filtered or unexported fields
}
SinkComponent is the dependency-bus integration for the observe runtime. Owns the TimescaleSink lifecycle so the app bootstrap can wire it in alongside the other components and shutdown flushes the buffered records cleanly.
func NewSinkComponent ¶
func NewSinkComponent(logger *slog.Logger, provider BunDBProvider, opts TimescaleSinkOptions, order int) *SinkComponent
NewSinkComponent constructs the dependency wrapper. Order should be higher than the database component's order so this component starts AFTER the DB; pass 0 to fall back to a default that's safely later than the standard memql startup phases.
func (*SinkComponent) ComponentName ¶
func (c *SinkComponent) ComponentName() common.ComponentName
func (*SinkComponent) IsRunning ¶
func (c *SinkComponent) IsRunning() bool
IsRunning, Order, ComponentName, Ready -- standard Dependency surface. Ready closes when Start has finished its setup regardless of whether the DB resolved (the runtime should not block other components on us being healthy).
func (*SinkComponent) Order ¶
func (c *SinkComponent) Order() int
func (*SinkComponent) Ready ¶
func (c *SinkComponent) Ready() <-chan struct{}
func (*SinkComponent) Start ¶
func (c *SinkComponent) Start(_ context.Context)
Start resolves the bun handle, constructs the sink, registers it as the active observe sink, and kicks off the buffered drain. Failure to resolve the DB is logged but not fatal -- the default slog sink stays active and the runtime keeps working at LevelOff / LevelCount without persistence.
func (*SinkComponent) Stop ¶
func (c *SinkComponent) Stop(ctx context.Context)
Stop flushes the remaining buffer and reinstalls the default slog sink so any in-flight callers writing through observe.Method don't NPE on a closed sink.
type TimescaleSink ¶
type TimescaleSink struct {
// contains filtered or unexported fields
}
TimescaleSink writes captured invocation records into the code_invocation hypertable backing the v1:observability:invocation concept. Writes are batched so the instrumentation hot path never blocks on Postgres; a single goroutine drains the buffer every FlushInterval (or whenever the buffer fills) and issues a single COPY-style multi-row insert via bun.
The sink is intentionally lossy under sustained pressure: when the buffer is full and a new record arrives, the record is dropped and the drop counter is incremented. The alternative -- blocking the instrumented call -- would turn observability into a head-of-line latency hazard, which is exactly what we want to avoid.
Lifecycle: NewTimescaleSink returns a sink + a Dependency the app bootstrap registers so Stop() flushes the remaining buffer on graceful shutdown.
func NewTimescaleSink ¶
func NewTimescaleSink(db *bun.DB, opts TimescaleSinkOptions) *TimescaleSink
NewTimescaleSink constructs the sink. It does not start the background drainer; call Start (typically from app bootstrap) to begin draining. Stop blocks until the buffer has flushed.
func (*TimescaleSink) Start ¶
func (s *TimescaleSink) Start()
Start begins the background drain. Safe to call multiple times; only the first invocation has effect.
func (*TimescaleSink) Stats ¶
func (s *TimescaleSink) Stats() (written, dropped, flushes, errs uint64)
Stats returns a snapshot of the sink's counters. Useful for tests and operational dashboards; not exposed via concept yet.
type TimescaleSinkOptions ¶
type TimescaleSinkOptions struct {
BufferSize int
FlushInterval time.Duration
MaxBatch int
Logger *slog.Logger
}
TimescaleSinkOptions configures the buffered writer. Zero values fall back to conservative defaults that work well on a dev machine: 1024-record buffer, 1-second flush, 256-row batches.
type TraceIDFromContext ¶
TraceIDFromContext is the hook the helpers use to stamp trace_id / span_id onto each record. Wired by callers that have distributed-trace context (OpenTelemetry, the gRPC interceptor, etc.) via SetTraceExtractor; default is a no-op so the package has no required dependency on a tracing system.