Documentation
¶
Index ¶
- Constants
- func ControlPlaneClientSecretName(egNamespace, egName string) string
- func EgressGatewayCASecretName(egName string) string
- func EnsureControlPlaneCABundle(ctx context.Context, r client.Reader, w client.Client, ns string, ...) error
- func EnsureControlPlaneRoot(ctx context.Context, r client.Reader, w client.Client, ns string) (rootCertPEM, rootKeyPEM []byte, err error)
- func EnsureControlPlaneServerCert(ctx context.Context, r client.Reader, w client.Client, ns, advertiseURI string, ...) (serverCertPEM, serverKeyPEM, caBundlePEM []byte, err error)
- func MintEgressGatewayCA(namespace, name string) (certPEM, keyPEM []byte, err error)
- type AIProviderRouteStatusReconciler
- type CLRKConfigReconciler
- type CredentialInjectionPolicyStatusReconciler
- type DaemonAgentRevisionReconciler
- type EgressGatewayOTLPForwardReconciler
- type EgressGatewayReconciler
- type FallbackRoutingPolicyStatusReconciler
- type HTTPInvoker
- type InvocationActiveCounter
- type TaskAgentCronReconciler
- type TaskAgentIngressReconciler
- type TaskAgentInvoker
- type TaskAgentRevisionReconciler
- type WorkerPoolDeploymentReconciler
- type WorkerPoolStatusReconciler
Constants ¶
const ( ControlPlaneRootSecretName = "clrk-control-plane-root" ControlPlaneServerSecretName = "clrk-control-plane-server" ControlPlaneCASecretName = "clrk-control-plane-ca" ControlPlaneSecretLabel = "clrk.apoxy.dev/control-plane-pki" )
Control-plane PKI Secrets + label. All live in the controller-manager's runtime namespace (POD_NAMESPACE). The root issues the :9443 server cert and every per-EG client cert; the CA bundle is the mount-only trust anchor the data-plane Envoys reference to verify the server cert.
const ( // EgressGatewayClassName is the shared GatewayClass we install for all // clrk EgressGateways. Envoy Gateway's Gateway controller binds Gateways // that reference this class to the clrk EnvoyProxy configuration. EgressGatewayClassName = "clrk-egress" // GatewayNamePrefix must match internal/egextension.GatewayNamePrefix. // Kept as a constant here to avoid importing the extension package into // the reconciler layer. GatewayNamePrefix = "clrk-eg-" // EgressGatewayFinalizer guards EgressGateway deletion so the // reconciler can reap the upstream-trust mirror Secret before the // object disappears. The mirror lives in the EG control-plane // namespace (EnvoyGatewayNamespace), not the EG's own namespace, so // it cannot carry an EG owner reference (SetControllerReference // rejects cross-namespace owners) and would otherwise leak on delete. EgressGatewayFinalizer = "clrk.apoxy.dev/egressgateway-upstream-trust" // EgressListenerBasePort is the first TCP port the EG-managed Envoy // binds. Each EgressListener in spec.listeners is assigned // EgressListenerBasePort+index — protocol shapes (TLS-Terminate, // plain TCP, etc.) cannot share a port, so each materializes as a // distinct Envoy listener and Service ServicePort. EgressListenerBasePort int32 = 18080 // MaxEgressListeners caps spec.listeners to keep the assigned port // range narrow and reviewable. Bump if a real workload needs more. MaxEgressListeners = 8 // LLMRevAnnotation carries a hash of the LLM routing inputs // (attached AIProviderRoutes, their referenced Backends, attached // FallbackRoutingPolicies) on the clrk-owned Gateway, for operator // observability. NOTE: the annotation alone does NOT trigger Envoy // Gateway retranslation — EG's watch predicate passes annotation // changes, but its gateway-api runner dedupes on the translated IR // and Gateway metadata never reaches the IR, so the xds-translator // (where the egextension hooks live) is never re-run. The actual // trigger is the revision header-match rule ensureCatchAllRoute // stamps on the catch-all HTTPRoute (see LLMRevHeaderName): route // rules ARE IR-visible, and the egextension replaces EG's whole // route configuration so the rule never serves traffic. LLMRevAnnotation = "clrk.apoxy.dev/llm-rev" // LLMRevHeaderName is the header name of the IR-perturbing revision // match on the catch-all HTTPRoute. Never sent by clients; exists // solely so a clrk CR change makes EG's translated IR differ and // forces the xds-translation pass that re-runs the extension hooks. LLMRevHeaderName = "x-clrk-llm-rev" )
const ( DispatchPort = ports.DispatchPort IngressExtProcBackendName = ports.IngressExtProcBackendName )
Re-exports of port + name consts so existing controller-package callers don't have to rewrite import paths. See internal/ports for the source of truth — that leaf package has no transitive deps, which is what lets cmd/clrk reach for them without pulling in generated protobuf code.
const ControlPlaneClientSecretLabel = "clrk.apoxy.dev/egressgateway-cp-client"
ControlPlaneClientSecretLabel marks a Secret as a per-EG control-plane mTLS client cert.
const EgressGatewayCASecretLabel = "clrk.apoxy.dev/egressgateway"
EgressGatewayCASecretLabel marks a Secret as the MITM CA for an EgressGateway.
Variables ¶
This section is empty.
Functions ¶
func ControlPlaneClientSecretName ¶
ControlPlaneClientSecretName is the deterministic name for a per-EG mTLS client-cert Secret. Like the upstream-trust mirror it lives in the EG control-plane namespace (r.EnvoyGatewayNamespace) so the EG-managed Envoy pod, which runs there, can mount it. Every EG across every namespace shares that one namespace, so the name embeds a short hash of the EG's namespaced name: two EGs of the same name in different namespaces must not collide onto a single Secret, or the second would mount the first's cert and present the wrong tenant identity (a namespace-qualified SAN) to the control plane.
func EgressGatewayCASecretName ¶
EgressGatewayCASecretName returns the deterministic Secret name holding the MITM CA keypair for the named EgressGateway. The Secret lives in the EgressGateway's namespace. The private key is consumed by the MITM handshaker in controller-manager; sandbox workers only need the certificate PEM.
func EnsureControlPlaneCABundle ¶
func EnsureControlPlaneCABundle(ctx context.Context, r client.Reader, w client.Client, ns string, caBundlePEM []byte) error
EnsureControlPlaneCABundle get-or-creates the shared CA-bundle Secret in ns that every data-plane Envoy mounts to verify the :9443 server cert. It holds only the root cert (no key) and is updated in place if the bundle changed (root rotation, rare).
func EnsureControlPlaneRoot ¶
func EnsureControlPlaneRoot(ctx context.Context, r client.Reader, w client.Client, ns string) (rootCertPEM, rootKeyPEM []byte, err error)
EnsureControlPlaneRoot get-or-creates the control-plane root CA Secret in ns and returns its cert + key PEM. Reads go through r (an uncached reader -- callers run this at startup, before the manager cache is started); writes go through w. A create race (another replica or a restart) is resolved by re-reading the winner's Secret so every replica trusts a single shared root.
func EnsureControlPlaneServerCert ¶
func EnsureControlPlaneServerCert(ctx context.Context, r client.Reader, w client.Client, ns, advertiseURI string, rootCertPEM, rootKeyPEM []byte) (serverCertPEM, serverKeyPEM, caBundlePEM []byte, err error)
EnsureControlPlaneServerCert get-or-creates the :9443 server-cert Secret in ns, signed by the given root, and returns the server cert/key PEM plus the CA bundle (the root cert). Desired SANs are derived from advertiseURI's host plus loopback names; an existing cert whose SANs no longer cover the desired set (advertiseURI changed across upgrades) is re-minted in place.
func MintEgressGatewayCA ¶
MintEgressGatewayCA generates a fresh ECDSA P-256 self-signed CA whose subject carries the EgressGateway coordinates for operator debuggability. It delegates to egpki.MintRootCA so the MITM CA and the control-plane PKI share one crypto shape (ECDSA P-256, 128-bit serial, SEC1 "EC PRIVATE KEY" PEM, 10-year validity); the only difference is the subject CommonName.
Types ¶
type AIProviderRouteStatusReconciler ¶
AIProviderRouteStatusReconciler writes the previously-unwritten AIProviderRoute Status.Parents: an Accepted condition per clrk EgressGateway parentRef (does the gateway exist?) and a ResolvedRefs condition (do all clrk Backend backendRefs resolve?). It is status-only — it never mutates spec or the data plane — so Backend/BackendSelector references become discoverable and a typo'd ref surfaces as ResolvedRefs=False instead of silently degrading to pass-through.
func (*AIProviderRouteStatusReconciler) SetupWithManager ¶
func (r *AIProviderRouteStatusReconciler) SetupWithManager(mgr ctrl.Manager) error
type CLRKConfigReconciler ¶
type CLRKConfigReconciler struct {
client.Client
Scheme *runtime.Scheme
// Phone is the api.apoxy.dev client; nil when --phone-home=false, which
// severs all outbound calls and reports Registered=False/PhoneHomeDisabled.
Phone *sentry.Client
// Namespace is the clrk system namespace where the CLRKConfig singleton and
// the token Secret live.
Namespace string
// SecretName names the token Secret written on registration.
SecretName string
// ClrkVersion is reported to api.apoxy.dev at registration.
ClrkVersion string
// Dropped, when set, returns the cumulative count of security reports the
// phone-home reporter has dropped under backpressure; it is stamped into
// status.notifications.reportsDropped each reconcile so the console health
// panel reflects real loss. Nil when no reporter is running (leader-only).
Dropped func() int64
}
CLRKConfigReconciler drives the notifications phone-home registration for the CLRKConfig singleton. On signup (spec.notifications.email set) it registers the deployment with api.apoxy.dev, persists the returned bearer token to a core/v1 Secret (via the cluster client -- the embedded apiserver serves no core/v1), and records status.notifications (deploymentID + registrationTokenSecretRef + Registered condition). The browser sees only the Secret REFERENCE, never the token.
It also exposes AuthState so the sentry Reporter/AdvisoryPoller can resolve the live token + gating flags without a second CLRKConfig/Secret client.
func (*CLRKConfigReconciler) AuthState ¶
AuthState resolves the current phone-home authorization by reading the CLRKConfig singleton and the token Secret. It is the AuthFunc for the sentry Reporter/AdvisoryPoller. A missing singleton or unset token yields an unauthorized (zero) state, not an error.
func (*CLRKConfigReconciler) SetupWithManager ¶
func (r *CLRKConfigReconciler) SetupWithManager(mgr ctrl.Manager) error
type CredentialInjectionPolicyStatusReconciler ¶
CredentialInjectionPolicyStatusReconciler writes the GEP-2649 PolicyStatus of a CredentialInjectionPolicy: one ancestor per targetRef with an Accepted condition reporting whether the (same-namespace) EgressGateway / AIProviderRoute / MCPRoute target exists. It is status-only — the worker resolves and applies the credential off the policy spec — so a typo'd or dangling targetRef surfaces as Accepted=False/TargetNotFound instead of silently never injecting.
func (*CredentialInjectionPolicyStatusReconciler) SetupWithManager ¶
func (r *CredentialInjectionPolicyStatusReconciler) SetupWithManager(mgr ctrl.Manager) error
type DaemonAgentRevisionReconciler ¶
type DaemonAgentRevisionReconciler struct {
client.Client
Scheme *runtime.Scheme
// Recorder emits rollout notifications (events.k8s.io/v1) on the
// RevisionReady False->True crossing. Nil-safe.
Recorder *notify.Recorder
}
DaemonAgentRevisionReconciler manages AgentSandboxRevision snapshots for a DaemonAgent. Phase / UpSince / RestartCount are written by the worker daemon loop — only the worker knows when its sandbox is alive.
func (*DaemonAgentRevisionReconciler) SetupWithManager ¶
func (r *DaemonAgentRevisionReconciler) SetupWithManager(mgr ctrl.Manager) error
type EgressGatewayOTLPForwardReconciler ¶
type EgressGatewayOTLPForwardReconciler struct {
client.Client
// Registry is the per-EG forwarder map the receiver reads from.
Registry *otelforward.Registry
}
EgressGatewayOTLPForwardReconciler watches EgressGateway objects and keeps internal/otelforward.Registry's per-EG forwarders in sync with each EG's Spec.OTLP. The cm OTLP receiver consults the registry per inbound request to decide whether to ship a copy of the captured signal to the customer's endpoint.
Cm always persists captured signals to the embedded ClickHouse via internal/chwriter regardless of forwarder state, so a missing or wedged forwarder never costs us the durable copy.
`clrk dev` doesn't go through this reconciler — the cm OTLP receiver is wired with an unconditional dev sink (see cmd/controller-manager/main.go, --dev-otlp-fallback-endpoint) that fans every inbound payload to the TUI receiver regardless of EG state. That way the TUI lights up even when no EgressGateway is applied.
func (*EgressGatewayOTLPForwardReconciler) Reconcile ¶
func (r *EgressGatewayOTLPForwardReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)
Reconcile applies the EG's Spec.OTLP to the registry: install or swap when the endpoint changes, remove when the EG is deleted or the endpoint is cleared. Spec-equality short-circuiting lives inside Registry.Apply.
func (*EgressGatewayOTLPForwardReconciler) SetupWithManager ¶
func (r *EgressGatewayOTLPForwardReconciler) SetupWithManager(mgr ctrl.Manager) error
SetupWithManager wires the reconciler onto EgressGateway events. No predicate: change detection happens inside Registry.Apply via the spec-DeepEqual short-circuit, so EG status churn is filtered without predicate boilerplate.
type EgressGatewayReconciler ¶
type EgressGatewayReconciler struct {
client.Client
Scheme *runtime.Scheme
// EnvoyImage is the container image used for the EG-managed Envoy
// fleet. Must contain the grpc_certificate_provider handshaker
// extension. Set from --envoy-image on controller-manager.
EnvoyImage string
// DevBackendHost, when non-empty, overrides
// Status.Listeners[*].BackendAddress to use this host with the EG
// Service NodePort instead of the in-cluster DNS name. Set by
// `clrk dev` to the docker hostname of the k3s container ("clrk-k3s")
// because workers run on the docker network and can't route to k3s
// ClusterIPs.
DevBackendHost string
// EnvoyGatewayNamespace is the namespace EG provisions per-EG
// Envoy Deployments + Services into. Equal to the controller-
// manager's runtime namespace (mirrors EG's
// ENVOY_GATEWAY_NAMESPACE env var). Used here to mirror
// upstream-CA Secrets and to list EG-managed Services for
// listener status. Required.
EnvoyGatewayNamespace string
// InsecureGRPC mirrors controller-manager's --insecure-grpc-no-mtls.
// When true (dev), the :9443 control plane is plaintext and identity is
// derived from :authority, so the per-EG client cert + CA mounts are
// skipped. When false (the secure default), the reconciler mints a
// per-EG client cert and the Envoy Deployment mounts it plus the shared
// control-plane CA.
InsecureGRPC bool
// ControlPlaneRootCert / ControlPlaneRootKey are the control-plane root
// CA the controller-manager minted at startup (see controlplane_pki.go).
// Held in memory so per-EG client certs mint without a cross-namespace
// read of the root private key. Empty when InsecureGRPC is true.
ControlPlaneRootCert []byte
ControlPlaneRootKey []byte
}
EgressGatewayReconciler provisions Envoy Gateway infrastructure for each clrk EgressGateway: a shared GatewayClass, an EnvoyProxy referencing our private Envoy image, a Gateway with one TCP listener, and the per-EG CA Secret consumed by both the sandbox trust store and the handshaker.
func (*EgressGatewayReconciler) SetupWithManager ¶
func (r *EgressGatewayReconciler) SetupWithManager(mgr ctrl.Manager) error
type FallbackRoutingPolicyStatusReconciler ¶
FallbackRoutingPolicyStatusReconciler writes the GEP-2649 PolicyStatus of a FallbackRoutingPolicy: one ancestor per targetRef with an Accepted condition reporting whether the (same-namespace) AIProviderRoute target exists. It is status-only — the egextension reads the policy off its spec when synthesizing the per-rule clusters — so a dangling targetRef surfaces as Accepted=False/TargetNotFound instead of silently never applying fallback.
func (*FallbackRoutingPolicyStatusReconciler) SetupWithManager ¶
func (r *FallbackRoutingPolicyStatusReconciler) SetupWithManager(mgr ctrl.Manager) error
type HTTPInvoker ¶
HTTPInvoker invokes a TaskAgent by POSTing directly to the WorkerPool's in-cluster dispatch Service. Reads the Service's EndpointSlices and dials a ready pod IP rather than the Service DNS — the controller-manager runs outside the cluster in `clrk dev` and so can't resolve `*.svc.cluster.local`, but the docker bridge / pod IPs in EndpointSlices are routable in both modes (production: shared pod CIDR; dev: shared docker network via the selectorless Service+EndpointSlice bridge that `clrk dev` materializes for the default WorkerPool).
func NewHTTPInvoker ¶
func NewHTTPInvoker(c client.Client) *HTTPInvoker
NewHTTPInvoker returns an HTTPInvoker with a default http.Client. The per-call deadline is supplied by the caller's context, so the client itself has no Timeout set.
func (*HTTPInvoker) Invoke ¶
func (h *HTTPInvoker) Invoke(ctx context.Context, ta *clrkv1alpha1.TaskAgent, body []byte) error
type InvocationActiveCounter ¶
type InvocationActiveCounter interface {
CountActive(ctx context.Context, namespace string, kind clrkv1alpha1.InvocationParentKind, name string) (int32, error)
}
InvocationActiveCounter reports the number of non-terminal invocations for a parent agent from the invocation read model. Injected by the controller-manager (backed by invocation.ActiveCounter). Defined here so the controller package needn't import the apiserver/ClickHouse stack; nil in tests / no-ClickHouse builds, where the reconciler falls back to the per-worker WorkerStatus sum.
type TaskAgentCronReconciler ¶
type TaskAgentCronReconciler struct {
client.Client
Scheme *runtime.Scheme
Invoker TaskAgentInvoker
// Now is overridable so tests can advance the clock without sleeping.
Now func() time.Time
// contains filtered or unexported fields
}
TaskAgentCronReconciler watches TaskAgents with a non-empty spec.schedule and fires them on a cron clock via Invoker. The fire loop runs as a separate Runnable that opts into leader election.
func (*TaskAgentCronReconciler) SetupWithManager ¶
func (r *TaskAgentCronReconciler) SetupWithManager(mgr ctrl.Manager) error
SetupWithManager registers the reconciler and the tick runnable on mgr.
func (*TaskAgentCronReconciler) Tick ¶
func (r *TaskAgentCronReconciler) Tick(ctx context.Context)
Tick walks all registered entries and fires those whose nextFire time has passed. Fires within a single tick run in parallel goroutines so one slow Invoker can't starve sibling agents; Tick returns once all in-flight fires complete (so the next ticker tick is paced to the slowest fire — good enough for minute-resolution cron).
Exported because tests drive it directly to avoid sleeping.
type TaskAgentIngressReconciler ¶
type TaskAgentIngressReconciler struct {
client.Client
Scheme *runtime.Scheme
// IngressExtProcHost / IngressExtProcPort point at the
// controller-manager's ingress ext_proc gRPC endpoint. Wired from
// `--ingress-extproc-host` / `--ingress-extproc-port`.
IngressExtProcHost string
IngressExtProcPort int32
}
TaskAgentIngressReconciler owns the k8s-side half of TaskAgent.
Per-TaskAgent it materializes:
- a Gateway (one HTTP listener) and HTTPRoute (per-TA path tree)
- an EG `Backend` of `type: DynamicResolver` — EG generates a dynamic_forward_proxy cluster that dials whatever `:authority` the request carries, so the HTTPRoute backendRef is this object instead of a static Service
- an `EnvoyExtensionPolicy` attaching the controller-manager's ingress ext_proc onto the per-TA HTTPRoute
Per TaskAgent's namespace it also materializes a shared `clrk-ingress-extproc` Backend pointing at the controller-manager's ingress ext_proc gRPC service (FQDN + port from IngressExtProc{Host,Port}). One Backend per namespace keeps the ExtensionPolicy backendRef short and re-targetable from a single place.
func (*TaskAgentIngressReconciler) SetupWithManager ¶
func (r *TaskAgentIngressReconciler) SetupWithManager(mgr ctrl.Manager) error
type TaskAgentInvoker ¶
type TaskAgentInvoker interface {
Invoke(ctx context.Context, ta *clrkv1alpha1.TaskAgent, body []byte) error
}
TaskAgentInvoker fires one execution of ta. Implementations route to the same path an HTTP-triggered TaskAgent would take. body is the verbatim scheduleInput (default empty JSON object when nil).
type TaskAgentRevisionReconciler ¶
type TaskAgentRevisionReconciler struct {
client.Client
Scheme *runtime.Scheme
// ActiveExec counts non-terminal invocations for the source of
// TaskAgent.Status.ActiveExecutions (APO-620). Nil falls back to the
// legacy per-worker sum off AgentSandboxRevision.Status.Workers — kept
// for the embedded-ClickHouse binary-layering rollout window, after
// which the fallback can be dropped.
ActiveExec InvocationActiveCounter
// Recorder emits rollout notifications (events.k8s.io/v1) on the
// RevisionReady False->True crossing. Nil-safe (tests / no-apiserver builds).
Recorder *notify.Recorder
}
TaskAgentRevisionReconciler manages AgentSandboxRevision snapshots for a TaskAgent. Gateway / HTTPRoute live in TaskAgentIngressReconciler so they can be skipped where gateway-api isn't installed.
func (*TaskAgentRevisionReconciler) SetupWithManager ¶
func (r *TaskAgentRevisionReconciler) SetupWithManager(mgr ctrl.Manager) error
type WorkerPoolDeploymentReconciler ¶
type WorkerPoolDeploymentReconciler struct {
client.Client
Scheme *runtime.Scheme
// CMOTLPEndpoint is the OTLP/HTTP URL of the controller-manager's
// receiver. The reconciler stamps it on every worker container as
// CLRK_CM_OTLP_ENDPOINT so the worker's OTLP emitter knows where
// to ship signals. Empty disables injection — the worker emitter
// falls back to noop.
CMOTLPEndpoint string
// CMNATSAddr is the controller-manager's NATS/JetStream client
// address (host:port). The reconciler stamps it on every worker
// container as CLRK_CM_NATS_ADDR so the dispatcher's invocation
// publisher can dial the cm's embedded JetStream over TCP. Empty
// disables injection — the worker simply doesn't publish lifecycle
// events.
CMNATSAddr string
// Recorder emits fleet-health notifications (events.k8s.io/v1) when a
// WorkerPool crosses the ready>=desired health boundary. Nil-safe.
Recorder *notify.Recorder
}
WorkerPoolDeploymentReconciler owns the k8s-side half of WorkerPool: it creates/updates the Deployment + Service that host worker pods and reports their health back as ReadyReplicas + Available/Progressing conditions.
func (*WorkerPoolDeploymentReconciler) SetupWithManager ¶
func (r *WorkerPoolDeploymentReconciler) SetupWithManager(mgr ctrl.Manager) error
type WorkerPoolStatusReconciler ¶
WorkerPoolStatusReconciler owns the portable half of WorkerPool: it aggregates execution capacity and per-agent active-execution counts, and writes the "Configured" condition. It never reads or writes Deployment / Service — that lives in WorkerPoolDeploymentReconciler. The `ReadyReplicas` and `Progressing` fields are left to that reconciler.
func (*WorkerPoolStatusReconciler) SetupWithManager ¶
func (r *WorkerPoolStatusReconciler) SetupWithManager(mgr ctrl.Manager) error
Source Files
¶
- aiproviderroute_status_controller.go
- clrkconfig_controller.go
- controlplane_pki.go
- credentialinjectionpolicy_status_controller.go
- daemonagent_revision_controller.go
- egressgateway_ca.go
- egressgateway_controller.go
- egressgateway_otlp_forward_reconciler.go
- fallbackroutingpolicy_status_controller.go
- labels.go
- policy_status.go
- retry_create_or_update.go
- revision_helpers.go
- taskagent_cron_controller.go
- taskagent_ingress_controller.go
- taskagent_invoker_http.go
- taskagent_revision_controller.go
- workerpool_deployment_controller.go
- workerpool_status_controller.go