feat(auth): support human and machine principals by EItanya · Pull Request #3358 · kagent-dev/kagent

EItanya · 2026-10-26T13:29:24Z

Scheduled execution needs an authenticated caller even when no user is present. This PR gives human, service-account, and Substrate Actor callers a common opaque Principal.ID, replacing the separate user and agent identity fields.

Authentication providers normalize identity and supply verified claims. Session ownership uses that ID, while credentials are supplied separately for authorized outbound requests. Only runtime callers carry a RuntimeBinding, which scopes access to their Session without inheriting its creator's permissions.

Kubernetes service-account authentication is opt-in, with Helm configuration and TokenReview RBAC. Identity uses the configured authority and verified UID, so rotation preserves identity and recreation changes it. Trusted-proxy IDs are scoped by issuer; the UI labels opaque values as creator IDs rather than names.

Successful TokenReviews are cached per controller by token hash and audience for up to 30 seconds, capped by token expiry, with at most 1,024 entries. Concurrent lookups share a review; failures are not cached and hits do not extend the TTL. Revocation can take up to 30 additional seconds to become visible. Outbound calls reuse the request credential and check expiry locally. Direct passthrough retains the caller's authority, so its destinations must be trusted with it. Scheduler projections are introduced with ScheduledRuns in #3360.

How caller identities become principals

flowchart LR
    human["Human caller"] -->|Trusted-proxy credential| auth["Authentication providers"]
    machine["Machine caller"] -->|Service-account token| auth
    actor["Substrate Actor"] -->|Actor JWT| auth
    auth -->|TokenReview on cache miss| kube["Kubernetes TokenReview"]
    kube -->|Verified identity| auth
    auth -->|Normalize identity and runtime binding| principal["Principal"]
    principal -->|ID and authenticated attributes| services["Control-plane authorization and services"]

Validation: focused authentication, runtime authorization, gateway, sandbox, and configuration race tests pass. Go lint, protobuf generation/lint/breaking checks, Helm tests, UI typecheck/lint, and conversation ownership tests pass. CI is running.

Stack 1/4. Targets main. Next: #3359.

Copilot

🔵 Needs a closer look

The cross-cutting authentication changes have unresolved credential-isolation and installation issues and require final human security review.

2 open issues

1 file not reviewed

go/api/gen/kagent/config/v1/config.pb.go: Generated file

What changed in this PR

Introduces a common identity model for human, service-account, and runtime callers, laying the authentication foundation for scheduled execution and the follow-up credential-passthrough work.

Changes:

Replaces user-specific identities with opaque principal IDs and explicit runtime Session bindings.
Adds Kubernetes TokenReview authentication, credential forwarding, projected tokens, and Helm RBAC.
Updates authorization consumers, tests, generated configuration, and architecture documentation.

File	Description
proto/kagent/config/v1/config.proto	Defines service-account authentication configuration.
helm/kagent/values.yaml	Adds authentication defaults.
helm/kagent/tests/invocation-auth_test.yaml	Tests token projection and TokenReview RBAC.
helm/kagent/templates/rbac/invocation-auth.yaml	Grants TokenReview permissions.
helm/kagent/templates/controller-deployment.yaml	Mounts the projected invocation token.
go/core/pkg/config/helm_test.go	Checks rendered authentication settings.
go/core/pkg/auth/credential.go	Adds principal IDs and credential forwarding.
go/core/pkg/auth/auth.go	Defines unified principals and runtime bindings.
go/core/pkg/auth/auth_test.go	Updates principal assertions.
go/core/pkg/app/app.go	Wires service-account authentication.
go/core/internal/service/tool/service_test.go	Updates tool-test identities.
go/core/internal/service/system/substrate_paging_test.go	Updates inventory-test identities.
go/core/internal/service/system/service.go	Uses principal IDs for identity fallback.
go/core/internal/service/system/service_test.go	Updates system-service identity tests.
go/core/internal/service/session/share.go	Checks runtime bindings for share restrictions.
go/core/internal/service/session/service.go	Separates caller identity from runtime data ownership.
go/core/internal/service/session/service_test.go	Updates Session authorization tests.
go/core/internal/service/scheduledrun/service.go	Uses principal IDs for schedule ownership.
go/core/internal/service/scheduledrun/service_test.go	Updates scheduled-run test identities.
go/core/internal/service/sandbox/service.go	Uses principal IDs for sandbox ownership.
go/core/internal/service/sandbox/service_test.go	Updates sandbox test identities.
go/core/internal/service/sandbox/guest.go	Adapts guest authentication to the new interface.
go/core/internal/service/sandbox/guest_test.go	Updates guest authentication fixtures.
go/core/internal/service/prompttemplate/service_test.go	Updates prompt-template test identities.
go/core/internal/service/model/service_test.go	Updates model-service test identities.
go/core/internal/service/model/discovery_test.go	Updates discovery-test identities.
go/core/internal/service/memory/service.go	Uses principal IDs for non-runtime memory access.
go/core/internal/service/memory/service_test.go	Updates memory identity and isolation assertions.
go/core/internal/service/kubecrud/service_test.go	Updates CRUD test identities.
go/core/internal/service/kubeauth/scope_test.go	Updates Kubernetes scope-test identities.
go/core/internal/service/inventoryauth/scope_test.go	Updates inventory scope-test identities.
go/core/internal/service/checkpoint/service.go	Uses principal IDs for checkpoint authorization.
go/core/internal/service/checkpoint/service_test.go	Updates checkpoint test identities.
go/core/internal/service/authscope/collection_test.go	Updates collection authorization fixtures.
go/core/internal/runtimeauth/provider.go	Separates runtime binding from upstream destination.
go/core/internal/runtimeauth/provider_test.go	Tests runtime identity and destination restrictions.
go/core/internal/runtimeauth/authorizer.go	Selects runtime policy through explicit bindings.
go/core/internal/runtimeauth/auth.go	Gives runtime callers independent principal IDs.
go/core/internal/mcp/tasks_test.go	Updates MCP task-test identities.
go/core/internal/httpserver/auth/serviceaccount.go	Implements verified service-account credential sources.
go/core/internal/httpserver/auth/serviceaccount_test.go	Tests rotation, audience isolation, and identity separation.
go/core/internal/httpserver/auth/proxy_authn.go	Namespaces proxy identities and checks token expiry.
go/core/internal/httpserver/auth/proxy_authn_test.go	Tests normalized identities and invalid credentials.
go/core/internal/httpserver/auth/authn.go	Adapts insecure authentication and bearer extraction.
go/core/internal/httpserver/auth/authn_test.go	Updates middleware identity assertions.
go/core/internal/grpcserver/task_store_test.go	Updates task-store test identities.
go/core/internal/grpcserver/session_test.go	Updates Session authorization policies.
go/core/internal/grpcserver/scheduledrun_controller_test.go	Updates scheduler authentication fixtures.
go/core/internal/grpcserver/policy_test.go	Updates runtime policy assertions.
go/core/internal/grpcserver/memory_test.go	Updates normalized memory partitions and JWT fixtures.
go/core/internal/grpcserver/interceptors.go	Requires bindings for runtime-only methods.
go/core/internal/grpcserver/interceptors_test.go	Tests the revised authentication contract.
go/core/internal/controller/scheduledrun/controller_test.go	Updates control-plane identity assertions.
go/core/internal/a2agateway/runtime.go	Passes destination Session IDs explicitly.
go/core/internal/a2agateway/runtime_test.go	Verifies forwarded destination IDs.
go/core/internal/a2agateway/gateway_test.go	Updates gateway authorization assertions.
go/core/cmd/controller/main_test.go	Updates proxy identity expectations and JWT fixtures.
go/api/gen/kagent/config/v1/config.pb.go	Generates service-account configuration bindings.
docs/architecture/scheduled-runs-and-caller-identity.md	Records identity decisions and planned scheduling work.
docs/architecture/README.md	Links the scheduling and identity decision.
docs/architecture/credential-injection.md	Distinguishes implemented behavior from planned work.
docs/architecture/a2a-gateway.md	Documents principal and runtime-binding semantics.

_{Review effort: Balanced}

EItanya · 2026-10-26T15:23:12Z

+  name: {{ include "kagent.fullname" . }}-invocation-auth
+  labels:
+    {{- include "kagent.controller.labels" . | nindent 4 }}
+rules:
+  - apiGroups: [authentication.k8s.io]
+    resources: [tokenreviews]
+    verbs: [create]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: {{ include "kagent.fullname" . }}-invocation-auth
+  labels:
+    {{- include "kagent.controller.labels" . | nindent 4 }}
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: {{ include "kagent.fullname" . }}-invocation-auth


Fixed in 4c6ad7798. The ClusterRole, binding, and roleRef now include the release namespace. Added chart tests for the same release name in two namespaces.

supreme-gg-gg

🤖 AI-generated review. I reviewed the findings before posting.

Two suggestions: update the OIDC authentication documentation and reuse the existing credential-session wrapper.

supreme-gg-gg · 2026-10-26T21:58:06Z

 	return &SimpleSession{
 		P: auth.Principal{
-			User:   auth.User{ID: userID},
+			ID:     auth.PrincipalID("trusted-proxy:"+issuer, userID),


Could you update the OIDC authentication guide alongside this change? It still describes selecting an acting user through X-Agent-Name, user_id, and X-User-Id, although this PR removes that path. It also says here and at the end of the guide that expired tokens are not rejected, while the updated provider checks expiry when exp is present. Operators following it could build integrations around behavior that no longer works. Please describe the issuer-scoped principal model and service-account TokenReview flow, while retaining the requirement for upstream signature verification.

supreme-gg-gg · 2026-10-26T21:58:06Z

+	return serviceAccountSession{principal: principal, source: source}, nil
+}
+
+type serviceAccountSession struct {


Optional simplification: this wrapper duplicates credentialSession in pkg/auth/credential.go, whose NewCredentialSession constructor currently has no callers. Both service-account authentication paths could return auth.NewCredentialSession(principal, source), nil, allowing this type and its methods to be deleted. Service-account verification would stay in the token source; this would just remove duplicate principal/credential plumbing.

EItanya

🤖 AI-generated review. Comment left by Claude on behalf of @EItanya.

Reviewed at f62eacc. Compatibility/regression concerns are intentionally out of scope (pre-1.0). Inline comments cover findings 1–3, 5 and 6; finding 4 is mostly outside the diff, so it's described here.

4. The opaque Principal.ID is used as a display value and as an external header contract.
For trusted-proxy users the ID is now base64(trusted-proxy:<iss>).base64(<sub>). That string is written to creator on sessions, sandboxes and scheduled runs.

UI: it renders creator verbatim, so users see strings like "Only dHJ1c3RlZC1wcm94eTo….YWxpY2VA… can open this conversation" (ui/src/pages/AgentPage.tsx:207,249, RenameConversationButton.tsx:40).
External consumers: ForwardCredential puts the same value in X-User-Id. The ADK forwards that header to MCP servers (go/adk/pkg/mcp/registry.go:333), which can no longer recognise the user.
Suggestion: keep ID opaque for ownership. Carry a separate display name (for example from email/preferred_username, or the service-account username), or stop sending X-User-Id to external servers.
Related: GetCurrentUser returns {username, groups} for service accounts and {sub: <opaque>} for claimless sessions, so its shape depends on the provider.

Checks: go test -race -count=2 on httpserver/auth, runtimeauth, and pkg/auth passes. Finding 1 is backed by the existing assertion at serviceaccount_test.go:59, not a new reproduction. E2E not run locally (CI is green).

EItanya · 2026-10-26T22:04:49Z

+}
+
+func (p *ServiceAccountProvider) UpstreamAuth(request *http.Request, session auth.Session, destinationSessionID string) error {
+	return p.users.UpstreamAuth(request, session, destinationSessionID)


1. The inbound service-account token is forwarded into agent runtimes and sandbox guests, contradicting the PR's own security model.

For service-account sessions this reaches ForwardCredential, and serviceAccountTokenSource.Token returns the caller's inbound token, whose audience is kagent-invocation. serviceaccount_test.go:59 asserts this.

Where the token lands: it is attached to every upstream A2A call (a2agateway/runtime.go:98) and to sandbox guest calls (sandbox/guest.go:65). Sandbox guests run arbitrary code.

Why it matters: any code there can replay the token against kagent as that machine principal, with all its permissions. With MCP token passthrough, it also reaches third-party MCP servers.

The contradiction: ProjectedSession exists to prevent exactly this. Its comment says "Neither the runtime nor downstream services receive the inbound token," and it rejects outbound credentials whose audience includes kagent-invocation. So a scheduled run sends a token scoped to an outbound audience, while a direct call from the same service account sends a kagent-scoped one. An MCP server that checks the outbound audience will accept scheduled runs and reject direct calls, or the audience split means nothing.

Pre-existing part: trusted-proxy human tokens were already forwarded. The new part is a token kagent itself accepts.

Smallest fix: for direct service-account sessions, forward the principal and no Authorization header (Token returns ""). Alternatively, require a separate outbound credential, as ProjectedSession does.

EItanya · 2026-10-26T22:04:49Z

+	if err != nil {
+		return "", err
+	}
+	principal, err := s.provider.verify(ctx, token, s.audience)


3. Two TokenReview calls per request, with no caching.

Authenticate runs a TokenReview, then this runs another on every upstream forward, and that happens on every A2A call made in the gateway. The inbound token is an immutable string that was just verified, and an API-server blip now fails every machine request.

If finding 1 is fixed, the second review goes away for direct callers. For the projected source, cache the result by token hash until exp or a short TTL.

EItanya · 2026-10-26T22:04:49Z

+		claims, _ := parseJWTPayload(token)
+		subject, _ := claims["sub"].(string)
+		_, kubernetes := claims["kubernetes.io"]
+		selected = selected || kubernetes || strings.HasPrefix(subject, "system:serviceaccount:")


6c. Any token carrying a kubernetes.io claim is sent to this cluster's TokenReview, so federated tokens from another cluster's workload identity can never take the trusted-proxy path. Worth a sentence in the docs if that's intended.

EItanya · 2026-10-26T22:04:49Z

  server:
    auth:
      mode: insecure
+      serviceAccount:


2. Machine authentication is on by default and partly dead in this PR.

On by default: every install, even with mode: insecure, gets a cluster-wide ClusterRole that can create tokenreviews, plus a mounted kagent-invocation projected token for the controller.

Dead here: nothing in this PR reads token_file, the mounted token, or ProjectedSession. The only consumer is feat(scheduling): invoke Agents from ScheduledRun CRDs #3360 (ProjectedSession(ctx, machine.GetTokenFile(), …)).

Smallest change: make serviceAccount opt-in here, and move ProjectedSession, token_file and the projected volume to #3360, where the scheduler actually uses them.

EItanya · 2026-10-26T22:04:49Z

 		return "", serviceerrors.NewPermissionDenied("Not authorized", err)
 	}
-	return principal.User.ID, nil
+	return principal.ID, nil


5. Deriving the data owner is left to each service.

Each service decides on its own whether a caller's data partition is principal.ID or RuntimeBinding.CreatorID. session/service.go:450-453 handles runtime callers, while this and scheduledrun/service.go:210 return principal.ID unconditionally.

That's safe today only because the runtime authorizer denies actors on these resource types. If someone later adds an actor grant, actor-owned checkpoints silently end up partitioned under the actor's ID. A method on Principal (say DataOwner()) or a shared helper would make that mistake hard to write (AGENTS.md §3).

EItanya · 2026-10-26T22:04:49Z

+// independent of the Kubernetes API token.
+message ServiceAccountAuthentication {
+  string authority = 1 [(buf.validate.field).string.min_len = 1];
+  string token_file = 3 [(buf.validate.field).string.min_len = 1];


6a. token_file = 3 skips field 2 in a brand-new message. Use 2, or add reserved 2;.

EItanya · 2026-10-26T22:04:49Z

          volumeMounts:
+            {{- with .Values.config.server.auth.serviceAccount }}
+            - name: invocation-token
+              mountPath: {{ dir .tokenFile | quote }}


6b. The template mounts the directory of tokenFile (dir .tokenFile), and validation only checks the field is non-empty. A value like /etc/kagent/token would shadow the controller config mount, and /token would mount at /. Constrain it, or use a fixed mount directory.

supreme-gg-gg

🤖 AI-generated review. I reviewed the finding before posting.

One reproduced issue in service-account claims serialization.

supreme-gg-gg · 2026-10-27T17:28:45Z

+		ID: auth.PrincipalID("kubernetes:"+p.authority, entry.user.UID),
+		Claims: map[string]any{
+			"username": entry.user.Username,
+			"groups":   slices.Clone(entry.user.Groups),


Could you normalize groups to a JSON-compatible []any here? slices.Clone(entry.user.Groups) produces a []string, which flows through system.Service.GetCurrentUser into structpb.NewStruct in grpcserver/system.go:40. That serializer does not accept []string, so a successfully authenticated service-account caller gets Internal from GetCurrentUser.

I reproduced this twice through the generated gRPC client using the real authentication interceptor, service, and protobuf encoding, with TokenReview mocked. Converting only this groups slice to []any in a scratch overlay made both runs pass. Please normalize the claims at the provider boundary and add a generated-client regression test for a service-account caller.

Fixed in cd63f4430. Groups are normalized to a fresh []any at the provider boundary. Added generated-client coverage for populated, empty, and nil groups on fresh and cached authentication, and retained the cache-isolation regression.

Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>

Downstream services need to identify the executing Actor independently of who invoked it. This adds `actorJwt` to ModelConfig and RemoteMCPServer, and to EgressPolicy's existing `authentication` object. Substrate mints, injects, and refreshes the token at the gateway. Managed clients use the existing SDK placeholders: models use their provider's credential header, and MCP uses `Authorization: Bearer`. Audiences are explicit, the lifetime defaults to 900 seconds, and `kagent-runtime` is reserved for internal access. Actor-authenticated MCP discovery runs inside the Actor, so those tools are absent from the global UI catalog. Bedrock, Gemini, and Ollama reject `actorJwt` because their credential paths target hosted services that require provider-issued credentials; this path does not exchange tokens. Existing Secret and token-passthrough options and grant lifetimes stay in place. Credential choices use field-presence unions, so omit inactive options rather than setting `tokenPassthrough: false` alongside another credential. Integration bindings remain pinned to revisions; EgressPolicy bindings remain live. This also lets the model editor preserve and edit Actor JWT settings. <details> <summary>How a downstream request receives the Actor identity</summary> ```mermaid flowchart LR config["ModelConfig, RemoteMCPServer, or EgressPolicy"] -->|Origin, header, audiences, lifetime| control["Kagent"] control -->|Credential binding| gateway["Substrate gateway"] runtime["Actor runtime"] -->|Request with placeholder| gateway gateway -->|Request with minted Actor JWT| service["Downstream service"] ``` </details> Validation: CRD admission tests, compiler and gateway tests, PostgreSQL persistence tests, Go race tests/lint, and model-editor tests/typecheck pass. Local gVisor E2E passes for model and MCP injection across all four harnesses, plus policy injection. CI is running. Stack 2/4. Depends on kagent-dev#3358. Next: kagent-dev#3362. Includes the Actor JWT work previously in kagent-dev#3374. Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>

EItanya requested a review from a team as a code owner October 26, 2026 13:29

github-actions Bot added the enhancement New feature or request label Oct 26, 2026

EItanya mentioned this pull request Oct 26, 2026

feat(auth): add gateway Actor JWT credentials #3359

Merged

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 26, 2026

EItanya force-pushed the feat/principals-and-machine-auth branch from 5fc32ef to 3fa3523 Compare October 26, 2026 13:34

EItanya added this pull request to stack #3363 October 26, 2026 13:37

EItanya requested a balanced review from Copilot October 26, 2026 13:51

EItanya force-pushed the feat/principals-and-machine-auth branch from 3fa3523 to cba2a8c Compare October 26, 2026 13:51

Copilot started reviewing on behalf of EItanya October 26, 2026 13:51

View session

EItanya force-pushed the feat/principals-and-machine-auth branch from cba2a8c to c7a0284 Compare October 26, 2026 13:57

Copilot AI reviewed Oct 26, 2026

View reviewed changes

EItanya force-pushed the feat/principals-and-machine-auth branch 3 times, most recently from 4c6ad77 to 66514f0 Compare October 26, 2026 15:43

This comment has been minimized.

Sign in to view

EItanya removed this pull request from stack #3363 October 26, 2026 16:04

EItanya added this pull request to stack #3375 October 26, 2026 16:07

EItanya force-pushed the feat/principals-and-machine-auth branch from 66514f0 to 4b61405 Compare October 26, 2026 17:11

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 26, 2026

EItanya removed this pull request from stack #3375 October 26, 2026 19:42

EItanya added this pull request to stack #3379 October 26, 2026 19:55

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 26, 2026

EItanya force-pushed the feat/principals-and-machine-auth branch from 4b61405 to f62eacc Compare October 26, 2026 20:00

supreme-gg-gg reviewed Oct 26, 2026

View reviewed changes

EItanya commented Oct 26, 2026

View reviewed changes

EItanya force-pushed the feat/principals-and-machine-auth branch from f62eacc to cc633bc Compare October 26, 2026 23:14

github-actions Bot removed the enhancement New feature or request label Oct 26, 2026

github-actions Bot added the enhancement New feature or request label Oct 26, 2026

EItanya force-pushed the feat/principals-and-machine-auth branch from cc633bc to febc121 Compare October 26, 2026 23:35

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 26, 2026

EItanya force-pushed the feat/principals-and-machine-auth branch from febc121 to 31f8a30 Compare October 27, 2026 14:46

supreme-gg-gg reviewed Oct 27, 2026

View reviewed changes

EItanya force-pushed the feat/principals-and-machine-auth branch from 31f8a30 to 871c32d Compare October 27, 2026 17:46

feat(auth): support human and machine principals

cd63f44

Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>

EItanya force-pushed the feat/principals-and-machine-auth branch from 871c32d to cd63f44 Compare October 27, 2026 17:52

supreme-gg-gg approved these changes Oct 27, 2026

View reviewed changes

EItanya added this pull request to the merge queue Oct 27, 2026

Merged via the queue into main with commit 261b58e Oct 27, 2026
30 checks passed

EItanya deleted the feat/principals-and-machine-auth branch October 27, 2026 18:29

This was referenced Oct 28, 2026

[FEATURE] Complete public cross-AgentInstance A2A delegation (K11) #2618

Open

将执行授权与运行时补偿上移原生层，并拆分站点发行版 pood1e/agentos-platform#191

Closed

BrewTestBot mentioned this pull request Nov 9, 2026

kagent 1.0.0 Homebrew/homebrew-core#324443

Merged

Conversation

EItanya commented Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

supreme-gg-gg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

EItanya left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

supreme-gg-gg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

EItanya commented Oct 26, 2026 •

edited

Loading