Repository navigation
Conversation
… Harness TestE2EClaudeTurnOverridesAndRuntimeUser sends three turns through the public A2A API against a recorded mock Anthropic API. A turn with kagent.dev/model and kagent.dev/effort uses them. The next turn, with neither key, uses the configured model and its default effort. An effort-only turn keeps the configured model. The first turn's Bash probe checks that the harness runs non-root and owns what it writes under Substrate's root-owned, mode 0777 durable directory. The claude-sdk-e2e Harness fixture sets KAGENT_CLAUDE_DRIVER=sdk. KAGENT_E2E_CLAUDE_HARNESS selects it, so every Claude case can run on the Agent SDK driver. Claude 5 family requests end with a system-role context message, which mockllm's last-message matching can't see past. The test strips that message before matching. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: James Olds <12104969+oldsj@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This adds E2E coverage for #69's per-turn override, the harness runtime user, and the Agent SDK driver.
TestE2EClaudeTurnOverridesAndRuntimeUsersends three turns through the public A2A API against a recorded mock Anthropic API, then checks what reached the model:kagent.dev/model=claude-opus-5-5andkagent.dev/effort=high, every agent-loop request carries that model and effort.claude-sonnet-5-5with its default effort. They never carry the previous turn's value.kagent.dev/effort=lowonly, requests keep the configured model and uselow.On the first turn, the harness's Bash tool also reports its uid and gid, and the owner and mode of the durable directories. It then writes a file to the workspace. The test requires a non-root user that owns what it writes. Substrate 1.0 creates
/dataroot-owned with mode 0777.claude-sdk-e2eHarness fixture (KAGENT_CLAUDE_DRIVER=sdk) inlifecycle.yaml.tmpl, plusKAGENT_E2E_CLAUDE_HARNESS. With that set toclaude-sdk-e2e, every Claude case runs on the SDK driver. The default staysclaude-e2e, so CI is unchanged. The upgrade test's fixture filter ignores the new Harness.With Claude 5 family models, the CLI ends each request with a
system-role context message (token budget and date). mockllm matches only the last message, and only on role. The new test's proxy therefore strips trailing system messages before matching, while the recorder keeps the original request. The other Claude E2E cases useclaude-sonnet-4-5and aren't affected.Testing
I ran this on kind-substrate-1-0 against the ported controller 0.0.0-7945f19 and harness image 0.0.0-c3cb83a-arm64, with no real model turns. It passes on both drivers:
Probe output was identical on both drivers:
go vetpasses, and lint reports 0 issues forcore/test/e2e.x-user-idauth.🤖 Generated with Claude Code
🤖 Generated with Claude Code