Repository navigation
MARSOHS-2084: --region on harness-runtime selects the regional API host - #2065
Conversation
A MARS resource lives in one region and is reachable only on that region's API host, so addressing the wrong one 404s. The region is carried by the hostname (api-<region>.digitalocean.com), and the region-less host is served by ric1. --region is persistent on the group, so every subcommand accepts it either side of the subcommand name. It is applied as a godo client option, which GetGodoClient installs before the --api-url override, so a host the user chose explicitly still wins and doctl only ever rewrites a default. Omitting the flag changes nothing: the default region needs no rewrite. The port-forward tunnel builds its own URL rather than going through the godo client, so it makes the same host choice; without it a tunnel would keep dialing the default region and 404 on a session held elsewhere. list says which region it is showing, since it never merges regions. doctl keeps no list of known regions, which would go stale as regions open; harness-api validates against its own registry and this checks only the shape of what is interpolated into a hostname. The create body region, following the create response's Location, and reading a region prefix off an id are not here; agents_region.go records what each is waiting on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
godo v1.233.0 (on main) carries the regional pieces doctl was waiting for, so the three parts left out of the first commit are no longer blocked. Region on a create-from-config request: a placement check, not a routing instruction. The host already decided the region, and the server rejects a body region that disagrees rather than re-placing anything. Location after a create: both create paths converge on createAgentSession, so the host that answered becomes the host every follow-up in that invocation uses. The server sends Location only for teams on the regional routing rollout, and an explicit --api-url is left alone, so this is inert in both cases. A Location that cannot be parsed is ignored rather than failing a session that was created successfully. Host selection now comes from godo.HostedAgentsRegionalBaseURL rather than a local format string. This changes --region ric1: it addresses api-ric1 instead of falling back to the region-less host, so an explicit region always names its own host. Omitting the flag is unchanged. The region is no longer carried in a request header. cns-lbaas asked for the forwarded Host instead, since it already selects the virtual host and cannot be forged past the edge, so harness-api reads it directly; comments updated to match, including that a misdirected regional host is a 421. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…gion-flag # Conflicts: # commands/agents.go
DO-rrao
left a comment
There was a problem hiding this comment.
Reviewing only the change that reaches outside harness-runtime, the client setup in NewCmdConfig. Details inline: a region set in config.yaml reroutes every doctl command, not just this group.
| // API host (agents_region.go). No other group registers the flag, | ||
| // so this is nil everywhere else, and GetGodoClient applies it | ||
| // before --api-url so an explicit host still wins. | ||
| regionOpts, err := agentRegionClientOpts() |
There was a problem hiding this comment.
This runs for every doctl command, and agentRegionClientOpts() reads the process-global viper key harness-runtime.region without checking which command is running.
The comment (and the one on agentRegionClientOpts) says the option is nil for every other command because only the harness-runtime group registers --region. That holds for the flag, but viper also loads the key from config.yaml, where users already put per-command defaults. With a config containing only:
harness-runtime:
region: mem1an unrelated command goes to the regional host:
$ doctl --config cfg.yaml compute droplet list --trace
[DEBUG] GET https://api-mem1.digitalocean.com/v2/droplets?page=1&per_page=200
The same key with a bad value (say MEM1) would make every doctl command fail with invalid region.
Suggest applying the option only when the command belongs to the group, for example:
var regionOpts []godo.ClientOpt
if c.NS == agentCmdName || strings.HasPrefix(c.NS, agentCmdName+".") {
regionOpts, err = agentRegionClientOpts()
if err != nil {
return err
}
}Plus a test that sets the key and checks that a non-agent namespace gets no option. Once it's scoped, a harness-runtime.region default in config.yaml becomes a nice feature for this group rather than a global side effect.
There was a problem hiding this comment.
You are correct, and the fix is in 598b6e33.
One note on the suggested check: a namespace test does not match the agent commands. agentsNS sets overrideCmdNS("agents"), thus create has the namespace agents.create, and each nested group sets its own (agents.template, agents.config, agents.files). A test for harness-runtime matches no command, and regional routing then stops fully.
The fix reads the command tree instead. agentRegionFlag() gives "" when the command that runs is not harness-runtime or one of its children, at any depth. The gate is in that function, thus all the readers obey one rule: the client options, the port-forward WebSocket URL and the region that doctl prints. A command outside the group also does not get a region error.
Test with a build, with harness-runtime: {region: mem1} in config.yaml:
compute droplet list→api.digitalocean.comharness-runtime list→api-mem1.digitalocean.comharness-runtime template list→api-mem1.digitalocean.com
With region: MEM1, only the harness-runtime commands fail.
TestAgentRegionIsScopedToTheGroup covers the four cases: a command outside the group, a bad value outside the group, no command, and a nested agent subcommand.
There was a problem hiding this comment.
Thanks. 598b6e3 is the right fix, and you were right about the namespace: the c.NS check in my snippet was wrong, because agent commands use agents.<name>.
But fe0559ec adds that check anyway, on top of 598b6e3, and it turns regional routing off for the API client. isAgentNS(c.NS) is never true for a real agent command, so agentRegionClientOpts() is never called. I built fe0559ec and traced each case:
| Command | Host reached | Expected |
|---|---|---|
harness-runtime --region mem1 list |
api.digitalocean.com |
api-mem1 |
harness-runtime --region mem1 template list |
api.digitalocean.com |
api-mem1 |
harness-runtime list, config region: mem1 |
api.digitalocean.com |
api-mem1 |
compute droplet list, config region: mem1 |
api.digitalocean.com |
api.digitalocean.com |
harness-runtime list, config region: MEM1 |
api.digitalocean.com, no error |
invalid region |
It also leaves the code paths disagreeing. port-forward goes through agentRegionFlag() directly, so it still honours the region, while create, list and get don't. port-forward --region mem1 would dial api-mem1 for a session that was created on the default host.
TestIsAgentNS passes namespaces like harness-runtime.list, which no real command has, so the tests stay green.
Suggested fix:
- Drop the
isAgentNSgate andTestIsAgentNS. The command-tree check inagentRegionFlag()already covers this, and it covers every reader. - Add a test that sets
activeCommandto a real agent subcommand (for exampletemplate list), builds the client the wayNewCmdConfigdoes, and checks the base URL ishttps://api-mem1.digitalocean.com/.
There was a problem hiding this comment.
You are correct. fe0559ec is reverted in 869330af.
That commit was a mistake. It applied the first suggestion again, on top of 598b6e3, which had already solved the same problem a different way. isAgentNS(c.NS) never matches a real command, so the client lost the region.
TestIsAgentNS did not catch this. The test used namespaces such as harness-runtime.list. No command has that namespace. The test made the same wrong assumption as the code, thus it stayed green. Both are removed.
The command-tree check in agentRegionFlag() is now the only gate again. It covers the client options, the port-forward URL and the printed region.
The test you asked for is already in 598b6e3: TestAgentRegionIsScopedToTheGroup/a nested agent subcommand still gets the option. It sets activeCommand to template list, builds the client from agentRegionClientOpts() and asserts the base URL is https://api-mem1.digitalocean.com/. Tell me if you want it to call NewCmdConfig directly instead.
Thank you for the trace table. It showed the fault clearly.
The region key is also settable in config.yaml, where viper serves it to every command, and the godo client is built for every command. A harness-runtime.region entry therefore rerouted all of doctl, and a bad value failed every command with an error no flag could clear. Membership is read from the command tree rather than the namespace: an agent command's namespace is agents.<name>, and a nested group such as template list carries one of its own.
…into MARSOHS-2084-doctl-region-flag
initServices runs for every command, and agentRegionClientOpts reads the
process-global viper key harness-runtime.region. Only this group
registers --region, but viper also serves the key from config.yaml, so
harness-runtime:
region: mem1
sent every other command to the regional host, and a bad value failed
all of them with "invalid region".
The namespace now decides. isAgentNS matches the group and its
subcommands, and only then is the option read, so a harness-runtime.region
default in config.yaml is a convenience for this group rather than a
global side effect.
Reported in review by @DO-rrao.
…oup" This reverts commit fe0559e.
MARSOHS-2084
A MARS session lives in one datacenter region and is only reachable on that
region's API host. Ask another region about it and you get a 404 — nothing is
redirected or forwarded. The hostname is the whole signal: harness-api reads the
forwarded
Hostitself, and no header carries the region.This adds
--regiontodoctl harness-runtimeso a user can say which regionto talk to.
How it works
--region mem1points the client atapi-mem1.digitalocean.com, viagodo.HostedAgentsRegionalBaseURL. The flag lives on the command group, soevery subcommand accepts it on either side of the subcommand name.
sent, and the server places in
ric1as it does today.--region ric1included.--api-url/DIGITALOCEAN_API_URLstill win. The region is applied as aclient option, which
GetGodoClientinstalls before the override, so a hostthe user chose explicitly is never rewritten.
port-forwardbuilds its own URL rather than going through the client, soit makes the same host choice. Without that, a tunnel would keep dialing the
default region and 404 on a session held elsewhere.
On a create from a config, the region also travels in the body. That is a
placement check rather than a routing instruction — the host already decided,
and a body region that disagrees is rejected rather than re-placed.
After a create, the host named by the response's
Locationbecomes the host thewait, prompt and attach use, so they land in the region that actually holds the
session. The server sends
Locationonly for teams on the regional routingrollout, and an explicit
--api-urlis left alone, so this is inert in bothcases.
doctl keeps no list of known regions — that would go stale the day a region
opens. harness-api validates against its own registry; this only checks the
shape of what gets put into a hostname.
Not here
Reading a region prefix off a session id (
mem1-sess_abc123). Whether ids carryone is still an open question in the design, so follow-up commands take
--region, or the host thatLocationnamed.Testing
commands/agents_region_test.gocovers host selection, the--api-urlprecedence, the
Locationhost, the tunnel URL, slug rejection, and that theflag is inherited by every subcommand. Verified by hand with
--tracethat eachof
--region ric1,--region mem1and no flag reachesapi-ric1,api-mem1and
api.digitalocean.comrespectively, and that a configured--api-urlstillwins.
🤖 Generated with Claude Code