Repository navigation
Description
Follow-up to #310. The #310 PR moves the hole-punch duplicate check under
the lifecycle write lock: two candidate winners can no longer both pass
the check and orphan each other. That PR deliberately leaves the
accepted-coordination insert as it is on master. It replaces the
winner-map slot under the lifecycle lock and closes nothing. This issue
tracks retiring the displaced connection safely.
1. The coordinator insert orphans a displaced untracked winner (#310,
bullet 2). establish_connection_to_validated_candidate_inner inserts
the dialled connection C into connections without lifecycle
registration. A hole-punch winner H that it displaces is not in the
lifecycle map either. H's detached close and observed-address watchers
keep it open until the peer closes it, the idle timeout fires or the
endpoint shuts down.
2. Closing H at the insert is wrong (Codex review of #310, P2-1). H can
be the connection the application reads from:
await_hole_punch_outcomeobtains H, starts its reader and returns
success (src/p2p_endpoint.rs~5600/5613/5630).- When the in-flight coordination dial then completes as C, C gets only
observed-address and close watchers. It is not queued for
accept_connection, and the event callback reports only progress
(p2p_endpoint.rs~2944/3120). - H's reader exit takes
ReaderExitOutcome::Noop(nat_traversal_api.rs
handle_reader_exit;p2p_endpoint.rs~10496). The outer peer state
stays connected.
So closing H kills live traffic and unfinished streams, and no reader
takes over. "Untracked" does not mean "unused". The #310 PR has a
regression guard for this:
coordinator_dial_leaves_a_displaced_untracked_winner_open_and_usable.
3. A locked custody check is not enough (Codex P2-2). Interleaving:
- A caller keeps H from
get_connection. - The coordinator replaces H with C, sees that H is untracked, and
releases the lock. - Before the close, another thread calls
add_connection(peer, H). This
records H as Live and puts it back in the winner map. - The stale decision then closes a tracked winner. C can also become an
untracked orphan through the registration insert.
Checking custody again outside the lock still races. Retirement needs an
atomic claim that registration respects, or a lifecycle transition that
stops resurrection between the decision and the close.
4. The two ends can disagree (Codex answer 2). Each end's choice of
untracked winner is local:
- hole-punch: the first winner keeps the slot;
- coordinator: the last insert takes the slot.
Neither is coordinated with the remote end's lifecycle tiebreak
(candidate_wins, register_connection_lifecycle_parts). For example,
local hole-punch first-wins can reject a second connection that the
remote's same-initiator registration chooses as its newer generation.
This predates #310. Rejecting every coordinator newcomer does not fix it
either.
5. The same orphan class exists at two more sites. The
lifecycle-registration insert (register_connection_lifecycle_parts, ~7473
on the #310 base) and the repromotion insert
(repromote_surviving_connection, ~7984 on the #310 base) can displace a
different untracked value without closing it. Any fix must keep two cases
distinct:
- tracked Superseded generations, kept open for stream draining;
- reinsertion of the same handle (same
stable_id).
Possible direction (not decided):
- Route hole-punch and accepted-coordination winners through lifecycle
registration. They then get generations, the same-initiator tiebreak and
the Superseded drain window. Alternatively, add an atomic retirement claim
that registration respects. - Hand the reader over to the replacement before the old winner is retired,
for example throughpending_acceptsor theconnection_promoted_tx
promotion signal. - Keep a drain window for the old winner's streams.
Acceptance tests:
- Finalize H at the P2P layer, with a
P2pEndpointreader running, before
releasing the coordinator dial. Check ongoing and later traffic and the
outerconnected_peersstate (P2-1). - Add a gate between the retirement decision and the close, and run a
concurrentadd_connection(H)there. Neither a tracked winner closed nor
an orphaned C is allowed (P2-2). - Same-peer, two-ended regression: both ends run the accepted coordination
and the hole-punch race, and both end on the same live connection
(answer 2). - An untracked value displaced at the registration or repromotion insert
is either closed or tracked, never orphaned. Superseded drain survivors
and same-handle reinsertion are left alone.
References:
- ant-quic nat_traversal: concurrent hole-punch winners can orphan a live connection (check-then-insert outside the lifecycle lock) #310 and its PR;
- Codex review of nat_traversal: concurrent hole-punch winners can orphan a live connection (check-then-insert outside the lifecycle lock) #310 round 1: P2-1, P2-2, answers 2, 4 and 5;
- Codex review of shutdown: rare residual socket holder after 0.27.55 (1/2500 in x0x workload) #309 round 1: P2-3.
Activity
dirvine commentedon Oct 9, 2026
#315 now depends on this issue. The parked fix makes an inbound winner actively close the superseded generation. Codex found that in the coordinator race this lets the remote end close the connection that carries A's only application reader, because A's coordinator path starts no reader for C. So the reader handover here is a prerequisite for #315.
Merge pull request #318 from saorsa-labs/fix/313-raw-winner-reader
dirvine commentedon Oct 9, 2026
Part 1 merged in #318 (master da719d0). Raw coordinator and hole-punch winners, plus materialize, now get a reader. That fixes the B→A non-delivery Codex confirmed in the coordinator race. Each connection also gets exactly one reader owner, kept until the reader has stopped. It adds no new close and makes no routing, lifecycle, wire or public-API change. Part 2 is still open and parked as an ADR-level decision: retiring displaced or superseded connections, and arbitration between the two ends. Plan and reviews: x0x
.planning/team-2026-10-05/s-313-plan.md, and thes-313-plan-review-codex.mdands-aq313-p1-review-codex.mdreports. Release: 0.27.57 is being prepared (D234).test(#313): allow the injected reader-task panic in the cfg(test) hook
Merge pull request #319 from saorsa-labs/backport/0.27.57
dirvine commentedon Oct 9, 2026
Part 1 (#318) is released in ant-quic 0.27.57 (crates.io, 2026-10-09). Part 2, retirement and two-ended arbitration, is still parked.
Merge pull request #320 from saorsa-labs/chore/forward-port-0.27.57
test(ci): match the #313 tuple return in the registration fixture's a…
test: gate three lifecycle tests on network-discovery
Merge pull request #326 from saorsa-labs/forward/0.27.58
test(p2p): race two reader starts for one connection (#284) (#343)
fix(nat): don't signal a #277 promotion for a winner already in place (…