Skip to content

Don't panic when a waiter gets a nil chunk from a failed request - #409

Merged
folbricht merged 1 commit into
masterfrom
dedupqueue-typed-nil
Sep 2, 2026
Merged

folbricht merged 1 commit into
masterfrom
dedupqueue-typed-nil

Conversation

@folbricht

Copy link
Copy Markdown
Owner

Fixes #408.

A store that fails returns a nil chunk alongside its error. DedupQueue.GetChunk publishes that result to every caller waiting on the same in-flight request, where it arrives as a *Chunk held in an any. A typed nil in an interface is not an untyped nil, so the case nil arm never matched it — case *Chunk did, with a nil b, and clone() dereferenced it:

github.com/folbricht/desync.(*Chunk).clone(...)
	chunk.go:32
github.com/folbricht/desync.(*DedupQueue).GetChunk(...)
	dedupqueue.go:42

The first requester already guarded this a few lines below; only the waiters were missing it. WriteDedupQueue.GetChunk carried a second copy of the same switch, so both waiter paths now share one helper and can't drift apart again. The case nil arm went with it: markDone is only ever handed a *Chunk, so it was unreachable, and its presence is what made the switch look like it covered the nil case.

Two ways to reach it:

  • untar -c <cache> on Windows, where cmd/desync/store.go wraps local stores in a WriteDedupQueue. Every cache miss on a fresh cache returns nil plus ChunkMissing, so any two goroutines asking for the same chunk at once can hit it.
  • chunk-server on any platform, where cmd/desync/chunkserver.go wraps the store unconditionally. There an upstream failure took down the server process.

Present since clone() was introduced in #372, so v1.0.4, v1.1.0 and v1.1.1 are all affected.

The regression test panics on master and passes here. It uses synctest like TestDedupQueueParallel, so the waiters are guaranteed to have piled onto the request before it completes rather than relying on a sleep.

A store that fails returns a nil chunk alongside its error. DedupQueue
publishes that to everyone waiting on the same in-flight request, where
it arrives as a *Chunk held in an interface. That is not an untyped nil,
so the `case nil` arm never matched it: the *Chunk arm did, and cloning
it dereferenced nil.

The first requester already guarded against this, the waiters did not.
Both waiter paths now share one helper, so they can't drift apart again,
and the unreachable `case nil` that made the switch look safe is gone.

Reachable through `untar -c <cache>` on Windows, where local stores are
wrapped in a WriteDedupQueue, and through chunk-server on any platform,
where an upstream failure took down the server process. Fixes #408.
@folbricht
folbricht merged commit 458fc17 into master Sep 2, 2026
17 of 18 checks passed
@folbricht
folbricht deleted the dedupqueue-typed-nil branch September 2, 2026 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DedupQueue.GetChunk panics on typed-nil *Chunk from store (Windows, -c cache path)

1 participant

⌂ 换网址