Repository navigation
Don't panic when a waiter gets a nil chunk from a failed request - #409
Merged
Merged
Conversation
A store that fails returns a nil chunk alongside its error. DedupQueue publishes that to everyone waiting on the same in-flight request, where it arrives as a *Chunk held in an interface. That is not an untyped nil, so the `case nil` arm never matched it: the *Chunk arm did, and cloning it dereferenced nil. The first requester already guarded against this, the waiters did not. Both waiter paths now share one helper, so they can't drift apart again, and the unreachable `case nil` that made the switch look safe is gone. Reachable through `untar -c <cache>` on Windows, where local stores are wrapped in a WriteDedupQueue, and through chunk-server on any platform, where an upstream failure took down the server process. Fixes #408.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #408.
A store that fails returns a nil chunk alongside its error.
DedupQueue.GetChunkpublishes that result to every caller waiting on the same in-flight request, where it arrives as a*Chunkheld in anany. A typed nil in an interface is not an untyped nil, so thecase nilarm never matched it —case *Chunkdid, with a nilb, andclone()dereferenced it:The first requester already guarded this a few lines below; only the waiters were missing it.
WriteDedupQueue.GetChunkcarried a second copy of the same switch, so both waiter paths now share one helper and can't drift apart again. Thecase nilarm went with it:markDoneis only ever handed a*Chunk, so it was unreachable, and its presence is what made the switch look like it covered the nil case.Two ways to reach it:
untar -c <cache>on Windows, wherecmd/desync/store.gowraps local stores in aWriteDedupQueue. Every cache miss on a fresh cache returns nil plusChunkMissing, so any two goroutines asking for the same chunk at once can hit it.chunk-serveron any platform, wherecmd/desync/chunkserver.gowraps the store unconditionally. There an upstream failure took down the server process.Present since
clone()was introduced in #372, so v1.0.4, v1.1.0 and v1.1.1 are all affected.The regression test panics on master and passes here. It uses
synctestlikeTestDedupQueueParallel, so the waiters are guaranteed to have piled onto the request before it completes rather than relying on a sleep.