Repository navigation
[lts-active] turbo-tasks snapshot consistency fixes (#99644, #99732, #99649) - #100002
Merged
lukesandberg merged 4 commits intoOct 14, 2026
Merged
lukesandberg merged 4 commits into
lukesandberg merged 4 commits into
Conversation
#99644) ### What? When a task that is already part of an in-progress persistence snapshot is modified again, `track_modification_internal` now bincode-encodes the task's pre-mutation state into the `SnapshotItem` that persistence will write, and stores that in the `snapshots` map. Before, it stored a `clone_snapshot()` copy of the `TaskStorage`. ### Why? The copy-on-write path exists so persistence sees a consistent snapshot even if a task is mutated while the snapshot is being written. Cloning `TaskStorage` doesn't guarantee that: cell contents are held behind `SharedReference` (`Arc`), so the clone shares them with the live task. If a cell value has interior mutability, a later mutation shows up in the "frozen" copy and the persisted state is no longer consistent. Encoding the state right away captures plain bytes that later mutations can't reach. The snapshot iterator hands the stored item straight to persistence, so nothing is encoded twice. The only cost is holding the encoded buffers in memory until the racing persistence picks them up, and this race is rare. <!-- NEXT_JS_LLM --> <!-- fleet 98a896e8-00fa-4f81-97f0-48bd49e2a9a2 --> --------- Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com> Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com> (cherry picked from commit a572b53)
### What? A failed persist (the snapshot cycle returning an error or panicking) disables persisting for the rest of the session. Later persistence cycles, including the one at shutdown, are skipped: `snapshot_and_persist` returns `Ok(None)` without starting a snapshot or GC. ### Why? A failed persist leaves the task graph and the database out of sync. For example, `save_snapshot` may already have consumed `persisted_task_cache_log` entries. Persisting again after that could write inconsistent state. Split out of #99649 as requested in review. ### How? `snapshot_in_progress` (the mutex that serializes persistence cycles) now holds a `LastPersistenceStatus { Ok, Failed }`. The backend starts at `Ok`. The status is set to `Failed` before a cycle starts and reset to `Ok` only when the cycle succeeds, so a panic also leaves it `Failed` (`parking_lot` mutexes don't poison). `Failed` is sticky for the rest of the session. Treating a disabled persist as a skipped cycle means `stop` needs no separate check. The test helper skips eviction on `None`, as it does on an error. What the existing design does with tasks the failed snapshot didn't yield is unchanged. A unit test checks that a failed persist through the noop backing storage sets the status to `Failed`, and that the next persist returns `None` and leaves the task's modification for later. The panic path has no test, since that would need a backend test hook; it was checked by reading the control flow. <!-- NEXT_JS_LLM --> <!-- fleet 98a896e8-00fa-4f81-97f0-48bd49e2a9a2 --> --------- Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com> Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com> (cherry picked from commit 0c729c5)
…#99649) ### What? During a snapshot, a task's first modification now copies the task's pre-mutation state whenever any of its categories is part of the snapshot, whichever category the modification touches. A `None` marker is inserted only for tasks with no modified category. ### Why? `track_modification` used to decide based on the touched category only. That let a snapshot persist inconsistent state. Take a task with dirty meta and clean data: 1. The snapshot begins. 2. Data is modified. Data wasn't part of the snapshot, so only a `None` marker is inserted. 3. Meta is modified. A marker is already present, so nothing is copied. 4. Persistence encodes the live meta, already mutated by step 3. ### How? The design is otherwise unchanged: `*_modified_during_snapshot` bits, `None` markers and the promotion in `end_snapshot` all stay, and nothing new happens inside the operation exclusion. Only the first-modification check in `track_modification_internal` changes, from "is the touched category modified" to "is any category modified". The copy still encodes with the live modified bits, so categories that weren't part of the snapshot aren't persisted. A regression test covers the sequence above. It checks that the persisted item carries the pre-snapshot meta and no data, and that both modifications are left for the next snapshot. <!-- NEXT_JS_LLM --> <!-- fleet 98a896e8-00fa-4f81-97f0-48bd49e2a9a2 --> Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com> Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com> (cherry picked from commit a7d6871)
Contributor
Tests PassedCommit: 99e0ce0 |
lukesandberg
marked this pull request as ready for review
October 12, 2026 00:20
sokra
approved these changes
Oct 12, 2026
Contributor
Author
|
waiting until after the security release |
…-releases/lts/active
lukesandberg
enabled auto-merge (squash)
October 14, 2026 20:01
lukesandberg
deleted the
backport-99644-99732-99649-to-releases/lts/active
branch
October 14, 2026 21:27
lukesandberg
added a commit
that referenced
this pull request
Oct 14, 2026
…ependency edges (#100209) Backports [[turbopack] Fix two GC paths that leave half-torn-down dependency edges (#99902)](#99902) to `releases/lts/active`. Builds on #100002, which brought over the snapshot rework (#99644, #99649) that this fix depends on. ## Conflict resolution `storage.rs` conflicted because #99747 (capture snapshot pending bits under exclusion) is not on this branch. The fix itself applies unchanged. The resolution only adjusts the surrounding code: - The snapshot scan takes #99902's loop as-is. The shard guard is already a write lock, so switching from `iter()` to `iter_mut()` is safe. - Added only the `is_deleted_new_task` helper. `debug_assert_persistable` does not exist on this branch. - Ported the three new storage tests to the `storage.take_snapshot(..)` method form. Dropped one `any_snapshot_pending()` assert, because that helper only exists after #99747. <!-- NEXT_JS_LLM -->
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backports three snapshot-consistency fixes to
releases/lts/active, in canary order:16.4.0 can persist inconsistent task state when a mutation races a background snapshot. The main case is #99649. A task that is meta-dirty and data-clean when the snapshot starts, then writes data before meta, has its live (post-mutation) meta persisted. One example is an invalidated task that re-runs while an idle snapshot is writing.
childrenandparent_countare both meta, so the cache on disk can list a child edge without the child's matchingparent_count.next devdoesn't persist on shutdown, so the next session restores that state. When the parent later drops the edge, the server aborts withparent_count underflow: decremented below the number of persistent parents. We have one report of this from a 16.4.0 dev run.The cherry-picks apply without conflicts. The six touched files are identical to
canaryat #99649.Verification
cargo test -p turbo-tasks-backend --libon this branch: 119 passed. This includes turbo-tasks: copy tasks on their first modification during a snapshot #99649's regression testmodify_other_category_first_during_snapshot_preserves_snapshot_metaand turbo-tasks: disable persisting after a failed snapshot #99732'sfailed_persist_disables_persisting.CARGO_TARGET_AARCH64_APPLE_DARWIN_LINKER=ccandRUSTFLAGSset to the repo's shared flags without--icf=all. The branch's pinnednightly-2026-08-20ships arust-lldthat can't read the current macOS SDK's.tbdfiles. This is unrelated to the change, and CI uses its normal config.