Repository navigation
verify: fix bandwidth asymmetry and memory leak with norandommap - #2107
Merged
Merged
Conversation
Collaborator
|
This seems like two distinct issues. Please split the PR into two patches, one for each issue. |
added 2 commits
June 10, 2026 09:25
When norandommap is set, io_pieces are stored in an RB tree sorted by offset. get_next_verify() only checked rb_first() for a completed entry. If the lowest-offset IO was still in-flight, it returned "nothing" even though completed entries existed at higher offsets. This starved verification reads during verify_backlog processing, causing severe read/write bandwidth asymmetry (e.g. 40 MiB/s read vs 700 MiB/s write) and unbounded io_piece accumulation in the tree. Fix this by walking the tree with rb_next() to find the first completed entry instead of giving up at rb_first(). This does not affect the linked-list path used when norandommap is not set, since list ordering matches submission order. Signed-off-by: Yehor Malikov <Yehor.Malikov@solidigm.com>
When an in-flight io_piece is evicted from the RB tree due to an overlapping write in log_io_piece(), the entry cannot be freed immediately. However, the IP_F_ONRB flag was not cleared on eviction. On IO completion, the entry still appeared to belong to the tree, so it was never freed. Clear IP_F_ONRB when evicting an in-flight entry from the tree. On IO completion, detect the orphaned io_piece (neither IP_F_ONRB nor IP_F_ONLIST set) and free it. This leak only occurs with norandommap or softrandommap where fio_offset_overlap_risk() is true, causing io_pieces to be stored in the RB tree with overlap detection. Signed-off-by: Yehor Malikov <Yehor.Malikov@solidigm.com>
malikoyv
force-pushed
the
pr-speed-up-norandommap-read-bw
branch
from
June 10, 2026 07:26
047c174 to
c0e4b6e
Compare
There was a problem hiding this comment.
Pull request overview
Fixes online verification backlog handling when norandommap/overlap detection stores io_pieces in an RB tree, addressing both verification starvation (read/write bandwidth asymmetry) and a long-run memory leak.
Changes:
verify.c: Walk the RB tree (rb_next()) to locate the first non-in-flightio_pieceinstead of giving up atrb_first().iolog.c: ClearIP_F_ONRBwhen evicting an overlapping in-flightio_piecefrom the RB tree.io_u.c: Detect an evicted/orphanedio_pieceat IO completion and free it to prevent leaks.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| verify.c | Avoid verify starvation by scanning RB tree for the first completed entry. |
| iolog.c | Fix eviction bookkeeping by clearing IP_F_ONRB for removed entries. |
| io_u.c | Free orphaned io_pieces on completion to prevent unbounded memory growth. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+2170
to
+2174
| if (io_u->error) { | ||
| unlog_io_piece(td, io_u); | ||
| else { | ||
| atomic_store_release(&io_u->ipo->flags, | ||
| io_u->ipo->flags & ~IP_F_IN_FLIGHT); | ||
| } else { | ||
| unsigned int flags = io_u->ipo->flags & ~IP_F_IN_FLIGHT; | ||
|
|
Collaborator
There was a problem hiding this comment.
A better solution would be to decrement td->io_hist_len in unlog_io_piece() only if the io_piece is removed from the tree or list.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When norandommap is set, io_pieces are stored in an RB tree sorted by offset to detect overlapping writes. This caused two problems with verify_backlog:
get_next_verify() only checked rb_first() for a completed entry. If
the lowest-offset IO was still in-flight, it gave up entirely even
though completed entries existed at higher offsets. This starved
verification reads, causing severe read/write bandwidth asymmetry
(e.g. 40 MiB/s read vs 700 MiB/s write).
When an in-flight io_piece was evicted from the RB tree due to an
overlapping write, the IP_F_ONRB flag was not cleared. On IO
completion the entry appeared to still belong to the tree, so it was
never freed. This caused unbounded memory growth over long runs.
Fix (1) by walking the tree with rb_next() to find the first non in-flight entry instead of giving up at rb_first().
Fix (2) by clearing IP_F_ONRB when evicting an in-flight entry from the tree, and detecting the orphaned entry on IO completion to free it.
Neither issue affects the linked-list path used when norandommap is not set, since list ordering matches submission order and no overlap eviction occurs.