Skip to content

Fix: parse.c opt_len() to use minimal distance to delimiter to determine option length - #1924

Merged
axboe merged 1 commit into
axboe:masterfrom
leonid-kozlov:fix-random-distribution-parsing-failure
Jun 23, 2025
Merged

axboe merged 1 commit into
axboe:masterfrom
leonid-kozlov:fix-random-distribution-parsing-failure

Conversation

@leonid-kozlov

Copy link
Copy Markdown
Contributor

Use minimal distance to delimiter to determine option length

Current implementation of opt_len() function is using array of possible delimiters for option name search.
They are {',', ':'}.

The loop at

fio/parse.c

Line 484 in 813445e

for (i = 0; i < FIO_ARRAY_SIZE(delimiter); i++) {
takes first found delimiter and return without trying to search by the rest of possibilities.

This causes __handle_option() function to falsely include into option name all chars up to first ',' ignoring any ':' chars.
Resulting not possible to parse options like zoned:50/5:30/15:20/,50/10:50/90.

Fixes: #1923

Signed-off-by: Leonid Kozlov leonid.e.kozlov@gmail.com

@leonid-kozlov

Copy link
Copy Markdown
Contributor Author

Tested by running following options:

  • with multiple zones as in example: ./fio --debug=parse --parse-only --random_distribution=zoned:50/5:30/15:20/,50/10:50/90
fio: set debug option parse
parse    190666 handle_option=random_distribution, ptr=zoned:50/5:30/15:20/,50/10:50/90
parse    190666 __handle_option=random_distribution, type=OPT_STR, ptr=zoned:50/5:30/15:20/,50/10:50/90
parse    190666 zone ddir 0 (nr=3): 
parse    190666         0: 50/5
parse    190666         1: 30/15
parse    190666         2: 20/80
parse    190666 zone ddir 1 (nr=2): 
parse    190666         0: 50/10
parse    190666         1: 50/90
parse    190666 zone ddir 2 (nr=2): 
parse    190666         0: 50/10
parse    190666         1: 50/90
parse    190666 free options
parse    190666 free options
  • with only one zone: ./fio --debug=parse --parse-only --random_distribution=zoned:50/5:30/15:20/
fio: set debug option parse
parse    191537 handle_option=random_distribution, ptr=zoned:50/5:30/15:20/
parse    191537 __handle_option=random_distribution, type=OPT_STR, ptr=zoned:50/5:30/15:20/
parse    191537 zone ddir 0 (nr=3): 
parse    191537         0: 50/5
parse    191537         1: 30/15
parse    191537         2: 20/80
parse    191537 zone ddir 1 (nr=3): 
parse    191537         0: 50/5
parse    191537         1: 30/15
parse    191537         2: 20/80
parse    191537 zone ddir 2 (nr=3): 
parse    191537         0: 50/5
parse    191537         1: 30/15
parse    191537         2: 20/80
parse    191537 free options
parse    191537 free options
  • with all 3 zones: ./fio --debug=parse --parse-only --random_distribution=zoned:50/5:30/15:20/,50/10:50/90,10/20:10/20:10/20:70/
fio: set debug option parse
parse    190843 handle_option=random_distribution, ptr=zoned:50/5:30/15:20/,50/10:50/90,10/20:10/20:10/20:70/
parse    190843 __handle_option=random_distribution, type=OPT_STR, ptr=zoned:50/5:30/15:20/,50/10:50/90,10/20:10/20:10/20:70/
parse    190843 zone ddir 0 (nr=3): 
parse    190843         0: 50/5
parse    190843         1: 30/15
parse    190843         2: 20/80
parse    190843 zone ddir 1 (nr=2): 
parse    190843         0: 50/10
parse    190843         1: 50/90
parse    190843 zone ddir 2 (nr=4): 
parse    190843         0: 10/20
parse    190843         1: 10/20
parse    190843         2: 10/20
parse    190843         3: 70/40
parse    190843 free options
parse    190843 free options
fio: set debug option parse
parse    190946 handle_option=sync_file_range, ptr=wait_before,write:8
parse    190946 __handle_option=sync_file_range, type=OPT_STR_MULTI, ptr=wait_before,write:8
parse    190946 __handle_option=sync_file_range, type=OPT_STR_MULTI, ptr=write:8
parse    190946 free options
parse    190946 free options
  • with regular options: ./fio --debug=parse --parse-only --rw=randrw --rwmixread=65 --bssplit=8k/5:32k/10:64k/15:128k/60:256k:10
fio: set debug option parse
parse    191232 handle_option=rw, ptr=randrw
parse    191232 __handle_option=rw, type=OPT_STR, ptr=randrw
parse    191232 handle_option=rwmixread, ptr=65
parse    191232 __handle_option=rwmixread, type=OPT_INT, ptr=65
parse    191232   ret=0, out=65
parse    191232 handle_option=bssplit, ptr=8k/5:32k/10:64k/15:128k/60:256k:10
parse    191232 __handle_option=bssplit, type=OPT_STR_ULL, ptr=8k/5:32k/10:64k/15:128k/60:256k:10
parse    191232 free options
parse    191232 free options

@leonid-kozlov
leonid-kozlov marked this pull request as draft June 18, 2025 08:32
@sitsofe

sitsofe commented Jun 18, 2025

Copy link
Copy Markdown
Collaborator

(Please don't worry about the Windows build failures. It looks like Cygwin now provides functions for nanosleep and clock_gettime (when it didn't in the past) so we'll have to think about how to test for those such that we only provide the fallbacks if they aren't implemented. nanosleep will be easy enough but clock_gettime will be trickier)

Comment thread parse.c Resolved
Comment thread parse.c Resolved
Comment thread parse.c Resolved
@leonid-kozlov
leonid-kozlov marked this pull request as ready for review June 19, 2025 09:12
@leonid-kozlov
leonid-kozlov requested a review from sitsofe June 19, 2025 09:12

@sitsofe sitsofe left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good code-wise. My next request is could you squash all these changes down to one commit and force push to update this branch? We're more interested in the final product than how you got there when we merge :-)

@leonid-kozlov
leonid-kozlov force-pushed the fix-random-distribution-parsing-failure branch 2 times, most recently from 2895e99 to eb5d564 Compare June 19, 2025 14:00
@leonid-kozlov

Copy link
Copy Markdown
Contributor Author

Unnecessary commits are squashed and commit message cleaned

@leonid-kozlov
leonid-kozlov requested a review from sitsofe June 19, 2025 14:06

@sitsofe sitsofe left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@leonid-kozlov one minor quibble:

Looking at the commit with git log --pretty=fuller I see this:

Author:     leonid-kozlov <[hidden]@gmail.com>
AuthorDate: Wed Jun 18 10:54:46 2025 +0300
Commit:     lkozlov <[hidden]@aubmc.local>
CommitDate: Thu Jun 19 16:55:34 2025 +0300

Would you be able to set the git username to your "Firstname Lastname" and your email to your gmail in both cases? Beyond that LGTM!

@leonid-kozlov
leonid-kozlov force-pushed the fix-random-distribution-parsing-failure branch from eb5d564 to 51796ae Compare June 19, 2025 23:17
Use minimal distance to delimiter to determine option length

Current implementation of opt_len() makes impossible to
locate option name in random_distribution zones list
combining ':' and ',' chars.
opt_len() function should try to locate option name
by all possible delimiters and return minimal length one
instead of returning first found.

Fixes: axboe#1923

Signed-off-by: Leonid Kozlov <leonid.e.kozlov@gmail.com>
@leonid-kozlov
leonid-kozlov force-pushed the fix-random-distribution-parsing-failure branch from 51796ae to 7d24088 Compare June 19, 2025 23:48
@leonid-kozlov

Copy link
Copy Markdown
Contributor Author

Fixed commit signature

@leonid-kozlov
leonid-kozlov requested a review from sitsofe June 19, 2025 23:53

@sitsofe sitsofe left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@leonid-kozlov Perfect, thank you for making the changes.

@axboe can you pull this (I think @vincentkfu is currently on holiday)?

@axboe
axboe merged commit 538db2e into axboe:master Jun 23, 2025
@vincentkfu

Copy link
Copy Markdown
Collaborator

Thanks to @leonid-kozlov for the fix and to @sitsofe for reviewing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

random_distribution option doesn't work with separate zones for reads, writes, and trims

4 participants

⌂ 换网址