Skip to content

Correct performance and option claims in the README - #394

Merged
folbricht merged 1 commit into
masterfrom
readme-accuracy
Aug 29, 2026
Merged

folbricht merged 1 commit into
masterfrom
readme-accuracy

Conversation

@folbricht

Copy link
Copy Markdown
Owner

Documentation only. Several statements no longer matched the code or measurement.

Performance claims

"up to 10x faster" had no qualifier. The speedup comes from chunking parts concurrently, so it scales with core count. On a 4-core machine make measured 1.8x–3.6x faster than casync 2 (Debian 2+20201210-2+b2), both writing zstd-compressed chunks to a local store. Run single-threaded, desync is in casync's range and can be slightly behind, since its rolling hash is pure Go against casync's optimized C — the advantage is concurrency, not a faster inner loop. The new paragraph says all of that, so a reader on a small machine isn't left expecting 10x.

The 0-byte row claimed make is "Slow (worst-case) — likely comparable to casync". The mechanism behind it is real — concurrent chunkers can't align when there are no split points — but the comparison to casync isn't supported by anything. Measured, desync was faster than casync on all-zero data, though that number doesn't settle it either: an all-zero file dedups to a single chunk, so the compress-and-store half of make is nearly free for both tools and the alignment penalty is masked. The row now keeps the mechanism and drops the unsupported comparison. Measuring a mostly-zero file with unique data mixed in would isolate it properly; that hasn't been done.

The "identical output to casync" bullet now says byte-identical, which #393 made literally true for blob indexes.

Four smaller corrections, each checked against the code

Claim Reality
info defaults to plain Defaults to json (desync info --help). Fixed the doc, not the flag — changing the default would break anyone parsing today's output.
min chunk size must be larger than the 48-byte window chunker.go:135 rejects min < ChunkerWindowSize, so exactly 48 is accepted
timeout store option, no scope given Only remotehttp.go, oci.go and (since #390) s3.go consult it. GCS, SFTP and local ignore it, so setting it there does nothing.
macOS catar caveat covers devices and filemodes #388 fixed device numbers. Dropped that half only; the filemode half is unverified either way, so it stays.

Not included

S3 and GCS silently ignore --ca-cert, --trust-insecure and the client certificate options — only remotehttp.go and oci.go build a TLS config. That's a bug with a contained fix rather than something to document, so it's left for its own PR.

Several statements no longer matched the code or measurement.

The "up to 10x faster" claim had no qualifier. The speedup comes from
chunking parts concurrently, so it scales with core count: on a 4-core
machine make measured 1.8x-3.6x faster than casync 2, and single
threaded desync is in casync's range or slightly behind, since its
rolling hash is pure Go against casync's optimized C. Say so, rather
than leaving readers to expect 10x anywhere.

The per-command table called all-zero data the worst case for make and
"likely comparable to casync". The mechanism it describes is real, but
nothing supports the comparison, so keep the former and drop the latter.

Also:

  - info defaults to json output, not plain
  - the chunker accepts a min chunk size equal to the rolling hash
    window, not only larger than it (chunker.go:135)
  - the timeout store option is only consulted by the HTTP(S), S3 and
    OCI stores; GCS, SFTP and local ignore it
  - device numbers on macOS were fixed in #388, so only the filemode
    half of the catar interop caveat still stands
@folbricht
folbricht merged commit 21a747b into master Aug 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

⌂ 换网址