Repository navigation
Comparing changes
Open a pull request
base repository: folbricht/desync
base: v1.1.0
head repository: folbricht/desync
compare: v1.1.1
- 10 commits
- 27 files changed
- 1 contributor
Commits on Aug 29, 2026
-
Drop the cgo zstd implementation (#386)
Compression had two implementations: the pure-Go klauspost/compress default, and a DataDog/zstd variant selected with the "datadog" build tag. The cgo variant meant the build could produce binaries needing a C toolchain, which complicates cross-compilation for no benefit that the pure-Go implementation doesn't already provide. Remove compress_datadog.go and the build constraint on compress.go, so the pure-Go implementation is the only one, and drop the now-unused DataDog/zstd dependency. The release matrix (linux, darwin, windows on amd64/arm64) plus linux/arm and linux/riscv64 all build with CGO_ENABLED=0, and no documentation referenced the build tag.
Configuration menu - View commit details
-
Copy full SHA for 6b6fcfa - Browse repository at this point
Copy the full SHA 6b6fcfaView commit details -
go-fuse v2.2.0 only builds on Linux and macOS, so the FreeBSD build was broken despite the README listing BSD as expected to work. Upgrade it to v2.11.0, which supports FreeBSD. No source changes were needed for the upgrade itself, the fs and fuse APIs desync uses are unchanged. That uncovered a second problem: createDeviceNode for non-Linux passed the device number to unix.Mknod as an int, but FreeBSD takes a uint64. FreeBSD has mknodat(2), so rather than widening the fallback, give it its own implementation using the same Root-relative creation as Linux, which also confines it better than the darwin path that resolves the parent and then creates on the real path. FreeBSD is now built by goreleaser and validated in CI: a cross-compile in the main matrix for a fast signal, and the full test suite in a FreeBSD VM. NetBSD, OpenBSD and DragonFly remain unsupported since go-fuse does not build there, so the README now says so instead of lumping all BSDs together as untested.
Configuration menu - View commit details
-
Copy full SHA for 9aafd9b - Browse repository at this point
Copy the full SHA 9aafd9bView commit details -
Use the platform's own device number encoding (#388)
mkdev built device numbers with Linux's dev_t layout on every platform, and the read side split Rdev with Linux's layout too. The encodings are not the same everywhere, so on macOS this produced wrong device nodes: a catar entry for major 1, minor 3 was created as major 0, minor 259, and reading /dev/null back reported major 0, minor 12290 rather than 3, 2. FreeBSD's encoding happens to agree with Linux's for the small major/minor values in practical use, so it was unaffected. Defer to x/sys/unix, which implements Mkdev, Major and Minor per platform. On Linux the encoding is unchanged; the read path there now also stops truncating majors above 12 bits. Encoding now returns an error when a major or minor from the archive is too large for the platform to represent, instead of silently truncating it to a different device. TestMkdev covers the encode/decode round-trip on every platform and fails on macOS with the old layout. TestCreateDeviceRoundTrip creates real device nodes and reads them back, which needs root, so it runs in the FreeBSD CI job and skips elsewhere.
Configuration menu - View commit details
-
Copy full SHA for 8949ae9 - Browse repository at this point
Copy the full SHA 8949ae9View commit details -
Upgrade minio-go from v6 to v7 (#389)
minio-go v6 has been end of life for years. v7 moves every object operation onto a context and renames a few things, so the store code changes with it: - minio.NewWithOptions is now minio.New - GetObject, PutObject, StatObject and RemoveObject take a context - ListObjectsV2 with a done channel is now ListObjects with a ListObjectsOptions, cancelled through its context - credentials.Provider gained RetrieveWithCredContext, which the static and refreshable providers now implement alongside Retrieve The store interface has no context to pass down, so the per-object calls use context.Background(), which is what v6 did internally. Prune already receives one and now uses it to bound the listing as well as the loop. Dependencies pulled up by v7's own requirements: klauspost/compress, logrus, cobra, pflag, x/crypto and x/sys. Verified against a real MinIO server as well as the existing fake-S3 tests: make, extract, info, prune and a prefixed store all round-trip, indexes stored in S3 are read back and survive a prune of the same bucket.Configuration menu - View commit details
-
Copy full SHA for 627a467 - Browse repository at this point
Copy the full SHA 627a467View commit details -
Apply the timeout store option to S3 stores (#390)
The timeout option was documented for chunk read/write but S3 stores ignored it: unlike the HTTP and OCI stores, which set it on their http.Client, the minio client takes a context per operation and was given context.Background(). An unresponsive endpoint would hang for as long as it held the connection open. Add StoreOptions.contextWithTimeout, which derives a context from the same effectiveTimeout() the HTTP stores use, and take care that a negative (infinite) timeout produces a context with no deadline rather than one that is already expired. Chunk reads and writes get a fresh timeout per attempt, so it bounds each try rather than the whole retry sequence, and it covers the body read since minio only issues the request on first read. Index reads return a reader that the caller consumes later, so the context is cancelled when that reader is closed. The listing in Prune keeps the caller's context, since a per-object timeout has no meaning for an operation that walks the whole store. Note this is a behaviour change for existing S3 users: operations that previously had no deadline now default to one minute, matching the HTTP stores. Set a negative timeout for the old behaviour.
Configuration menu - View commit details
-
Copy full SHA for 1aeccdc - Browse repository at this point
Copy the full SHA 1aeccdcView commit details -
There was no way to tell which build of desync was running, which makes a bug report hard to place. Add a 'version' command printing the version, commit, build time, Go toolchain and platform, in plain text or JSON, plus --version on the root command as a shorthand. The version is taken from -X main.version at link time when present, and otherwise from the build information the toolchain embeds, so every install path reports something useful without a build step: - 'go install ...@v1.1.0' has no ldflags, but the module version is embedded, giving v1.1.0 - a build from a checkout gets a pseudo-version derived from the last tag plus vcs.revision and vcs.time, and +dirty for a modified tree - release builds get the exact tag, since the ldflags are now set explicitly in .goreleaser.yml rather than left to its defaults A build from a source tarball has neither, and reports the toolchain's own "(devel)". --version is registered explicitly rather than left to cobra, which would otherwise claim -v for it. Users would reasonably read -v as --verbose.Configuration menu - View commit details
-
Copy full SHA for 00b3c6b - Browse repository at this point
Copy the full SHA 00b3c6bView commit details -
Release binaries for 32-bit ARM and riscv64 (#392)
Both architectures build cleanly and are a natural fit for the embedded/appliance update case, but neither was in the release matrix, so those users had to build from source. Add arm and riscv64, restricted to Linux. Of the other platforms only freebsd/arm is a combination Go supports at all, and FreeBSD is already untested at runtime, so a second architecture there would be an artifact nobody exercises. goreleaser skips the combinations Go doesn't support on its own; the ignore rule is for the deliberate omission. Both ARM variants are built: v6 runs anywhere 32-bit ARM does, and v7 is faster on hardware that supports it, which is worth an extra artifact for a tool whose chunking is CPU-bound.
Configuration menu - View commit details
-
Copy full SHA for f70904e - Browse repository at this point
Copy the full SHA f70904eView commit details -
Produce byte-identical indexes to casync (#393)
Two differences kept desync's blob indexes from matching casync's for the same input, despite the README claiming identical output. The chunker never tested the rolling hash window that ends exactly at the min chunk size. Both tools apply the same optimization of leaving the first min-48 bytes unhashed, but casync's ca_chunker_scan() seeds the window and immediately tests it, so a boundary there yields a chunk of exactly min bytes. desync seeded the identical hash and rolled a byte in before its first test, making min+1 the smallest chunk it could produce. Whenever a boundary fell on min the two disagreed: on a 123MB tar that happened once in 1699 chunks, casync emitting 16384+56320 where desync emitted a single 72704. Test the seeded window before entering the loop. The blob index header also omitted CaFormatExcludeFile, which casync includes in CA_FORMAT_DEFAULT and stamps into every index, so the feature flags read 0xa000000000000000 rather than 0xb000000000000000 and the files differed in one byte even when the chunking agreed. The flag means the encoder honors .caexclude files while walking a directory tree, so it is inert for a blob and only set on that path. Archive indexes keep their current flags: desync doesn't implement .caexclude (#114) and shouldn't claim to, and their flags differ from casync's regardless because desync stores a smaller catar feature set. With both fixed, casync and desync produce byte-identical caibx files for the same input, verified for default and custom chunk sizes and for both digest algorithms. Note for existing stores: the chunker change alters boundaries in the rare case above, so re-chunking data already in a store will not dedup against those few chunks. Existing indexes and chunks stay valid and readable.
Configuration menu - View commit details
-
Copy full SHA for 6696dc1 - Browse repository at this point
Copy the full SHA 6696dc1View commit details -
Correct performance and option claims in the README (#394)
Several statements no longer matched the code or measurement. The "up to 10x faster" claim had no qualifier. The speedup comes from chunking parts concurrently, so it scales with core count: on a 4-core machine make measured 1.8x-3.6x faster than casync 2, and single threaded desync is in casync's range or slightly behind, since its rolling hash is pure Go against casync's optimized C. Say so, rather than leaving readers to expect 10x anywhere. The per-command table called all-zero data the worst case for make and "likely comparable to casync". The mechanism it describes is real, but nothing supports the comparison, so keep the former and drop the latter. Also: - info defaults to json output, not plain - the chunker accepts a min chunk size equal to the rolling hash window, not only larger than it (chunker.go:135) - the timeout store option is only consulted by the HTTP(S), S3 and OCI stores; GCS, SFTP and local ignore it - device numbers on macOS were fixed in #388, so only the filemode half of the catar interop caveat still standsConfiguration menu - View commit details
-
Copy full SHA for 21a747b - Browse repository at this point
Copy the full SHA 21a747bView commit details -
Stop calling desync a drop-in replacement for casync (#395)
The intro claimed compatibility with casync's data structures "to function as a drop-in replacement". The data compatibility is real and worth stating, but the CLI is not interchangeable: the options differ and desync has commands casync doesn't. The CLI Reference already said as much, so the intro contradicted it. The remaining mention, in Design Philosophy, is a narrow and accurate one - desync can stand in for the casync binary on an SSH server serving chunks read-only, which is what the pull command implements. Only its bold lead-in read like the general claim, so scope that to what it describes.
Configuration menu - View commit details
-
Copy full SHA for 1b2038d - Browse repository at this point
Copy the full SHA 1b2038dView commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff v1.1.0...v1.1.1