Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: folbricht/desync
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v1.1.3
Choose a base ref
...
head repository: folbricht/desync
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v1.1.4
Choose a head ref
  • 18 commits
  • 44 files changed
  • 2 contributors

Commits on Sep 3, 2026

  1. Let 'config -w' create the config file it is pointed at (#417)

    A config file named with --config had to exist before any command ran,
    including the one whose job is to write it. 'desync --config new.json
    config -w', the example in the command's own help and in the docs, failed
    with "no such file or directory" and wrote nothing.
    
    Record the missing file during initialization instead of exiting there,
    and report it before running any command other than 'config'. A config
    file that exists but can't be read or parsed remains fatal.
    folbricht authored Sep 3, 2026
    Configuration menu
    Copy the full SHA
    d12697b View commit details
    Browse the repository at this point in the history
  2. Reject a --concurrency below 1 (#419)

    Commands start one worker per --concurrency and then feed chunks into an
    unbuffered channel. With -n 0 or a negative value there are no workers to
    read from it, so the command blocks forever, and because the main
    goroutine never reaches its context check it can't be interrupted with
    SIGINT or SIGTERM either.
    
    Reject the value up front in the store options, and validate those in
    'verify' too, which has its own --concurrency flag.
    folbricht authored Sep 3, 2026
    Configuration menu
    Copy the full SHA
    5b1c8b2 View commit details
    Browse the repository at this point in the history
  3. Test the idle transfer guard on a simulated clock (#420)

    The guard test had a handler write a byte every three quarters of the
    timeout, leaving 50ms of slack per gap before a transfer that was in fact
    progressing got cancelled. That is thinner than a loaded VM can promise,
    and it failed intermittently on the OpenBSD runner.
    
    Drive the guard through a stub transport inside a synctest bubble, where
    the gaps are exact and the seconds they simulate cost no wall time. That
    buys a longer timeout and more of them than the real clock allowed.
    
    A stand-in body can't show that cancelling the context aborts a read
    blocked on a socket, so a smaller test over a real connection still
    covers that. Both stalled cases now read in a goroutine and fail on a
    deadline rather than hanging when the guard never fires.
    folbricht authored Sep 3, 2026
    Configuration menu
    Copy the full SHA
    61fee4a View commit details
    Browse the repository at this point in the history
  4. Exit non-zero when a server fails to start (#418)

    * Exit non-zero when a server fails to start
    
    The chunk and index servers printed the error from ListenAndServe to
    stderr and then returned nil, so a server that never came up exited with
    status 0. Supervisors and CI read that as a clean shutdown.
    
    Hand the error back to the caller instead, and keep returning nil when
    the context is cancelled, which is how a signal shuts the server down.
    
    * Wait for the signal and the servers in one select
    
    The derived context only existed to merge the two ways serve() can
    finish. A select on the error channel and the caller's context does that
    directly, so the context, its cancel and the second, non-blocking select
    all go.
    folbricht authored Sep 3, 2026
    Configuration menu
    Copy the full SHA
    d9973b7 View commit details
    Browse the repository at this point in the history
  5. Don't fail 'cat' when --length runs past the end of the blob (#421)

    io.CopyN returns io.EOF when the source has fewer bytes left than were
    asked for, and that was handed straight back. 'desync cat -o <near the
    end> -l 4096' wrote the bytes that were there and then exited 1 with
    "Error: EOF". Reaching the end of the data is not a failure.
    folbricht authored Sep 3, 2026
    Configuration menu
    Copy the full SHA
    5646c1e View commit details
    Browse the repository at this point in the history
  6. Take the default when the prune prompt gets an empty line (#422)

    fmt.Fscanln reports an empty line as "unexpected newline", so pressing
    enter at the "[y/N]:" prompt exited 1 with that message and the case
    meant to handle it was unreachable. Read the whole line instead, so the
    default the prompt offers is the one it takes, and closed input declines
    rather than erroring.
    
    An index read from STDIN consumes the same input the prompt needs, which
    left it answering EOF. Ask for --yes in that case instead.
    
    The prompt writes to the package's stdout writer and reads a new stdin
    one, which is what lets the tests answer it.
    folbricht authored Sep 3, 2026
    Configuration menu
    Copy the full SHA
    64fd0dc View commit details
    Browse the repository at this point in the history
  7. Let the config file set the concurrency for a store (#423)

    MergedWith assigned the concurrency from the command line unconditionally,
    unlike every other option, which is only taken when its flag was given.
    A "n" in store-options was therefore always replaced with the flag's
    default of 10 and never had an effect.
    
    The flag carries a default rather than a zero value, so "not given" can't
    be told from "given as 10" by the value alone. Use the same Changed check
    as the other options, falling back to the flag when the config doesn't
    name a concurrency any store could use.
    folbricht authored Sep 3, 2026
    Configuration menu
    Copy the full SHA
    d0f0e31 View commit details
    Browse the repository at this point in the history
  8. Pass the tar error over a channel in 'mtree' (#425)

    Walking a directory runs Tar in a goroutine writing into a pipe while
    UnTar reads it. UnTar can stop before the stream ends, a cancelled
    context being one way, and the main goroutine then read tarErr while the
    Tar goroutine was still assigning it.
    
    Send the error over a channel instead, and close the read end of the pipe
    first so Tar isn't left blocked writing into it and can be waited for.
    
    The command writes to the package's stdout writer, which is what lets it
    be tested at all; these are its first tests.
    folbricht authored Sep 3, 2026
    Configuration menu
    Copy the full SHA
    ddc1695 View commit details
    Browse the repository at this point in the history
  9. Report close failures when writing a tar archive (#424)

    * Report close failures when writing a tar archive
    
    The gnu-tar output path closed both the tar writer and the file it wrote
    to with the error discarded. Closing the tar writer is what writes the
    padding and the two end-of-archive blocks, so a failure there, on a full
    disk or a broken pipe, left a truncated archive behind and still exited
    0. Readers accept an archive with no end-of-archive marker, so nothing
    downstream reports it either.
    
    Both closes now go into the returned error, the way the disk output path
    already handled its own. The shared helper for that replaces the three
    copies this would otherwise need.
    
    * Drop the tests added with the close error change
    folbricht authored Sep 3, 2026
    Configuration menu
    Copy the full SHA
    76d89ac View commit details
    Browse the repository at this point in the history
  10. Accept an index from anywhere in 'mtree' (#426)

    * Accept an index from anywhere in 'mtree'
    
    The input was passed to os.Stat before anything looked at whether it was
    meant to be an index, so 'mtree -i' only ever accepted a local file.
    'mtree -i -s store -' and an index named by URL both failed with "no such
    file or directory", where every other command reading an index takes
    both.
    
    Handle the index case first and hand the location to the index store, and
    only go to the filesystem for a catar or a directory, which are always
    local. The friendly message for '-i' on a directory is kept.
    
    The mtree header is no longer written before the input is known to be
    readable, so a failure doesn't leave one on stdout with nothing under it.
    
    * Check the close in the STDIN test cleanup
    folbricht authored Sep 3, 2026
    Configuration menu
    Copy the full SHA
    0b72d29 View commit details
    Browse the repository at this point in the history
  11. Report output write errors, and don't print a size that isn't known (#…

    …427)
    
    MtreeFS discarded the error from every line it wrote and returned nil, so
    'desync mtree' against output that stopped accepting writes produced a
    truncated listing and exited 0. Each entry now reports its own write.
    
    The plain output of 'info' and the output of 'chunk' went straight to
    os.Stdout rather than the package writer the JSON branch uses, which is
    also why neither had any tests.
    
    'info' reported a compressed size of 0 both when there was nothing to
    transfer and when it had deliberately given up on the estimate, either
    because no chunks info file was given or because one was missing a chunk.
    Report it only when it was worked out, so the JSON field is absent and
    the plain line missing when it isn't known, and label it as the size it
    is rather than a count.
    folbricht authored Sep 3, 2026
    Configuration menu
    Copy the full SHA
    9b52269 View commit details
    Browse the repository at this point in the history
  12. Keep 'chop --ignore' reading the file in order (#428)

    * Keep 'chop --ignore' reading the file in order
    
    ChopFile's workers seek to each chunk's offset, so the order the chunks
    are handed over in is the order the file is read in. Filtering rebuilt
    that list by ranging over a map, and Go randomizes map iteration, so
    --ignore turned a front-to-back read into a different random walk over
    the whole file on every run.
    
    Sort the survivors by offset again. Keying the filter by chunk ID also
    collapses repeated chunks, which saves re-reading them, so that stays.
    
    The filtering moved into its own function to make the order assertable.
    
    * Keep 'chop --ignore' reading the file in order
    
    ChopFile's workers seek to each chunk's offset, so the order the chunks
    are handed over in is the order the file is read in. Filtering rebuilt
    that list by ranging over a map, and Go randomizes map iteration, so
    --ignore turned a front-to-back read into a different random walk over
    the whole file on every run.
    
    Sort the survivors by offset again. Keying the filter by chunk ID also
    collapses repeated chunks, which saves re-reading them, so that stays.
    folbricht authored Sep 3, 2026
    Configuration menu
    Copy the full SHA
    b9d6117 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2026

  1. Bump google.golang.org/grpc from 1.83.1 to 1.83.2 (#429)

    Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.83.1 to 1.83.2.
    - [Release notes](https://github.com/grpc/grpc-go/releases)
    - [Commits](grpc/grpc-go@v1.83.1...v1.83.2)
    
    ---
    updated-dependencies:
    - dependency-name: google.golang.org/grpc
      dependency-version: 1.83.2
      dependency-type: indirect
    ...
    
    Signed-off-by: dependabot[bot] <support@github.com>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored Sep 9, 2026
    Configuration menu
    Copy the full SHA
    6314450 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2026

  1. Bump go.opentelemetry.io/otel/sdk from 1.44.0 to 1.45.0 (#431)

    Bumps [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go) from 1.44.0 to 1.45.0.
    - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
    - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
    - [Commits](open-telemetry/opentelemetry-go@v1.44.0...v1.45.0)
    
    ---
    updated-dependencies:
    - dependency-name: go.opentelemetry.io/otel/sdk
      dependency-version: 1.45.0
      dependency-type: indirect
    ...
    
    Signed-off-by: dependabot[bot] <support@github.com>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored Sep 20, 2026
    Configuration menu
    Copy the full SHA
    cc07416 View commit details
    Browse the repository at this point in the history
  2. Open SFTP and SSH store connections on demand (#432)

    * Open SFTP and SSH store connections on demand
    
    The SFTP and SSH stores opened all N connections when the store was
    created, whether they were needed or not. They now share a pool that
    opens the first connection up front, to confirm the store can be
    reached, and further ones only when all existing connections are busy,
    up to N. Connections are still opened one at a time.
    
    The S3 store kept minio's default of 16 idle connections per host, so
    with a concurrency above 16 the extra connections were closed and
    reopened for every request. It now keeps one per concurrent request.
    
    * Raise the total idle connection cap for S3 and OCI stores
    
    Raising the per-host idle limit isn't enough on its own. minio's
    transport caps idle connections at 256 in total and Go's default
    transport, which the OCI store clones, at 100. Above that, connections
    were still closed and reopened for every request.
    folbricht authored Sep 20, 2026
    Configuration menu
    Copy the full SHA
    f22b340 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2026

  1. Record the digest in use in index feature flags (#435)

    Indexes produced by 'desync tar -i' always carried CaFormatSHA512256,
    even with '--digest sha256', because ChunkStream hardcoded the flag and
    the tar command ORed in TarFeatureFlags, which includes it. 'desync make'
    on a catar had the same problem since it copied the archive's feature
    flags, including the digest bit, into the index. Such indexes were
    rejected when read with the matching digest and failed chunk
    verification otherwise.
    
    Set the digest flag from Digest in both ChunkStream and IndexFromFile,
    and don't take it from TarFeatureFlags or the catar entry.
    
    Fixes #434
    folbricht authored Sep 21, 2026
    Configuration menu
    Copy the full SHA
    9305870 View commit details
    Browse the repository at this point in the history
  2. Set the digest flag in catar entries from Digest (#436)

    Tar wrote CaFormatSHA512256 into every catar entry, even with
    '--digest sha256'. Drop the digest flag from TarFeatureFlags and set it
    per entry from the digest in use, matching how indexes record it.
    folbricht authored Sep 21, 2026
    Configuration menu
    Copy the full SHA
    5f24084 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2026

  1. Rework the README pitch around use cases and a multi-version measurem…

    …ent (#437)
    
    Replace "What it's for" and "What it saves" with scenarios tied to the
    features that enable them, and a measurement across six Debian cloud image
    builds updating from different starting points. Show seeded updates in the
    Quick Start, link Key Features to their docs, and rename Design Philosophy
    to Differences from casync.
    folbricht authored Sep 22, 2026
    Configuration menu
    Copy the full SHA
    ad89f62 View commit details
    Browse the repository at this point in the history
Loading
⌂ 换网址