Skip to content
This repository was archived by the owner on Nov 19, 2024. It is now read-only.

Fix create archive to a continuous writing source file failed - #388

Merged
mholt merged 2 commits into
mholt:masterfrom
halfcrazy:fix-too-long
Dec 13, 2023
Merged

mholt merged 2 commits into
mholt:masterfrom
halfcrazy:fix-too-long

Conversation

@halfcrazy

Copy link
Copy Markdown
Contributor

Fix #387

mholt#387
Signed-off-by: Yan Zhu <hackzhuyan@gmail.com>
@mholt

mholt commented Sep 21, 2023

Copy link
Copy Markdown
Owner

Thanks! Why does this fix it, though? Wouldn't Copy() just copy the 2 GB and finish at EOF? Why do we need to limit the number of bytes?

@halfcrazy

Copy link
Copy Markdown
Contributor Author

I suppose the problem is caused by writing a header with the original file size and the reader reading the new append content.

image
0880c4c80710ddd68215
0880c4c80710ddd6a214

https://github.com/mholt/archiver/blob/v4.0.0-alpha.8/tar.go#L74-L100

func (Tar) writeFileToArchive(ctx context.Context, tw *tar.Writer, file File) error {
	if err := ctx.Err(); err != nil {
		return err // honor context cancellation
	}

	hdr, err := tar.FileInfoHeader(file, file.LinkTarget)
	if err != nil {
		return fmt.Errorf("file %s: creating header: %w", file.NameInArchive, err)
	}
	hdr.Name = file.NameInArchive // complete path, since FileInfoHeader() only has base name

==>	if err := tw.WriteHeader(hdr); err != nil {
		return fmt.Errorf("file %s: writing header: %w", file.NameInArchive, err)
	}

	// only proceed to write a file body if there is actually a body
	// (for example, directories and links don't have a body)
	if hdr.Typeflag != tar.TypeReg {
		return nil
	}

	if err := openAndCopyFile(file, tw); err != nil {
		return fmt.Errorf("file %s: writing data: %w", file.NameInArchive, err)
	}

	return nil
}

@halfcrazy

Copy link
Copy Markdown
Contributor Author

ping @mholt

@mholt

mholt commented Dec 1, 2023

Copy link
Copy Markdown
Owner

Thanks, sorry for the delay. Just had a baby. I just need to think about this one more time but I suppose it is likely to be merged soon 🙂

@mholt mholt left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I mean, I think I have no problems with this, but I also don't really know what the implications are for not your use case (which I don't fully understand still from code screenshots).

Could we add a comment maybe explaining why we're doing it this way?

Even better, tests to ensure correct behavior in your use case and the 'standard' use cases. I haven't really gotten around to doing a lot of testing in this package yet but if you want to make me even more confident in this change, then that'd be good.

Comment thread archiver.go Resolved

@mholt mholt left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the improvement, we'll try it out

@mholt
mholt merged commit 09bbccc into mholt:master Dec 13, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

file 2G.file: writing data: archive/tar: write too long

2 participants

⌂ 换网址