chunked: flush the input tarball to the output #1772
Conversation
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: giuseppe The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
LGTM |
|
the lint failure is unrelated, I've a patch in https://github.com/containers/storage/pull/1769/commits to fix it |
|
|
||
| // make sure the entire tarball is flushed to the output as it might contain | ||
| // some trailing zeros that affect the checksum. | ||
| if _, err := io.Copy(zstdWriter, its); err != nil { |
There was a problem hiding this comment.
For the record: This looked very surprising to me (if notthing else, I doubted that after passing its to tar.NewReader there are any promises at all about the position of its)…
But tar-split internally does the same thing in https://github.com/vbatts/tar-split/blob/9a95e026024d3929928021244d72e59f758616b3/tar/asm/disassemble.go#L121-L151 .
(And for the purposes of this PR, we don’t care whether the unexpected trailing padding is recorded into tar-split metadata (that is important, but hopefully ensured by existing code), but whether we write it into the zstdWriter unmodified; so we must do that separately.)
|
I merged the other PR which I believe fixes the lint, could you rebase to make sure tests pass. |
close the pipe immediately when the file is closed to avoid a possible deadlock if the other end is waiting for EOF on the writer side. Signed-off-by: Giuseppe Scrivano <[email protected]>
Flush the entire input tarball to the output in the zstd:chunked stream writer. This is needed to include any trailing zeros that affect the uncompressed digest. Closes: containers#1771 Signed-off-by: Giuseppe Scrivano <[email protected]>
giuseppe
force-pushed
the
fix-diffid-zstd-chunked
branch
from
79dee19 to
6a7ab2b
Compare
|
/lgtm |
Flush the entire input tarball to the output in the zstd:chunked stream writer. This is needed to include any trailing zeros that affect the uncompressed digest.
Closes: #1771
Signed-off-by: Giuseppe Scrivano [email protected]