Make compressed log rotation atomic

[chancez]

If another process is watching for *.gz files then it's possible to
begin reading the archive before it has been completely created,
resulting in corruption if the other process is copying the archive to
another location (for example: archival to s3).

To resolve this, we can use a different suffix when writing the file so
that other programs do not read it while it's being created. Once the
archive has been completely created, we atomically rename it to the
desired file name with the *.gz extension, ensuring external programs
only ever see the finished archive.

Signed-off-by: Chance Zibolski [email protected]

[pchaigno]

@natefinch 👋 Did you get a chance to look into this?

[natefinch]

I mean, I guess it's ok? but then what if code is looking for .tmp files?

What problem is this solving? What program do you have that is looking for *.gz files and what does it do?

[chancez]

@natefinch I considered making the tmp suffix configurable. Basically, in cilium we are using lumberjack for log rotation, and have a daemon watching for *.gz files and is uploading them to s3 for archival. What's happening without this is that the daemon is seeing the files get created, and it's uploading the files before they've been completely written to, so we've got corrupted .gz files in s3.

Also, I need to push up another fix to this PR, I forgot to Flush the writer before calling Sync on the underlying file. (Done)

[pchaigno]

Hi @natefinch 👋 Does the above look good?

[pchaigno]

@natefinch Did you get a chance to look into this? Is there anything else we'd need to change?

[kaworu]

@natefinch could you take a quick look at this PR please? 🙏