Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Corrupt TIFFs created for incompressible data using zlib compression #68

Open
twotabbies opened this issue Oct 9, 2020 · 1 comment

Comments

@twotabbies
Copy link

https://github.com/jai-imageio/jai-imageio-core/blob/master/src/main/java/com/github/jaiimageio/impl/plugins/tiff/TIFFDeflater.java#L90 counts the number of 32Ki blocks based on the input size, and then allocates a compression buffer assuming that zlib compression will, in the worst case, add at most 5 bytes per 32k block, plus a 6 byte header. This is incorrect: https://www.zlib.net/zlib_tech.html says "the only expansion is an overhead of five bytes per 16 KB

block (about 0.03%), plus a one-time overhead of six bytes for the entire stream" - note the block size!

If the input data is incompressible, the resulting TIFF file is corrupt as the zlib data is truncated, due to the buffer being too small. An example file is attached (gzipped because github won't accept a plain .tif).
test.tif.gz

@schwehr
Copy link

schwehr commented Oct 14, 2020

From @twotabbies fix:

int blocks = (inputSize + 32767)/32768;

// Worst case for Zlib deflate is input size + 5 bytes per 32k

Should be

int blocks = (inputSize + 16383)/16384;

// Worst case for Zlib deflate is input size + 5 bytes per 16k

Patch style:

--- a/imageio_ext/plugin/tiff/src/main/java/it/geosolutions/imageioimpl/plugins/tiff/TIFFDeflater.orig
+++ b/imageio_ext/plugin/tiff/src/main/java/it/geosolutions/imageioimpl/plugins/tiff/TIFFDeflater.java
@@ -116,10 +116,12 @@ public class TIFFDeflater extends TIFFCo
                       int scanlineStride) throws IOException {
 
         int inputSize = height*scanlineStride;
-        int blocks = (inputSize + 32767)/32768;
+        int blocks = (inputSize + 16383)/16384;
 
-        // Worst case for Zlib deflate is input size + 5 bytes per 32k
+        // Worst case for Zlib deflate is input size + 5 bytes per 16k
         // block, plus 6 header bytes
         byte[] compData = new byte[inputSize + 5*blocks + 6];
 
         int numCompressedBytes = 0;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants