Insufficient integer bit width used

[mossblaser]

It appears that this implementation does not use integers with enough bits to guarantee freedom from integer overflows for all of the wavelet and transform depth configurations it supports.

Is this a big problem? (TL;DR: Not really)
Typical pictures do not frequently result in very large signal levels within a decoder. As such, since no overflow issues with this codec have been spotted during development it is highly likely that such issues will be very rare in practice. However, they could still occur and when they do they tend to be fairly easy to spot as illustrated by the image below:

NB: The example above was produced by a different VC-2 implementation with an artificially reduced integer bit width and not by this codec.

So to restate, it is unlikely this codec will encounter an integer overflow in practice but it is not guaranteed.

Proof of existence
The following zip file contains a series of VC-2 bitstreams produced by a "well-behaved" VC-2 encoder which trigger integer overflows in vc2hqdecode:

signal_range_decoder_tests.zip

The expected and actual outputs produced by vc2hqdecode are attached below:

signal_range_expected_vs_actual.zip

As a sample, the following shows a fragment of the decoded output with the expected output on the top and the actual output on the bottom:

Looking closely at these you should be able to see that some of the patterns differ from the expectation, typically taking the form of a small square being black rather than white (or visa versa). This is fairly typical of an overflow causing an integer wrap-around.

Test sequence notes

This test sequence was designed for a 2-level symmetric Le Gall (5, 3) transform on 10-bit video content. As such, the bug reported here is only proven to exist for that specific configuration. It is very likely, however, that the same problem applies to some or all other wavelet transforms and depths.

The test sequence itself consists of a series of test patterns which look something like the following image:

These test patterns consist of entirely valid (i.e. in-range) video signals encoded using a "well-behaved" VC-2 encoder. Well-behaved here means that this encoder operates by applying a wavelet transform and then quantizing the coefficients as suggested informatively in the VC-2 specification. That is, the encoder does not 'invent' any arbitrary, large transform coefficients. The encoder does, however, tend to use much heavier quantisation than typical VC-2 applications -- but still does so entirely within the limits of the VC-2 specification.

For more details on these sequences please contact @mossblaser and/or read the manual for the vc2_bit_widths software which generates these test patterns.

Should this be fixed?
In an ideal world, this bug should be fixed. In practice, doing so would almost certainly require a significant amount of effort given the low-level nature of this implementation. In practice I would expect that this bug could safely remain with this issue remaining as a warning should anybody encounter problems in practice.