Mind interpolation type when resizing / rotating masks

[maxfreu]

This issue is more a reminder to myself, haven't tested yet. But with the new feature for masks we have to keep in mind that masks could come as e.g. color coded and each exact value has a specific meaning. Therefore we must use nearest interpolation when rotating / resizing etc. I think, so far eg imresize uses linear, which could introduce label noise.

[johnnychen94]

The best time to support this might be ImageTransformations v0.9 release because it's the first version that supports method keyword for imresize and imrotate.

For Augmentor, we need to, e.g., pass method=BSpline(Constant()) to

Augmentor.jl/src/operations/resize.jl

Lines 69 to 75 in b3a1873

	function applyeager(op::Resize, img::AbstractArray, param)
	plain_array(imresize(img, op.size))
	end
	function applylazy(op::Resize, img::AbstractArray, param)
	applyaffineview(op, prepareaffine(img), param)
	end

[barucden]

Oh, yeah! I did not consider this. Just to confirm:

julia> using Augmentor

julia> img = rand(5,5)
5×5 Matrix{Float64}:
 0.135452  0.215202  0.474272  0.0512992  0.481672
 0.78849   0.272553  0.758369  0.950766   0.130598
 0.565686  0.924896  0.768007  0.90122    0.657758
 0.446789  0.454401  0.951394  0.981122   0.268825
 0.738076  0.235141  0.678847  0.162833   0.789674

julia> mask = img .> 0.5
5×5 BitMatrix:
 0  0  0  0  0
 1  0  1  1  0
 1  1  1  1  1
 0  0  1  1  0
 1  0  1  0  1

julia> augment(img => mask, Rotate(30))[2]
7×7 Matrix{Float64}:
 0.0        0.0        0.0       0.0          0.0       0.0       0.0
 0.0        0.0        0.0       0.267949     0.102886  0.267949  0.767949
 0.0        0.0310889  0.401924  1.0          0.866025  0.866025  0.366025
 0.5        0.732051   0.566987  1.0          1.0       0.267949  0.5
 1.0        1.0        0.633975  0.566987     0.866025  0.303848  1.0
 0.767949   0.267949   0.200962  8.88178e-16  0.866025  0.267949  0.598076
 0.0980762  0.598076   1.0       0.5          0.366025  0.767949  0.0980762

[maxfreu]

The best time to support this might be ImageTransformations v0.9 release because it's the first version that supports method keyword for imresize and imrotate.

Any estimate when this will land? As an ugly fix, NNlib also supports nearest neighbour resizing, but no rotations.

Edit: Actually it could be a cool thing to integrate with NNlib in order to offload augmentations to the gpu. E.g. pipeline = Rotate90() |> Crop(...) |> GPU() |> ColorJitter() |> ...

[johnnychen94]

Hopefully before Sep. Most work is done already, but needs a more careful backward compatibility check because the API is revised a lot.

(I find myself involved in too many projects and can't find a large amount of time to just sit down and shakes it out)

[maxfreu]

(I find myself involved in too many projects and can't find a large amount of time to just sit down and shakes it out)

No worries, I'm primarily doing other work too.