[CPU][Ref] Fix Reduce ops to produce stable (default val) output for empty input

[mitruska]

Details:

• Fix Reduce ops to produce stable output (aligned with default values instead of random values) for an empty input:
  including update of CPU and reference implementations.

  openvino/src/plugins/intel_cpu/src/nodes/reduce.cpp

  Lines 2949 to 3023 in eabe528

  	inline void Reduce::init_dst_data(uint8_t *out_ptr, size_t dst_size) {
  	switch (algorithm) {
  	case Algorithm::ReduceL1:
  	case Algorithm::ReduceL2:
  	case Algorithm::ReduceLogSum:
  	case Algorithm::ReduceLogSumExp:
  	case Algorithm::ReduceMean:
  	case Algorithm::ReduceOr:
  	case Algorithm::ReduceSum:
  	case Algorithm::ReduceSumSquare:
  	memset(out_ptr, 0, dst_size);
  	break;
  	case Algorithm::ReduceAnd:
  	case Algorithm::ReduceProd:
  	if (output_prec == ov::element::f32) {
  	auto out_p = reinterpret_cast<float *>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = static_cast<float>(1); });
  	} else if (output_prec == ov::element::i32) {
  	auto out_p = reinterpret_cast<int32_t *>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = static_cast<int32_t>(1); });
  	} else if (output_prec == ov::element::bf16) {
  	auto out_p = reinterpret_cast<bfloat16_t*>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = static_cast<bfloat16_t>(1); });
  	} else if (output_prec == ov::element::f16) {
  	auto out_p = reinterpret_cast<ov::float16*>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = static_cast<ov::float16>(1); });
  	} else if (output_prec == ov::element::u8) {
  	auto out_p = reinterpret_cast<uint8_t *>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = static_cast<uint8_t>(1); });
  	} else if (output_prec == ov::element::i8) {
  	auto out_p = reinterpret_cast<int8_t *>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = static_cast<int8_t>(1); });
  	}
  	break;
  	case Algorithm::ReduceMax:
  	if (output_prec == ov::element::f32) {
  	auto out_p = reinterpret_cast<float *>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = std::numeric_limits<float>::lowest(); });
  	} else if (output_prec == ov::element::i32) {
  	auto out_p = reinterpret_cast<int32_t *>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = std::numeric_limits<int32_t>::min(); });
  	} else if (output_prec == ov::element::bf16) {
  	auto out_p = reinterpret_cast<bfloat16_t*>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = std::numeric_limits<bfloat16_t>::lowest(); });
  	} else if (output_prec == ov::element::f16) {
  	auto out_p = reinterpret_cast<ov::float16*>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = std::numeric_limits<ov::float16>::lowest(); });
  	} else if (output_prec == ov::element::u8) {
  	auto out_p = reinterpret_cast<uint8_t *>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = std::numeric_limits<uint8_t>::min(); });
  	} else if (output_prec == ov::element::i8) {
  	auto out_p = reinterpret_cast<int8_t *>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = std::numeric_limits<int8_t>::min(); });
  	}
  	break;
  	case Algorithm::ReduceMin:
  	if (output_prec == ov::element::f32) {
  	auto out_p = reinterpret_cast<float *>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = std::numeric_limits<float>::max(); });
  	} else if (output_prec == ov::element::i32) {
  	auto out_p = reinterpret_cast<int32_t *>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = std::numeric_limits<int32_t>::max(); });
  	} else if (output_prec == ov::element::bf16) {
  	auto out_p = reinterpret_cast<bfloat16_t*>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = std::numeric_limits<bfloat16_t>::max(); });
  	} else if (output_prec == ov::element::f16) {
  	auto out_p = reinterpret_cast<ov::float16*>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = std::numeric_limits<ov::float16>::max(); });
  	} else if (output_prec == ov::element::u8) {
  	auto out_p = reinterpret_cast<uint8_t *>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = std::numeric_limits<uint8_t>::max(); });
  	} else if (output_prec == ov::element::i8) {
  	auto out_p = reinterpret_cast<int8_t *>(out_ptr);
  	parallel_for(dst_size / dst_data_size, [&](size_t i) { out_p[i] = std::numeric_limits<int8_t>::max(); });
  	}

• Stable behavior for empty input has been requested for NNCF

• Originally random values has been produced, because common shape inference of the Reduce can produce non empty output shape for an empty input, for example Reduce with keep_dims=False:
  InputShape{2, 0}, reduce_axes=1, OutputShape{2}

• Common CPU Reduce::isExecutable was returning False for an empty input tensor (with 0 dim in the input Shape), and the Reduce::execute has been not executed at all for such case.
  The proposal of updated behavior is to return False for an empty output shape (with 0 dim in the output Shape),
  but when the input is empty, and the output is not, the output will be filled with default value.

The approach I've tested within the second PR #27423 is similar, but it forces CPU Reduce output value to be always 0 (which is not fully compatible with the reference impl, TF and ONNX).

Note: This change is considered as backward compatible with relatively low risk.
Alternatively Reduce ops common shape_inference can be updated:

• A. To produce empty output for empty input and keep 0 dim in the output shape even if requested by "axes" to be reduced.
  or
• B. Throw an error when reduction over 0 dimension is detected.

But those options are currently considered as not backward compatible, and would require new version of each Reduce* op.

Tickets:

• 117469

[dmitry-gorokhov]

I am afraid this is not fully correct fix.
Reduce node semantics support post operations which mean several ops should be executed within single kernel.
For example Reduce + Add. It means we shouln't only fill ouput with correct default value, but also apply Add oepration to it.
@xuchen-intel Could you please take a look? Would appreciate if you can help to finilize the fix on CPU side.

[xuchen-intel]

@dmitry-gorokhov I agree. If the expected output shape is non-empty, then we should apply post ops fusion.
@mitruska Could you please add reduce_kernel_post_process(dst_data); before the new added early return?

[xuchen-intel]

An alternative way would be to disable post ops fusion here if input is empty, which would be more safe. After all this is a corner case to reduce dim size 0 to 1, any incompatibility to the variants of such case from the computation module would lead to incorrect/undefined results.

[dmitry-gorokhov]

An alternative way would be to disable post ops fusion here if input is empty, which would be more safe. After all this is a corner case to reduce dim size 0 to 1, any incompatibility to the variants of such case from the computation module would lead to incorrect/undefined results.

@xuchen-intel Shapes might be dynamic. So we don't know in advance should the fusion be disabled or not.

[xuchen-intel]

Right! Then @mitruska please let's add reduce_kernel_post_process(dst_data);, with some test cases (e.g. here) to reproduce this issue beforehand if possible.
Applying reduce_kernel_post_process(dst_data); should work for post ops cases. Otherwise, please let me know, if you need me to fix it in a separate PR for the CPU part. @dmitry-gorokhov

[mitruska]

@dmitry-gorokhov, @xuchen-intel Thank you for your support! I've added the reduce_kernel_post_process(dst_data), but it needed some special conditions to make the tests passed.

The condition:

openvino/src/plugins/intel_cpu/src/nodes/reduce.cpp

Lines 2281 to 2283 in 99c1e89

	const bool skip_post_process = getAlgorithm() == Algorithm::ReduceMean || attr.get()->post_ops_.len() == 0;
	if (!skip_post_process) {
	reduce_kernel_post_process(dst_data);

was inspired by:

openvino/src/plugins/intel_cpu/src/nodes/reduce.cpp

Lines 2564 to 2565 in 99c1e89

	apply_division = getAlgorithm() == Algorithm::ReduceMean && attr.get()->post_ops_.len() == 0;
	apply_post_kernel = !apply_division;

Moreover I was able to use only fuse "Swish" CPU tests, because other fusion tests pipeline was failing for 0 and dynamic cases. Those cases probably need some fusing test pipeline adjustment.

[praasz]

LGTM core part.

[mitruska]

Agreed with CPU maintainers that the work will be continued within a separate PR.