[Bug]: The prediction results using the PaddleOCR model are inconsistent between CPU and GPU inference in version 2024.04.

[Algabeno]

OpenVINO Version

2024.04

Operating System

Windows System

Device used for inference

CPU

Framework

ONNX

Model used

PadleOCR

Issue description

When using the 2024.04 version of the C++ OpenVINO framework for image inference, I found that the results vary significantly between CPU and Intel GPU. Additionally, I observed that different Intel GPUs produce different results, while the CPU results remain consistent.

model use

inference.zip

Step-by-step reproduction

Due to confidentiality requirements, I am unable to provide my code. However, I found that when I switch between CPU and GPU, the results are inconsistent.

compiled_model_ = core_.compile_model(model, "GPU", compile_options);

use intel GPU Intel® Iris® Xe Graphics eligible

use intel GPU Intel@ UHD Graphics for 10th Gen Intel@ Processors

use intel CPU

Relevant log output

No response

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[vladimir-paramuzov]

@Algabeno Could you try to run GPU with f32 precision? You need to add ov::hint::inference_precision(ov::element::f32) to model compile options. Note that GPU by default uses f16 precision, so maybe that causes difference between GPU and CPU inference results if model is sensitive to the precision of intermediate tensors

[Algabeno]

@Algabeno Could you try to run GPU with f32 precision? You need to add ov::hint::inference_precision(ov::element::f32) to model compile options. Note that GPU by default uses f16 precision, so maybe that causes difference between GPU and CPU inference results if model is sensitive to the precision of intermediate tensors

Thank you very much for your response. This issue has been successfully resolved.

[vladimir-paramuzov]

@Algabeno FYI, we have an ongoing work on activations scaling feature (#27265) which may help to improve accuracy with fp16 inference precision in some cases. Might be useful if fp32 latency is not good enough.

[Algabeno]

@Algabeno FYI, we have an ongoing work on activations scaling feature (#27265) which may help to improve accuracy with fp16 inference precision in some cases. Might be useful if fp32 latency is not good enough.

Great! I was just troubled by the fact that using FP32 makes the model's inference speed several times slower than FP16. I'll give this a try.

[Algabeno]

@Algabeno FYI, we have an ongoing work on activations scaling feature (#27265) which may help to improve accuracy with fp16 inference precision in some cases. Might be useful if fp32 latency is not good enough.

Hello, how can I use the code from (#27265)? I tried compiling the code from https://github.com/e-ddykim/openvino/tree/static_scaling, but I encountered errors during the compilation：

Error:

C:\Users\Administrator\Desktop\openvino\src\core\include\openvino\op\asinh.hpp(16,41): error C2504: “UnaryElementwiseArithmetic”: base class undefined [C:\Users\Administrator\Desktop\openvino\build\src\common\transformations\openvino_transformations_obj.vcxproj]

[vladimir-paramuzov]

@Algabeno The error doesn't seem to be related to the changes on the branch. Does the build work for ov master in your environment? If no, please check the documentation page here https://github.com/openvinotoolkit/openvino/blob/master/docs/dev/build.md

@e-ddykim FYI