[PT2.7][Torch.complie] Performance analysis and optimization

[riverliuintel]

🚀 The feature, motivation and pitch

Analyze Triton kernels data and report to Triton XPU.

1. Recollect reasonable competitive GPU performance data
2. Use TorchInductor built-in benchmark tool to detect slower XPU triton kernels.

Alternatives

No response

Additional context

No response

[jianyizh]

scatter op issue: intel/intel-xpu-backend-for-triton#2665,
In general, should we always use fp32 instead of fp16/bf16 for atomic related ops regardless accuracy change?

[jianyizh]

layout issue

1. when number of conv is small, inductor will close layout opt. We have to force it open by TORCHINDUCTOR_FORCE_LAYOUT_OPT, otherwise we may meet inefficient kernel like cat_layernorm in this issue Some triton kernels gernerated by inductors have low efficiency on PVC 1550 compare to A100 intel-xpu-backend-for-triton#2229
2. For both xpu and cuda, when there are more nodes between conv, there will be unnecessary transpose. For example, conv (channel last) + bias and leakyrelu fusion (to channel first) + avg_pool (to channel last) + conv. It seems inductor does not propagate layout. Fuse bias and activation into conv will mitigate.
3. Currently we do not open the graph freezing feature. It should be helpful for channel last inference
4. for models like pytorch_unet that contain conv which in-channel > out-channel, inductor will choose channel first. conv takes 36.5ms out of e2e ~75ms for fp16 training. After force channel last, conv takes 15.5 ms, but batch norm become much slower. We should use channel last in this case depend on Channel last batch norm have bad performance intel-xpu-backend-for-triton#3001

[jianyizh]

RNN related ops: #1109
We should fuse it using onednn instead of using torch compile for these small ops.

[jianyizh]

sdpa pattern: #1128
We need align with cuda in https://github.com/pytorch/pytorch/blob/v2.5.1/torch/_inductor/fx_passes/fuse_attention.py

[jianyizh]

pad mm: #1129