Too many repeats in kineto_traces for pt2 compiled ops

[FindHao]

pt2 compiled version:

liger version:

There are many extra repeats in kineto_trace for pt2 compiled version.

[xuzhao9]

Are they from the warmup iterations? https://github.com/pytorch-labs/tritonbench/blob/main/docs/kineto_trace.md

[FindHao]

Are they from the warmup iterations? https://github.com/pytorch-labs/tritonbench/blob/main/docs/kineto_trace.md

yes. So the warmup section in kineto traces are expected? but liger's kineto trace doesn't have such warmup section.

[xuzhao9]

My mental model is that liger's kineto trace should also have warmup section. It is something weird if not.

Can I have the reproduction command line?

[FindHao]

My mental model is that liger's kineto trace should also have warmup section. It is something weird if not.

Can I have the reproduction command line?

python run.py --op kl_div --mode fwd_bwd  --precision fp32 --metrics kineto_trace --csv  --num-inputs 1

[xuzhao9]

@FindHao This is the liger kernel trace with warmup iteration profiled:

It shows that when the profile iteration starts, the warm up iterations have finished. Therefore in the kineto trace, it only shows "single iteration", because all previous warmup iterations have finished.

For PT2 and Torch, when the last iteration on CPU launches the kernel, the GPU kernels from previous warm up iterations are still running on GPU, so it looks like "too many repeats for pt2 compiled ops".

Therefore, to me it is expected behavior.