Inline PTX

[andrii0lomakin]

Good day.

Have you ever considered finding it useful to provide the possibility to provide inline PTX code in the same way as that is done in examples here https://github.com/sschaetz/nvidia-opencl-examples/blob/master/OpenCL/src/oclInlinePTX/inlinePTX.cl and here https://docs.nvidia.com/cuda/inline-ptx-assembly/index.html?

I am asking because, for example, support for tensor cores is absent at the moment, and it will likely be possible to squash all possible GPU performance using the given approach while still keeping the kernel's maintainability at a decent level.

P.S. I have seen PR about the addition of precompiled PTX code, but from my point of view, that noticeably decreases the maintainability of kernel code.

[jjfumero]

Hi @andrii0lomakin , the #487 PR is for internal testing. From my point of view, this API is difficult to use. We use the pre-built API for analysing potential optimisations that we will include in the compiler. We also use it in another project for internal testing as well.

Regarding the inlining of the PTX, you can still use it with the pre-built API. But a better way is to extend the JIT compiler to insert tensor operations directly in the Graal IR and generate the corresponding code from it (e.g., to PTX, SPIR-V, etc).

[andrii0lomakin]

Hi @jjfumero

. From my point of view, this API is difficult to use. We use the pre-built API for analysing potential optimisations that we will include in the compiler. We also use it in another project for internal testing as well.

Yes, that is exactly what I meant.

Regarding the inlining of the PTX, you can still use it with the pre-built API.
I am afraid that will kill any maintainability.

But a better way is to extend the JIT compiler to insert tensor operations directly in the Graal IR and generate the corresponding code from it (e.g., to PTX, SPIR-V, etc).

Yeah, I suppose you are right, even inline PTX (the same idea as code snippets in Panama) seems to be quite an error-prone approach.