-
Notifications
You must be signed in to change notification settings - Fork 72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC-0031: ZenDNN Integration #52
base: master
Are you sure you want to change the base?
Conversation
Signed-off-by: Naveen Kumar <[email protected]>
Hi @naveenthangudu! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at [email protected]. Thanks! |
Hi @naveenthangudu See |
@SherlockNoMad, Thanks for links. |
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
1 similar comment
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
Thanks for filing this RFC. Just to connect the dots for reviewers this is a follow up to pytorch/pytorch#76244 and pytorch/pytorch#76242 +1 Sherlock's comment that integration with the new compile() and export() stack is probably a more fruitful approach than torchscript. |
@naveenthangudu this is being looked into. Also given that there are no maintainers from the core team working on it, I have no idea who could review a PR integrating a new backend there. So it is very possible for it to stay in limbo for a while :/ |
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
3 similar comments
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
Hello all, I wanted to follow up on this request. We would like to upstream AMD EPYC processor specific optimizations termed as ZenDNN to PyTorch. PyTorch 2.0 is fine. Should we file a new request for PT2.0? Kindly advise. |
cc @SherlockNoMad @ezyang for questions about torch.compile integrations |
We are resuming the integration, since CPU optimization becomes quite important for Meta internal use cases. For AMD CPU use case, at least there are two feasible paths: 1) integrate the related optimization in the triton CPU backend, or 2) we integrate ZenDNN as a backend for inductor. |
I guess we are talking about Conv/GEMM optimizations for inductor, right? I'm not sure how mature Triton CPU backend is for Conv and GEMMs. For x86 and ARM CPUs, we are leveraging oneDNN Conv and GEMMs ATen fusion ops in inductor. For AMD CPUs, perhaps, integrating AMD-specific optimizations into oneDNN is a better way to go, which can leverage existing PyTorch integration. Or, if that is harder, perhaps, integrating ZenDNN the same way as how we integrate oneDNN might work. |
We do have triton-cpu: https://github.com/triton-lang/triton-cpu. The goal is using the Inductor (AOT mode) to 1) leverage/generate high performance kernels; 2) remove the framework overhead. The model is similar to model arches like dlrm (https://github.com/facebookresearch/dlrm), not CV models like ResNet. |
If it is all about GEMMs, we also have CPP GEMM template support in inductor, for which we are actively developing: pytorch/pytorch#125683 |
Hi @houseroad According to @SherlockNoMad's suggestion, we have introduced a CPU inference extension based on torch compile for the AMD EPYC™️ series (known as zentorch). This extension combines ZenDNN and optimizes deep learning inference workloads. Our team has recently launched the first version of this extension. For more information, please visit this link: https://www.amd.com/en/developer/resources/technical-articles/supercharge-your-ai-inference-with-zendnn-on-amd-epyc-cpus.html. |
Thanks for the update (@naveenthangudu ), does this work with AOTInductor? We probably need to give it a try. |
Hi @houseroad We started a POC of using extension with export and AOT Inductor. Would you be interested in collaborating? |
Hi @naveenthangudu , sure, we would like to see if we can adopt it for Meta's use cases. |
This RFC proposes an approach for integrating ZenDNN library into PyTorch.
This integration will enable inference optimizations for deep learning workloads on AMD CPUs.