-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make Enzyme the only compatible AD engine #364
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #364 +/- ##
==========================================
- Coverage 92.50% 89.19% -3.31%
==========================================
Files 30 30
Lines 1574 1583 +9
==========================================
- Hits 1456 1412 -44
- Misses 118 171 +53 ☔ View full report in Codecov by Sentry. |
…de.jl into ptiede-enzymeswitch
I am going to merge this now. All the Enzyme issues either have rules now to get around them or utilize a ChainRulesRule |
Ok so from testing, Enzyme looks ready to go for almost everything! Almost all of the examples are switched over to Enzyme and some of the testing still needs to be patched.
Before merging the following issues/PR's will need to be solved
unsafe_execute
EnzymeAD/Enzyme.jl#1717Now for the exciting part I am seeing pretty large performance boosts both for the forward and reverse passes. The forward pass is faster because Enzyme makes me able to do a lot more in place and revert some coding styles that were needed to get Zygote to be somewhat efficient.
For the reverse pass before this PR I would typically see reverse pass speeds that were 4-6x slower than the forward pass. After this PR it is typically between 2-4x. The biggest gains are really for polarized modeling.
This will be a breaking change however so this will push a new Comrade version (sorry people)