Replies: 1 comment
-
I agree, I also found out about this and arrived at your same conclusion. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello, it seems to me that the formula given in notebook 5 for the surrogate gradient is wrong.
The function which approximates the Heavisde function is
and therefore the derivative wrt U should be:
Which is consistent with the backward specified in the LeakySurrogate.
Beta Was this translation helpful? Give feedback.
All reactions