MoE gate logging #4907
LorenzoBoccaccia
started this conversation in
Ideas
MoE gate logging
#4907
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
would it be possible log gate layer model decision to understand what percentage of each expert is responsible for each token?
Beta Was this translation helpful? Give feedback.
All reactions