-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fixed Llama 2 to 3.2 NBs #388
Conversation
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
About RoPe, I cannot follow the code implementation compared to the formula. This also applies to converting-gpt-to-llama2.ipynb where I think it would be beneficial to add the RoPE formula too. |
Thanks for these refinements! Regarding
Yeah, the formula is only shown to highlight the "theta" and "base" aspects but the one formula is not enough to follow the implementation of the complete RoPE method. I was originally thinking of explaining RoPE here, but this would be better for a separate article because it's a whole topic (or even paper) in itself. |
Alright, thanks! |
-2
)safetensors
missing,ipywidgets
for HF progress bar)local_dir
for llama 3.1 and 3.2 files