Skip to content

Issues: CStanKonrad/long_llama

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

18 Open 6 Closed
18 Open 6 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Compared to RAG techniques
#25 opened Jan 8, 2024 by leezythu
It's questionable whether the context window has truly been expanded？
#24 opened Jan 3, 2024 by Vincentyua
Need clarification on token limit of input used for fine tuning
#20 opened Oct 11, 2023 by lokesh-iterate
2
0-shot long-context summarization / QA inference
#19 opened Oct 10, 2023 by shi-kejian
4
utilizing Long Llama with Mojo Framework and applying 4-bit quantization and is it possible to use flash attention 2 and your thoughts about Speculative execution for LLM
#17 opened Oct 9, 2023 by myname36
1
I have some questions
#16 opened Oct 6, 2023 by dziulatex
1
How much vram needed to finetune 3b model? Is 12gb enough?
#15 opened Sep 28, 2023 by universewill
1
FoT can only be used for pre-training, can't it be used for instruction fine-tuning?
#13 opened Sep 1, 2023 by wujiekd
How is the contrastive data pipeline implemented?
#12 opened Aug 27, 2023 by MarkYangjiayi
8
Code for zero-shot arxiv evaluation
#10 opened Jul 17, 2023 by bronyayang
1
Support for gradient_checkpointing
#9 opened Jul 13, 2023 by Richar-Du
3
About the use of rotary position coding.
#8 opened Jul 13, 2023 by tianyabanbu
2
Does each token requires KNN search during inference?
#7 opened Jul 11, 2023 by noanti
3
How's the speed droping when length get large compare with vanilla llama?
#5 opened Jul 10, 2023 by lucasjinreal
11
FoT attention and the scaling trick
#4 opened Jul 10, 2023 by StrangeTcy
3
Would LongNet be easily applied to the attention with FoT
#3 opened Jul 8, 2023 by jebarpg
1
How would you go about instruction finetuning?
#2 opened Jul 7, 2023 by jordancole21
13
Finetuning code?
#1 opened Jul 7, 2023 by StrangeTcy
7
ProTip! Adding no:label will show everything without a label.