Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

memory out of error. Hardware requirements #68

Open
Kushalamummigatti opened this issue Jun 16, 2023 · 1 comment
Open

memory out of error. Hardware requirements #68

Kushalamummigatti opened this issue Jun 16, 2023 · 1 comment

Comments

@Kushalamummigatti
Copy link

I have tried to finetune Codegen 2B Mono on 40GB GPU (single card) with sequence length set as 256. It gave CUDA memory out of error. What is the GPU memory required to finetune 2B and above models?
@Fusseldieb
Copy link

Fusseldieb commented Jul 24, 2023 •
edited
Loading

Seems like an issue on your side. A 7Billion parameter model fits on a 12GB GPU (the 7B 4-Bit one a 8GB GPU).

The 2B model should fit on a 6GB card just fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Fusseldieb @Kushalamummigatti