remove cpu restriction for bnb training #3062
Conversation
|
Would it make sense to add a version check on bitsandbytes before exempting CPU usage from raising, for those users who are still using older versions? |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Hi @BenjaminBossan , thanks for your review, I have fixed it. As transformers has supported CPU BNB path. I think we can start to review this PR. |
src/accelerate/accelerator.py
Outdated
| try: | ||
| from transformers.utils import is_bitsandbytes_multi_backend_available | ||
| bnb_multi_backends = is_bitsandbytes_multi_backend_available() | ||
| except ImportError: | ||
| bnb_multi_backends = False |
There was a problem hiding this comment.
Let's not use the transformers import in accelerate. You can just create the same function in accelerate. This way, we don't need to have the try/catch also.
|
Hi @SunMarc . Thanks, can you merge it since all tests passed? |
|
Oh just a small nit before merging ! Also, I don't think this is specific to 8-bit training, |
SunMarc
changed the title
Yes, I have added 4-bit in the log. |
Please let me know if there is anything I need to change, thx! |
|
Nothing ! I was just waiting the CI to finish. Merging ! |
Refer to cpu_bnb, we will support bnb 8bit and 4bit training on CPU.