ImportError: cannot import name '_expand_mask' from 'transformers.models.bloom.modeling_bloom

[gameveloster]

When starting finetune_qlora.sh and using transformers==4.34.0, it crashes with the error

TypeError: forward() got an unexpected keyword argument 'padding_mask'

bitsandbytes              0.41.0
peft                      0.4.0
torch                     2.0.1
torchvision               0.15.2
transformers              4.34.0

---

Updated to transformers==4.35.0 and a new error occurs

ImportError: cannot import name '_expand_mask' from 'transformers.models.bloom.modeling_bloom'

---

Updated to transformers==4.36.0 and 4.37.0 and another error occurs

ValueError: 'llava' is already used by a Transformers config, pick another name.

Changed the named used in the AutoConfig and we are back to the same error as before:

ImportError: cannot import name '_expand_mask' from 'transformers.models.bloom.modeling_bloom'

Anyone knows how to fix this problem?

[BinZhu-ece]

I have the same problem, have you solved it? Thank you!