Known Issue: Transformers 4.35+ Serialization Issues

[hannanjgaws]

In versions of transformers>=4.35 model.save_pretrained serializes models as safetensors instead of pytorch_model.bin files by default. This change breaks the PretrainedModel.from_pretrained and save_pretrained_split methods in transformers-neuronx, since they expect a pytorch_model.bin file.

This issue can be avoided by using an older transformers version:

pip install "transformers<4.35.0"

This limitation will be addressed in a future Neuron release.

[ernesttam]

Yes, I also bumped into this problem.

[dennj]

Hello, I made a pull request for a CR to add support for safetensors in from_pretrained().
With this change I am able to use the last version of Huggingface transformers.

aws-neuron/transformers-neuronx#62

[mrnikwaws]

Safe tensor loads are now fully supported in transformer_neuronx - closing. Please re-open if there are further issues