You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Jun 24, 2024. It is now read-only.
We can provide parameters during model loading, such as "vocab_only" and "tensor_only". Loading only the vocabulary can be useful in cases where only the tokenizer is needed, while loading only tensors would be suitable for certain visual models.
I'm open to hearing any suggestions or advice regarding whether it would be better to implement a separate TokenizerModel or make modifications within the existing Model. Any suggestions would be greatly appreciated.
The text was updated successfully, but these errors were encountered:
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
We can provide parameters during model loading, such as "vocab_only" and "tensor_only". Loading only the vocabulary can be useful in cases where only the tokenizer is needed, while loading only tensors would be suitable for certain visual models.
I'm open to hearing any suggestions or advice regarding whether it would be better to implement a separate TokenizerModel or make modifications within the existing Model. Any suggestions would be greatly appreciated.
The text was updated successfully, but these errors were encountered: