pretraining the nucleotide-transformer

[mpage21]

Hi, I would like to pre-train the model myself to gain a better understanding of machine learning models.

Specifically, could you provide the code that was used to pre-train the v2 500m multi-species model?

I am new to this so any and all help is appreciated.

thank you!

[mpage21]

Okay, I think I might have found what I'm looking for. It looks like the code in the nucleotide_transformer directory is what was used to build the models (maybe I just need to tweak it a bit?) and then I think the configurations are found here: https://huggingface.co/InstaDeepAI/nucleotide-transformer-v2-500m-multi-species/tree/main

Is this correct?

[mpage21]

Actually, I'm not sure I see a training script. Is it named something else?