<QUESTION> Does llama3 add <EOS> token when do pretraining?

[bugm]

Hello,
I noticed that the llama3 tokenizer loaded with hf transformers.AutoTokenizer only add a token when call the encode function. May I ask during llama3 pretraining, which behavior is taken? only add token or add both and tokens for each training document.