Purpose of unks in skipgram function

[mmlynarik]

Hi,

Can you please explain, what is the purpose of including the <UNK> tokens in the owords vector produced by skipgram function? What should model learn by using these as training examples?

Also, what is the purpose of variable ws in train function, if it's not used anywhere after its definition?

[theeluwin]

<UNK> means unknown token. This should be defined in the preprocessing step. ws means word score, and it is calculated heuristically, so you can use it if you want, or not.

[mmlynarik]

Hi, yes, I understand what <UNK> means. What I don't understand is why these dummy tokens (btw, I suppose they probably were meant to represent padding and not unknown words) are included in the output of skipgram function. I checked the original implementation of Word2Vec and I can't see there this step of including padding nor unknown tokens. What information should model learn from them being included as training examples?