A bug(maybe)

[shaoyangxu]

Hi, dear ziyi~
I found in your code, the bert output weights are not set to be the same as the input embedding, which can be proved in here(In detail, the code didnt set the weight of BertLMPredictionHead.decoder to be the same as the weight of BertModel.embeddings).
I think maybe it is you who deliberately modified the source code of bert LM into this way. Why? Will it influence the final result?

[zdou0830]

Hi, yes the input and output embedding layers are not shared because I didn't define get_input/output_embeddings in BERT. This was not intentional but I found this has almost no impact on the alignment performance.