You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sorry to bother you again. It's really a nice work! I have found two problem when I was training the main.py.
I think this code is wrong,because the unpad_input function has five outputs。
So,it will report an error when I run the training code of main.py. I guess it should be revised to 'q, indices_q, cu_seqlens_q, max_len_q, _ = unpad_input(q, mask_q)' and 'k, indices_kv, cu_seqlens_kv, max_len_kv, _ = unpad_input(k, mask_kv)'. Is that correct?
I found that the tokenized sequences obtained by mesh through tokenizer always have vertices 3 and 4. I counted the number of vertices 3 and 4. It even exceeds the sum of the occurrences of the other top 10 vertices. As a result, the final predicted results are all 3 and 4. Have you ever encountered such a problem during training? How to solve it?
The text was updated successfully, but these errors were encountered:
Sorry to bother you again. It's really a nice work! I have found two problem when I was training the main.py.
I think this code is wrong,because the unpad_input function has five outputs。
So,it will report an error when I run the training code of main.py. I guess it should be revised to 'q, indices_q, cu_seqlens_q, max_len_q, _ = unpad_input(q, mask_q)' and 'k, indices_kv, cu_seqlens_kv, max_len_kv, _ = unpad_input(k, mask_kv)'. Is that correct?
I found that the tokenized sequences obtained by mesh through tokenizer always have vertices 3 and 4. I counted the number of vertices 3 and 4. It even exceeds the sum of the occurrences of the other top 10 vertices. As a result, the final predicted results are all 3 and 4. Have you ever encountered such a problem during training? How to solve it?
The text was updated successfully, but these errors were encountered: