Revert "Fix the names of two sets of weight and bias in mcore_to_nemo…

…_mapping" (NVIDIA#11560) * Revert "Fix the names of two sets of weight and bias in mcore_to_nemo_mapping (NVIDIA#9628)" This reverts commit 6784db5. * keep underscores Signed-off-by: ashors1 <[email protected]> --------- Signed-off-by: ashors1 <[email protected]>
tomlifu · Dec 13, 2024 · 6932216 · 6932216
1 parent 849e58e
commit 6932216
Showing 1 changed file with 2 additions and 2 deletions.
diff --git a/scripts/checkpoint_converters/convert_gpt_nemo_to_mcore.py b/scripts/checkpoint_converters/convert_gpt_nemo_to_mcore.py
@@ -158,8 +158,8 @@ def build_key_mapping(nemo_cfg):
         for wb in ('weight', 'bias') if has_layernorm_bias else ('weight',):
             mcore_to_nemo_mapping.update(
                 {
-                    f"{mcore_prefix}.{i}.input_layernorm.{wb}": f"{nemo_prefix}.{i}.input_layernorm.{wb}",
-                    f"{mcore_prefix}.{i}.pre_mlp_layernorm.{wb}": f"{nemo_prefix}.{i}.post_attention_layernorm.{wb}",
+                    f"{mcore_prefix}.{i}.self_attention.linear_qkv.layer_norm_{wb}": f"{nemo_prefix}.{i}.input_layernorm.{wb}",
+                    f"{mcore_prefix}.{i}.mlp.linear_fc1.layer_norm_{wb}": f"{nemo_prefix}.{i}.post_attention_layernorm.{wb}",
                 }
             )