addressing Anna Shor's comment

Signed-off-by: Lifu Zhang <[email protected]>
tomlifu · Dec 12, 2024 · 9c526ad · 9c526ad
1 parent 097266a
commit 9c526ad
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/scripts/nlp_language_modeling/prepare_packed_ft_dataset.py b/scripts/nlp_language_modeling/prepare_packed_ft_dataset.py
@@ -55,7 +55,7 @@
 python scripts/nlp_language_modeling/prepare_packed_ft_dataset.py \
     model.data.train_ds.file_names=[/path/to/training.jsonl] \
     model.data.train_ds.max_seq_length=4096 \
-    +model.context_parallel_size=2 \
+    ++model.context_parallel_size=2 \
     +tokenizer_path=<see note 1 below> \
     +output_dir=/path/to/output_folder \
     +pack_sizes=[4096]