Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mismatch in -valid_src and -valid_tgt in the provided training script for forward and backward tasks #8

Open
EronHou opened this issue Dec 12, 2024 · 0 comments

Comments

@EronHou
Copy link

EronHou commented Dec 12, 2024

Hello,

Thank you for sharing this great repository. I have been following the instructions in the training script provided, but I noticed a potential mismatch in the validation dataset configuration for both the forward and backward tasks.

Forward Task:
In the onmt_preprocess command for the forward task:
-valid_src "${DATASET}/src-valid.txt" -valid_tgt "${DATASET_TRANSFER}/tgt-valid.txt"
Here, the -valid_src comes from ${DATASET}, while the -valid_tgt comes from ${DATASET_TRANSFER}. Shouldn't both src and tgt for validation come from the same dataset?

Backward Task:
Similarly, in the backward task:
-valid_src "${DATASET}/tgt-valid.txt" -valid_tgt "${DATASET_TRANSFER}/src-valid.txt"
Here, the -valid_src comes from ${DATASET}/tgt-valid.txt, and -valid_tgt comes from ${DATASET_TRANSFER}/src-valid.txt. This seems inconsistent, as the validation source and target do not match in a straightforward way.

Questions:

  1. Is this mismatch intentional? If yes, could you explain why the validation datasets for src and tgt are taken from different sources (${DATASET} and ${DATASET_TRANSFER})?
  2. If this is an error, how should we correctly set the -valid_src and -valid_tgt for both forward and backward tasks?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant