Low BLEU on PBSMT

[kellymarchisio]

I ran PBSMT for en-fr but only got a BLEU score of ~9.5 instead of the ~13.5 mentioned in the readme
kmarc@c16:~/experiment/unsup/UnsupervisedMT/PBSMT/moses_train_en-fr$ cat eval.t* BLEU = 9.45, 46.1/16.0/6.6/2.9 (BP=0.869, ratio=0.877, hyp_len=71143, ref_len=81098) BLEU = 9.60, 46.9/16.2/6.7/2.9 (BP=0.869, ratio=0.877, hyp_len=71143, ref_len=81098)

The only thing I can think of is that downloading the semeval2017 data for MUSE failed since http://alt.qcri.org is down. Could this have caused my considerably depreciated BLEU, and is there another source of this data?

[kellymarchisio]

Update: Seems multiple files for MUSE weren't downloaded because I was rate-limited. I'll reopen the issue after I debug.

[kellymarchisio]

OK, turns out there are some access issues with the data requested in MUSE's get_evaluation.sh. I'll file the issue with MUSE and retry PBSMT training.

[kellymarchisio]

I've downloaded the data and retrained the system, and I get the same BLEU scores. Do you have any indication why this could be? I simply ran run.sh