Switch to ICU tokenizer #939
Open
firefoxci-taskcluster / clean-corpus-opus-ada83_v1-ru-en
succeeded
Nov 22, 2024 in 5m 41s
FirefoxCI (pull_request)
Clean opus ada83_v1 dataset ru-en use OpusCleaner true
Details
View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster
Task Status
Started: 2024-11-22T22:55:01.482Z
Resolved: 2024-11-22T22:56:12.325Z
Task Execution Time: 1 minute, 10 seconds, 843 milliseconds
Task Status: completed
Reason Resolved: completed
RunId: 0
Artifacts
- public/build/ada83_v1.en.zst
- public/build/ada83_v1.ru-en.filters.json
- public/build/ada83_v1.ru.zst
- public/logs/live_backing.log
- public/logs/live.log
[taskcluster 2024-11-22 22:55:01.573Z] Task ID: R12XdGPpRoCqwNhtpLocRw
[taskcluster 2024-11-22 22:55:01.573Z] Worker ID: 4891173128478963054
[taskcluster 2024-11-22 22:55:01.573Z] Worker Group: us-central1-b
[taskcluster 2024-11-22 22:55:01.573Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2024-11-22 22:55:01.573Z] Worker Pool: translations-1/b-linux-large-gcp-300gb
[taskcluster 2024-11-22 22:55:01.573Z] Worker Version: 38.0.5
[taskcluster 2024-11-22 22:55:01.573Z] Public IP: 34.56.6.144
[taskcluster 2024-11-22 22:55:01.573Z] Hostname: translations-1-b-linux-large-gcp-300gb-nebgdu5etayv9h0ofgiocq
[taskcluster 2024-11-22 22:55:01.573Z] using cache "translations-level-1-checkouts-v3-7afeb851dd97df8f3607-KnyIE1GvSz67R9mjL97Now" -> /builds/worker/checkouts
[taskcluster 2024-11-22 22:55:04.867Z] Downloading artifact "public/image.tar.zst" from task ID: KnyIE1GvSz67R9mjL97Now.
[taskcluster 2024-11-22 22:55:09.870Z] Download Progress: 90.53%
[taskcluster 2024-11-22 22:55:10.416Z] Downloaded artifact successfully.
[taskcluster 2024-11-22 22:55:10.416Z] Downloaded 287.207 mb
[taskcluster 2024-11-22 22:55:10.417Z] Decompressing downloaded image
[taskcluster 2024-11-22 22:55:12.473Z] Loading docker image from downloaded archive.
[taskcluster 2024-11-22 22:55:26.870Z] Image 'public/image.tar.zst' from task 'KnyIE1GvSz67R9mjL97Now' loaded. Using image ID sha256:d31e1900b8212f46ff27eab4217df610f5d7a124bb4975b4b8ea07a64443f3ba.
[taskcluster 2024-11-22 22:55:27.039Z] === Task Starting ===
[setup 2024-11-22T22:55:36.631Z] run-task started in /builds/worker
[setup 2024-11-22T22:55:36.631Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ -- bash -c pip install -r $VCS_PATH/pipeline/clean/requirements/clean.txt && if [ ${USE_OPUSCLEANER} == "true" ]; then dir="clean/opuscleaner"; else dir="clean"; fi && $VCS_PATH/pipeline/${dir}/clean-corpus.sh $MOZ_FETCHES_DIR/ada83_v1 $TASK_WORKDIR/artifacts/ada83_v1 auto opus_ada83/v1 ${OPUSCLEANER_MODE} 2>&1
...(2606 lines hidden)...
[task 2024-11-22T22:56:03.693Z] 120550K .......... .......... .......... .......... .......... 94% 230M 0s
[task 2024-11-22T22:56:03.693Z] 120600K .......... .......... .......... .......... .......... 94% 193M 0s
[task 2024-11-22T22:56:03.693Z] 120650K .......... .......... .......... .......... .......... 94% 223M 0s
[task 2024-11-22T22:56:03.693Z] 120700K .......... .......... .......... .......... .......... 94% 236M 0s
[task 2024-11-22T22:56:03.694Z] 120750K .......... .......... .......... .......... .......... 94% 242M 0s
[task 2024-11-22T22:56:03.694Z] 120800K .......... .......... .......... .......... .......... 94% 213M 0s
[task 2024-11-22T22:56:03.694Z] 120850K .......... .......... .......... .......... .......... 94% 239M 0s
[task 2024-11-22T22:56:03.694Z] 120900K .......... .......... .......... .......... .......... 94% 235M 0s
[task 2024-11-22T22:56:03.695Z] 120950K .......... .......... .......... .......... .......... 94% 222M 0s
[task 2024-11-22T22:56:03.695Z] 121000K .......... .......... .......... .......... .......... 94% 198M 0s
[task 2024-11-22T22:56:03.695Z] 121050K .......... .......... .......... .......... .......... 94% 232M 0s
[task 2024-11-22T22:56:03.695Z] 121100K .......... .......... .......... .......... .......... 94% 223M 0s
[task 2024-11-22T22:56:03.695Z] 121150K .......... .......... .......... .......... .......... 94% 232M 0s
[task 2024-11-22T22:56:03.696Z] 121200K .......... .......... .......... .......... .......... 94% 212M 0s
[task 2024-11-22T22:56:03.696Z] 121250K .......... .......... .......... .......... .......... 94% 240M 0s
[task 2024-11-22T22:56:03.696Z] 121300K .......... .......... .......... .......... .......... 94% 233M 0s
[task 2024-11-22T22:56:03.696Z] 121350K .......... .......... .......... .......... .......... 94% 231M 0s
[task 2024-11-22T22:56:03.696Z] 121400K .......... .......... .......... .......... .......... 94% 209M 0s
[task 2024-11-22T22:56:03.697Z] 121450K .......... .......... .......... .......... .......... 94% 234M 0s
[task 2024-11-22T22:56:03.697Z] 121500K .......... .......... .......... .......... .......... 94% 240M 0s
[task 2024-11-22T22:56:03.697Z] 121550K .......... .......... .......... .......... .......... 94% 245M 0s
[task 2024-11-22T22:56:03.697Z] 121600K .......... .......... .......... .......... .......... 94% 218M 0s
[task 2024-11-22T22:56:03.697Z] 121650K .......... .......... .......... .......... .......... 94% 249M 0s
[task 2024-11-22T22:56:03.698Z] 121700K .......... .......... .......... .......... .......... 94% 239M 0s
[task 2024-11-22T22:56:03.698Z] 121750K .......... .......... .......... .......... .......... 95% 245M 0s
[task 2024-11-22T22:56:03.698Z] 121800K .......... .......... .......... .......... .......... 95% 204M 0s
[task 2024-11-22T22:56:03.698Z] 121850K .......... .......... .......... .......... .......... 95% 230M 0s
[task 2024-11-22T22:56:03.699Z] 121900K .......... .......... .......... .......... .......... 95% 234M 0s
[task 2024-11-22T22:56:03.699Z] 121950K .......... .......... .......... .......... .......... 95% 242M 0s
[task 2024-11-22T22:56:03.699Z] 122000K .......... .......... .......... .......... .......... 95% 202M 0s
[task 2024-11-22T22:56:03.699Z] 122050K .......... .......... .......... .......... .......... 95% 238M 0s
[task 2024-11-22T22:56:03.699Z] 122100K .......... .......... .......... .......... .......... 95% 237M 0s
[task 2024-11-22T22:56:03.700Z] 122150K .......... .......... .......... .......... .......... 95% 233M 0s
[task 2024-11-22T22:56:03.700Z] 122200K .......... .......... .......... .......... .......... 95% 202M 0s
[task 2024-11-22T22:56:03.700Z] 122250K .......... .......... .......... .......... .......... 95% 239M 0s
[task 2024-11-22T22:56:03.700Z] 122300K .......... .......... .......... .......... .......... 95% 232M 0s
[task 2024-11-22T22:56:03.701Z] 122350K .......... .......... .......... .......... .......... 95% 239M 0s
[task 2024-11-22T22:56:03.701Z] 122400K .......... .......... .......... .......... .......... 95% 220M 0s
[task 2024-11-22T22:56:03.701Z] 122450K .......... .......... .......... .......... .......... 95% 256M 0s
[task 2024-11-22T22:56:03.701Z] 122500K .......... .......... .......... .......... .......... 95% 240M 0s
[task 2024-11-22T22:56:03.701Z] 122550K .......... .......... .......... .......... .......... 95% 241M 0s
[task 2024-11-22T22:56:03.702Z] 122600K .......... .......... .......... .......... .......... 95% 201M 0s
[task 2024-11-22T22:56:03.702Z] 122650K .......... .......... .......... .......... .......... 95% 251M 0s
[task 2024-11-22T22:56:03.702Z] 122700K .......... .......... .......... .......... .......... 95% 244M 0s
[task 2024-11-22T22:56:03.702Z] 122750K .......... .......... .......... .......... .......... 95% 225M 0s
[task 2024-11-22T22:56:03.702Z] 122800K .......... .......... .......... .......... .......... 95% 212M 0s
[task 2024-11-22T22:56:03.703Z] 122850K .......... .......... .......... .......... .......... 95% 241M 0s
[task 2024-11-22T22:56:03.703Z] 122900K .......... .......... .......... .......... .......... 95% 260M 0s
[task 2024-11-22T22:56:03.703Z] 122950K .......... .......... .......... .......... .......... 95% 239M 0s
[task 2024-11-22T22:56:03.703Z] 123000K .......... .......... .......... .......... .......... 95% 206M 0s
[task 2024-11-22T22:56:03.703Z] 123050K .......... .......... .......... .......... .......... 96% 254M 0s
[task 2024-11-22T22:56:03.704Z] 123100K .......... .......... .......... .......... .......... 96% 245M 0s
[task 2024-11-22T22:56:03.704Z] 123150K .......... .......... .......... .......... .......... 96% 252M 0s
[task 2024-11-22T22:56:03.704Z] 123200K .......... .......... .......... .......... .......... 96% 223M 0s
[task 2024-11-22T22:56:03.704Z] 123250K .......... .......... .......... .......... .......... 96% 231M 0s
[task 2024-11-22T22:56:03.704Z] 123300K .......... .......... .......... .......... .......... 96% 235M 0s
[task 2024-11-22T22:56:03.705Z] 123350K .......... .......... .......... .......... .......... 96% 246M 0s
[task 2024-11-22T22:56:03.705Z] 123400K .......... .......... .......... .......... .......... 96% 89.8M 0s
[task 2024-11-22T22:56:03.707Z] 123450K .......... .......... .......... .......... .......... 96% 29.5M 0s
[task 2024-11-22T22:56:03.707Z] 123500K .......... .......... .......... .......... .......... 96% 213M 0s
[task 2024-11-22T22:56:03.707Z] 123550K .......... .......... .......... .......... .......... 96% 233M 0s
[task 2024-11-22T22:56:03.707Z] 123600K .......... .......... .......... .......... .......... 96% 214M 0s
[task 2024-11-22T22:56:03.708Z] 123650K .......... .......... .......... .......... .......... 96% 57.0M 0s
[task 2024-11-22T22:56:03.709Z] 123700K .......... .......... .......... .......... .......... 96% 217M 0s
[task 2024-11-22T22:56:03.709Z] 123750K .......... .......... .......... .......... .......... 96% 228M 0s
[task 2024-11-22T22:56:03.710Z] 123800K .......... .......... .......... .......... .......... 96% 37.3M 0s
[task 2024-11-22T22:56:03.710Z] 123850K .......... .......... .......... .......... .......... 96% 149M 0s
[task 2024-11-22T22:56:03.711Z] 123900K .......... .......... .......... .......... .......... 96% 230M 0s
[task 2024-11-22T22:56:03.711Z] 123950K .......... .......... .......... .......... .......... 96% 228M 0s
[task 2024-11-22T22:56:03.711Z] 124000K .......... .......... .......... .......... .......... 96% 156M 0s
[task 2024-11-22T22:56:03.711Z] 124050K .......... .......... .......... .......... .......... 96% 226M 0s
[task 2024-11-22T22:56:03.712Z] 124100K .......... .......... .......... .......... .......... 96% 246M 0s
[task 2024-11-22T22:56:03.712Z] 124150K .......... .......... .......... .......... .......... 96% 249M 0s
[task 2024-11-22T22:56:03.712Z] 124200K .......... .......... .......... .......... .......... 96% 190M 0s
[task 2024-11-22T22:56:03.712Z] 124250K .......... .......... .......... .......... .......... 96% 222M 0s
[task 2024-11-22T22:56:03.712Z] 124300K .......... .......... .......... .......... .......... 97% 252M 0s
[task 2024-11-22T22:56:03.713Z] 124350K .......... .......... .......... .......... .......... 97% 227M 0s
[task 2024-11-22T22:56:03.713Z] 124400K .......... .......... .......... .......... .......... 97% 216M 0s
[task 2024-11-22T22:56:03.714Z] 124450K .......... .......... .......... .......... .......... 97% 73.8M 0s
[task 2024-11-22T22:56:03.714Z] 124500K .......... .......... .......... .......... .......... 97% 230M 0s
[task 2024-11-22T22:56:03.715Z] 124550K .......... .......... .......... .......... .......... 97% 243M 0s
[task 2024-11-22T22:56:03.715Z] 124600K .......... .......... .......... .......... .......... 97% 53.6M 0s
[task 2024-11-22T22:56:03.715Z] 124650K .......... .......... .......... .......... .......... 97% 79.1M 0s
[task 2024-11-22T22:56:03.716Z] 124700K .......... .......... .......... .......... .......... 97% 230M 0s
[task 2024-11-22T22:56:03.716Z] 124750K .......... .......... .......... .......... .......... 97% 106M 0s
[task 2024-11-22T22:56:03.716Z] 124800K .......... .......... .......... .......... .......... 97% 215M 0s
[task 2024-11-22T22:56:03.717Z] 124850K .......... .......... .......... .......... .......... 97% 121M 0s
[task 2024-11-22T22:56:03.717Z] 124900K .......... .......... .......... .......... .......... 97% 242M 0s
[task 2024-11-22T22:56:03.718Z] 124950K .......... .......... .......... .......... .......... 97% 33.9M 0s
[task 2024-11-22T22:56:03.719Z] 125000K .......... .......... .......... .......... .......... 97% 192M 0s
[task 2024-11-22T22:56:03.719Z] 125050K .......... .......... .......... .......... .......... 97% 233M 0s
[task 2024-11-22T22:56:03.719Z] 125100K .......... .......... .......... .......... .......... 97% 223M 0s
[task 2024-11-22T22:56:03.719Z] 125150K .......... .......... .......... .......... .......... 97% 219M 0s
[task 2024-11-22T22:56:03.720Z] 125200K .......... .......... .......... .......... .......... 97% 207M 0s
[task 2024-11-22T22:56:03.720Z] 125250K .......... .......... .......... .......... .......... 97% 240M 0s
[task 2024-11-22T22:56:03.720Z] 125300K .......... .......... .......... .......... .......... 97% 227M 0s
[task 2024-11-22T22:56:03.720Z] 125350K .......... .......... .......... .......... .......... 97% 224M 0s
[task 2024-11-22T22:56:03.720Z] 125400K .......... .......... .......... .......... .......... 97% 194M 0s
[task 2024-11-22T22:56:03.721Z] 125450K .......... .......... .......... .......... .......... 97% 231M 0s
[task 2024-11-22T22:56:03.721Z] 125500K .......... .......... .......... .......... .......... 97% 223M 0s
[task 2024-11-22T22:56:03.721Z] 125550K .......... .......... .......... .......... .......... 97% 240M 0s
[task 2024-11-22T22:56:03.721Z] 125600K .......... .......... .......... .......... .......... 98% 211M 0s
[task 2024-11-22T22:56:03.722Z] 125650K .......... .......... .......... .......... .......... 98% 243M 0s
[task 2024-11-22T22:56:03.722Z] 125700K .......... .......... .......... .......... .......... 98% 238M 0s
[task 2024-11-22T22:56:03.722Z] 125750K .......... .......... .......... .......... .......... 98% 224M 0s
[task 2024-11-22T22:56:03.722Z] 125800K .......... .......... .......... .......... .......... 98% 195M 0s
[task 2024-11-22T22:56:03.722Z] 125850K .......... .......... .......... .......... .......... 98% 238M 0s
[task 2024-11-22T22:56:03.723Z] 125900K .......... .......... .......... .......... .......... 98% 240M 0s
[task 2024-11-22T22:56:03.723Z] 125950K .......... .......... .......... .......... .......... 98% 246M 0s
[task 2024-11-22T22:56:03.723Z] 126000K .......... .......... .......... .......... .......... 98% 201M 0s
[task 2024-11-22T22:56:03.723Z] 126050K .......... .......... .......... .......... .......... 98% 223M 0s
[task 2024-11-22T22:56:03.723Z] 126100K .......... .......... .......... .......... .......... 98% 232M 0s
[task 2024-11-22T22:56:03.724Z] 126150K .......... .......... .......... .......... .......... 98% 258M 0s
[task 2024-11-22T22:56:03.724Z] 126200K .......... .......... .......... .......... .......... 98% 218M 0s
[task 2024-11-22T22:56:03.724Z] 126250K .......... .......... .......... .......... .......... 98% 229M 0s
[task 2024-11-22T22:56:03.724Z] 126300K .......... .......... .......... .......... .......... 98% 241M 0s
[task 2024-11-22T22:56:03.725Z] 126350K .......... .......... .......... .......... .......... 98% 252M 0s
[task 2024-11-22T22:56:03.725Z] 126400K .......... .......... .......... .......... .......... 98% 203M 0s
[task 2024-11-22T22:56:03.725Z] 126450K .......... .......... .......... .......... .......... 98% 253M 0s
[task 2024-11-22T22:56:03.725Z] 126500K .......... .......... .......... .......... .......... 98% 239M 0s
[task 2024-11-22T22:56:03.725Z] 126550K .......... .......... .......... .......... .......... 98% 251M 0s
[task 2024-11-22T22:56:03.726Z] 126600K .......... .......... .......... .......... .......... 98% 209M 0s
[task 2024-11-22T22:56:03.726Z] 126650K .......... .......... .......... .......... .......... 98% 125M 0s
[task 2024-11-22T22:56:03.726Z] 126700K .......... .......... .......... .......... .......... 98% 224M 0s
[task 2024-11-22T22:56:03.726Z] 126750K .......... .......... .......... .......... .......... 98% 234M 0s
[task 2024-11-22T22:56:03.727Z] 126800K .......... .......... .......... .......... .......... 98% 215M 0s
[task 2024-11-22T22:56:03.727Z] 126850K .......... .......... .......... .......... .......... 98% 236M 0s
[task 2024-11-22T22:56:03.727Z] 126900K .......... .......... .......... .......... .......... 99% 209M 0s
[task 2024-11-22T22:56:03.727Z] 126950K .......... .......... .......... .......... .......... 99% 222M 0s
[task 2024-11-22T22:56:03.728Z] 127000K .......... .......... .......... .......... .......... 99% 91.5M 0s
[task 2024-11-22T22:56:03.728Z] 127050K .......... .......... .......... .......... .......... 99% 234M 0s
[task 2024-11-22T22:56:03.728Z] 127100K .......... .......... .......... .......... .......... 99% 232M 0s
[task 2024-11-22T22:56:03.728Z] 127150K .......... .......... .......... .......... .......... 99% 245M 0s
[task 2024-11-22T22:56:03.729Z] 127200K .......... .......... .......... .......... .......... 99% 65.0M 0s
[task 2024-11-22T22:56:03.729Z] 127250K .......... .......... .......... .......... .......... 99% 231M 0s
[task 2024-11-22T22:56:03.730Z] 127300K .......... .......... .......... .......... .......... 99% 226M 0s
[task 2024-11-22T22:56:03.730Z] 127350K .......... .......... .......... .......... .......... 99% 243M 0s
[task 2024-11-22T22:56:03.730Z] 127400K .......... .......... .......... .......... .......... 99% 115M 0s
[task 2024-11-22T22:56:03.731Z] 127450K .......... .......... .......... .......... .......... 99% 152M 0s
[task 2024-11-22T22:56:03.731Z] 127500K .......... .......... .......... .......... .......... 99% 154M 0s
[task 2024-11-22T22:56:03.731Z] 127550K .......... .......... .......... .......... .......... 99% 234M 0s
[task 2024-11-22T22:56:03.732Z] 127600K .......... .......... .......... .......... .......... 99% 51.8M 0s
[task 2024-11-22T22:56:03.732Z] 127650K .......... .......... .......... .......... .......... 99% 225M 0s
[task 2024-11-22T22:56:03.732Z] 127700K .......... .......... .......... .......... .......... 99% 237M 0s
[task 2024-11-22T22:56:03.733Z] 127750K .......... .......... .......... .......... .......... 99% 226M 0s
[task 2024-11-22T22:56:03.733Z] 127800K .......... .......... .......... .......... .......... 99% 67.5M 0s
[task 2024-11-22T22:56:03.734Z] 127850K .......... .......... .......... .......... .......... 99% 184M 0s
[task 2024-11-22T22:56:03.734Z] 127900K .......... .......... .......... .......... .......... 99% 236M 0s
[task 2024-11-22T22:56:03.735Z] 127950K .......... .......... .......... .......... .......... 99% 50.7M 0s
[task 2024-11-22T22:56:03.735Z] 128000K .......... .......... .......... .......... .......... 99% 194M 0s
[task 2024-11-22T22:56:03.735Z] 128050K .......... .......... .......... .......... .......... 99% 220M 0s
[task 2024-11-22T22:56:03.736Z] 128100K .......... .......... .......... .......... .......... 99% 241M 0s
[task 2024-11-22T22:56:03.736Z] 128150K .......... .......... .......... ......... 100% 224M=0.9s
[task 2024-11-22T22:56:03.736Z]
[task 2024-11-22T22:56:03.736Z] 2024-11-22 22:56:03 (134 MB/s) - ‘/builds/worker/.local/lib/python3.10/site-packages/opuscleaner/filters/large.bin’ saved [131266198/131266198]
[task 2024-11-22T22:56:03.736Z]
[task 2024-11-22T22:56:03.737Z] + echo '### Generating cleaning config: opus_ada83/v1.ru-en.filters.json'
[task 2024-11-22T22:56:03.737Z] ### Generating cleaning config: opus_ada83/v1.ru-en.filters.json
[task 2024-11-22T22:56:03.737Z] + filter_path=/builds/worker/artifacts/ada83_v1.ru-en.filters.json
[task 2024-11-22T22:56:03.737Z] + python3 generate_filters.py /builds/worker/fetches/ada83_v1 ru en opus_ada83/v1 /builds/worker/artifacts/ada83_v1.ru-en.filters.json custom
[task 2024-11-22T22:56:03.766Z] Using filter /builds/worker/checkouts/vcs/pipeline/clean/opuscleaner/configs/default.filters.json
[task 2024-11-22T22:56:03.770Z] + test -s /builds/worker/artifacts/ada83_v1.ru-en.filters.json
[task 2024-11-22T22:56:03.770Z] + echo '### Cleaning /builds/worker/fetches/ada83_v1 with filter /builds/worker/artifacts/ada83_v1.ru-en.filters.json'
[task 2024-11-22T22:56:03.770Z] ### Cleaning /builds/worker/fetches/ada83_v1 with filter /builds/worker/artifacts/ada83_v1.ru-en.filters.json
[task 2024-11-22T22:56:03.771Z] + opuscleaner-clean --parallel 32 --batch-size=50000 --input=- /builds/worker/artifacts/ada83_v1.ru-en.filters.json ru en
[task 2024-11-22T22:56:03.771Z] ++ zstdmt -dc /builds/worker/fetches/ada83_v1.ru.zst
[task 2024-11-22T22:56:03.771Z] + cut -f2
[task 2024-11-22T22:56:03.771Z] + paste /dev/fd/63 /dev/fd/62
[task 2024-11-22T22:56:03.771Z] + tee /dev/fd/63
[task 2024-11-22T22:56:03.771Z] ++ zstdmt -dc /builds/worker/fetches/ada83_v1.en.zst
[task 2024-11-22T22:56:03.771Z] + zstdmt
[task 2024-11-22T22:56:03.771Z] ++ cut -f1
[task 2024-11-22T22:56:03.772Z] ++ zstdmt
[task 2024-11-22T22:56:04.659Z] + echo '### Checking length of the files'
[task 2024-11-22T22:56:04.659Z] ### Checking length of the files
[task 2024-11-22T22:56:04.659Z] + test -s /builds/worker/artifacts/ada83_v1.ru.zst
[task 2024-11-22T22:56:04.659Z] + test -s /builds/worker/artifacts/ada83_v1.en.zst
[task 2024-11-22T22:56:04.659Z] ++ zstdmt -dc /builds/worker/artifacts/ada83_v1.ru.zst
[task 2024-11-22T22:56:04.660Z] ++ wc -l
[task 2024-11-22T22:56:04.663Z] + new_len_src=3988
[task 2024-11-22T22:56:04.664Z] ++ zstdmt -dc /builds/worker/artifacts/ada83_v1.en.zst
[task 2024-11-22T22:56:04.664Z] ++ wc -l
[task 2024-11-22T22:56:04.667Z] + new_len_trg=3988
[task 2024-11-22T22:56:04.667Z] ++ zstdmt -dc /builds/worker/fetches/ada83_v1.ru.zst
[task 2024-11-22T22:56:04.667Z] ++ wc -l
[task 2024-11-22T22:56:04.671Z] + orig_len_src=4122
[task 2024-11-22T22:56:04.671Z] + [[ 3988 -ge 1 ]]
[task 2024-11-22T22:56:04.671Z] + [[ 3988 -ge 1 ]]
[task 2024-11-22T22:56:04.671Z] + [[ 3988 = \3\9\8\8 ]]
[task 2024-11-22T22:56:04.671Z] + echo '### Filtered length: 3988 / 4122'
[task 2024-11-22T22:56:04.671Z] ### Filtered length: 3988 / 4122
[task 2024-11-22T22:56:04.671Z] + echo '### Clean /builds/worker/fetches/ada83_v1 is written to /builds/worker/artifacts/ada83_v1'
[task 2024-11-22T22:56:04.671Z] ### Clean /builds/worker/fetches/ada83_v1 is written to /builds/worker/artifacts/ada83_v1
[task 2024-11-22T22:56:04.671Z] + echo '###### Done: Cleaning corpus with OpusCleaner'
[task 2024-11-22T22:56:04.671Z] ###### Done: Cleaning corpus with OpusCleaner
[fetches 2024-11-22T22:56:04.671Z] removing /builds/worker/fetches
[fetches 2024-11-22T22:56:04.672Z] finished
[taskcluster 2024-11-22 22:56:10.972Z] === Task Finished ===
[taskcluster 2024-11-22 22:56:11.690Z] Successful task run with exit code: 0 completed in 70.118 seconds
Loading