Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to ICU tokenizer #939

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Use ICU system package

d585a63
Select commit
Loading
Failed to load commit list.
Open

Switch to ICU tokenizer #939

Use ICU system package
d585a63
Select commit
Loading
Failed to load commit list.
firefoxci-taskcluster / analyze-mono-opus-ru-tldr-pages_v2023-08-29 succeeded Nov 22, 2024 in 5m 26s

FirefoxCI (pull_request)

Analyze opus tldr-pages_v2023-08-29 dataset mono-src ru

Details

View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster

Task Status

Started: 2024-11-22T22:54:44.294Z
Resolved: 2024-11-22T22:55:57.507Z
Task Execution Time: 1 minute, 13 seconds, 213 milliseconds
Task Status: completed
Reason Resolved: completed
RunId: 0

Artifacts

- public/build/tldr-pages_v2023-08-29.ru.distribution-codepoints.png
- public/build/tldr-pages_v2023-08-29.ru.distribution-words.png
- public/logs/live_backing.log
- public/logs/live.log


[taskcluster 2024-11-22 22:54:44.355Z] Task ID: dVDYmS_ZRO2FvIX099r4mA
[taskcluster 2024-11-22 22:54:44.356Z] Worker ID: 8111982370903340398
[taskcluster 2024-11-22 22:54:44.356Z] Worker Group: us-central1-a
[taskcluster 2024-11-22 22:54:44.356Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2024-11-22 22:54:44.356Z] Worker Pool: translations-1/b-linux-large-gcp-300gb
[taskcluster 2024-11-22 22:54:44.356Z] Worker Version: 38.0.5
[taskcluster 2024-11-22 22:54:44.356Z] Public IP: 35.238.19.32
[taskcluster 2024-11-22 22:54:44.356Z] Hostname: translations-1-b-linux-large-gcp-300gb-tzr4yc2crsm0zvbyae8-cq
[taskcluster 2024-11-22 22:54:44.356Z] using cache "translations-level-1-checkouts-v3-7afeb851dd97df8f3607-KnyIE1GvSz67R9mjL97Now" -> /builds/worker/checkouts

[taskcluster 2024-11-22 22:54:47.046Z] Downloading artifact "public/image.tar.zst" from task ID: KnyIE1GvSz67R9mjL97Now.
[taskcluster 2024-11-22 22:54:51.287Z] Downloaded artifact successfully.
[taskcluster 2024-11-22 22:54:51.288Z] Downloaded 287.207 mb
[taskcluster 2024-11-22 22:54:51.289Z] Decompressing downloaded image
[taskcluster 2024-11-22 22:54:53.052Z] Loading docker image from downloaded archive.
[taskcluster 2024-11-22 22:55:25.856Z] Image 'public/image.tar.zst' from task 'KnyIE1GvSz67R9mjL97Now' loaded.  Using image ID sha256:d31e1900b8212f46ff27eab4217df610f5d7a124bb4975b4b8ea07a64443f3ba.
[taskcluster 2024-11-22 22:55:26.024Z] === Task Starting ===
[setup 2024-11-22T22:55:28.050Z] run-task started in /builds/worker
[setup 2024-11-22T22:55:28.050Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ -- bash -c pip3 install --upgrade pip setuptools && pip3 install -r $VCS_PATH/pipeline/data/requirements/analyze.txt && export PYTHONPATH=$PYTHONPATH:$VCS_PATH && python3 $VCS_PATH/pipeline/data/analyze.py --file_location $MOZ_FETCHES_DIR/tldr-pages_v2023-08-29.ru.zst --output $TASK_WORKDIR/artifacts --dataset "opus_tldr-pages/v2023-08-29" --language ru
[setup 2024-11-22T22:55:28.050Z] Python version: 3.10.12
[cache 2024-11-22T22:55:28.052Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2024-11-22T22:55:28.052Z] volume /builds/worker/checkouts is a cache
[setup 2024-11-22T22:55:28.052Z] running as worker:worker
[vcs 2024-11-22T22:55:28.053Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2024-11-22T22:55:28.054Z] executing ['git', 'clone', 'https://github.com/mozilla/translations', '/builds/worker/checkouts/vcs']
[vcs 2024-11-22T22:55:28.056Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2024-11-22T22:55:29.732Z] executing ['git', 'fetch', '--tags', '--force', 'https://github.com/mozilla/translations', 'icu_tokenizer']
[vcs 2024-11-22T22:55:29.910Z] From https://github.com/mozilla/translations
[vcs 2024-11-22T22:55:29.910Z]  * branch            icu_tokenizer -> FETCH_HEAD
[vcs 2024-11-22T22:55:29.917Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/mozilla/translations', 'icu_tokenizer']
[vcs 2024-11-22T22:55:30.081Z] From https://github.com/mozilla/translations
[vcs 2024-11-22T22:55:30.081Z]  * branch            icu_tokenizer -> FETCH_HEAD
[vcs 2024-11-22T22:55:30.088Z] executing ['git', 'checkout', '-f', '-B', 'icu_tokenizer', 'd585a63a6abc04ece83e26ce51a0caa2f7fa21e6']
[vcs 2024-11-22T22:55:30.814Z] Switched to a new branch 'icu_tokenizer'
[vcs 2024-11-22T22:55:30.834Z] executing ['git', 'submodule', 'init']
[vcs 2024-11-22T22:55:30.852Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2024-11-22T22:55:30.853Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2024-11-22T22:55:30.853Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2024-11-22T22:55:30.854Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2024-11-22T22:55:30.854Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2024-11-22T22:55:30.855Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2024-11-22T22:55:30.855Z] Submodule 'inference/3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path 'inference/3rd_party/browsermt-marian-dev'
[vcs 2024-11-22T22:55:30.856Z] Submodule 'inference/3rd_party/emsdk' (https://github.com/emscripten-core/emsdk.git) registered for path 'inference/3rd_party/emsdk'
[vcs 2024-11-22T22:55:30.856Z] Submodule 'inference/3rd_party/ssplit-cpp' (https://github.com/browsermt/ssplit-cpp) registered for path 'inference/3rd_party/ssplit-cpp'
[vcs 2024-11-22T22:55:30.857Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2024-11-22T22:55:30.877Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2024-11-22T22:55:31.927Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2024-11-22T22:55:32.141Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2024-11-22T22:55:32.433Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2024-11-22T22:55:33.008Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2024-11-22T22:55:34.349Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2024-11-22T22:55:34.775Z] Cloning into '/builds/worker/checkouts/vcs/inference/3rd_party/browsermt-marian-dev'...
[vcs 2024-11-22T22:55:35.857Z] Cloning into '/builds/worker/checkouts/vcs/inference/3rd_party/emsdk'...
[vcs 2024-11-22T22:55:36.324Z] Cloning into '/builds/worker/checkouts/vcs/inference/3rd_party/ssplit-cpp'...
[vcs 2024-11-22T22:55:36.686Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2024-11-22T22:55:36.696Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2024-11-22T22:55:36.705Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2024-11-22T22:55:36.729Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2024-11-22T22:55:36.827Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2024-11-22T22:55:36.845Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2024-11-22T22:55:36.908Z] Submodule path 'inference/3rd_party/browsermt-marian-dev': checked out '2781d735d4a10dca876d61be587afdab2726293c'
[vcs 2024-11-22T22:55:36.924Z] Submodule path 'inference/3rd_party/emsdk': checked out '2346baa7bb44a4a0571cc75f1986ab9aaa35aa03'
[vcs 2024-11-22T22:55:36.937Z] Submodule path 'inference/3rd_party/ssplit-cpp': checked out 'a311f9865ade34db1e8e080e6cc146f55dafb067'
[vcs 2024-11-22T22:55:36.937Z] cleaning git checkout...
[vcs 2024-11-22T22:55:36.937Z] executing ['git', 'clean', '-nxdff']
[vcs 2024-11-22T22:55:36.940Z] removing []
[vcs 2024-11-22T22:55:36.940Z] successfully cleaned git checkout!
[vcs 2024-11-22T22:55:36.941Z] TinderboxPrint:<a href='https://github.com/mozilla/translations/commit/d585a63a6abc04ece83e26ce51a0caa2f7fa21e6' title='Built from translations commit d585a63a6abc04ece83e26ce51a0caa2f7fa21e6'>d585a63a6abc04ece83e26ce51a0caa2f7fa21e6</a>
[setup 2024-11-22T22:55:36.941Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[fetches 2024-11-22T22:55:36.941Z] fetching artifacts
[fetches 2024-11-22T22:55:36.942Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/An7a68SXTrOfwsG2BYveCA/artifacts/public/build/tldr-pages_v2023-08-29.ru.zst to /builds/worker/fetches/tldr-pages_v2023-08-29.ru.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/An7a68SXTrOfwsG2BYveCA/artifacts/public/build/tldr-pages_v2023-08-29.ru.zst
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/An7a68SXTrOfwsG2BYveCA/artifacts/public/build/tldr-pages_v2023-08-29.ru.zst resolved to 5026 bytes with sha256 30c6aaaa43262f10dd625ce6a2289cbecc9f937a7525c35e4c41ba6291bd39f3 in 0.362s
Verified size of https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/An7a68SXTrOfwsG2BYveCA/artifacts/public/build/tldr-pages_v2023-08-29.ru.zst
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 0.3640117390000057, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-11-22T22:55:37.383Z] finished fetching artifacts
[task 2024-11-22T22:55:37.383Z] executing ['bash', '-c', 'pip3 install --upgrade pip setuptools && pip3 install -r $VCS_PATH/pipeline/data/requirements/analyze.txt && export PYTHONPATH=$PYTHONPATH:$VCS_PATH && python3 $VCS_PATH/pipeline/data/analyze.py --file_location $MOZ_FETCHES_DIR/tldr-pages_v2023-08-29.ru.zst --output $TASK_WORKDIR/artifacts --dataset "opus_tldr-pages/v2023-08-29" --language ru']
[task 2024-11-22T22:55:37.706Z] WARNING: The directory '/builds/worker/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[task 2024-11-22T22:55:37.707Z] Defaulting to user installation because normal site-packages is not writeable
[task 2024-11-22T22:55:37.728Z] Requirement already satisfied: pip in /usr/lib/python3/dist-packages (22.0.2)
[task 2024-11-22T22:55:37.901Z] Collecting pip
[task 2024-11-22T22:55:38.016Z]   Downloading pip-24.3.1-py3-none-any.whl (1.8 MB)
[task 2024-11-22T22:55:38.317Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 6.1 MB/s eta 0:00:00
[task 2024-11-22T22:55:38.327Z] Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (59.6.0)
[task 2024-11-22T22:55:38.647Z] Collecting setuptools
[task 2024-11-22T22:55:38.668Z]   Downloading setuptools-75.6.0-py3-none-any.whl (1.2 MB)
[task 2024-11-22T22:55:38.780Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 11.0 MB/s eta 0:00:00
[task 2024-11-22T22:55:38.908Z] Installing collected packages: setuptools, pip
[task 2024-11-22T22:55:40.139Z] Successfully installed pip-24.3.1 setuptools-75.6.0
[task 2024-11-22T22:55:40.476Z] WARNING: The directory '/builds/worker/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[task 2024-11-22T22:55:40.476Z] Defaulting to user installation because normal site-packages is not writeable
[task 2024-11-22T22:55:40.652Z] Collecting certifi==2024.2.2 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 7))
[task 2024-11-22T22:55:40.713Z]   Downloading certifi-2024.2.2-py3-none-any.whl.metadata (2.2 kB)
[task 2024-11-22T22:55:40.844Z] Collecting charset-normalizer==3.3.2 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 9))
[task 2024-11-22T22:55:40.864Z]   Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
[task 2024-11-22T22:55:40.970Z] Collecting contourpy==1.2.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 11))
[task 2024-11-22T22:55:40.992Z]   Downloading contourpy-1.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.8 kB)
[task 2024-11-22T22:55:41.019Z] Collecting cycler==0.12.1 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 13))
[task 2024-11-22T22:55:41.040Z]   Downloading cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)
[task 2024-11-22T22:55:41.188Z] Collecting fonttools==4.49.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 15))
[task 2024-11-22T22:55:41.210Z]   Downloading fonttools-4.49.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (159 kB)
[task 2024-11-22T22:55:41.281Z] Collecting idna==3.6 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 17))
[task 2024-11-22T22:55:41.301Z]   Downloading idna-3.6-py3-none-any.whl.metadata (9.9 kB)
[task 2024-11-22T22:55:41.374Z] Collecting kiwisolver==1.4.5 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 19))
[task 2024-11-22T22:55:41.395Z]   Downloading kiwisolver-1.4.5-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (6.4 kB)
[task 2024-11-22T22:55:41.562Z] Collecting matplotlib==3.8.3 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 21))
[task 2024-11-22T22:55:41.583Z]   Downloading matplotlib-3.8.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.8 kB)
[task 2024-11-22T22:55:41.813Z] Collecting numpy==1.26.4 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 23))
[task 2024-11-22T22:55:41.834Z]   Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
[task 2024-11-22T22:55:41.876Z] Collecting packaging==24.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 28))
[task 2024-11-22T22:55:41.896Z]   Downloading packaging-24.0-py3-none-any.whl.metadata (3.2 kB)
[task 2024-11-22T22:55:42.114Z] Collecting pillow==10.2.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 30))
[task 2024-11-22T22:55:42.136Z]   Downloading pillow-10.2.0-cp310-cp310-manylinux_2_28_x86_64.whl.metadata (9.7 kB)
[task 2024-11-22T22:55:42.190Z] Collecting pyparsing==3.1.2 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 32))
[task 2024-11-22T22:55:42.210Z]   Downloading pyparsing-3.1.2-py3-none-any.whl.metadata (5.1 kB)
[task 2024-11-22T22:55:42.243Z] Collecting python-dateutil==2.9.0.post0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 34))
[task 2024-11-22T22:55:42.264Z]   Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
[task 2024-11-22T22:55:42.314Z] Collecting requests==2.31.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 36))
[task 2024-11-22T22:55:42.334Z]   Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
[task 2024-11-22T22:55:42.363Z] Collecting six==1.16.0 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 38))
[task 2024-11-22T22:55:42.383Z]   Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
[task 2024-11-22T22:55:42.442Z] Collecting urllib3==2.2.1 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/analyze.txt (line 40))
[task 2024-11-22T22:55:42.464Z]   Downloading urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB)
[task 2024-11-22T22:55:42.566Z] Downloading certifi-2024.2.2-py3-none-any.whl (163 kB)
[task 2024-11-22T22:55:42.591Z] Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
[task 2024-11-22T22:55:42.616Z] Downloading contourpy-1.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (310 kB)
[task 2024-11-22T22:55:42.643Z] Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB)
[task 2024-11-22T22:55:42.666Z] Downloading fonttools-4.49.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB)
[task 2024-11-22T22:55:42.722Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.6/4.6 MB 86.2 MB/s eta 0:00:00
[task 2024-11-22T22:55:42.742Z] Downloading idna-3.6-py3-none-any.whl (61 kB)
[task 2024-11-22T22:55:42.766Z] Downloading kiwisolver-1.4.5-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
[task 2024-11-22T22:55:42.771Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 435.5 MB/s eta 0:00:00
[task 2024-11-22T22:55:42.794Z] Downloading matplotlib-3.8.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)
[task 2024-11-22T22:55:42.861Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.6/11.6 MB 174.7 MB/s eta 0:00:00
[task 2024-11-22T22:55:42.882Z] Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
[task 2024-11-22T22:55:42.980Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 188.3 MB/s eta 0:00:00
[task 2024-11-22T22:55:43.000Z] Downloading packaging-24.0-py3-none-any.whl (53 kB)
[task 2024-11-22T22:55:43.023Z] Downloading pillow-10.2.0-cp310-cp310-manylinux_2_28_x86_64.whl (4.5 MB)
[task 2024-11-22T22:55:43.047Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 202.2 MB/s eta 0:00:00
[task 2024-11-22T22:55:43.067Z] Downloading pyparsing-3.1.2-py3-none-any.whl (103 kB)
[task 2024-11-22T22:55:43.087Z] Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
[task 2024-11-22T22:55:43.108Z] Downloading requests-2.31.0-py3-none-any.whl (62 kB)
[task 2024-11-22T22:55:43.128Z] Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
[task 2024-11-22T22:55:43.149Z] Downloading urllib3-2.2.1-py3-none-any.whl (121 kB)
[task 2024-11-22T22:55:43.358Z] Installing collected packages: urllib3, six, pyparsing, pillow, packaging, numpy, kiwisolver, idna, fonttools, cycler, charset-normalizer, certifi, requests, python-dateutil, contourpy, matplotlib
[task 2024-11-22T22:55:47.407Z] Successfully installed certifi-2024.2.2 charset-normalizer-3.3.2 contourpy-1.2.0 cycler-0.12.1 fonttools-4.49.0 idna-3.6 kiwisolver-1.4.5 matplotlib-3.8.3 numpy-1.26.4 packaging-24.0 pillow-10.2.0 pyparsing-3.1.2 python-dateutil-2.9.0.post0 requests-2.31.0 six-1.16.0 urllib3-2.2.1
[task 2024-11-22T22:55:48.002Z] Matplotlib created a temporary cache directory at /tmp/matplotlib-kvvape4k because the default path (/builds/worker/.cache/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
[task 2024-11-22T22:55:48.394Z] [analyze] file_location: /builds/worker/fetches/tldr-pages_v2023-08-29.ru.zst
[task 2024-11-22T22:55:48.394Z] [analyze] output_dir: /builds/worker/artifacts
[task 2024-11-22T22:55:48.394Z] [analyze] dataset: opus_tldr-pages/v2023-08-29
[task 2024-11-22T22:55:48.394Z] [analyze] language: ru
[task 2024-11-22T22:55:48.552Z] [analyze] Saving plot to: /builds/worker/artifacts/tldr-pages_v2023-08-29.ru.distribution-words.png
[task 2024-11-22T22:55:48.960Z] [analyze] Saving plot to: /builds/worker/artifacts/tldr-pages_v2023-08-29.ru.distribution-codepoints.png
[fetches 2024-11-22T22:55:49.280Z] removing /builds/worker/fetches
[fetches 2024-11-22T22:55:49.280Z] finished
[taskcluster 2024-11-22 22:55:56.531Z] === Task Finished ===
[taskcluster 2024-11-22 22:55:56.911Z] Successful task run with exit code: 0 completed in 72.557 seconds