Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to ICU tokenizer #939

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Use ICU system package

d585a63
Select commit
Loading
Failed to load commit list.
Open

Switch to ICU tokenizer #939

Use ICU system package
d585a63
Select commit
Loading
Failed to load commit list.
firefoxci-taskcluster / tests-lint-eslint succeeded Nov 22, 2024 in 3m 27s

FirefoxCI (pull_request)

Run tests

Details

View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster

Task Status

Started: 2024-11-22T22:52:23.707Z
Resolved: 2024-11-22T22:53:58.520Z
Task Execution Time: 1 minute, 34 seconds, 813 milliseconds
Task Status: completed
Reason Resolved: completed
RunId: 0

Artifacts

- public/logs/live_backing.log
- public/logs/live.log


[taskcluster 2024-11-22 22:52:23.735Z] Task ID: Hh9E_87oSLuDxLEPHh2idA
[taskcluster 2024-11-22 22:52:23.735Z] Worker ID: 6885756460432405546
[taskcluster 2024-11-22 22:52:23.735Z] Worker Group: us-west1-a
[taskcluster 2024-11-22 22:52:23.735Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2024-11-22 22:52:23.735Z] Worker Pool: translations-1/b-linux-large-gcp
[taskcluster 2024-11-22 22:52:23.735Z] Worker Version: 38.0.5
[taskcluster 2024-11-22 22:52:23.735Z] Public IP: 34.127.105.194
[taskcluster 2024-11-22 22:52:23.735Z] Hostname: translations-1-b-linux-large-gcp-iuxgh2-ms6irn-clsmbetw
[taskcluster 2024-11-22 22:52:23.735Z] using cache "translations-level-1-checkouts-v3-7afeb851dd97df8f3607-Z1btq_81SkuLThe6rhEeOg" -> /builds/worker/checkouts

[taskcluster 2024-11-22 22:52:26.923Z] Downloading artifact "public/image.tar.zst" from task ID: Z1btq_81SkuLThe6rhEeOg.
[taskcluster 2024-11-22 22:52:31.926Z] Download Progress: 31.28%
[taskcluster 2024-11-22 22:52:36.926Z] Download Progress: 66.85%
[taskcluster 2024-11-22 22:52:41.194Z] Downloaded artifact successfully.
[taskcluster 2024-11-22 22:52:41.195Z] Downloaded 837.647 mb
[taskcluster 2024-11-22 22:52:41.196Z] Decompressing downloaded image
[taskcluster 2024-11-22 22:52:48.220Z] Loading docker image from downloaded archive.
[taskcluster 2024-11-22 22:53:34.384Z] Image 'public/image.tar.zst' from task 'Z1btq_81SkuLThe6rhEeOg' loaded.  Using image ID sha256:cb3c49d22c8dd3fa5a03e47e47f31a22b8a0bba094ca40fc7bd822c69e3e2c08.
[taskcluster 2024-11-22 22:53:35.640Z] === Task Starting ===
[setup 2024-11-22T22:53:43.069Z] run-task started in /builds/worker
[setup 2024-11-22T22:53:43.069Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ --task-cwd /builds/worker/checkouts/vcs -- bash -c task lint-eslint
[setup 2024-11-22T22:53:43.069Z] Python version: 3.10.12
[cache 2024-11-22T22:53:43.071Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2024-11-22T22:53:43.071Z] changing ownership of volume /builds/worker/.cache to 1000:1000
[volume 2024-11-22T22:53:43.071Z] volume /builds/worker/checkouts is a cache
[setup 2024-11-22T22:53:43.071Z] running as worker:worker
[vcs 2024-11-22T22:53:43.071Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2024-11-22T22:53:43.073Z] executing ['git', 'clone', 'https://github.com/mozilla/translations', '/builds/worker/checkouts/vcs']
[vcs 2024-11-22T22:53:43.075Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2024-11-22T22:53:44.927Z] executing ['git', 'fetch', '--tags', '--force', 'https://github.com/mozilla/translations', 'icu_tokenizer']
[vcs 2024-11-22T22:53:45.171Z] From https://github.com/mozilla/translations
[vcs 2024-11-22T22:53:45.171Z]  * branch            icu_tokenizer -> FETCH_HEAD
[vcs 2024-11-22T22:53:45.180Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/mozilla/translations', 'icu_tokenizer']
[vcs 2024-11-22T22:53:45.416Z] From https://github.com/mozilla/translations
[vcs 2024-11-22T22:53:45.416Z]  * branch            icu_tokenizer -> FETCH_HEAD
[vcs 2024-11-22T22:53:45.425Z] executing ['git', 'checkout', '-f', '-B', 'icu_tokenizer', 'd585a63a6abc04ece83e26ce51a0caa2f7fa21e6']
[vcs 2024-11-22T22:53:46.215Z] Switched to a new branch 'icu_tokenizer'
[vcs 2024-11-22T22:53:46.239Z] executing ['git', 'submodule', 'init']
[vcs 2024-11-22T22:53:46.261Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2024-11-22T22:53:46.261Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2024-11-22T22:53:46.261Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2024-11-22T22:53:46.265Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2024-11-22T22:53:46.265Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2024-11-22T22:53:46.266Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2024-11-22T22:53:46.266Z] Submodule 'inference/3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path 'inference/3rd_party/browsermt-marian-dev'
[vcs 2024-11-22T22:53:46.266Z] Submodule 'inference/3rd_party/emsdk' (https://github.com/emscripten-core/emsdk.git) registered for path 'inference/3rd_party/emsdk'
[vcs 2024-11-22T22:53:46.267Z] Submodule 'inference/3rd_party/ssplit-cpp' (https://github.com/browsermt/ssplit-cpp) registered for path 'inference/3rd_party/ssplit-cpp'
[vcs 2024-11-22T22:53:46.267Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2024-11-22T22:53:46.290Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2024-11-22T22:53:47.576Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2024-11-22T22:53:47.941Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2024-11-22T22:53:48.322Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2024-11-22T22:53:48.959Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2024-11-22T22:53:50.513Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2024-11-22T22:53:51.338Z] Cloning into '/builds/worker/checkouts/vcs/inference/3rd_party/browsermt-marian-dev'...
[vcs 2024-11-22T22:53:52.585Z] Cloning into '/builds/worker/checkouts/vcs/inference/3rd_party/emsdk'...
[vcs 2024-11-22T22:53:53.112Z] Cloning into '/builds/worker/checkouts/vcs/inference/3rd_party/ssplit-cpp'...
[vcs 2024-11-22T22:53:53.605Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2024-11-22T22:53:53.618Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2024-11-22T22:53:53.631Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2024-11-22T22:53:53.661Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2024-11-22T22:53:53.784Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2024-11-22T22:53:53.807Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2024-11-22T22:53:53.889Z] Submodule path 'inference/3rd_party/browsermt-marian-dev': checked out '2781d735d4a10dca876d61be587afdab2726293c'
[vcs 2024-11-22T22:53:53.912Z] Submodule path 'inference/3rd_party/emsdk': checked out '2346baa7bb44a4a0571cc75f1986ab9aaa35aa03'
[vcs 2024-11-22T22:53:53.929Z] Submodule path 'inference/3rd_party/ssplit-cpp': checked out 'a311f9865ade34db1e8e080e6cc146f55dafb067'
[vcs 2024-11-22T22:53:53.930Z] cleaning git checkout...
[vcs 2024-11-22T22:53:53.930Z] executing ['git', 'clean', '-nxdff']
[vcs 2024-11-22T22:53:53.934Z] removing []
[vcs 2024-11-22T22:53:53.934Z] successfully cleaned git checkout!
[vcs 2024-11-22T22:53:53.936Z] TinderboxPrint:<a href='https://github.com/mozilla/translations/commit/d585a63a6abc04ece83e26ce51a0caa2f7fa21e6' title='Built from translations commit d585a63a6abc04ece83e26ce51a0caa2f7fa21e6'>d585a63a6abc04ece83e26ce51a0caa2f7fa21e6</a>
[task 2024-11-22T22:53:53.936Z] executing ['bash', '-c', 'task lint-eslint']
[task 2024-11-22T22:53:53.943Z] task: [lint-eslint] cd ./inference/wasm/tests && npm install && npm run lint
[task 2024-11-22T22:53:54.353Z] (node:474) ExperimentalWarning: CommonJS module /builds/worker/.nvm/versions/node/v23.1.0/lib/node_modules/npm/node_modules/debug/src/node.js is loading ES Module /builds/worker/.nvm/versions/node/v23.1.0/lib/node_modules/npm/node_modules/supports-color/index.js using require().
[task 2024-11-22T22:53:54.353Z] Support for loading ES Module in require() is an experimental feature and might change at any time
[task 2024-11-22T22:53:54.353Z] (Use `node --trace-warnings ...` to show where the warning was created)
[task 2024-11-22T22:53:56.413Z] 
[task 2024-11-22T22:53:56.413Z] added 134 packages, and audited 135 packages in 2s
[task 2024-11-22T22:53:56.413Z] 
[task 2024-11-22T22:53:56.413Z] 38 packages are looking for funding
[task 2024-11-22T22:53:56.413Z]   run `npm fund` for details
[task 2024-11-22T22:53:56.414Z] 
[task 2024-11-22T22:53:56.414Z] 1 low severity vulnerability
[task 2024-11-22T22:53:56.414Z] 
[task 2024-11-22T22:53:56.414Z] To address all issues, run:
[task 2024-11-22T22:53:56.414Z]   npm audit fix
[task 2024-11-22T22:53:56.414Z] 
[task 2024-11-22T22:53:56.414Z] Run `npm audit` for details.
[task 2024-11-22T22:53:56.541Z] 
[task 2024-11-22T22:53:56.541Z] > [email protected] lint
[task 2024-11-22T22:53:56.541Z] > eslint .
[task 2024-11-22T22:53:56.541Z] 
[taskcluster 2024-11-22 22:53:58.060Z] === Task Finished ===
[taskcluster 2024-11-22 22:53:58.060Z] Successful task run with exit code: 0 completed in 94.326 seconds