Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for ICU tokenizer #61

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

Add support for ICU tokenizer #61

wants to merge 5 commits into from

Commits on Nov 22, 2024

  1. Add support for ICU tokenizer

    @eu9ene
    eu9ene committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    ee534f3 View commit details
    Browse the repository at this point in the history

  2. Fix pyicu installation in CI

    @eu9ene
    eu9ene committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    a397864 View commit details
    Browse the repository at this point in the history

  3. Fix benchmark

    @eu9ene
    eu9ene committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    cdb3abd View commit details
    Browse the repository at this point in the history

  4. Fix test package installation

    @eu9ene
    eu9ene committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    4e6ac75 View commit details
    Browse the repository at this point in the history

Commits on Nov 23, 2024

  1. Disable ICU tokenizer test

    @eu9ene
    eu9ene committed Nov 23, 2024
    Configuration menu
    Copy the full SHA
    ea357e8 View commit details
    Browse the repository at this point in the history