Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deprecate tokenizeSentences #254

Merged
merged 5 commits into from
Dec 2, 2024
Merged

Deprecate tokenizeSentences #254

merged 5 commits into from
Dec 2, 2024

Conversation

mh-northlander
Copy link
Collaborator

@mh-northlander mh-northlander commented Nov 28, 2024 •
edited
Loading

resolve #251, #250

  • remove Tokenizer.tokenizeSentences and SentenceSplittingAnalysis
  • replace Tokenizer.tokenizeSentences by current Tokenizer.lazyTokenizeSentences
    • and mark Tokenizer.lazyTokenizeSentences deprecated
  • modify signatures of Tokenizer methods: use List<Morpheme>
  • add Tokenizer.split(List<Morpheme>, mode) instead of MorphemeList.split
  • providing null as a split mode now errors

@mh-northlander mh-northlander added this to the 0.8 milestone Nov 28, 2024
This was linked to issues Nov 28, 2024
Update Tokenizer method signature #250
Open
Deprecate Tokenizer.tokenizeSentences #251
Open
@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Passed Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
97.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@mh-northlander mh-northlander merged commit 239d02b into develop-v0.8 Dec 2, 2024
4 checks passed
@mh-northlander mh-northlander deleted the feature/251-deprecate-tokenize-sentences branch December 2, 2024 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kazuma-t kazuma-t kazuma-t approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.8
Development

Successfully merging this pull request may close these issues.

Deprecate Tokenizer.tokenizeSentences Update Tokenizer method signature
2 participants
@mh-northlander @kazuma-t