该脚本有过滤语料的作用
过滤特征有：(Features)
1.双语句子长度比率(length ratio of src and tgt sentences)
2.重复句子(repeated sentences)
目前支持的语种类：(Supported LANGs)
- 英文EN<-->中文ZH
- 日文JP<-->中文ZH
- 韩文KR<-->中文ZH
9th/Aug/2018
-
Notifications
You must be signed in to change notification settings - Fork 2
alphadl/corpus_filter
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
Scripts for machine translation corpora filtering/ 机器翻译平行语料过滤的脚本
Topics
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published