Releases
v.4.9.1
Added pretrained KenLM models, trained on https://github.com/huseinzol05/malay-dataset/tree/master/dumping/clean , https://malaya.readthedocs.io/en/latest/load-kenlm.html
Improved spelling correction interface, under malaya.spelling_correction.*
.
Improved JamSpell spelling correction interface, https://malaya.readthedocs.io/en/latest/load-spelling-correction-jamspell.html
Improved speed and accuracy Probability spelling correction, https://malaya.readthedocs.io/en/latest/load-spelling-correction-probability.html
Added Probability LM, probability + KenLM spelling correction, a better scoring based on sentence context, https://malaya.readthedocs.io/en/latest/load-spelling-correction-probability-lm.html
Improved Spylls spelling correction interface, https://malaya.readthedocs.io/en/latest/load-spelling-correction-probability-lm.html
Improved SymSpeller spelling correction interface, https://malaya.readthedocs.io/en/latest/load-spelling-correction-symspell.html
Improved Transformer Encoder spelling correction interface, https://malaya.readthedocs.io/en/latest/load-spelling-correction-encoder-transformer.html
Improved Seq2Seq Transformer spelling correction interface, https://malaya.readthedocs.io/en/latest/load-spelling-correction-transformer.html
Added Syllable tokenizer, https://malaya.readthedocs.io/en/latest/load-tokenizer-syllable.html
Added stemmer trained on noisy dataset to achieve better stemming for local language structure, https://malaya.readthedocs.io/en/latest/load-stemmer.html#Sensitive-towards-local-language-structure
Improved normalizer, now able to add stemmer and add more parameters, https://malaya.readthedocs.io/en/latest/load-normalizer.html
You can’t perform that action at this time.