Skip to content

Commit

Permalink
explain "hidden entry"
Browse files Browse the repository at this point in the history
  • Loading branch information
mh-northlander committed Nov 28, 2024
1 parent 86128e5 commit 73d1978
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 5 deletions.
9 changes: 5 additions & 4 deletions src/main/java/com/worksap/nlp/sudachi/Dictionary.java
Original file line number Diff line number Diff line change
Expand Up @@ -60,11 +60,12 @@ public interface Dictionary extends AutoCloseable {
/**
* Create a parallel stream of all words in the dictionary as morphemes.
*
* Corresponds to the lines in the lexicon csv, i.e. includes hidden entries and
* excludes entries for normalization form. Entries in the stream are not
* sorted.
* Corresponds to the lines in the lexicon csv, i.e. it includes entries that
* appear only when refered from other words (e.g. as constitution) during an
* analysis and excludes entries that automatically added to store a
* normalization form of another word. Entries in the stream are not sorted.
*
* @return a stream of morphemes.
* @return a parallel stream of morphemes.
*/
public Stream<Morpheme> entries();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ class JapaneseDictionaryTest {
assertEquals(1, sudachi.size)
assertEquals("徳島県産", sudachi[0].getUserData())

// cannot find hidden entry
// cannot find entry with -1 conjunction cost
val hidden = dict.lookup("隠し")
assertTrue(hidden.isEmpty())

Expand Down

0 comments on commit 73d1978

Please sign in to comment.