Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vocab index ordering? #75

Open
Tahlor opened this issue Apr 19, 2018 · 0 comments
Open

Vocab index ordering? #75

Tahlor opened this issue Apr 19, 2018 · 0 comments

Comments

@Tahlor
Copy link

Tahlor commented Apr 19, 2018

Why do we calculate word counts if we ultimately include all words and sort the vocab alphabetically? Mostly I'm wondering if you ordered it with the most common words first, and later reverted to alphabetical ordering for some reason. Would it be bad to have the most common words first? Or do we expect the model would learn better with a random ordering?

In utils.py:

        vocabulary_inv = [x[0] for x in word_counts.most_common()]
        vocabulary_inv = list(sorted(vocabulary_inv))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant