Language Modeling with Gated Convolutional Networks Yann N. Dauphin, Angela Fan, Michael Auli, David Grangier https://arxiv.org/abs/1612.08083
This is a PyTorch implementation of Facebook AI Research Lab's paper: Language Modeling with Gated Convolutional Networks. This paper applies a convolutional approach to language modelling with a novel Gated-CNN model.
- Download Google 1 Billion Word dataset.
- PyTorch 0.2.0_3
- impl adaptive softmax
- bug? accuracy didn't increase from epoch#1
- support variable sentence length using padding
- Understand padding at the beginning of sentence
- impl checkpoints