Skip to content

Latest commit

 

History

History
130 lines (100 loc) · 2.76 KB

README.md

File metadata and controls

130 lines (100 loc) · 2.76 KB
Raw

Episodic Memory Reader (EMR)

This work has been published on The Meeting of the Association for Computational Linguistics (ACL 2019).
Here is the paper of arxiv version in https://arxiv.org/abs/1903.06164.
or you can find the paper of ACL anthology version in https://aclweb.org/anthology/P19-1434

Requirements

  • Python 3.6.4
  • Pytorch >= 1.0.0
  • tensorboardX >= 1.6
  • tqdm
  • termcolor
  • boto3
  • numpy
  • (below is for only tvqa)
  • pysrt
  • h5py
  • image
  • seaborn
  • matplotlib

How to use

Recommend running with at least three GPUs.

bAbI

cd babi

To pre-process and train the model,

python main-a3c.py \
  --task-id=[2, 22] \
  --prepro \
  --model=[FIFO, UNIFORM, LIFO, LRU_DNTM, R_EMR, T_EMR] \
  --log-dir=$DIR

You can see other configuration in 'main-a3c.py'.
If you have a pre-process, you do not have to do it again.

TriviaQA

cd trivia

Download TriviaQA dataset from http://nlp.cs.washington.edu/triviaqa

To pre-process TriviaQA dataset,

python prepro.py

To train the model,

python main-a3c.py \
  --task=[web, wikipedia] \
  --model=[FIFO, UNIFORM, LIFO, LRU_DNTM, R_EMR, T_EMR] \
  --rl-method=[a3c, policy, discrete] \
  --log-dir=$DIR

You can see other configuration in 'main-a3c.py'.

Demo

python main-a3c.py --demo --ckpt=TRAINED_MODEL

Test

python main-a3c.py --test --ckpt=TRAINED_MODEL

TVQA

This code is mostly based on the original TVQA code (https://github.com/jayleicn/TVQA).

In case of Transformer code, it is based on the BERT code (https://github.com/huggingface/pytorch-pretrained-BERT).

cd tvqa

Dataset

To prepare the TVA dataset, please follow the instructions "Usage 1~3" of the original TVQA code (https://github.com/jayleicn/TVQA).

Codes for preprocessing are also included in this repository.

Preparing ImageNet feature is also needed.

In case of ImageNet feature, please divide it in to pieces using "sep_large.py".

Usage:

  1. Pretraining 
    python main-a3c.py --pretrain \
	--log-dir=$PRE_DIR

Pretraining the TVQA model is needed to get right results.

  1. Training

    python main-a3c.py --input_streams sub imagenet \
	--no_ts \
	--pretrain-dir=$PRE_DIR \
	--model=[FIFO, UNIFORM, LIFO, LRU_DNTM, R_EMR, T_EMR] \
	--log-dir=$DIR

  2. Test

    python main-a3c.py --test --ckpt=$DIR

  3. Demo

    python main-a3c.py --demo --ckpt=$DIR

Acknowledgements

This work was supported by Clova, NAVER Corp. and Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government(MSIT) (No.2016-0-00563, Research on Adaptive Machine Learning Technology Development for Intelligent Autonomous Digital Companion).