Number of Epochs to replicate the acc results

[afzalxo]

Hi,
I am currently training your model to replicate the accuracy results. Could you kindly let me know the number of epochs you trained for for each of the three datasets (MetaQA, PathQuestions, WC2014) in order to get the Top 1 scores in Table 2 in your paper. I know that WC2014 saturates to ~100% within the first few Epochs. Don't know about the remaining two datasets.
Thanks,

[afzalxo]

Also, It'd be great if you could mention the batch sizes you utilized for each of the three datasets and the time it took for training. Thanks

[lanyunshi]

Hi,

Sorry for the epoch number, I don't remember it accurately. You can set your epoch number by observing the log printout.

For the batch size, I remember it's 1 as the number of iteration for each question is different so we only set it as 1.

Thanks!

[afzalxo]

Hey thanks for the clarification. I used batch size of 1. I am able to get the accuracy mentioned in the paper on MovieQA and WC datasets but for PathQuestions, it saturates at roughly 87%. I trained for 600 epochs using batch size 1 and default parameters. Here's the run in question:

https://wandb.ai/afzal/kbqa-proj/runs/2p0cu4tg/overview

Could you kindly make a wild guess as to what I might be doing wrong?

[lanyunshi]

Did you solve the issue? I tried to find my previous log file but I cannot find it as previously I deleted some folders by accident. Can you share your log file to me?

[afzalxo]

Hi,
Here is a short log: https://wandb.ai/afzal/kbqa-proj/runs/2p0cu4tg/logs?workspace=user-afzal
You can download the full log in the files section here: https://wandb.ai/afzal/kbqa-proj/runs/2p0cu4tg/files?workspace=user-afzal
The relevant loss/accuracy plots are located here: https://wandb.ai/afzal/kbqa-proj/runs/2p0cu4tg?workspace=user-afzal

[lanyunshi]

Hi,

I have read your plots. Can you slightly increase the dropout ratio? You can slightly tune it as I feel it might be a little bit overfitting.