-
Notifications
You must be signed in to change notification settings - Fork 205
1602AnsselRNN
Petr Baudis edited this page Mar 17, 2016
·
6 revisions
RNN is a pretty powerful model that's also quite simple and has low number of parameters. We study it on the anssel task, looking to find an optimal configuration that may serve as a good default for other tasks then.
Settings for configuration "rnn1":
{"Ddim": "2",
"balance_class": "True",
"batch_size": "160",
"dropout": "2/3",
"dropoutfix_inp": "0",
"dropoutfix_rec": "0",
"e_add_flags": "True",
"embdim": "300",
"epoch_fract": "0.25",
"inp_e_dropout": "2/3",
"inp_w_dropout": "0",
"l2reg": "0.0001",
"loss": "<function ranknet at 0xb25f938>",
"mlpsum": "sum",
"nb_epoch": "16",
"pact": "linear",
"pdim": "2.5", # 5/2
"project": "True",
"ptscorer": "<function mlp_ptscorer at 0xb26b2a8>",
"rnn": "<class 'keras.layers.recurrent.GRU'>",
"rnnact": "tanh",
"rnnbidi": "True",
"rnnbidi_mode": "sum",
"rnninit": "glorot_uniform",
"rnnlevels": "1",
"sdim": "2"})
Baseline with no dropout - 0.341914 (95% [0.335095, 0.348732]).
10x pact="tanh", ptscorer=B.dot_ptscorer ay_1rnnd0_tdot - 0.351119 (95% [0.339229, 0.363009]):
10731538.arien.ics.muni.cz.ay_1rnnd0_tdot etc.
[0.324377, 0.356488, 0.359251, 0.376279, 0.378426, 0.337559, 0.357010, 0.344303, 0.337580, 0.339916, ]
8x "mini" sdim=1/6, pdim=1/6 tdot configuration (as favored for Ubuntu experiments) ay_1rnnd0_s16p16tdot - 0.316167 (95% [0.298347, 0.333986]):
10681203.arien.ics.muni.cz.ay_1rnnd0_s16p16tdot etc.
[0.291450, 0.321549, 0.360601, 0.321138, 0.314937, 0.319708, 0.315095, 0.284857, ]
7x sdim=2, pdim=1, tdot (which is also used in the Tranfer experiments) ay_1rnnd0_s2p1tdot - 0.332909 (95% [0.317992, 0.347826]):
10681212.arien.ics.muni.cz.ay_1rnnd0_s2p1tdot etc.
[0.325728, 0.337246, 0.314658, 0.360484, 0.312650, 0.331085, 0.348513, ]
Baseline for non-Bayesian dropout 1/2 - 0.365131 (95% [0.356652, 0.373611]) (from BayesDropout).
8x ay_1rnn_i12d12_s1 - 0.368005 (95% [0.345081, 0.390928]):
10745738.arien.ics.muni.cz.ay_1rnn_i12d12_s1 etc.
[0.338554, 0.367423, 0.363592, 0.415391, 0.395761, 0.346984, 0.385336, 0.330996, ]
17x sdim=2, pdim=5/2, pact="tanh" ay_1rnn_i12d12_t - 0.370692 (95% [0.359962, 0.381421]):
10731552.arien.ics.muni.cz.ay_1rnn_i12d12_t etc.
[0.413446, 0.317608, 0.357394, 0.381370, 0.356894, 0.390496, 0.376919, 0.366056, 0.357054, 0.358958, 0.364415, 0.385996, 0.378332, 0.398782, 0.356024, 0.361949, 0.380064, ]
8x sdim=1, pdim=5/2=2.5 (default value), pact="tanh" ay_1rnn_i12d12_s1p52t - 0.374482 (95% [0.361503, 0.387461]):
10711143.arien.ics.muni.cz.ay_1rnn_i12d12_s1p52t etc.
[0.378060, 0.370266, 0.376403, 0.393484, 0.359925, 0.401185, 0.351863, 0.364670, ]
8x above + dot ptscorer ay_1rnn_i12d12_s1p52tdot - 0.366771 (95% [0.357422, 0.376119]):
10711135.arien.ics.muni.cz.ay_1rnn_i12d12_s1p52tdot etc.
[0.364200, 0.362318, 0.350010, 0.362913, 0.368420, 0.382481, 0.358206, 0.385617, ]
8x above + smaller pdim ay_1rnn_i12d12_s1p1tdot - 0.343655 (95% [0.318951, 0.368358]):
10711102.arien.ics.muni.cz.ay_1rnn_i12d12_s1p1tdot etc.
[0.320096, 0.331607, 0.322086, 0.324384, 0.325787, 0.412256, 0.349718, 0.363304, ]
8x above + larger sdim ay_1rnn_i12d12_s2p1tdot - 0.351717 (95% [0.343524, 0.359910]):
10684792.arien.ics.muni.cz.ay_1rnn_i12d12_s2p1tdot etc.
[0.344882, 0.360933, 0.336029, 0.369034, 0.354446, 0.348257, 0.355818, 0.344340, ]
everything hardly distinguishable...
Baseline for non-Bayesian dropout 2/3 - 0.387441 (95% [0.380025, 0.394857]) (from BayesDropout).
5x loss='binary_crossentropy' ay_1rnn_i23d23_lbc - 0.356821 (95% [0.327733, 0.385909]):
10711083.arien.ics.muni.cz.ay_1rnn_i23d23_lbc etc.
[0.343136, 0.369890, 0.389565, 0.360727, 0.320787, ]
7x balance_class=True ay_1rnn_i23d23_b - 0.381474 (95% [0.363936, 0.399012]):
10711091.arien.ics.muni.cz.ay_1rnn_i23d23_b etc.
[0.402060, 0.403580, 0.351338, 0.382750, 0.396561, 0.362050, 0.371980, ]
These are not statistically conclusive, but it's likely they aren't better than the baseline.
Then, some automated tuning with an updated configuration:
rs = RandomSearch(modelname+'_rlog.txt',
dropout=[2/3], inp_e_dropout=[2/3], l2reg=[1e-6, 1e-5, 1e-4, 1e-3, 1e-2],
rnnact=['tanh'], rnninit=['glorot_uniform'],
sdim=[1/6, 1/2, 1, 2, 3, 4],
project=[True, True, False], pdim=[1/2, 1, 2, 2.5, 3, 4], pact=['tanh'], # pact=relu -> fail, no learning at all
ptscorer=[B.dot_ptscorer])
(so, "tdot" setup) Absolutely non-converging configurations not listed.
30032560de759d19 0.258189 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fr
act": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.0001", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2", "project": "Tr
ue", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1",
"sdim": "0.166666666667"}
-4b70bfe186f14ac4 0.278552 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_f
ract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.0001", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2.5", "project":
"True", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1
", "sdim": "0.166666666667"}
-527890ab171156bf 0.281506 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_f
ract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.01", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2", "project": "Tru
e", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "
sdim": "0.166666666667"}
-7fe585b769cb6f70 0.289304 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_f
ract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.0001", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "0.5", "project":
"False", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "
1", "sdim": "0.166666666667"}
4552ca1a19a4e5b1 0.295363 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fr
act": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "1e-06", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "3", "project": "Tru
e", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "
sdim": "0.166666666667"}
-5a77bcb2bc9ab012 0.298944 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_f
ract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.0001", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2.5", "project":
"True", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1
", "sdim": "0.5"}
-30d68c482f7814c4 0.301972 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_f
ract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "1e-06", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2", "project": "Tr
ue", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1",
"sdim": "0.166666666667"}
-40bf08e4e7df4c16 0.302068 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_f
ract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.0001", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "3", "project": "T
rue", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1",
"sdim": "0.166666666667"}
-42e02894889f5d02 0.310917 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_f
ract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.001", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2", "project": "Fa
lse", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1",
"sdim": "0.5"}
-2fecd7127d655ea2 0.312127 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_f
ract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.001", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2", "project": "Tr
ue", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1",
"sdim": "0.333333333333"}
5312e64aeaabd203 0.315111 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fr
act": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "1e-06", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "3", "project": "Tru
e", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "
sdim": "0.166666666667"}
1d93ad0a93e9fb3e 0.315273 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fr
act": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "1e-06", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "4", "project": "Fal
se", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1",
"sdim": "0.5"}
-40a3c338377c2de3 0.319094 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_f
ract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "1e-06", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2.5", "project": "
True", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1"
, "sdim": "0.166666666667"}
-648717ad159f50f4 0.319186 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.0001", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "0.5", "project": "True", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "0.5"}
389f6e88f3adb663 0.319643 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "1e-06", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "1", "project": "True", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "1"}
5d1a79328ff1a2a2 0.327586 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "1e-06", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2", "project": "True", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "0.5"}
-68585c43a0260a79 0.327915 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.01", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2", "project": "True", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "0.5"}
34a4dab47dae9899 0.334086 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.001", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "3", "project": "False", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "1"}
-67aa14b031843a3a 0.340372 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.0001", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "0.5", "project": "True", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "0.5"}
2497416c6af3b2b0 0.342282 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.01", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "1", "project": "True", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "3"}
6833abf71e6b0e2a 0.342784 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.001", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "1", "project": "True", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "2"}
-4521206ad640dfd8 0.348176 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.01", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2.5", "project": "True", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "3"}
7d1cb65b0b5b723f 0.349520 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.001", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2.5", "project": "True", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "3"}
db3fae0aa57627a 0.354631 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "1e-06", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "1", "project": "True", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "2"}
57ebd3f1df0f9630 0.358292 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "1e-05", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "0.5", "project": "False", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "3"}
-1c8f8972db7a0176 0.359372 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.0001", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "1", "project": "True", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "3"}
3e80778fbcd6a280 0.360672 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "1e-06", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "3", "project": "True", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "2"}
3e8076a6e8130e0d 0.368596 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "1e-06", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "3", "project": "True", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "3"}
-7b3a603dc21c6e37 0.372511 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.0001", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2", "project": "True", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "1"}
-777b0154f85570e8 0.389973 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "1e-05", "loss": "<function ranknet at 0xc322e60>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "3", "project": "True", "ptscorer": "<function dot_ptscorer at 0xc322758>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "3"}
3b268e8c7096578 0.406663 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "1e-05", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "0.5", "project": "True", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "3"}
-7b3a64c9eae274d8 0.407353 {"Ddim": "2", "balance_class": "False", "batch_size": "160", "dropout": "0.666666666667", "dropoutfix_inp": "0", "dropoutfix_rec": "0", "e_add_flags": "True", "embdim": "300", "epoch_fract": "0.25", "inp_e_dropout": "0.666666666667", "inp_w_dropout": "0", "l2reg": "0.0001", "loss": "<function ranknet at 0xbd0f668>", "mlpsum": "sum", "nb_epoch": "16", "pact": "tanh", "pdim": "2", "project": "True", "ptscorer": "<function dot_ptscorer at 0xbd08ed8>", "rnn": "<class 'keras.layers.recurrent.GRU'>", "rnnact": "tanh", "rnnbidi": "True", "rnnbidi_mode": "sum", "rnninit": "glorot_uniform", "rnnlevels": "1", "sdim": "4"}
TODO process this with some nice tool. Distilled:
- Benchmark sdim=4, pdim=2 compared to 1rnn and see how long it takes
23x ay_1rnn_i23d23_s4p2tdot - 0.354171 (95% [0.344686, 0.363656]):
10731587.arien.ics.muni.cz.ay_1rnn_i23d23_s4p2tdot etc.
[0.375070, 0.369584, 0.316354, 0.372787, 0.392013, 0.342979, 0.345659, 0.360379, 0.339608, 0.358889, 0.361726, 0.379280, 0.360529, 0.346524, 0.362308, 0.322179, 0.340513, 0.298692, 0.385529, 0.363035, 0.366788, 0.349447, 0.336063, ]
Interesting! did tdot mess things up for the dropout tuning?
or we won't be able to guess anything from individual samples.
Baseline (from BayesDropout) 0.415827 (95% [0.408099, 0.423554]).
16x ay_1rnn_i45d45_s2p2t (i.e. p 2.5 to 2, +t) - TODO.
8x transfer learning baseline ay_1rnn_i45d45_s2p1tdot - 0.350665 (95% [0.336821, 0.364508]):
10783017.arien.ics.muni.cz.ay_1rnn_i45d45_s2p1tdot etc.
[0.348711, 0.317360, 0.363682, 0.342676, 0.358955, 0.377087, 0.354354, 0.342493, ]
20x ay_1rnn_i45d45_s4p2tdot - 0.350500 (95% [0.341328, 0.359671]):
10739214.arien.ics.muni.cz.ay_1rnn_i45d45_s4p2tdot etc.
[0.371234, 0.338251, 0.381913, 0.369576, 0.355388, 0.330645, 0.349966, 0.353879, 0.351151, 0.363340, 0.379282, 0.344599, 0.366437, 0.333251, 0.322626, 0.353949, 0.366681, 0.342111, 0.302228, 0.333485, ]
24x ay_1rnn_i45d45_s3p2tdot - 0.341946 (95% [0.333953, 0.349938]):
10731642.arien.ics.muni.cz.ay_1rnn_i45d45_s3p2tdot etc.
[0.341057, 0.316921, 0.351668, 0.317970, 0.316332, 0.348401, 0.337350, 0.318023, 0.337985, 0.338498, 0.333824, 0.336234, 0.343213, 0.359052, 0.360386, 0.332316, 0.328997, 0.356152, 0.367019, 0.360280, 0.398054, 0.317150, 0.349487, 0.340323, ]
24x ay_1rnn_i45d45_s3p2dot - 0.334360 (95% [0.325937, 0.342784]):
10745682.arien.ics.muni.cz.ay_1rnn_i45d45_s3p2dot etc.
[0.358457, 0.320285, 0.340341, 0.297953, 0.319354, 0.335538, 0.323518, 0.337880, 0.309650, 0.346792, 0.330609, 0.359314, 0.345065, 0.361806, 0.331816, 0.309322, 0.309736, 0.361158, 0.360009, 0.359615, 0.357373, 0.316627, 0.312461, 0.319971, ]
24x ay_1rnn_i45d45_s3p2 - 0.413996 (95% [0.399175, 0.428816]):
10745706.arien.ics.muni.cz.ay_1rnn_i45d45_s3p2 etc.
[0.369562, 0.446025, 0.416261, 0.433968, 0.449057, 0.458304, 0.425981, 0.311952, 0.389022, 0.403008, 0.425823, 0.355724, 0.421814, 0.388889, 0.414412, 0.455312, 0.404979, 0.438039, 0.445879, 0.428172, 0.439621, 0.439441, 0.411093, 0.363564, ]
8x ay_1rnn_i45d45_s4p2 - 0.394235 (95% [0.368002, 0.420469]):
10776629.arien.ics.muni.cz.ay_1rnn_i45d45_s4p2 etc.
[0.378254, 0.398227, 0.407026, 0.451275, 0.423937, 0.369771, 0.343743, 0.381650, ]
8x ay_1rnn_i45d45_s4p2relu - 0.393240 (95% [0.350593, 0.435887]):
10776650.arien.ics.muni.cz.ay_1rnn_i45d45_s4p2relu etc.
[0.396775, 0.404625, 0.418166, 0.473242, 0.429899, 0.377101, 0.289903, 0.356208, ]