forked from suamin/ICD-BERT
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Results.txt
407 lines (373 loc) · 18.5 KB
/
Results.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
______________________________________
baseline-en
______________________________________
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=476
FPs=63
FNs=307
precision=0.8831168831168831
recall=0.6079182630906769
fscore=0.7201210287443267
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=476
FPs=44
FNs=307
precision=0.9153846153846154
recall=0.6079182630906769
fscore=0.7306216423637759
______________________________________
baseline-de
______________________________________
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=413
FPs=48
FNs=370
precision=0.89587852494577
recall=0.5274584929757343
fscore=0.6639871382636656
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=413
FPs=31
FNs=370
precision=0.9301801801801802
recall=0.5274584929757343
fscore=0.6731866340668297
______________________________________
biobert-en
______________________________________
Evaluating: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 26/26 [00:03<00:00, 7.82it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=670
FPs=382
FNs=113
precision=0.6368821292775665
recall=0.855683269476373
fscore=0.7302452316076293
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=670
FPs=205
FNs=113
precision=0.7657142857142857
recall=0.855683269476373
fscore=0.8082026537997588
______________________________________
bert-en
______________________________________
Evaluating: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 26/26 [00:03<00:00, 7.84it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=652
FPs=398
FNs=131
precision=0.6209523809523809
recall=0.8326947637292464
fscore=0.7114020731042009
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=652
FPs=215
FNs=131
precision=0.7520184544405998
recall=0.8326947637292464
fscore=0.7903030303030303
______________________________________
multi-bert-de
______________________________________
Evaluating: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 26/26 [00:03<00:00, 8.02it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=653
FPs=554
FNs=130
precision=0.5410107705053853
recall=0.8339719029374202
fscore=0.6562814070351759
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=653
FPs=304
FNs=130
precision=0.6823406478578893
recall=0.8339719029374202
fscore=0.7505747126436783
______________________________________
cnn-fasttext-en
______________________________________
Iteration: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 361.24it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=460
FPs=129
FNs=323
precision=0.7809847198641766
recall=0.5874840357598978
fscore=0.6705539358600584
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=460
FPs=92
FNs=323
precision=0.8333333333333334
recall=0.5874840357598978
fscore=0.6891385767790262
______________________________________
han-fasttext-en
______________________________________
Iteration: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 82.14it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=491
FPs=156
FNs=292
precision=0.758887171561051
recall=0.6270753512132823
fscore=0.6867132867132867
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=491
FPs=107
FNs=292
precision=0.8210702341137124
recall=0.6270753512132823
fscore=0.7110789283128168
______________________________________
slstm-fasttext-en
______________________________________
Iteration: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 80.97it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=515
FPs=109
FNs=268
precision=0.8253205128205128
recall=0.6577266922094508
fscore=0.7320540156361052
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=515
FPs=82
FNs=268
precision=0.8626465661641541
recall=0.6577266922094508
fscore=0.7463768115942028
______________________________________
clstm-fasttext-en
______________________________________
Iteration: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 56.70it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=509
FPs=163
FNs=274
precision=0.7574404761904762
recall=0.6500638569604087
fscore=0.6996563573883161
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=509
FPs=107
FNs=274
precision=0.8262987012987013
recall=0.6500638569604087
fscore=0.727662616154396
______________________________________
cnn-pubmed-en
______________________________________
Iteration: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 346.98it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=504
FPs=119
FNs=279
precision=0.8089887640449438
recall=0.6436781609195402
fscore=0.716927453769559
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=504
FPs=77
FNs=279
precision=0.8674698795180723
recall=0.6436781609195402
fscore=0.7390029325513195
______________________________________
han-pubmed-en
______________________________________
Iteration: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 77.13it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=520
FPs=134
FNs=263
precision=0.7951070336391437
recall=0.6641123882503193
fscore=0.7237299930410577
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=520
FPs=93
FNs=263
precision=0.8482871125611745
recall=0.6641123882503193
fscore=0.7449856733524354
______________________________________
slstm-pubmed-en
______________________________________
Iteration: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 79.09it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=533
FPs=158
FNs=250
precision=0.7713458755426917
recall=0.6807151979565773
fscore=0.723202170963365
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=533
FPs=108
FNs=250
precision=0.8315132605304212
recall=0.6807151979565773
fscore=0.7485955056179775
______________________________________
clstm-pubmed-en
______________________________________
Iteration: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 53.32it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=534
FPs=116
FNs=249
precision=0.8215384615384616
recall=0.6819923371647509
fscore=0.7452896022330775
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=534
FPs=81
FNs=249
precision=0.8682926829268293
recall=0.6819923371647509
fscore=0.7639484978540774
______________________________________
cnn-fasttext-de
______________________________________
Iteration: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 411.96it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=428
FPs=105
FNs=355
precision=0.8030018761726079
recall=0.5466155810983397
fscore=0.6504559270516718
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=428
FPs=64
FNs=355
precision=0.8699186991869918
recall=0.5466155810983397
fscore=0.6713725490196079
______________________________________
han-fasttext-de
______________________________________
Iteration: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 71.06it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=428
FPs=171
FNs=355
precision=0.7145242070116862
recall=0.5466155810983397
fscore=0.6193921852387845
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=428
FPs=103
FNs=355
precision=0.8060263653483992
recall=0.5466155810983397
fscore=0.6514459665144597
______________________________________
slstm-fasttext-de
______________________________________
Iteration: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 78.12it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=502
FPs=132
FNs=281
precision=0.7917981072555205
recall=0.6411238825031929
fscore=0.7085391672547636
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=502
FPs=86
FNs=281
precision=0.8537414965986394
recall=0.6411238825031929
fscore=0.7323121808898615
Evaluating: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 41/41 [00:05<00:00, 8.25it/s]
Iteration: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 11/11 [00:00<00:00, 53.29it/s]
***** best f1-score=0.869 @ k=0.633 *****
Evaluating: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 26/26 [00:03<00:00, 7.99it/s]
Iteration: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 54.32it/s]
***** Test results (Original) *****
Number of dev documents: 407
Number of documents with predictions: 407
TPs=641
FPs=220
FNs=142
precision=0.7444831591173054
recall=0.8186462324393359
fscore=0.7798053527980536
***** Test results (Modified) *****
Number of dev documents: 407
Number of documents with predictions: 314
TPs=641
FPs=130
FNs=142
precision=0.8313878080415046
recall=0.8186462324393359
fscore=0.8249678249678251