Fixed Major Bugs
We finally fixed some major bugs regarding the INRIA evaluation system. To demonstrate the impact of this fixes, the following table shows the results of a simple, string based approach before and after the fix.
| algo |
3-gram – old(broken) |
3-Gram-Fixed |
| test |
Prec. |
Rec. |
Fall. |
FMeas. |
Over. |
Prec. |
Rec. |
Fall. |
FMeas. |
Over. |
| 101 |
89.00% |
34.00% |
11.00% |
49.00% |
30.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 102 |
n/a |
n/a |
0.00% |
NaN |
100.00% |
NaN |
NaN |
|
|
|
| 103 |
89.00% |
34.00% |
11.00% |
49.00% |
30.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 104 |
89.00% |
34.00% |
11.00% |
49.00% |
30.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 201 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
7.00% |
1.00% |
93.00% |
2.00% |
0.0-13 |
| 202 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
7.00% |
1.00% |
93.00% |
2.00% |
0.0-13 |
| 203 |
89.00% |
34.00% |
11.00% |
49.00% |
30.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 204 |
73.00% |
28.00% |
27.00% |
40.00% |
18.00% |
93.00% |
91.00% |
7.00% |
92.00% |
84.00% |
| 205 |
29.00% |
10.00% |
71.00% |
15.00% |
0.0-15 |
42.00% |
38.00% |
58.00% |
40.00% |
0.0-14 |
| 208 |
73.00% |
28.00% |
27.00% |
40.00% |
18.00% |
93.00% |
91.00% |
7.00% |
92.00% |
84.00% |
| 209 |
29.00% |
10.00% |
71.00% |
15.00% |
0.0-15 |
42.00% |
38.00% |
58.00% |
40.00% |
0.0-14 |
| 221 |
89.00% |
34.00% |
11.00% |
49.00% |
30.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 222 |
81.00% |
31.00% |
19.00% |
45.00% |
24.00% |
97.00% |
100.00% |
3.00% |
98.00% |
97.00% |
| 223 |
89.00% |
34.00% |
11.00% |
49.00% |
30.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 224 |
89.00% |
34.00% |
11.00% |
49.00% |
30.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 225 |
89.00% |
34.00% |
11.00% |
49.00% |
30.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 228 |
89.00% |
100.00% |
11.00% |
94.00% |
88.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 230 |
74.00% |
35.00% |
26.00% |
47.00% |
22.00% |
77.00% |
100.00% |
23.00% |
87.00% |
71.00% |
| 231 |
89.00% |
34.00% |
11.00% |
49.00% |
30.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 232 |
89.00% |
34.00% |
11.00% |
49.00% |
30.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 233 |
89.00% |
100.00% |
11.00% |
94.00% |
88.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 236 |
89.00% |
100.00% |
11.00% |
94.00% |
88.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 237 |
81.00% |
31.00% |
19.00% |
45.00% |
24.00% |
97.00% |
100.00% |
3.00% |
| 238 |
89.00% |
34.00% |
11.00% |
49.00% |
30.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 239 |
81.00% |
100.00% |
19.00% |
89.00% |
76.00% |
76.00% |
100.00% |
24.00% |
87.00% |
69.00% |
| 240 |
89.00% |
100.00% |
11.00% |
94.00% |
88.00% |
85.00% |
100.00% |
15.00% |
92.00% |
82.00% |
| 241 |
89.00% |
100.00% |
11.00% |
94.00% |
88.00% |
100.00% |
100.00% |
0.00% |
100.00% |
100.00% |
| 246 |
81.00% |
100.00% |
19.00% |
89.00% |
76.00% |
76.00% |
100.00% |
24.00% |
87.00% |
69.00% |
| 247 |
89.00% |
100.00% |
11.00% |
94.00% |
88.00% |
85.00% |
100.00% |
15.00% |
| 248 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
7.00% |
1.00% |
93.00% |
2.00% |
0.0-13 |
| 249 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
7.00% |
1.00% |
93.00% |
2.00% |
0.0-13 |
| 250 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
| 251 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
7.00% |
1.00% |
93.00% |
2.00% |
0.0-13 |
| 252 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
4.00% |
1.00% |
96.00% |
2.00% |
0.0-24 |
| 253 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
7.00% |
1.00% |
93.00% |
2.00% |
0.0-13 |
| 254 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
| 257 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
| 258 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
7.00% |
1.00% |
93.00% |
2.00% |
0.0-13 |
| 259 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
4.00% |
1.00% |
96.00% |
2.00% |
0.0-24 |
| 260 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
| 261 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
| 262 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
| 265 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
| 266 |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
0.00% |
0.00% |
100.00% |
NaN |
NaN |
| 301 |
42.00% |
21.00% |
58.00% |
28.00% |
0.0-8 |
55.00% |
79.00% |
45.00% |
64.00% |
13.00% |
| 302 |
35.00% |
23.00% |
65.00% |
28.00% |
0.0-18 |
NaN |
0.00% |
NaN |
NaN |
NaN |
| 303 |
n/a |
n/a |
NaN |
0.00% |
NaN |
NaN |
NaN |
|
|
|
| 304 |
78.00% |
37.00% |
22.00% |
50.00% |
26.00% |
77.00% |
95.00% |
23.00% |
85.00% |
67.00% |
