wiki:evaluation

Evaluation

This is the preliminary evaluation of some of the algorithms on the Ontology Alignment Evaluation Initiative Test library, compared to some of the most successful competitors. Please note, that these are not yet the results of the final algorithm setup.

The algorithms evaluated her are the following:

  • 2005-xxx - Competitors from the 2005 OAEI challenge
  • 3-gram - Simple n-gram matching with n=3.
  • PhaseTab-3G-SF - A PhaseTab algorithm with a 3-Gram and SimilarityFlooding setup.
algo 3-Gram PhaseTab-3G+SF 2005-isti-cnr 2005-falcon 2005-foam 2005-dublin20
test Rec. Prec. Fall. FMeas. Over. Rec. Prec. Fall. FMeas. Over. Rec. Prec. Fall. FMeas. Over. Rec. Prec. Fall. FMeas. Over. Rec. Prec. Fall. FMeas. Over. Rec. Prec. Fall. FMeas. Over.
101 1 1 0 1 1 1 1 0 1 1 1 0.96 0.04 0.98 0.96 1 1 0 1 1 n/a n/a 0.99 1 0 0.99 0.99
102 NaN 0 1 NaN NaN NaN NaN NaN NaN NaN n/a n/a NaN NaN NaN NaN NaN NaN 0 1 NaN NaN n/a n/a





103 1 1 0 1 1 1 1 0 1 1 1 0.96 0.04 0.98 0.96 1 1 0 1 1 0.98 0.98 0.02 0.98 0.96 0.99 1 0 0.99 0.99
104 1 1 0 1 1 1 1 0 1 1 1 0.96 0.04 0.98 0.96 1 1 0 1 1 0.98 0.98 0.02 0.98 0.96 0.99 1 0 0.99 0.99
201 0.01 0.11 0.89 0.02 0.0-7 0.39 0.55 0.45 0.46 0.07 0.38 0.88 0.12 0.53 0.33 0.98 0.98 0.02 0.98 0.96 n/a n/a 0.96 0.96 0.04 0.96 0.92


202 0.01 0.11 0.89 0.02 0.0-7 0.39 0.55 0.45 0.46 0.07 0.24 0.85 0.15 0.37 0.2 0.87 0.87 0.13 0.87 0.73 0.52 0.79 0.21 0.63 0.38 0.28 0.75 0.25 0.41 0.19
203 1 1 0 1 1 1 1 0 1 1 1 0.96 0.04 0.98 0.96 1 1 0 1 1 1 1 0 1 1 0.99 1 0 0.99 0.99
204 0.89 1 0 0.94 0.89 0.95 1 0 0.97 0.95 0.89 0.95 0.05 0.91 0.84 1 1 0 1 1 0.97 1 0 0.98 0.97 0.98 0.98 0.02 0.98 0.96
205 0.34 0.62 0.38 0.44 0.13 0.56 0.66 0.34 0.6 0.27 0.66 0.63 0.37 0.64 0.27 0.87 0.88 0.13 0.87 0.74 0.73 0.89 0.11 0.8 0.64 0.97 0.98 0.02 0.97 0.95
208 0.89 1 0 0.94 0.89 0.95 1 0 0.97 0.95 0.9 0.96 0.04 0.93 0.86 1 1 0 1 1 0.89 0.96 0.04 0.92 0.85 0.96 0.99 0.01 0.97 0.95
209 0.34 0.62 0.38 0.44 0.13 0.56 0.66 0.34 0.6 0.27 0.6 0.43 0.57 0.5 0.0-20 0.86 0.86 0.14 0.86 0.72 0.58 0.78 0.22 0.66 0.41 0.56 0.68 0.33
221 1 1 0 1 1 1 1 0 1 1 1 0.96 0.04 0.98 0.96 1 1 0 1 1 1 1 0 1 1 0.99 1 0
222 1 1 0 1 1 1 1 0 1 1 1 0.96 0.04 0.98 0.96 1 1 0 1 1 0.98 0.98 0.02 0.98 0.96 0.99 1 0 0.99 0.99
223 1 1 0 1 1 1 1 0 1 1 1 0.96 0.04 0.98 0.96 1 1 0 1 1 0.98 0.99 0.01 0.98 0.97 0.98 0.99 0.01 0.98 0.97
224 1 1 0 1 1 1 1 0 1 1 1 0.96 0.04 0.98 0.96 1 1 0 1 1 n/a n/a 0.99 1 0 0.99 0.99
225 1 1 0 1 1 1 1 0 1 1 1 0.96 0.04 0.98 0.96 1 1 0 1 1 0 0 1 NaN NaN 0.99 1 0 0.99 0.99
228 1 1 0 1 1 1 1 0 1 1 1 0.92 0.08 0.96 0.91 1 1 0 1 1 1 1 0 1 1 1 1 0 1 1
230 1 0.94 0.06 0.97 0.93 1 0.97 0.03 0.99 0.97 1 0.91 0.09 0.95 0.9 1 0.94 0.06 0.97 0.93 1 0.94 0.06 0.97 0.93 0.99 0.95 0.05 0.97 0.93
231 1 1 0 1 1 1 1 0 1 1 1 0.96 0.04 0.98 0.96 1 1 0 1 1 0.98 0.98 0.02 0.98 0.96 0.99 1 0 0.99 0.99
232 1 1 0 1 1 1 1 0 1 1 1 0.96 0.04 0.98 0.96 1 1 0 1 1 0.99 1 0 0.99 0.99 0.99 1 0
233 1 1 0 1 1 1 1 0 1 1 1 0.92 0.08 0.96 0.91 1 1 0 1 1 1 1 0 1 1 1 1 0 1 1
236 1 1 0 1 1 1 1 0 1 1 1 0.92 0.08 0.96 0.91 1 1 0 1 1 1 1 0 1 1 1 1 0
237 1 1 0 1 1 1 1 0 1 1 1 0.95 0.05 0.97 0.95 1 1 0 1 1 0.99 1 0 0.99 0.99
238 1 1 0 1 1 1 1 0 1 1 1 0.96 0.04 0.98 0.96 0.99 0.99 0.01 0.99 0.98 0.99 1 0 0.99 0.99 0.98 0.99 0.01
239 1 0.97 0.03 0.98 0.97 1 0.97 0.03 0.98 0.97 1 0.85 0.15 0.92 0.83 1 0.97 0.03 0.98 0.97 1 0.97 0.03 0.98 0.97 1 0.97 0.03 0.98 0.97
240 1 0.97 0.03 0.99 0.97 1 0.97 0.03 0.99 0.97 1 0.87 0.13 0.93 0.85 1 0.97 0.03 0.99 0.97 0.97 0.94 0.06 0.96 0.91 0.97 0.94 0.06
241 1 1 0 1 1 1 1 0 1 1 1 0.92 0.08 0.96 0.91 1 1 0 1 1 1 1 0 1 1 1 1 0 1 1
246 1 0.97 0.03 0.98 0.97 1 0.97 0.03 0.98 0.97 1 0.85 0.15 0.92 0.83 1 0.97 0.03 0.98 0.97 1 0.97 0.03 0.98 0.97 1 0.97 0.03 0.98 0.97
247 1 0.97 0.03 0.99 0.97 1 0.97 0.03 0.99 0.97 1 0.87 0.13 0.93 0.85 0.97 0.94 0.06 0.96 0.91 n/a n/a 0.97 0.94 0.06
248 0.01 0.11 0.89 0.02 0.0-7 0.34 0.51 0.49 0.41 0.01 0.24 0.85 0.15 0.37 0.2 0.82 0.84 0.16 0.83 0.67 n/a n/a 0.25 0.71 0.29 0.37 0.14


249 0.01 0.11 0.89 0.02 0.0-7 0.39 0.55 0.45 0.46 0.07 0.23 0.85 0.15 0.36 0.19 0.86 0.86 0.14 0.86 0.71 0.51 0.8 0.2 0.62 0.38 0.29 0.74 0.26 0.41 0.19
250 0 0 1 NaN NaN 0.21 0.3 0.7 0.25 0.0-27 0.45 0.05 0.95 0.09 -7.0-96 0.7 0.77 0.23 0.73 0.48 0.55 1 0 0.71 0.55 0.09 1 0 0.17 0.09
251 0.01 0.11 0.89 0.02 0.0-7 0.17 0.24 0.76 0.2 0.0-36 0.25 0.85 0.15 0.38 0.2 0.69 0.69 0.31 0.69 0.38 n/a n/a 0.32 0.79 0.21 0.46 0.24


252 0.01 0.1 0.9 0.02 0.0-8 0.21 0.3 0.7 0.25 0.0-26 0.24 0.85 0.15 0.37 0.2 0.67 0.67 0.33 0.67 0.34 0.35 0.67 0.33 0.46 0.18 0.22 0.57 0.43
253 0.01 0.11 0.89 0.02 0.0-7 0.34 0.51 0.49 0.41 0.01 0.23 0.85 0.15 0.36 0.19 0.85 0.86 0.14 0.85 0.71 0.4 0.8 0.2 0.53 0.3 0.27 0.76 0.24 0.4 0.19
254 0 0 1 NaN NaN 0 0 1 NaN NaN 1 0.03 0.97 0.06 -31.0-9 0.27 1 0 0.43 0.27 0.21 0.78 0.22 0.33 0.15 0 NaN NaN NaN NaN
257 0 0 1 NaN NaN 0.21 0.3 0.7 0.25 0.0-27 0.45 0.05 0.95 0.09 -7.0-96 0.64 0.7 0.3 0.67 0.36 0.64 1 0 0.78 0.64 0.09 1 0 0.17 0.09
258 0.01 0.11 0.89 0.02 0.0-7 0.17 0.24 0.76 0.2 0.0-36 0.25 0.85 0.15 0.38 0.2 0.7 0.7 0.3 0.7 0.4 n/a n/a 0.32 0.79 0.21 0.46 0.24


259 0.01 0.1 0.9 0.02 0.0-8 0.21 0.3 0.7 0.25 0.0-26 0.24 0.85 0.15 0.37 0.2 0.68 0.68 0.32 0.68 0.36 0 NaN NaN NaN NaN 0.21 0.59 0.41 0.31 0.06
260 0 0 1 NaN NaN 0.07 0.1 0.9 0.08 0.0-58 0.86 0.05 0.95 0.09 -16.0-79 0.48 0.52 0.48 0.5 0.03 0.31 0.75 0.25 0.44 0.21 0.1 0.75 0.25 0.18 0.07
261 0 0 1 NaN NaN 0.03 0.04 0.96 0.04 0.0-66 0.15 0.01 0.99 0.03 -10.0-39 0.48 0.5 0.5 0.49 0 0.3 0.63 0.38 0.41 0.12 0.06 0.33 0.67 0.1 0.0-6
262 0 0 1 NaN NaN 0 0 1 NaN NaN 1 0.03 0.97 0.06 -31.0-9 0.24 0.89 0.11 0.38 0.21 0.21 0.78 0.22 0.33 0.15 0 NaN NaN NaN NaN
265 0 0 1 NaN NaN 0.07 0.1 0.9 0.08 0.0-58 0.86 0.05 0.95 0.09 -16.0-79 0.45 0.48 0.52 0.46 0.0-3 0.31 0.75 0.25 0.44 0.21 0.1 0.75 0.25 0.18 0.07
266 0 0 1 NaN NaN 0.03 0.04 0.96 0.04 0.0-66 0.15 0.01 0.99 0.03 -10.0-39 0.48 0.5 0.5 0.49 0 0.36 0.67 0.33 0.47 0.18 0.06 0.33 0.67 0.1 0.0-6
301 0.74 0.92 0.08 0.82 0.67 0.31 0.35 0.65 0.33 0.0-26 0.25 0.94 0.06 0.39 0.23 0.8 0.96 0.04 0.88 0.77 n/a n/a 0.64 0.74 0.26 0.68 0.41


302 0 NaN NaN NaN NaN 0 NaN NaN NaN NaN 0.58 1 0 0.74 0.58 0.67 0.97 0.03 0.79 0.65 0.65 0.97 0.03 0.78 0.63 n/a n/a


303 0 NaN NaN NaN NaN NaN NaN 0.8 0.89 0.11 n/a n/a 0.82 0.8 0.2 0.81 0.61 0 NaN NaN 0.84 0.69 n/a n/a





304 0.95 0.96 0.04 0.95 0.91 0.93 0.92 0.08 0.93 0.86 0.91 0.91 0.09 0.91 0.82 0.96 0.97 0.03 0.97 0.93 0.96 0.95 0.05 0.95 0.91 0.7 0.75 0.25 0.72 0.46
H-mean 0.59 0.91 0.09 0.71 0.53 0.67 0.79 0.21 0.72 0.49 0.71 0.34 0.66 0.46 0.0-68 0.88 0.91 0.09 0.89 0.79 0.59 0.9 0.1 0.71 0.52 0.69 0.93 0.07 0.79 0.64
Last modified 11 years ago Last modified on 08/10/06 12:13:56