Evaluation

This is the preliminary evaluation of some of the algorithms on the ​Ontology Alignment Evaluation Initiative Test library, compared to some of the most successful competitors. Please note, that these are not yet the results of the final algorithm setup.

The algorithms evaluated her are the following:

• 2005-xxx - Competitors from the 2005 OAEI challenge
• 3-gram - Simple n-gram matching with n=3.
• PhaseTab-3G-SF - A PhaseTab algorithm with a 3-Gram and SimilarityFlooding setup.

algo	3-Gram					PhaseTab-3G+SF					2005-isti-cnr					2005-falcon					2005-foam					2005-dublin20
test	Rec.	Prec.	Fall.	FMeas.	Over.	Rec.	Prec.	Fall.	FMeas.	Over.	Rec.	Prec.	Fall.	FMeas.	Over.	Rec.	Prec.	Fall.	FMeas.	Over.	Rec.	Prec.	Fall.	FMeas.	Over.	Rec.	Prec.	Fall.	FMeas.	Over.
101	1	1	0	1	1	1	1	0	1	1	1	0.96	0.04	0.98	0.96	1	1	0	1	1	n/a	n/a	0.99	1	0	0.99	0.99
102	NaN	0	1	NaN	NaN	NaN	NaN	NaN	NaN	NaN	n/a	n/a	NaN	NaN	NaN	NaN	NaN	NaN	0	1	NaN	NaN	n/a	n/a
103	1	1	0	1	1	1	1	0	1	1	1	0.96	0.04	0.98	0.96	1	1	0	1	1	0.98	0.98	0.02	0.98	0.96	0.99	1	0	0.99	0.99
104	1	1	0	1	1	1	1	0	1	1	1	0.96	0.04	0.98	0.96	1	1	0	1	1	0.98	0.98	0.02	0.98	0.96	0.99	1	0	0.99	0.99
201	0.01	0.11	0.89	0.02	0.0-7	0.39	0.55	0.45	0.46	0.07	0.38	0.88	0.12	0.53	0.33	0.98	0.98	0.02	0.98	0.96	n/a	n/a	0.96	0.96	0.04	0.96	0.92
202	0.01	0.11	0.89	0.02	0.0-7	0.39	0.55	0.45	0.46	0.07	0.24	0.85	0.15	0.37	0.2	0.87	0.87	0.13	0.87	0.73	0.52	0.79	0.21	0.63	0.38	0.28	0.75	0.25	0.41	0.19
203	1	1	0	1	1	1	1	0	1	1	1	0.96	0.04	0.98	0.96	1	1	0	1	1	1	1	0	1	1	0.99	1	0	0.99	0.99
204	0.89	1	0	0.94	0.89	0.95	1	0	0.97	0.95	0.89	0.95	0.05	0.91	0.84	1	1	0	1	1	0.97	1	0	0.98	0.97	0.98	0.98	0.02	0.98	0.96
205	0.34	0.62	0.38	0.44	0.13	0.56	0.66	0.34	0.6	0.27	0.66	0.63	0.37	0.64	0.27	0.87	0.88	0.13	0.87	0.74	0.73	0.89	0.11	0.8	0.64	0.97	0.98	0.02	0.97	0.95
208	0.89	1	0	0.94	0.89	0.95	1	0	0.97	0.95	0.9	0.96	0.04	0.93	0.86	1	1	0	1	1	0.89	0.96	0.04	0.92	0.85	0.96	0.99	0.01	0.97	0.95
209	0.34	0.62	0.38	0.44	0.13	0.56	0.66	0.34	0.6	0.27	0.6	0.43	0.57	0.5	0.0-20	0.86	0.86	0.14	0.86	0.72	0.58	0.78	0.22	0.66	0.41	0.56	0.68	0.33
221	1	1	0	1	1	1	1	0	1	1	1	0.96	0.04	0.98	0.96	1	1	0	1	1	1	1	0	1	1	0.99	1	0
222	1	1	0	1	1	1	1	0	1	1	1	0.96	0.04	0.98	0.96	1	1	0	1	1	0.98	0.98	0.02	0.98	0.96	0.99	1	0	0.99	0.99
223	1	1	0	1	1	1	1	0	1	1	1	0.96	0.04	0.98	0.96	1	1	0	1	1	0.98	0.99	0.01	0.98	0.97	0.98	0.99	0.01	0.98	0.97
224	1	1	0	1	1	1	1	0	1	1	1	0.96	0.04	0.98	0.96	1	1	0	1	1	n/a	n/a	0.99	1	0	0.99	0.99
225	1	1	0	1	1	1	1	0	1	1	1	0.96	0.04	0.98	0.96	1	1	0	1	1	0	0	1	NaN	NaN	0.99	1	0	0.99	0.99
228	1	1	0	1	1	1	1	0	1	1	1	0.92	0.08	0.96	0.91	1	1	0	1	1	1	1	0	1	1	1	1	0	1	1
230	1	0.94	0.06	0.97	0.93	1	0.97	0.03	0.99	0.97	1	0.91	0.09	0.95	0.9	1	0.94	0.06	0.97	0.93	1	0.94	0.06	0.97	0.93	0.99	0.95	0.05	0.97	0.93
231	1	1	0	1	1	1	1	0	1	1	1	0.96	0.04	0.98	0.96	1	1	0	1	1	0.98	0.98	0.02	0.98	0.96	0.99	1	0	0.99	0.99
232	1	1	0	1	1	1	1	0	1	1	1	0.96	0.04	0.98	0.96	1	1	0	1	1	0.99	1	0	0.99	0.99	0.99	1	0
233	1	1	0	1	1	1	1	0	1	1	1	0.92	0.08	0.96	0.91	1	1	0	1	1	1	1	0	1	1	1	1	0	1	1
236	1	1	0	1	1	1	1	0	1	1	1	0.92	0.08	0.96	0.91	1	1	0	1	1	1	1	0	1	1	1	1	0
237	1	1	0	1	1	1	1	0	1	1	1	0.95	0.05	0.97	0.95	1	1	0	1	1	0.99	1	0	0.99	0.99
238	1	1	0	1	1	1	1	0	1	1	1	0.96	0.04	0.98	0.96	0.99	0.99	0.01	0.99	0.98	0.99	1	0	0.99	0.99	0.98	0.99	0.01
239	1	0.97	0.03	0.98	0.97	1	0.97	0.03	0.98	0.97	1	0.85	0.15	0.92	0.83	1	0.97	0.03	0.98	0.97	1	0.97	0.03	0.98	0.97	1	0.97	0.03	0.98	0.97
240	1	0.97	0.03	0.99	0.97	1	0.97	0.03	0.99	0.97	1	0.87	0.13	0.93	0.85	1	0.97	0.03	0.99	0.97	0.97	0.94	0.06	0.96	0.91	0.97	0.94	0.06
241	1	1	0	1	1	1	1	0	1	1	1	0.92	0.08	0.96	0.91	1	1	0	1	1	1	1	0	1	1	1	1	0	1	1
246	1	0.97	0.03	0.98	0.97	1	0.97	0.03	0.98	0.97	1	0.85	0.15	0.92	0.83	1	0.97	0.03	0.98	0.97	1	0.97	0.03	0.98	0.97	1	0.97	0.03	0.98	0.97
247	1	0.97	0.03	0.99	0.97	1	0.97	0.03	0.99	0.97	1	0.87	0.13	0.93	0.85	0.97	0.94	0.06	0.96	0.91	n/a	n/a	0.97	0.94	0.06
248	0.01	0.11	0.89	0.02	0.0-7	0.34	0.51	0.49	0.41	0.01	0.24	0.85	0.15	0.37	0.2	0.82	0.84	0.16	0.83	0.67	n/a	n/a	0.25	0.71	0.29	0.37	0.14
249	0.01	0.11	0.89	0.02	0.0-7	0.39	0.55	0.45	0.46	0.07	0.23	0.85	0.15	0.36	0.19	0.86	0.86	0.14	0.86	0.71	0.51	0.8	0.2	0.62	0.38	0.29	0.74	0.26	0.41	0.19
250	0	0	1	NaN	NaN	0.21	0.3	0.7	0.25	0.0-27	0.45	0.05	0.95	0.09	-7.0-96	0.7	0.77	0.23	0.73	0.48	0.55	1	0	0.71	0.55	0.09	1	0	0.17	0.09
251	0.01	0.11	0.89	0.02	0.0-7	0.17	0.24	0.76	0.2	0.0-36	0.25	0.85	0.15	0.38	0.2	0.69	0.69	0.31	0.69	0.38	n/a	n/a	0.32	0.79	0.21	0.46	0.24
252	0.01	0.1	0.9	0.02	0.0-8	0.21	0.3	0.7	0.25	0.0-26	0.24	0.85	0.15	0.37	0.2	0.67	0.67	0.33	0.67	0.34	0.35	0.67	0.33	0.46	0.18	0.22	0.57	0.43
253	0.01	0.11	0.89	0.02	0.0-7	0.34	0.51	0.49	0.41	0.01	0.23	0.85	0.15	0.36	0.19	0.85	0.86	0.14	0.85	0.71	0.4	0.8	0.2	0.53	0.3	0.27	0.76	0.24	0.4	0.19
254	0	0	1	NaN	NaN	0	0	1	NaN	NaN	1	0.03	0.97	0.06	-31.0-9	0.27	1	0	0.43	0.27	0.21	0.78	0.22	0.33	0.15	0	NaN	NaN	NaN	NaN
257	0	0	1	NaN	NaN	0.21	0.3	0.7	0.25	0.0-27	0.45	0.05	0.95	0.09	-7.0-96	0.64	0.7	0.3	0.67	0.36	0.64	1	0	0.78	0.64	0.09	1	0	0.17	0.09
258	0.01	0.11	0.89	0.02	0.0-7	0.17	0.24	0.76	0.2	0.0-36	0.25	0.85	0.15	0.38	0.2	0.7	0.7	0.3	0.7	0.4	n/a	n/a	0.32	0.79	0.21	0.46	0.24
259	0.01	0.1	0.9	0.02	0.0-8	0.21	0.3	0.7	0.25	0.0-26	0.24	0.85	0.15	0.37	0.2	0.68	0.68	0.32	0.68	0.36	0	NaN	NaN	NaN	NaN	0.21	0.59	0.41	0.31	0.06
260	0	0	1	NaN	NaN	0.07	0.1	0.9	0.08	0.0-58	0.86	0.05	0.95	0.09	-16.0-79	0.48	0.52	0.48	0.5	0.03	0.31	0.75	0.25	0.44	0.21	0.1	0.75	0.25	0.18	0.07
261	0	0	1	NaN	NaN	0.03	0.04	0.96	0.04	0.0-66	0.15	0.01	0.99	0.03	-10.0-39	0.48	0.5	0.5	0.49	0	0.3	0.63	0.38	0.41	0.12	0.06	0.33	0.67	0.1	0.0-6
262	0	0	1	NaN	NaN	0	0	1	NaN	NaN	1	0.03	0.97	0.06	-31.0-9	0.24	0.89	0.11	0.38	0.21	0.21	0.78	0.22	0.33	0.15	0	NaN	NaN	NaN	NaN
265	0	0	1	NaN	NaN	0.07	0.1	0.9	0.08	0.0-58	0.86	0.05	0.95	0.09	-16.0-79	0.45	0.48	0.52	0.46	0.0-3	0.31	0.75	0.25	0.44	0.21	0.1	0.75	0.25	0.18	0.07
266	0	0	1	NaN	NaN	0.03	0.04	0.96	0.04	0.0-66	0.15	0.01	0.99	0.03	-10.0-39	0.48	0.5	0.5	0.49	0	0.36	0.67	0.33	0.47	0.18	0.06	0.33	0.67	0.1	0.0-6
301	0.74	0.92	0.08	0.82	0.67	0.31	0.35	0.65	0.33	0.0-26	0.25	0.94	0.06	0.39	0.23	0.8	0.96	0.04	0.88	0.77	n/a	n/a	0.64	0.74	0.26	0.68	0.41
302	0	NaN	NaN	NaN	NaN	0	NaN	NaN	NaN	NaN	0.58	1	0	0.74	0.58	0.67	0.97	0.03	0.79	0.65	0.65	0.97	0.03	0.78	0.63	n/a	n/a
303	0	NaN	NaN	NaN	NaN	NaN	NaN	0.8	0.89	0.11	n/a	n/a	0.82	0.8	0.2	0.81	0.61	0	NaN	NaN	0.84	0.69	n/a	n/a
304	0.95	0.96	0.04	0.95	0.91	0.93	0.92	0.08	0.93	0.86	0.91	0.91	0.09	0.91	0.82	0.96	0.97	0.03	0.97	0.93	0.96	0.95	0.05	0.95	0.91	0.7	0.75	0.25	0.72	0.46
H-mean	0.59	0.91	0.09	0.71	0.53	0.67	0.79	0.21	0.72	0.49	0.71	0.34	0.66	0.46	0.0-68	0.88	0.91	0.09	0.89	0.79	0.59	0.9	0.1	0.71	0.52	0.69	0.93	0.07	0.79	0.64