Joint modality fusion and temporal context exploitation for semantic video analysis

Papadopoulos, Georgios Th; Mezaris, Vasileios; Kompatsiaris, Ioannis; Strintzis, Michael G.

doi:10.1186/1687-6180-2011-89

EURASIP Journal on Advances in Signal Processing

Table 3 Semantic class association results using multiple modalities and temporal context.

From: Joint modality fusion and temporal context exploitation for semantic video analysis

Domain	Measure		Audio	Motion	Color	Integrated BN for TW = 3 (1)	SVM for TW = 1 (2)	Method of[12](3)	Method of[26](4)	Method of[12]using proposed features (5)	Method of[26]using proposed features (6)
	Class recognition rate (%)	e ₁	80.43	97.83	92.03	96.38	96.38	92.75	84.78	93.48	88.41
		e ₂	13.46	78.85	76.92	86.54	71.15	53.85	28.85	67.31	57.69
		e ₃	17.39	43.48	52.17	69.57	13.04	21.74	17.39	21.74	52.17
		e ₄	40.76	68.25	69.19	88.15	94.31	88.15	97.63	88.63	90.52
Tennis	Overall accuracy (%)		49.06	77.83	76.65	89.62	87.74	81.84	80.66	83.96	83.73
	Average precision (%)		54.98	80.54	85.02	90.75	87.41	82.22	81.96	84.38	83.86
	Class recognition rate (%)	e ₁	72.88	83.05	54.24	93.22	91.53	52.54	54.24	59.32	77.97
		e ₂	67.86	75.00	14.29	67.86	10.71	7.14	21.43	14.29	21.43
		e ₃	62.64	83.91	87.36	93.68	98.85	99.43	97.13	99.43	93.10
		e ₄	12.50	62.50	78.13	96.88	93.75	81.25	81.25	81.25	84.38
News	Overall accuracy (%)		59.73	80.55	72.70	91.47	88.40	79.18	79.52	81.23	82.25
	Average precision (%)		71.24	84.09	76.67	91.58	86.03	81.51	80.88	83.75	81.50
	Class recognition rate (%)	e ₁	68.87	90.73	99.34	100.00	98.68	80.13	90.73	97.35	98.01
		e ₂	64.86	81.08	58.11	91.89	100.00	68.92	83.78	85.14	89.19
		e ₃	12.68	64.79	59.15	90.14	87.32	15.49	21.13	33.80	35.21
		e ₄	63.56	89.83	91.95	94.07	90.25	97.88	96.19	94.49	94.92
Volleyball-I	Overall accuracy (%)		58.46	85.53	84.96	94.92	93.61	77.82	82.89	85.90	87.03
	Average precision (%)		59.90	87.10	84.81	95.14	94.24	81.78	85.30	85.38	87.04
	Class recognition rate (%)	e ₁	74.59	88.52	81.15	96.72	99.18	99.18	96.72	99.18	99.18
		e ₂	41.18	88.24	58.82	88.24	35.29	23.53	23.53	64.71	23.53
		e ₃	61.67	83.33	81.67	96.67	95.00	96.67	90.00	95.00	90.00
		e ₄	36.84	57.89	47.37	78.95	84.21	68.42	57.89	78.95	52.63
		e ₅	19.72	78.87	76.06	94.37	83.10	90.14	78.87	85.92	76.06
		e ₆	79.79	54.26	80.85	94.68	93.62	93.62	91.49	96.81	96.81
Volleyball-II		e ₇	29.82	63.16	35.09	59.65	5.26	33.33	29.82	28.07	42.11
		e ₈	57.14	47.62	71.43	95.24	9.52	57.14	19.05	61.90	28.57
		e ₉	22.08	62.34	63.64	77.92	66.23	55.84	46.75	75.32	55.84
	Overall accuracy (%)		51.49	71.56	70.82	88.48	74.91	78.44	71.75	82.34	75.65
	Average precision (%)		57.18	75.66	73.75	88.86	78.74	77.86	66.19	82.49	72.65

Numbers in bold indicate the best performance among the considered methods, according to a given measure.

Back to article page