Content Analysis Showcase and Evaluation Web Portal (CASEWP)

This Showcase was integrated into the Evaluation Showcase in JPA4. Please find the results on the Evaluation Showcase page.

Leaders: Andreas Rauber, TU WIEN and Joly Philippe, institut de Recherche en Informatique de Toulouse
Partners and participants:

• TU Vienna–IFS (Thomas Lidy, Jakob Frank)
• UVA: (Nicu Sebe, Cees Snoek)
• IRIT (Julien Pinquier, Thomas Foures)
• AIIA – AUTH (Costas Kotropoulos, Emmanouil Benetos)
• University of Surrey (Bill Christmas)
• TAU (Arie Yeredor)

Goals: The goal of this showcase is to bring together and demonstrate the wide range of semantic analysis and annotation capabilities that are present within MUSCLE. Based on contributions from the participants, a video data base was compiled from TV short recordings of different genres (e.g. news reports, music clips, commercials, etc.). These were integrated and shared by all team members as well as provided in the evaluation web portal.

Showcase participants will perform whatever semantic extraction and analysis (single or multimodal) they can apply to the videos, such as all kinds of low-level feature extraction, face detection, moving objects, fire and smoke detection, logo detection, music genre analysis, music segment clustering, speech recognition, text detection and recognition, etc. To achieve this, partners are allowed to use all kind of algorithms, additional external information, as well as additional data they may have and use within their own labs to enhance the information extracted from the video.

The CASEWP web portal is particularly meant to evaluate results of temporal segmentation tools. The web portal will provide all the necessary environment resources (e.g. free data sets and annotations) and will enable evaluations of state-of-the-art methods outside the constrained timelines of scientific evaluation campaigns. The goal here is to propose an online evaluation tool to the research community, which objectively measure various temporal segmentation results “on demand”, and which indirectly promote the best technology.

Results of the showcase demonstrate the variety of current semantic analysis algorithms developed within MUSCLE on a single common database.

Showcase Tasks:

• Data collection: short video recordings from TV
• Setup of a common server directory for the database and features
• Data cleansing and conversion into common format
• Separation of modalities (audio, still images)
• Preparation of the Evaluation Web Portal
• Extraction of features from the individual modalities (audio, images, video)
• Applcation of all different kinds of description & analysis methods available at labs of the participants
• Exchange descriptors
• Demonstration of results of semantic extraction (classification, clustering) on a web page
• Creation of an interactive web page allowing browsing of content according to different modalities and content descriptors
• Analysis and classification, description and clustering results based on combinations of descriptors
• Publication of Evaluation Web Portal