Main Menu

Home
Coordination
Research & Integration
Dissemination
Community
News
Events
Links

Workpackages

WP2: Evaluation, Integration and Standards
WP3: Visual Content Indexing
WP4: Content Description for Audio, Speech and Text
WP5: Multimodal Processing and Interaction
WP6: Machine Learning and Computation Applied to Multimedia
WP7: Dissemination towards Industry


Logo IST
Documents & Deliverables

Documents and Deliverables

Report on Benchmark-Based Evaluations

MUSCLE, the Network of Excellence on Multimedia Understanding through Semantics, Computation and Learning consolidates research groups that are exploring methods for knowledge extraction from multimedia data. Research within work-package 4 focusses on processing of text and audio data (including speech, sound and music). The aim of this work-package is to extract semantic concepts from these modalities which are then utilized for knowledge extraction, for instance in topic or genre recognition tasks. The extracted numerical data from the different modalities can be used individually, or in a combined manner, which is of particular value to the Cross-Modal Integration research work of work-package 5.

Comparability of results is a key issue in many disciplines and has proven to be extremely helpful for both collaboration and competition among research groups. Benchmarking corpora are clearly essential in order to devise well-defined tasks on which researchers can test their algorithms. In this context, we can define a corpus as an annotated collection of files, documents, or digital objects, where the annotations represent the criteria the algorithms will be evaluated against.
Deliverable JPA4-DN 4.2 (M48): Report on Benchmark-Based Evaluations (pdf)

Audio Feature Extraction, Description and Recognition Tools 

Within the MUSCLE Network of Excellence on multimedia understanding, datamining and machine learning researchers have developed a range of tools for audio analysis, speech recognition, sound description and music retrieval.
 
This deliverable represents an inventory of MUSCLE audio tools. The links from the following list will bring you to a detailed description of the individual tools, and possibilites to download them:
 
Deliverable JPA3-DN4.2 (M33): Feature Extraction Tools for Audio (pdf)
Deliverable JPA4-DN4.1 (M45) (v1.2): Audio Indexing and Recognition Tools (pdf)

Text Analysis Tools

Within the MUSCLE Network of Excellence on multimedia understanding, datamining and machine learning researchers have developed a range of tools for text analysis, text annotation, Natural Language Processing text classification and semantic indexing.
 
This deliverable represents the final inventory of text analysis tools developed:
 
Deliverable JPA3-DN4.3: Text Analysis Tools (pdf) 

3D Multimedia Environments


  • The MediaSquare

Within the showcase "Shaping 3-dimensional Multimedia Environments", a 3D Multimedia Environment named The MediaSquare was developed.

The MediaSquare is a synthetic 3D multimedia environment, that takes advantage of these spatial metaphors and allows users to explore multimedia information that is structured and organized within space. Users may engage in conversations with other members of the community, exchange experiences or simply enjoy the featured content. The MediaSquare features different scenarios, which enable the user to virtually immerse in multimedia content:
  • A 3D Music Showroom that enables users to browse and listen to songs by literally walking through them.
  • A 3D Image/Video Showroom that enables users to browse and experience visual content such as images and videos
  • The Muscle Scientific Results Showroom that enables users to explore the scientific results (i.e. documents such as Posters, paper abstracts, etc...) of Muscle in an immersive 3D environment.

 
Deliverable JPA4-DN4.3 (M40): The Media Square (download) (pdf) 


  • MUSCLE in SecondLife

For the Showcase BEYOND: The Virtual MUSCLE Experience a MUSCLE environment in the popular online 3D environment SecondLife was realized. In order to bring scientific research results to a wide public we developed virtual showrooms in SecondLife.
Users may engage in conversations with other users, exchange experiences as well as collectively explore and enjoy the featured content. On the one hand, such 3D virtual worlds address the aspect of social interaction by providing instruments to interact and to exchange experiences with other users that go beyond the possibilities of conventional text-based chat rooms. Especially ones inherent presence in space and the awareness of others facilitate the initiation of social contacts. On the other hand, using 3D virtual worlds has the advantage of communicating via commonly accepted spatial metaphors. Beyond these aspects, Virtual MUSCLE allows to browse and experience multimedia content by literally walking through it.

MUSCLE's virtual showrooms in SecondLife provide presentations of scientific results, videos and other (showcase) content developed within MUSCLE:

The MUSCLE virtual showrooms are accessible via: secondlife://MUSCLE/127/127/22.

Deliverable JPA4-DN4.3 (M48):  BEYOND: The Virtual MUSCLE Experience (pdf)

also see: