MUSCLE aims at creating and supporting a pan-European Network of Excellence to foster close collaboration between research groups in multimedia datamining on the one hand, and machine learning on the other in order to make breakthrough progress towards the following objectives: - Harnessing the full potential of machine learning and cross-modal interaction for the (semi-) automatic generation of meta-data with high semantic content for multimedia documents.
- Applying machine learning for the creation of expressive, context-aware, self-learning, and human-centered interfaces that will be able to effectively assist users in the exploration of complex and rich multimedia content.
- Improving interoperability and exchangeability of heterogeneous and distributed (meta)data by enabling data descriptions of high semantic content (e.g. ontologies, MPEG7 and XML schemata) and inference schemes that can reason about these at the appropriate levels.
- Ensure durable integration and collaboration through the creation of a virtual lab that facilitates the easy and immediate access to people, data and ideas.
- Through dissemination, training and industrial liaison, contribute to the distribution and uptake of the technology by relevant end-users such as industry, education, and the service sector. In particular, close interactions with other IPs and NoEs in this and related activity fields are planned.
- Through accomplishing the above, facilitate the broad and democratic (i.e. obviating the need for special expertise) access to information and knowledge for all European citizens (e.g. e-Education, enriched cultural heritage).
Network cohesion and integration based on two Grand Challenges
To stimulate cohesion, the NoE will set itself two grand challenges. - Grand Challenge #1: Natural high-level interaction with multimedia databases In this vision it is possible to query a multimedia database at a high semantic level. Think Ask Jeeves for multimedia content, where one can address a search engine using natural language and it will take appropriate action, or at least ask intelligent, clarifying questions. This is an extremely challenging problem and will involve a wide range of techniques: natural language processing, interfacing technology, learning and inferencing, merging of different modalities, federation of complex meta-data, appropriate representation and interfaces, etc.
- Grand Challenge #2: Detecting and interpreting humans and human behaviour in videos Many important applications of multimedia data mining revolve around the detection and interpretation of human behaviour. Applications are legion: surveillance and intrusion detection, face recognition and registration of emotion or affect, automatic analysis of sports videos and movies, etc. Again, success will depend heavily on the integration and interpretation of various modalities such as vision, audio and speech.
|
|
With the start of JPA3 the Network is entering the second half of its lifecycle it therefore becomes increasingly more important to engage outside parties, industry in particular. To this end it is clearly helpful to take advantage of a number of EC-backed initiatives currently underway aimed at drawing in more substantial involvement and commitment from commercial partners. Particularly noteworthy in this respect is the Networked and Electronic Media Platform (NEM, www.nem-initiative.org) which has been set up by the European Commission and brings together most of Europe's main industrial stakeholders in ICT and consumer electronics. Of the five priority areas, at least two are of particular interest to the Muscle Consortium: - Content: Adaptation, personalisation, context awareness, ambient intelligence, content summarising and indexation, semantic searching for content;
- Enabling Technologies:Metadata, multimedia search engines, natural and multimodal user interfaces, human language technologies, multimedia analysis and computer vision (object recognition and tracking, data fusion).
It goes without saying that this sort of industry-driven technology pull offers the MUSCLE consortium a timely and welcome opportunity to intensify its knowledge transfer and forge new research partnerships in anticipation of emerging FP7 opportunities. The Steering Committee has therefore decided that the best way to engage the industry's interest is to create a number of showcases centered around the Grand Challenges to be used as proof of concept. Ideally, these showcases should be at the level of systems, rather than components, and as a consequence involve middle-sized groups of contributors (e.g. E-teams). It is felt that although a number of components ( typically created by single-lab teams) are already available, the integration into more complex systems that transcend individual efforts is still largely lacking. Furthermore, stimulating cross-lab integration of software components will undoubtedly result in closer ties between the partners with exchange visits and transfer of expertise and software. Information about Workpackages: here |
|
WP1: Coordination- Scientific Coordination: Eric Pauwels (CWI)
- Administrative Coordination: Remi Ronchaud (ERCIM)
WP2: Building the Virtual Lab- WP Leader: Eric Pauwels (CWI)
- The primary objective of the NoE Virtual Lab is to facilitate collaboration and integration by providing an electronic portal to the knowledge available in the MUSCLE network and to encourage the sharing of expertise and ideas among the members of the network.
- Web site: n/a
WP3: Benchmarking- WP Leader: Allan Hanbury (PRIP TU-Vienna)
- During the lifetime of the MUSCLE NoE, a large number of algorithms and complete systems for use in the automatic extraction of semantic information from multimedia data will be developed. This workpackage aims to develop objective methods for comparing these algorithms, and to encourage the use of these methods.
- Web site
WP4: Dissemination and Training- WP Leader: Panos Trahanias (FORTH)
- Expertise accumulated in the Network will be disseminated through contacts with industry and academia, Fellowships, summerschools and contacts with other Networks and Integrated Projects.
- Web site
WP5: Content-based Description- WP Leader: Nozha Boujemaa (INRIA-IMEDIA)
- This workpackage addresses the problem of designing various content-based descriptors for MM modalities, such as still images and video, speech and audio, text and natural language.
- Web site
WP6: Cross-Modal Integration for Multimedia Content- WP Leader: Petros Maragos (NTUA-ICCS)
- This work package addresses research on the theory and applications of multimedia analysis approaches that improve robustness and performance through cross-modal interaction and/or integration.
- Web site
WP7: Computation Intensive Methods - WP Leader: Simon Wilson (TCD)
- The challenge for this WP is to develop computational methods that successfully implement solutions to problems in MM-understanding. These solutions contain large and/or complex data, and require complex modeling.
- Web site : n/a
WP8: Machine Learning for Multimedia Content- WP Leader: Padraig Cunningham (TCD)
- This workpackage aims to explore different ways in which machine learning (ML) can contribute to the automatic categorization, marking up and exploration of multimedia data.
- Web site: n/a
WP9: Representation and Communication of Data and Meta-Data- WP Leader: Ovidio Salvetti (CNR)
- Enabling interaction and exchange of meta-data emanating from different MM modalities requires standardization of data and meta-data formats.
- Web site
WP10: Human-Computer Interface for Multimedia Retrieval - WP Leader: Alexandros Potamianos (TSI-TUC)
- Multimedia information retrieval via an interactive human-computer interface is a complex task that requires feedback from the user and a complex negotiation between the user and the machine. We propose to research, design and build natural and efficient human-computer interfaces for performing multimedia information retrieval tasks that allow for negotiation (dialogue) between the user and the system.
- Web site
WP11: Integration and Grand Challenges- WP Leader: Enis Cetin (Bilkent Univ)
- The main objective of this WP will be to stimulate integration and collaboration between various partners by conducting research related to the Grand Challenges that have been put forward by Consortium. The two Grand Challenges of the Network are
1. Natural high-level interaction with multimedia databases, and
2. Detecting and interpreting humans and human behaviour in videos.
|
| |
|
|
Research & Integration 