2018 ICPR Multimedia Information Processing for Personality & Social Networks Analysis Challenge - DivFusion

Track description

Diversification of image search results is now a hot research problem in multimedia. Search engines are fostering techniques that allow for providing the user with a diverse representation of his search results, rather than providing redundant information, e.g. the same perspective of a monument, or location etc. The DivFusion task builds on the MediaEval Retrieving Diverse Social Images Tasks and challenges the participants to develop highly effective information fusion techniques for social image search results diversification.

Participation in this task involves the following steps:

1. Design your algorithms on the development data (devset): download the development data and design your approach for the task. These data comes with ground truth;
2. Validate and optimize your algorithms on the validation data (validset): download the validation data and evaluate the performance of your algorithms. Optimize the parameters and the performance. These data comes with ground truth;
3. Test your algorithms on the test data (testset): download the test data, build your final runs and submit them to the challenge. You are allowed to submit only 5 runs during the entire duration of the task. The ground truth for this data is not available; 
4. Receive your evaluation results: your results on test data are available in real-time on the challenge leaderboard, where you can compare them against a baseline and other participant results.

The DivFusion Task is run via the Codalab platform. To access the task follow this link (click on the link).

Goal of task

Participants will receive a list of image search queries with up to 300 photos retrieved from Flickr and ranked with Flickr’s default "relevance" algorithm. These data are accompanied by various metadata and content descriptors. Each query comes also with a variety of diversification system outputs (participant runs from previous years).

______________________________________________________________________________

The requirements of the task are to fuse the provided systems' outputs and return a ranked list of up to 50 photos that are both relevant and diverse representations of the query.

______________________________________________________________________________

Relevance: a photo is considered to be relevant for the query if it is a common photo representation of all query concepts at once. Low quality photos (e.g., severely blurred, out of focus, etc.) are not considered relevant in this scenario.

Diversity: a set of photos is considered to be diverse if it depicts different visual characteristics of the query topics and subtopics, e.g., sub-locations, temporal information, typical actors/objects, genesis information, different views at different times of the day/year and under different weather conditions, close-ups on architectural details, sketches, creative views, with a certain degree of complementarity, i.e., most of the perceived visual information is different from one photo to another.

To have an illustration of what relevance and diversified results are, participants may refer to the ground truth of the development and validation data.

Use Scenario

The provided data are using two use case scenarios: (i) a tourist (single-topic query) scenario where a person tries to find more information about a place or event she might visit or attend and is interested in getting a more complete visual description of the target; (ii) a general ad-hoc (multi-topic query) scenario where the user searches for general-purpose images.

Submissions

Participants can submit results on testset data during the entire duration of the challenge. You are allowed a total of 5 runs. Submissions are to be uploaded in the CodaLab platform (see the track  description). Results are available in real-time on the provided leaderboard (see the results section in CodaLab).

Recommended Reading

[1] C. G. M. Snoek, M. Worring, and A. W. M. Smeulders, Early Versus Late Fusion in Semantic Video Analysis, Proceedings of ACM International Conference on Multimedia (MM), 2005.

[2] X. Benavent, A. Garcia-Serrano, R. Granados, J. Benavent and E. de Ves, Multimedia Information Retrieval Based on Late Semantic Fusion Approaches: Experiments on a Wikipedia Image Collection, IEEE Transactions on Multimedia, 15(8), pp. 2009-2021, 2013.

[3] B. Ionescu, A. L. Gînscă, M. Zaharieva, B. Boteanu, M. Lupu, and H. Müller, Retrieving Diverse Social Images at MediaEval 2016: Challenge, Dataset and Evaluation, Proceedings of MediaEval Benchmarking Initiative for Multimedia Evaluation, vol. 1739, 2016.

[4] B. Ionescu, A. L. Gînscă, B. Boteanu, M. Popescu and H. Müller, Div150multi: A Social Image Retrieval Result Diversification Dataset with Multi-topic Queries, Proceedings of ACM International Conference on Multimedia Systems (MMSys), 2016.

[5] B. Ionescu, A. Popescu, M. Lupu, A. L. Gînscă, B. Boteanu and H. Müller, Div150cred: A Social Image Retrieval Result Diversification with User Tagging Credibility Dataset, Proceedings of ACM International Conference on Multimedia Systems (MMSys), 2015.

[6] B. Ionescu, A. Popescu, A.-L. Radu, H. Müller, Result Diversification in Social Image Retrieval: a Benchmarking Framework. Multimedia Tools and Applications, 75(2), pp. 1301–1331, 2016.

[7] B. Ionescu, A. Popescu, A.-L. Radu, M.  Menéndez, M. Müller, H. Popescu, and A. Loni, Div400: A Social Image Retrieval Result Diversification Dataset, Proceedings of ACM International Conference on Multimedia Systems (MMSys), 2014.