Just out of the labs

Here are some of our recent research, experiments and prototypes.




Bag-of-colors for improved image search open source package

Christian Wengert, Matthijs Douze and Hervé Jégou

Overview
This paper investigates the use of color information when used within a state-of-the-art large scale image search system. We introduce a simple yet effective and efficient color signature generation procedure. It is used either to produce global or local descriptors. As a global descriptor, it outperforms several state-of-the-art color description methods, in particular the bag-of-words method based on color SIFT. As a local descriptor, our signature is used jointly with SIFT descriptors (no color) to provide complementary information. This significantly improves the recognition rate, outperforming the state of the art on two image search benchmarks. Here we provide an open source package of our signature.

Supplemental material
You can find the open source package here and the corresponding paper here. The resized and pre-processed dataset (INRIA holidays) is here. The code relies also on the yael framework, here

Shooting Star - Effortless Photo Management (2010)

Shooting Star tags your photos automatically using image recognition. Labels landmarks and places in your holiday snaps. With names, GPS location, and even related websites. Works with your favorite photo sharing platforms such as Flickr.

Try Shooting Star.

Mobile Augmented Reality (2009)

You may know that we have been playing around with live augmented reality on mobile phones for a while. While we see the whole project currently more as a technical fun project, we are still convinced that there is or will be no serious augmented reality application without image recognition capabilities. Thus, with our friends at ETH Zurich we have invested some time in a new prototype. This one runs on the Google Nexus One, and in theory would be ready for public release from a technical point of view.

More information can also be found in this blogpost.

Patents

All elements of our image recognition platform are covered by patents:

  1. "Method and system for generating a pictorial reference database", 2008.
  2. "Method and System for Image-based Information Retrieval", 2007.
  3. "Robust Interest Point Detector and Descriptor", 2006.

Scientific Publications

Our team members published and are still actively publishing research results at scientific conferences and in journals. Below follows a selection:

  1. Matthias Dantone, Lukas Bossard, Till Quack, Luc Van Gool
    Augmented Faces
    IEEE International Workshop on Mobile Vision at ICCV 2011
  2. Christian Wengert, Matthijs Douze and Hervé Jégou
    Bag-of-colors for improved image search (pdf)
    Proc. ACM Multimedia'11, November, 2011.
  3. Qin Danfeng, Stephan Gammeter, Lukas Bossard, Till Quack, Luc Van Gool
    Hello neighbor: accurate object retrieval with k-reciprocal nearest neighbors (pdf)
    CVPR 2011, June 2011, Colorado Springs, USA.
  4. Stephan Gammeter, David Tingdahl Till Quack and Luc Van Gool
    Size does matter: improving object recognition and 3D reconstruction with cross-media analysis of image clusters
    ECCV 2010, Heraklion, Creete, Greece.
  5. Stephan Gammeter, Alexander Gassmann, Lukas Bossard, Till Quack and Luc Van Gool
    Server-side object recognition and client-side object tracking for mobile augmented reality
    IEEE International Workshop on Mobile Vision, San Francisco, USA, June 13-18 2010.
  6. Stephan Gammeter, Lukas Bossard, Till Quack and Luc Van Gool
    I know what you did last summer: object-level auto-annotation of holiday snaps (pdf)
    ICCV 2009, Kyoto, Japan, 27.9.2009 - 4.10.2009
  7. Stephan Gammeter and Till Quack
    PhotoContext: An Objectlevel Autoannotation System for Your Holiday Snaps
    Demo @ ICCV 2009, Kyoto, Japan, 27.9.2009 - 4.10.2009
  8. Till Quack, Bastian Leibe and Luc Van Gool
    World-scale Mining of Objects and Events from Community Photo Collections (pdf)
    CIVR 2008, Niagara Falls, Canada, 7.-9. July 2008
  9. Till Quack, Herbert Bay and Luc Van Gool
    Object Recognition for the Internet of Things (pdf)
    Internet of Things 2008, Zurich, Switzerland
  10. Herbert Bay, Tinne Tuytelaars and Luc Van Gool
    SURF: Speeded Up Robust Features (pdf)
    ECCV 2006, Graz, Austria
  11. Till Quack, Ullrich Mönich, Lars Thiele, B.S. Manjunath,
    "Cortina: A System for Large-scale, Content-based Web Image Retrieval"
    ACM Multimedia 2004, New York, NY, USA, Oct. 2004.
Follow us!