About Behold

Behold^TM is a search engine for high-quality Flickr images. It aims to answer your queries based on what is inside the images -- at the pixel level. It offers a completely new way to search for images, using techniques of computer vision. It is different to standard image search engines, such as Flickr or Google, because those search through images using only image tags and filenames.

Behold looks for high quality images, so you don't have to sift through hundreds of poorly taken pictures to find a good one. Behold uses both aesthetic and technical quality indicators to find some of the best images available online.

Behold draws computational power from Amazon Elastic Compute Cloud (EC2) to handle large volumes of images.

Features

Behold is capable of recognising a number of visual concepts in pictures. You can ask Behold to return images that look like one of these concepts. This new type of search can be flexibly combined with regular text-based search. For example you can ask Behold to return images tagged with the word 'london' that look like pictures of buildings (try it!). You can also filter text-based image search results based on what the images actually look like. Both of these features are demonstrated in these videos.

With a newly introduced feature, Behold goes one step further and automatically suggests visual filters after analysing the words in your query. It shows you what your search results would look like if you apply one of these filters, so you save time on finding the right one. This feature is demonstrated in these two videos.

Examples

Recognising visual concepts in pictures

The video clip below shows how Behold can visually filter tag-based search on 100,000 Flickr photos from the Flickr Unofficial JPEG Magazine group.

The Camtasia Studio video content presented here requires JavaScript to be enabled and the latest version of the Macromedia Flash Player. If you are you using a browser with JavaScript disabled please enable it now. Otherwise, please update your version of the free Flash Player by downloading here.

And the following video clip shows another way to combine Behold's image recognition with tag-based search when looking for dog faces in the same set of photos.

The table below demonstrates how Behold's image analysis improves a number of standard tag-only searches on Flickr.

Behold search Standard tag-only search

Bus Bus

Paris building Paris building

Coast Coast

Tokyo skyline Tokyo skyline

Automatic visual filter suggestions based on query analysis

The following two videos demonstrate how queries eagle and skyscraper are processed by Behold.

When searching for eagle, Behold offers visual filters animal, face and bird to refine the search further.

And when searching for skyscraper, Behold offers visual filters building, skyline, texture and tower.

What happens now?

Behold's technology is in the early stages of its development. We expect significant improvements in Behold's visual image search quality over the coming weeks and months. We are currently working on three fronts:

teaching Behold to recognise more concepts in images
improving the quality of concept recognition
searching larger volumes of images

How does it work?

Behold's visual concept search is based on the technique called "automated image annotation". This technique calculates probabilities of concepts being relevant to images based on the pixel content of each image. The models for calculating concept probabilities are estimated using manually annotated training images. Behold implements the automated annotation model proposed in Alexei Yavlinsky's PhD thesis.

Contact

Alexei Yavlinsky