If you’ve ever visualized something in your head but couldn’t think of its name, you might appreciate a new method of online discovery: visual search.
Screenshot of Google Image Swirl
This week, I tested forms of visual search from two companies that hold some serious clout when it comes to hunting around online–Google and Microsoft. Although Google has become our go-to site for looking anything up on the Internet, its searches are dense with text. Microsoft’s Bing search engine, which was introduced last spring, is marketed as a Google alternative that aims to return more useful query data on the first results page.
Both companies know there are times when text, alone, just won’t do. Google (GOOG) and Microsoft (MSFT) have long offered options for searching the Images section of almost any search term to find a visual representation of it. But now the companies are allowing visually minded users to scour through images to more efficiently pinpoint the picture or information they want. These new visual searches are a bit different. And they also differ from one another.
Users can use Google’s Image Swirl search to sift through some 200,000 queries of images. And Microsoft offers Bing Visual Search as a way of performing searches on images that are tagged with useful data. Google Image Swirl still requires you to input text search terms, but Bing Visual Search lets you select images the whole time, without typing search terms. The ability to search using images alone is also being explored, and a number of mobile apps make this possible, which I’ll briefly talk about in a bit.
Google’s Image Swirl, http://image-swirl.googlelabs.com/, is currently categorized by the company as a Google Labs project, meaning that it’s in an experimental stage. It lets users search for images in certain categories that, according to computer vision algorithms, look like they would fit into the search results. Unlike Google queries using the “Images” section, Image Swirl sorts results into several stacks of images, with the most relevant results on the top of each stack. This makes for less image repetition in results, compared with regular image searches.
These stacks of images come in handy in cases where one word has two meanings, so users can select the one that represents what they’re searching for. Image Swirl also can be used to discover images of a place or thing that you didn’t originally associate with the search term.
By clicking on the top image in a stack, users can see a diagram of the main image positioned in a center circle and related images connected by lines that resemble bicycle spokes. Selecting one image pulls it to the center of the circle and repositions its surrounding photos. A search for “Robert Downey, Jr.” displayed several stacks—each topped with different images of him. There was a stack of pictures of him dressed as different movie characters, one of him at movie premieres, and a stack of his mug-shot arrest photos.
Hometown Search
Presumably because it’s an experiment, Image Swirl doesn’t cover a lot of topics. I typed “Allentown, PA,” the name of my hometown, into the Image Swirl search box and received a message that said my query wasn’t included in the demo.
Since computer vision algorithms can make mistakes, Image Swirl can pull up images that aren’t relevant to the intended search. My search for “George Washington Bridge” pulled up photos of the bridge at different times of the day from different angles, divided into stacks. But one photo was of a Marvel Comics character named G.W. Bridge. Another was of bikes on pavement, a photo from a Web site for “Bike Month NYC” that mentioned the bridge.
While Google’s Image Swirl works well as an image search engine, Bing Visual Search is a collection of 48 galleries of photos and is designed to be a data search engine by associating each image with specific data.
For example, a search for “Famous Directors” is sorted alphabetically. Each image displays data about the person it represents when you hover over it with a cursor. Steven Spielberg’s image text tells me he’s 63 years old, directed 26 films and won two Oscars, and that his highest grossing film was “Jurassic Park,” at $919.7 million. A list on the left side provides categories with which I can narrow the search results. In the case of the “Famous Directors” gallery, these categories include gender, country of origin, and what genre he or she is best known for directing.
Some of the Visual Search galleries include digital cameras, dog breeds, world leaders, top iPhone apps and yoga poses. Each has its own detailed description and left-side subcategories that can be selected for narrowing down the results. But these Bing Visual Search categories represent images only from sources that have teamed up with Bing, like Fox Sports, Billboard and the American Film Institute. Google searches a larger pool of data from Google Images, which crawls the entire Web.
The Bing Visual Search results have all been pre-sorted and tagged to associate with a search term. Bing Visual Search is especially helpful with product searches, since each image has a good deal of information associated with it, including price, product reviews and brand. Some items can even be purchased directly from these links.
After searching with either Google Image Swirl or Bing Visual Search, the final click on an item often takes users to a more text-based Web page, where people can dig deeper into the details of the searched item, like a plain, text search. But first seeing an image could help to narrow the field—or expand a search to include something else that wasn’t originally intended.
Augmented Reality
For people looking to take visual search quite literally (without typing any text at all), mobile devices with built-in cameras can let people point and search in a different way from either Image Swirl or Visual Search.Thanks to the integration of augmented reality (AR)—a way of matching real-world photos with computer-generated images—into mobile apps, users can aim their device at something and the image can then be used to identify the subject, as well as details about it.
Screenshot of Bing Visual Search
I tried three apps on Google’s Nexus One mobile device and Apple’s (AAPL) iPhone: Google Goggles, SnapTell and Layar. SnapTell retrieved much search data about two books I captured in photos.
Google Goggles is a visual-search application that works on phones running Google’s Android operating system. With Goggles, people could take photos of the outside of a restaurant and learn its name, menu or read customer reviews. Likewise, snapping a photo of a piece of art will return details like its title and artist, as well as a Web link to more information. Google says Goggles will be coming to other mobile platforms in the future.
This technology brings up a potential privacy issue: Could you some day take a photo of someone and then search for information on that person?
A Google spokesperson says this app has the ability to use facial recognition with Goggles, but hasn’t launched this feature because it hasn’t been built into an app that would provide real value for users. The spokesperson also cites “some important transparency and consumer-choice issues we need to think through.”
A Walk With the Beatles
SnapTell (http://snaptell.com/apps) is another app that uses AR on Android devices as well as Apple’s iPhone. It allows you to snap a photo of a book, CD, videogame or DVD, and get information about it. Layar (http://layar.com) is an app that lets people point their Android devices at locations to get more information. You could see an on-screen visual of a completed structure by pointing the camera at a construction site, or look at a representation of the Beatles on Abbey Road by pointing your phone at the famous crosswalk.
If you’re a visual thinker and you work well by seeing illustrations of the things for which you search, Bing Virtual Search or Google Image Swirl might help. Or consider using an app with your mobile device that takes advantage of AR technology if you want fast information about something while you’re on the go. As all of these products improve, they’ll include more categories and images to aid online explorations.
Edited by Walter S. Mossberg. Email mossbergsolution@wsj.com
