AllThingsD » Amit Singhal

The Evolution of Search (As Told by Google)

Liz Gannes — Mon, 28 Nov 2011 17:59:33 +0000

Google has been the major force in the technology, interface design and business of search over the last 15 years. But even so, the premise of its new infomercial seems to be based a bit more in creationism than evolution.

The company this morning released a video, titled “The Evolution of Search in Six Minutes,” that compiles interviews with its search leaders and takes a historical look at the major breakthroughs in search — well, at least those that have occurred since 1997, when the Google.com domain was registered.

So, what would be a hard query that Google wants to answer in the future? Complex questions that take reasoning, says Google Fellow Amit Singhal. “In my ideal world, I would be able to walk up to a computer and say, ‘Hey, what is the best time for me to sow seeds in India, given that monsoon was early this year?’” Singhal says in the video.

Nerd Out With Google's Search Gurus (Video)

Liz Gannes — Mon, 08 Aug 2011 15:30:43 +0000

Amit Singhal, Ben Gomes and Matt Cutts are longtime leaders of Google’s search team. Search analyst Danny Sullivan calls them “the brains” (Singhal does Google’s search ranking algorithm); “the looks” (Gomes works on the interface); and “the brawn” (Cutts fights spam) of Google search.

In a rare joint public appearance, the four men dove deeply into the big issues facing search at a Churchill Club event last week.

For instance, Singhal fended off Eli Pariser’s “filter bubble” critique, the idea that online personalization presents a skewed view of the world that leaves out important things like opposing viewpoints.

Singhal said that personalization is a big factor in Google results for some queries, like restaurants, but not at all for others~~, like banks~~. “Our algorithms are tremendously balanced to give a mix of what you want and what the world says you should at least know,” he said.

There wasn’t a ton of discussion of Google+ and social, but Singhal affirmed Google plans to revive its recently closed real-time search feature, and said that “who knows who and who knows what” can be a powerful combination of signals about what information is important.

Google to FTC: Bring It On

Ina Fried — Fri, 24 Jun 2011 16:50:39 +0000

Google on Friday staked out its position in what could be a long antitrust battle over how the company conducts its core business.

The company confirmed in a regulatory filing that it has indeed received a subpoena from the Federal Trade Commission. In a blog post, Google laid out how it sees things.

“At Google, we’ve always focused on putting the user first,” the company said in the post, written by Amit Singhal, one of the key creators of Google’s search business. “We aim to provide relevant answers as quickly as possible — and our product innovation and engineering talent have delivered results that users seem to like, in a world where the competition is only one click away.”

Singhal notes that Google is aware that its success would lead to greater scrutiny, but argues that everything it has done has been for the benefit of users, rather than to reduce competition. Singhal promises the company will aid in the inquiry.

“It’s still unclear exactly what the FTC’s concerns are, but we’re clear about where we stand,” he said. “Since the beginning, we have been guided by the idea that, if we focus on the user, all else will follow.”

It’s worth a read, as I suspect this is a topic we are going to be hearing a ton about in the coming months.

Here’s the full blog post:

Google Makes Search Faster With Instant Pages, Desktop Voice Search & Image Queries

Liz Gannes — Tue, 14 Jun 2011 16:35:00 +0000

Google gathered press today to speak with its search expert Amit Singhal and other members of its team about its latest progress in search. Before the event started, the company posted an “Inside Search” Web page reviewing the history of its search innovations. Here’s what happened next:

Google Fellow Amit Singhal introduces search as “breaking barriers between you and the knowledge you seek.” He’ll talk about three areas: mobile, desktop, Google Instant.

9:36 am: Now he’s talking about knowledge, Google’s new favorite word. “With search we strive to make sure that there are no derailments in your train of thought … Search is about getting you all the knowledge instantaneously.”

First barrier to knowledge: being away from a computer — a.k.a. mobile. On weekends, Google search traffic from desktop computers dips. But mobile searches actually peak on Saturday.

Singhal: If you look by time of day, highest search volume is around 3 pm. People work later and later these days but desktop search traffic drops off at the end of the night. But mobile search peaks at night.

Search traffic goes down seasonally, but “there is no summer slump or Christmas break in mobile search,” says Singhal.

Singhal now onto another metaphor: Google’s efforts at search are like trying to get a hole in one every time. “On mobile, it’s even more critical that you get the first result right.”

9:48 am: Next up: Scott Huffman and Steve Chang for a mobile search dive.

Huffman: Mobile searches used to be really simple. But then mobile phones became more powerful computers and “sort of overnight we saw our mobile search stream transform into something interesting.”

Launching today are icons at the bottom of Google mobile Web search interface. And a better mobile local search UI shows your location on a map and dynamically changes the map with AJAX to highlight each result as you look at it.

Also, they have integrated query suggestions from a user’s history so you can more quickly get to things you search for a lot. Searching for the S&P 500 brings up an interactive widget that was launched back in March, Huffman says.

Still demoing features that were previously launched. A “spyglass icon” next to mobile search results lets you scan through previews of Web pages.

9:56 am: Now a sneak preview of a refreshed tablet search UI: bigger images, etc. “On these tablet devices, people really do like to consume a lot of information,” Huffman says.

Also announcing Google Goggles with translation for Russian.

9:58 am: Mike Cohen, manager of speech technology, is next. “Arguably, speaking is the most natural way we learn to express our needs,” he says. “When it comes to mobile, speech input is particularly important.”

Mobile speech input for search has grown by a factor of six in the last year, Cohen says.

Cohen: To make a speech interface successful, it must be accurate and it needs to be available for every application, language, platform and device.

Speech interfaces that are more accurate promote more usage. This requires massive amounts of data. Just for U.S. English Google has the system parse roughly 230 billion words of data from real queries, taking many CPUs decades of processing time to do that learning (but multiple machines are run simultaneously so it doesn’t take that long).

In Android, whenever the keypad button pops up there’s also a microphone button — for all applications, all Web pages, without the developer needing to do anything. Developers may not realize their apps are speech enabled, but they are just by virtue of being on Android.

10:05 am: Currently released languages cover about two thirds of the world’s population, and constantly releasing more.

Next: Johanna Wright, director of product management for search. “Why is it that we can’t take these same innovations and bring them back to the computer to find knowledge there?”

She tells a story about proving she was right in an argument (Which is really the point of everything, huh?) by using voice search to get the translation of squirrel into Spanish (“ardilla”).

So today: voice search for desktop.

This is evoking the first applause of the day, from the planted Google employees. We press folks are a surly bunch.

Now showing off voice search disambiguation of Worcester, Massachusetts and College of Wooster.

Also announcing: search by image on desktop.

Next demo is of a blurry image of a guy standing on a trail in front of an island in Greece. You can now drag an image from your desktop into the search query box. This returns the name of the island: Nea Kameni. (Whoa, kinda crazy!)

Wright says, I see at work people passing around all these images with cute animals and block white text on them, what is this all about?

Searching for a “Y U No” image produces the Know Your Meme entry for it, and shows visually similar images even though they have different text in them.

The image search feature is rolling out globally on images.google.com over the next few days. It can copy/paste URL, upload from desktop, drag and drop, or use a new Chrome or Firefox extension.

10:17 am: Singhal is back to talk about what’s next for Google Instant.

“What is the biggest barrier to getting the knowledge you want? It’s time.”

Google Instant will be available “in the coming weeks” on Google Image Search. Now showing video demo of this.

10:21 am: Flipping channels on the TV or pages on a magazine is still faster than loading a Web page, Singhal says.

Singhal: Autocomplete and Google Instant help save users time to enter a query and select a result. But once you select the result, it takes another 5 seconds on average for that page to arrive on your browser.

But what can Google do about third-party Web site load times? Next announcement: Google Instant Pages.

“With Instant Pages, sometimes when you click on a result the page will be there just instantaneously.”

Instant pages takes 0 seconds for the Washington Post homepage to load, versus 3.2 seconds without it. This is done by pre-rendering in Chrome triggered by Web search when Google is “very confident” in results.

Singhal: This wouldn’t be possible without our relevance technology. We don’t want to waste bandwidth.

Every query on Google is now potentially 4-10 seconds faster with the combination of Google Instant and Instant Pages. Available this week in Chrome Beta, and available today in Chrome’s developer version. Launching on mobile in coming weeks.

10:34 am: Singhal wrapping up: it’s all about faster and faster. Now time for Q&A.

Q: Will Instant Pages come to Firefox?
Singhal: It’s in the open source code, so we are expecting other browsers to take this standard and incorporate it.

Q: What about ads?
Wright: Expect more innovations soon.
Singhal: Ads are an integral part of that experience.

Q: How is Instant Pages different from pre-rendering?
Peter Linsley: It’s similar to Firefox pre-fetching, but beyond downloading just the HTML page it gets images, style sheets and executes Javascripts.

Q: What about TV?
Singhal: We do want to extend to every Android device which would include Google TV.

Q: Couple detail checks:
Wright: Voice search on the desktop is Chrome only now.
Huffman: Yes, we will see recommendations from friends in local search.

Q: How often will Instant Pages be triggered? How do you keep from skewing analytics?
Singhal: Webmasters will be told through analytics. We are very pleased that the number of times we can predict confidently is far more than we expected.

Q: What’s launching today?
A: Query-building capability, icons, local maps, Russian translation, voice search in English, search by image. Instant Pages for stable Chrome “in coming weeks.”

Q: Bing?
Singhal: Competition is good for users.
Alan Eustace: Search is the core of our business and the investment levels are continuing and increasing.

Q (from me!): Is search speed a competitive advantage for your other products, like Chrome and Android?
Singhal: Our focus on speed is not only launched on things like Android and so on. Google Instant is everywhere. With the launch of Instant Pages we are opening up the source code and would like all browsers to take on these advances because it’s good for the users, it’s good for the world. In search it’s all about the users.
Wright: In Google’s DNA is speed. Every time you speed up an application people use it more.

Q: Face recognition?
A: Nope.

Q: Reorg?
Eustace: We used to have a lot of “time-slicing,” now are narrowed down into product areas instead of multiplexing over a lot of things.

The reason Larry created a title of “Knowledge” is his view of search is much broader than just a query finding a result in pages. He thinks Google should understand how things are related. He wants us to know more rather than just find better.

Long answer from Eustace continues: Amit leads search … For me it’s elevated the importance of search in the organization.

Singhal: Focusing on knowledge is just a natural progression of that information pyramid.

Q: Would you submit voice search sound files to authorities?
Cohen: Voice searches are anonymized and saved for 24 months, we don’t associate them with individuals.

Q: More clarifications about Instant Pages…
Singhal: Instant Pages is usually just for the first result.

Q: Can search by image be used to find things that are private, like a person’s house?
Wright: If you take a picture of something that’s not common, that there aren’t a lot of photos of, it’s probably not going to work very well.

Q: Why no social?
Wright: Social is important to us, but today was around speed. Great year for social: +1 buttons, liked content in search results.
Singhal: We have seen social search grow and we are very pleased with what we’re seeing.

Singhal’s ending note on speed: Every time we shave even 50 milliseconds off the search process, users search more and more. It’s great for the users, and it’s great for Google.

Also, Instant Pages not personalized, doesn’t matter if you’re logged in or not, says Singhal.

PageYank: As New SVPs Are Born at Google in CEO Reorg, What Happens to the Old Ones?

John Paczkowski — Fri, 08 Apr 2011 07:15:40 +0000

Things are sure shaking over at Google, since the sudden departure on Monday of Jonathan Rosenberg, Google’s head of product management and one of its most senior executives.

While his exit was portrayed as friendly all around, sources with knowledge of the dicey situation said that was definitely not the case.

Instead, moving aside Rosenberg was newly installed CEO and Co-founder Larry Page’s first parry at remaking the search giant in his own image.

Moving management chairs around is one of the tried-and-true way new leaders often try to effect that kind of dramatic change and several sources said Page has been tossing them about rather than just rearranging them.

That was certainly clear in last night’s knighting of six new SVP titles upon a group of execs, all very close to Page.

The promoted in new business units: Sundar Pichai, SVP of Chrome; Vic Gundotra, SVP of social; Andy Rubin SVP of mobile; Salar Kamangar SVP of YouTube and video; Alan Eustace SVP of search; Susan Wojcicki SVP of ads.

Of them, Eustace was previously an SVP, in charge of engineering and research, and Wojcicki had recently held the title SVP of product management.

It’s all the next step in Page’s overhauling the company’s management structure, as I reported in this column earlier this week was in the works.

As I wrote:

This, of course, brings into focus that fates of several other SVPs on the formal management structure list on Google’s Web site and still serving on that OC.

Leaving Eustace off, since he has a new SVP title, they are: Nikesh Arora, SVP and Chief Business Officer; David Drummond SVP, Corporate Development, and Chief Legal Officer; Shona Brown, SVP, Business Operations; and Patrick Pichette, SVP and Chief Financial Officer.

How their roles evolve or do not–all might stay as is, of course–will be the next interesting part of what I am calling PageYank:

Nikesh Arora:

In a widely read column earlier this week, investing gadfly Eric Jackson argued that Arora is probably the most vulnerable of all the senior executives at the company.

The high-profile Arora is well known both inside and outside the company as both highly ambitious and consistently pugnacious.

While that is not necessarily a bad thing to be, that style has garnered him some criticism and he is often referred to as “Darth Vader” among detractors (and even some supporters).

Still, Arora has been a consistent producer of results over his tenure, which might be all that matters. In fact, it might also make him an attractive candidate for a CEO job outside Google.

But, perhaps most important right now though, is that Arora is “definitely not part of Larry’s inner circle,” said one source, adding “and that’s a very important place to be right now.”

Incidentally, that inner circle currently seems to consist of many of those promoted last night–Kamangar, Rubin, Pichai and Gundotra–as well as search leads Udi Manber and Amit Singhal and, of course, Co-founder Sergey Brin.

And not, it seems, Arora.

David Drummond:

With Kent Walker recently promoted to an SVP title, along with being Google’s general counsel, does the company need a Chief Legal Officer or does it need to winnow down another layer of management?

As one source told me, “Why do you need a Drummond, when you’ve got a Walker?” It’s a fair point.

While also in charge of both public policy and corporate development, Drummond has been known more for benign absence at Google than for aggressive presence.

Some also suggest that the affable exec, who has been at Google since early on and is presumably very wealthy, might also not want to sign up for the long-term commitment that Page now expects of his top managers.

Shona Brown:

Before she came to Google, Brown spent a decade consulting for McKinsey and is widely credited with optimizing Google’s internal structure.

But Page is not a McKinsey guy and he’s obviously not a big fan of Google’s current management organization anymore.

That might not bode well for the legendarily sharp-elbowed Brown who most sources describe as highly strategic but also as extremely difficult to work with.

Still, if Page is tinkering with the way Google is organized, Brown might also be the one he turns to find a new structure.

That said, he seems to be fine doing it on his own and some suggest Brown will move to another role within the company rather than leaving.

Not all agree.

Said one source: “I wouldn’t be shocked to see Shona go. Frankly, I’m surprised she survived as long as she did, but then I didn’t think Rosenberg would last this long either.”

But, said another about Brown, who has previously taken time off from Google and returned: “I’d never count Shona out.”

Patrick Pichette:

He’s not going anywhere, as far as I can tell. The friendly and erudite Pichette is widely admired at the company and by Page–the most important admirer of all at Google these days.

He’s also been a smart and stable presence on earnings calls and does a job with Wall Street analysts and investors that Page is pretty much uninterested in and–more to the point–completely incapable of doing well.

Let’s be honest about the socially awkward CEO: Page’s frequently prickly and robotic style makes Facebook’s Mark Zuckerberg look like Cary Grant.

Pichette stays.

As for everyone else, as Page reaches even further down into the organization at Google, it will be interesting to see where the next chair will fall.

One thing is clearest of all: Page is positioning himself as the centerpoint of the entire company.

Because make no mistake, these new autonomous divisions all report to him, in a system that mimics Apple and its legendary leader Steve Jobs.

A tough act to follow, to be sure.

"Beyond the Search Box": The White Pleather Honeypot Smackdown

Kara Swisher — Tue, 01 Feb 2011 19:53:20 +0000

Perusing AOL’s leaked damn-the-journalism-full-speed-ahead business plan, BoomTown was a little late to the Microsoft Bing event this morning called “Farsight: Beyond the Search Box.”

But things had certainly been cooking with gas when I walked into the meeting room at the University of San Francisco, which the organizers had decked out in white nubby rugs, white pleather couches and those white egg-shaped chairs found only in 1970s decor.

Schweeet!

First up was well-known investor and entrepreneur Peter Thiel, poo-poohing Microsoft’s prospects of ever making money in search.

“It’s difficult to produce a new search company,” said Thiel, noting that even with a growing market share it’s curtains for Bing, given the huge fixed costs. “As far as I can tell, it’s still not breaking even.”

Ouch!

By the way, Thiel sold semantic search engine Powerset to Microsoft for upward of $100 million in 2008 to help it, you know, get ahead in search.

Way to insult your money-bearing hosts!

Then, moderator Vivek Wadhwa harangued the panelists from Google, Microsoft and Blekko in the session “Who Will Win the Spam Wars?”

And they say I’m a snarky moderator! Wadhwa is snarktastic!

Wadhwa did not like any of it–not crappy content sites that sully Web search, not the efforts the companies were making to fix things, not the vision the trio had of the future.

And, by the way, Microsoft was not ever going to make money off all the company’s efforts.

Way to insult your hosts! I like this event!

Of course, what everyone was interested in was a smackdown between Google and Microsoft, given that the search giant accused the software giant of stealing its results today.

In an excellent, if exhaustive, post by Search Engine Land’s Danny Sullivan, Google said Bing was cheating by lifting its search results, which Google said it had proved via a “honeypot” sting operation.

“I’ve spent my career in pursuit of a good search engine,” Google’s Amit Singhal told Search Engine Land. “I’ve got no problem with a competitor developing an innovative algorithm. But copying is not innovation, in my book.”

The very presence of the word “honeypot” in any story about search algorithms is superb, in my book, even though this “controversy” is pretty much a he-said-he-said geek-off.

Google’s Matt Cutts kept up the cheater pressure at the Bing event, in a short debate with Microsoft’s Harry Shum, who was not having any of it.

“It’s not like we actually copy anything,” he said.

Translation: Actually, we do borrow, just like Facebook’s Mark Zuckerberg did to the Winklevii, resulting in a social networking behemoth that will soon take over all search and make this whole debate moot.

Microsoft is rubber, Google is glue. And Facebook, which was not present at the search event, is the real sticky honeypot.

Is Google a Monopolist? A Debate

Amit Singhal and Charles Rule — Fri, 17 Sep 2010 22:13:37 +0000

Amit Singhal of Google (GOOG) argues the competition is one click away. Charles Rule, an attorney whose firm represents corporations suing Google, counters that the company commands a share of search advertising in excess of 70 percent—the threshold for monopoly under the Sherman Act.

Competition in an Instant
by Amit Singhal

Last week, “Googling something” took on a whole new meaning. Instead of typing your question into the search box and hitting Enter, our newest invention—Google Instant—shows constantly evolving results based on the individual letters you type.

Instant is just the latest in a long line of search improvements. Five years ago, search results were just “ten blue links”—simple web pages with some text. Today search engines provide answers in the form of images, books, news, music, maps and even “real time” results from sites such as Twitter.

Read the rest of this post on the original site

QOTD

John Paczkowski — Mon, 23 Aug 2010 12:00:10 +0000

— Google search guru Amit Singhal on sci-fi’s role in innovation

A Couple of Google Geeks Sitting Around Talking About Search: Video Interviews With Gundotra and Singhal!

Kara Swisher — Tue, 08 Dec 2009 14:00:41 +0000

Please see this disclosure related to me and Google.

Here’s the video of interviews BoomTown did with the two Google execs most responsible for the rollout of its real-time search and other new mobile search features announced yesterday.

That would be VP of Engineering Vic Gundotra and Google Fellow Amit Singhal, who is known as the master of ranking algorithm and search quality.

The smooth-talking Gundotra and the jumping-bean Singhal make an interesting pair of nerdish bookends, but are essentially pushing the same idea that Google is as innovative as ever.

Along with Google (GOOG) VP of Search Products & User Experience Marissa Mayer, the pair unveiled the updates at an event the search giant held in Silicon Valley.

Here is the video of the interviews (and here is another video I did at the event):

[ See post to watch video ]

Liveblogging the Google Search Event: Gutenberg, Goggles and Scrolling Real-Time Search!

Kara Swisher — Mon, 07 Dec 2009 18:08:53 +0000

Please see this disclosure related to me and Google.

Right now, BoomTown is sitting right behind the very affable Jason Hirschhorn, chief product officer of MySpace, who is here to make one of the many partner announcements with Google at its “search event” in Silicon Valley today.

I also ran right into Twitter’s Biz Stone at the coffee stand. He is also here to talk about the new features Google (GOOG) is adding to its search repertoire, although he is remaining mum until the program starts in five minutes.

It’s about real-time search, of course, given that the partners visiting today are all real-time search folks.

The confab–being held at the Computer History Museum near the Googleplex HQ–is essentially Google’s rejoinder to last week’s event by Microsoft (MSFT), which announced a bunch of new features for its Bing search service, including mapping updates.

Of course, because it is Google, the sound system rocks, the food is better and it is more overproduced than “Dancing With the Stars.”

10:13 am PT: The event is opened by Marissa Mayer, who runs search products and user experience for Google.

And it takes exactly 13 seconds for there to be a classic Silicon Valley buzzword. Modes! Translation: It is how we use the Web.

Mayer is outlining Google’s key components in the future of search. Along with modes, they are media, language and personalization.

“We are a company that likes to launch early and often,” she said, adding that Google has launched 33 search innovations in 67 days.

In other words, take that, Bing. Oh, dear, giant Google just boasted about its innovation cred and is apparently a little worried about weensie Bing.

10:18 am: Mayer welcomes Vic Gundotra, VP of engineering, who will talk about mobile search.

He begins by noting that no one knows where all the new innovations in computing will lead, much as no one got the Gutenberg press way back in the olden days.

Professor Gundotra then launches into a computing history lesson, with stops at Moore’s Law (better, faster, cheaper) and how one understood all the zillions of computing connections that would occur.

The “missing ingredient,” noted Gundotra, is the cloud.

Next, he moves to a demo to show where Google is headed. Gundotra nails a voice query on an Android phone about President Obama at the G8 Summit with the French president. Everyone cheers.

Gundotra now tries to top himself with a Mandarin query for McDonald’s in Beijing. He sticks it.

He then announces support for the voice search on mobile devices for Japan, bringing up a Japanese speaker.

One voice query is a very long one for a favorite restaurant in Tokyo near the Google office there. Does Google find it? Of course Google does.

“Our dreams at Google go way beyond what you just saw,” says Gundotra, who opines on a real-time interpreter on the phone. Of course, he demos the interpreter, which he said will show up sometime in 2010.

It works, again. Natch! These are big-brained dudes here at Google, so don’t mess with them.

10:30 am: Gundotra moves to locations, which he says will be a key element of future versions of Google search. You know, Red Sox comes up in Boston, data appear for nearby stores for digital cameras.

He shows off the “Near Me Now” feature, which is kind of like those many Apple (AAPL) iPhone apps, like Yelp. It explores stuff nearby. It will be available on Google mobile maps for Android right away.

Next, he announces a Google Labs project called Google Goggles, which takes pictures of something and then identifies it. I have seen this kind of thing in a lot of labs at various tech companies.

Gundotra, who is a slick dude at presentations, uses the example of being a wine expert without being one. He scans a wine bottle and then Google quickly shows info on it.

Oooooh, aaaaaah.

Gundotra uses the service to identify a Japanese landmark successfully.

Someday, he predicts, your phone will be a “mouse pointer” to the world.

10:42 am: Back to Mayer, who talks about media relevancy in search. Google Fellow Amit Singhal is the man on deck.

“What we’re going to announce today is one of the most exciting things in my career,” said Singhal, who first launches into a short history of information flow.

Campfires, more Gutenberg! Also some pictures of old Google servers. I feel so educated; plus, Singhal is pretty funny for a supergeek.

Now, he gets to the news: “We are here today to announce Google real-time search.”

The demo is launched and it shows news scrolling as it is produced. “This is the first time ever,” enthuses Singhal.

It looks cool, but reminds me a lot of old tickers that used to be in the newsroom at the Washington Post. You know, the kind of newspaper that Google is often accused of killing off.

Irony alert! I wonder if that will scroll up soon.

The scrolling also includes Twitter updates. One tweet by Googler Matt Cutts about the Google real-time search launch showed up immediately.

The latest results will be available on the search options and in preferences and will also be hyperlocal and mobile on the iPhone and Android.

“Real-time search becomes incredibly powerful, since it shows you exactly what you need in your geography,” said Singhal.

Singhal is a font of news. He also announces that Google Trends is moving out of the labs and will also show real-time results.

He launches into the “how” of how Google did all this. Well, it was really, really hard, said Singhal, because there are a badillion real-time pieces of data out there to analyze and render.

And which company, with its massive computing power, can make this relevant and hand over the info quickly? Three guesses, and the first two don’t count.

Recap: Real-time search, latest search option, update option, mobile real-time search and Google Trends in the real-time world.

“At Google we will not be satisfied,” said Singhal, until Google can get you info at the speed of light.

11:07 am: Just to stick a true fork into anything Microsoft could come up with, Mayer comes back up and announces Google’s Facebook, MySpace and Twitter partnerships as part of the launch of real-time search.

Facebook will be sending in public feeds and MySpace is providing all of them, as is Twitter.

Google now has eyes and ears, says Mayer. When it gets a whole body, get ready to run for your life.

Q&A time!

The first question is about whether Goggles could have facial recognition. Gundotra says Google could do that, but will not until the privacy issues are worked out. Operative thought here: Google is capable of doing this. Eek!

The next question is about advertising opportunities in these new features. Singhal does not really answer, but says businesses will develop.

The next question is about how much content Google is crawling. Answer: About a billion pages a day.

Gundotra adds that the first launch is only available on English-speaking locales. But it will move into other languages next year.

What about spammers taking advantage of real-time search? Oh, says Singhal, they will get a beat-down from Matt Cutts, who is in charge of spam-killing at Google.

Wouldn’t that make a good reality show? “The Spam Hunters!”

About questions on real-time partnerships, Mayer said Google wanted to be comprehensive.

Mayer will not disclose the details of any financial payments for these real-time feeds. Of course, Google is paying up.

And now a question about whether Google will limit development on non-Android phones. “Absolutely not,” says Gundrotra.

At last, a zinger question: Do you feel that Google will be responsible for the death of journalism and doesn’t that make Google a scary black hole of, presumably, evil?

Awkward!

Singhal casts about for an answer, which is mostly about bringing info to users, which is not an answer.

“It’s really about user empowerment,” he says. Uh-oh, we’re doomed!

Mayer jumps in nervously to shoot this meme down and says Google is about facilitation and not decimation.

The PR dude onstage also throws in the boilerplate about Google sending gazillions of clicks all over.

But the point is made: Today Google–which owns universal search–just made its big move in real-time search.

The next question is about the difference between Google’s practice of wanting people off the page and onto the Web and Microsoft Bing’s focus on topic pages of rich information.

Mayer is sticking with quick on and off for Google.

And what about junk information on the silly side that comes with more real-time search, like dead celebs who are not dead, or really untrue information on important issues?

It’s a hard problem, says Singhal, who says Google is working on it.

What about disabling the real-time updates rather than just being able to turn them on and off. Nope, says Singhal. Mayer notes that this may change.

But the truth is: With the big search giant jumping in, real-time search is most definitely here to stay.

Liveblogging the Google Confab at 10 am PT: Searchtastic?

Kara Swisher — Mon, 07 Dec 2009 13:23:54 +0000

Please see this disclosure related to me and Google.

BoomTown will be motoring the MINI–top up, it’s pouring rain here!–down to Silicon Valley this morning to liveblog a “search event” that Google is holding at the Computer History Museum.

Last week, Microsoft (MSFT) unveiled some new features for its Bing search service, so turnabout is fair play for Google (GOOG).

Lots of news is promised, which is likely to mean a bunch of new features, partners and more to be announced (perhaps yet another acquisition or two?), showing that the search giant can keep innovating from its lofty perch.

Included in the event will be Google VP of Search Products & User Experience Marissa Mayer, VP of Engineering Vic Gundotra and Google Fellow Amit Singhal, who is known as the master of ranking algorithm and search quality.

Liveblogging begins at 10 am PT, with video to follow.

Google to Hold an Anything-Bing-Can-Do-We-Can-Do-Better Search Event (and Singalong?) Monday

Kara Swisher — Fri, 04 Dec 2009 11:28:25 +0000

Please see this disclosure related to me and Google.

Google answered BoomTown’s blatantly begging plea to call, write or just say hi after I was inundated with news about Bing search from Microsoft (MSFT) this week–emailing yesterday about a “search event” to be held Monday in Silicon Valley.

It will be at the Computer History Museum, and the PR minion who wrote me (see email below) noted “there’ll be lots of news.”

News? Well, you don’t have to ask me and my Flip twice!

Included in the confab will be Google VP of Search Products & User Experience Marissa Mayer, VP of Engineering and Renaissance man Vic Gundotra and Google Fellow Amit Singhal, who is known as the master of ranking algorithm and search quality.

Already, I feel dumber than a box of hammers. But, can I resist? Of course not!

Sources tell me the event is for unveiling a wide range of feature updates to the Google service, probably to take some of its search innovations out for a showy walk around the park before Bing sucks up all the PR oxygen.

Here’s the note I got:

Now, if Yahoo (YHOO) would only throw a look-at-me-too search event soon, it will be a trifecta!

Until Monday, please enjoy a video of one of my favorite musical scenes and songs–mostly due to the line, “You couldn’t give me a lesson in long-distance spitting!”–from the movie version of “Annie Get Your Gun,” sung by the perfect Betty Hutton and Howard Keel:

Google and the Evolution of Search III: What's Next in Search? Much, Much Better Search

John Paczkowski — Fri, 05 Jun 2009 12:00:03 +0000

For many years, Google (GOOG), on its Explanation of Our Search Results page, claimed that “a site’s ranking in Google’s search results is automatically determined by computer algorithms using thousands of factors to calculate a page’s relevance to a given query.” Then in May of 2007, that statement changed: “A site’s ranking in Google’s search results relies heavily on computer algorithms using thousands of factors to calculate a page’s relevance to a given query.” In this third and final interiew with Google’s search team, Google Fellow Amit Singhal helps us understand why. Previously, Google Engineering director Scott Huffman talked about the company’s human evaluators and software engineer Matt Cutts discussed search quality and spam.

Part III: Amit Singhal

John Paczkowski: Talk a bit about the history of search evaluation and your role in it.

Amit Singhal: Search evaluation was born in the late 50′s and the early 60′s in the U.K. In the beginning it was very basic because back then, search was Boolean. The first evaluation measure was recall. You take a query and 100 documents relevant to it. How many of those documents does your search on that query retrieve? We quickly found out that it was very easy to get 100 percent recall. But we also found that our searches often returned a lot of irrelevant documents along with the relevant ones. So we came up with a second measure: Precision. That tells us what percentage of our search returns is actually good. So if a search returns 100 out of 100 relevant documents for a query, but it returns 1,000 documents total, its recall is 100 percent, but its precision is only 10 percent.

And those two measures or some combination thereof have evolved over time, and even modern search engines like Google use them. So since search began, there have always been teams in the lab judging how relevant a search return is to a human query.

JP: But relevance is a subjective notion.

AS: Right. But these evaluation measures don’t directly affect the results returned to our users. They are only used to evaluate whether an algorithm is working well or whether a new algorithm is working better than an old one. They don’t directly impact user experience. They are simply just calibration tools.

JP:Matt and Scott spoke at length about human search evaluators. Just how broad is their role at Google?

AS: Well, our search evaluation is based on many components. And one of those components is human evaluation. We have automatic systems as well–things that tell us if, for example, users suddenly stop clicking on a number-one result and instead begin clicking on the number-five result. Together these techniques tell us how well our system is doing at any point. And we do this in over a hundred languages.

JP: How do you balance fresh results with more historical ones?

AS: When is a fresh result more relevant than a historical result? That is a question… very important for our users and thus for our algorithms. So we evaluate queries for freshness—this query deserves freshness today, but it did not three weeks back. We do the same thing with documents. We are always asking how fresh is the document? How relevant? How useful? And we put the answers to those questions together purely algorithmically and present them to users in our universal search results. All this is done automatically. No human being is sitting there and saying GM is important today or Mumbai is important today. Because at the end of the day, human beings are far too prone to subjectivity to do it. Algorithms are not. And they can make the same sorts of determinations in hundreds of languages.

JP: How far have we come in search?

AS: We are still barely at the beginning. We are nowhere close to being done. Search is a hard problem, and the hard part about it is that user expectations are deep, and they keep going higher and higher as you keep improving search. And so search by no means is a solved problem.

JP: So what’s next?

AS: What is next in search? Much, much better search… universality of search, and by that I mean search where the user doesn’t have to go to YouTube specifically to search for video or to Google for documents. Whatever type of content is relevant to you should just show up in your search results. So search becomes focused on who you are and where you are. So it would be local to you as a person and it would be local to your geography as well. And those two things combined will give you universally relevant results much more relevant to you and to your locality.

JP: Circling back a bit to the role of human evaluators in search, do you think they will always be necessary? Will they be more necessary in the future or less necessary? How will their role change?

AS: I believe that the role of the human evaluator in search will be there until we can understand language by computers, which is a far distance from where we are today. You know, we have made great advances but by no means is our language understanding technology close to saying this person really meant to get this document or not.

Google and the Evolution of Search III: What's Next in Search? Much, Much Better Search

John Paczkowski — Fri, 05 Jun 2009 12:00:03 +0000

Part III: Amit Singhal

John Paczkowski: Talk a bit about the history of search evaluation and your role in it.

JP: But relevance is a subjective notion.

JP:Matt and Scott spoke at length about human search evaluators. Just how broad is their role at Google?

JP: How do you balance fresh results with more historical ones?

JP: How far have we come in search?

JP: So what’s next?

Google and the Evolution of Search II: Cheating the System

John Paczkowski — Thu, 04 Jun 2009 11:30:15 +0000

This is the second of three interviews with members of the Google (GOOG) team responsible for overseeing search algorithms at the company. The introduction and Part I, an interview with Scott Huffman, appeared yesterday. In today’s installment Google software engineer Matt Cutts talks about search quality and spam. In Part III tomorrow, Google Fellow Amit Singhal will wrap up the series.

Part II: Matt Cutts

John Paczkowski: How do you maintain quality in search?

Matt Cutts: Well, broadly, we improve our algorithms and hopefully, every so often, develop some punctuated equilibrium where we create totally new ways to improve our relevance. My contribution… is ensuring that people who try to cheat the system don’t show up higher than they deserve to in our results. We want sites ranking high based on merit, not based on shortcuts.

JP: OK, so how do you do that?

MC: Essentially we look at a wide variety of input. We look at user complaints, for example. We also have a variety of internal metrics we use to track current trends. They help show us what people are using to spam right now. What’s getting past our defenses. And when we detect those things, we write some new algorithms or develop some tool that helps us detect and, hopefully, counteract them. So a large part of what we do is simply spotting trends in spam.

JP: Is there a human evaluation element here as well?

MC: Each team is responsible for general search-quality evaluations, but it’s not like they’re changing rankings or anything like that. That said, there are some policy violations that are pretty egregious. So, for example, if you type in your name and instead of getting All Things Digital, you got a porn site, you would get pretty angry about that. And you might complain to Google. And it would be frustrating if our reply was, “Yeah, well, we think we might have an algorithm that might fix that problem in five or six months, so we’re just going to leave that porn site as the top result for All Things D until we get an algorithm up to help you out.” Obviously, that’s a deeply dissatisfying answer.

So in spam, we are sometimes willing to take manual action on those sorts of policy violations. But Google’s philosophy is that wherever you can use machines and algorithms, it is much better, more robust, more scalable. And so, to the extent that we can, we always want to rely on the computers as our first line of defense.

JP: But you’re willing to remove spam manually until you can find an algorithm to counteract it. Do you think that will always be the case? Will we some day reach a point where human intervention of the sort you just described won’t be necessary or are we headed toward increasing human intervention?

MC: That’s a really fascinating question, but I don’t know the answer. What’s interesting to think about is that page rank, the raw page rank algorithm, actually improves as it ranks more pages. So the more pages you add to it, the easier it is to determine how reputable a particular page is without human intervention.

But as the Web grows in size we also encounter new and different policy violations–hidden text, cloaking. Those are the sorts of things that humans are very good at spotting. You can certainly identify some of them with a computer algorithm, but not all. And so our intent is always to try to make sure that we handle things efficiently with machines and algorithms. But I don’t know that we will ever get there completely.