In a rare joint public appearance, the four men dove deeply into the big issues facing search at a Churchill Club event last week.

For instance, Singhal fended off Eli Pariser’s “filter bubble” critique, the idea that online personalization presents a skewed view of the world that leaves out important things like opposing viewpoints.

Singhal said that personalization is a big factor in Google results for some queries, like restaurants, but not at all for others, like banks. “Our algorithms are tremendously balanced to give a mix of what you want and what the world says you should at least know,” he said.

There wasn’t a ton of discussion of Google+ and social, but Singhal affirmed Google plans to revive its recently closed real-time search feature, and said that “who knows who and who knows what” can be a powerful combination of signals about what information is important.

The basic +1 function allows users to recommend a Web page by clicking on a small +1 button next to search results. These votes are aggregated globally, but logged-in users will see the pictures and names of their connections who have “+1′ed” a link.

Just as with Facebook’s “like” button, all +1′s are public. But +1 doesn’t have the social feedback you might get by sharing a link on Facebook or Twitter, or the option to annotate links with your comments.

This is only rolling out gradually, though users can opt in to try +1 at www.google.com/experimental. It’s part of a larger effort to get Google users to start maintaining their Google Profiles–which are obviously key to the grand Google social plan.

For now, users can choose to make their +1′s available as a tab on their Google Profile, but there’s no activity stream that brings together friends’ likes.

Another limitation: right now +1 is only for users’ connections on Gmail, Google Contacts, Google Reader and Google Buzz. Support for connections on other services like Twitter is “coming soon.”

Yet another feature coming soon: +1 buttons for publishers, which they can add alongside the other colorful doodads for sharing on Twitter, Facebook, Digg, StumbleUpon, and perhaps even Google Buzz.

One feature that’s ready at launch is +1 for ads, a highly unusual move in Silicon Valley where monetization is usually relegated to a lower priority. +1 buttons will appear next to Google ads and show which users have clicked on them, just like +1 for search. Advertisers don’t have to pay for the feature but will get reporting on how many +1s they get.

Google already has multiple social search features currently rolled out, and has experimented with users voting on search results in the past through tools like Google SearchWiki (which is no longer available).

In another significant search announcement yesterday, Google said it was revising its algorithm to target makers of low-quality content.

The search giant has been criticized by many of late for the presence of too much spam in its results, which degrades the consumer experience on the powerful site.

Thus, “pretty big algorithmic improvement to our ranking–a change that noticeably impacts 11.8% of our queries,” said Google in a blog post.

The company continued:

“This update is designed to reduce rankings for low-quality site–sites which are low-value add for users, copy content from other websites or sites that are just not very useful. At the same time, it will provide better rankings for high-quality sites—sites with original content and information such as research, in-depth reports, thoughtful analysis and so on.”

Who Google is aiming at is unclear–some point to Demand Media, whose top exec recently said the content company welcomed any improvements to the search results in its recent quarterly call.

“We consider ourself very white hat,” declared CEO Richard Rosenblatt, who has often touted the Demand’s good relations with Google, to a question from a Wall Street analyst about the series of recent declarations by Google to clean up its search results.

That was further underscored yesterday.

Interestingly, the Google post about the changes, titled “Finding More High-Quality Sites,” was authored by Google’s Amit Singhal and Matt Cutts–who have cut a high profile of late in the search arena.

Both Singhal and Cutts were quite vocal recently in loopy accusations about Microsoft’s Bing lifting Google’s search results.

And Cutts has been a frequent visitor to Washington, D.C. of late, to defend Google over its controversial acquisition of the ITA Software flight information company, as well as its search ranking process.

At a January 13 meeting, in an email obtained by BoomTown, Cutts was the draw:

Please join us!

You’re invited to learn

How Google’s Search Engine Works

Myth-busting and Q&A for House/Senate staff members

with

Matt Cutts

Principal Search Engineer, Google

Thursday, January 13, 2011

2:30 – 3:30 PM

House Visitor Center Room 201

How does Google’s search engine really work? Can websites pay Google to improve their ranking in Google results? What’s the difference between the “natural” results and the ads on the right hand side? And why does a particular website rank #1 or #3 when you do a Google search for your boss’ name? You’re invited to join Matt Cutts, one of Google’s top search engine engineers and the company’s ambassador to webmasters for a session on Capitol Hill where Matt will explain how Google ranks websites, address common myths about Google’s search results, and answer your questions. Please join us!

In another invite, low-quality content was the topic:

Matt Cutts is one of Google’s top search engineers who heads up the team ensuring that spam and low-quality sites don’t game search results. He is going to be here in DC to talk with folks around town about some of the recent calls for government to police or regulate the “fairness” of search results. Matt is a bit of a rock star in the search world and spends a lot of time speaking and blogging about these issues. Basically he’ll talk about how Google goes about ranking websites, how his team fights webspam, and he’ll provide a closer look at sites like Foundem and MyTriggers (who have filed antitrust actions against Google).

Finally, he’ll talk about the recent calls by some for Google’s search results to be regulated.

Perhaps I’m being cynical, but the noisy search algorithm changes, while welcome to those using Google, also have a pretty clear goal to burnish the Silicon Valley company’s image.

Please see this disclosure related to me and Google.

Perusing AOL’s leaked damn-the-journalism-full-speed-ahead business plan, BoomTown was a little late to the Microsoft Bing event this morning called “Farsight: Beyond the Search Box.”

But things had certainly been cooking with gas when I walked into the meeting room at the University of San Francisco, which the organizers had decked out in white nubby rugs, white pleather couches and those white egg-shaped chairs found only in 1970s decor.

Schweeet!

First up was well-known investor and entrepreneur Peter Thiel, poo-poohing Microsoft’s prospects of ever making money in search.

“It’s difficult to produce a new search company,” said Thiel, noting that even with a growing market share it’s curtains for Bing, given the huge fixed costs. “As far as I can tell, it’s still not breaking even.”

Ouch!

By the way, Thiel sold semantic search engine Powerset to Microsoft for upward of $100 million in 2008 to help it, you know, get ahead in search.

Way to insult your money-bearing hosts!

Then, moderator Vivek Wadhwa harangued the panelists from Google, Microsoft and Blekko in the session “Who Will Win the Spam Wars?”

And they say I’m a snarky moderator! Wadhwa is snarktastic!

Wadhwa did not like any of it–not crappy content sites that sully Web search, not the efforts the companies were making to fix things, not the vision the trio had of the future.

And, by the way, Microsoft was not ever going to make money off all the company’s efforts.

Way to insult your hosts! I like this event!

Of course, what everyone was interested in was a smackdown between Google and Microsoft, given that the search giant accused the software giant of stealing its results today.

In an excellent, if exhaustive, post by Search Engine Land’s Danny Sullivan, Google said Bing was cheating by lifting its search results, which Google said it had proved via a “honeypot” sting operation.

“I’ve spent my career in pursuit of a good search engine,” Google’s Amit Singhal told Search Engine Land. “I’ve got no problem with a competitor developing an innovative algorithm. But copying is not innovation, in my book.”

The very presence of the word “honeypot” in any story about search algorithms is superb, in my book, even though this “controversy” is pretty much a he-said-he-said geek-off.

Google’s Matt Cutts kept up the cheater pressure at the Bing event, in a short debate with Microsoft’s Harry Shum, who was not having any of it.

“It’s not like we actually copy anything,” he said.

Translation: Actually, we do borrow, just like Facebook’s Mark Zuckerberg did to the Winklevii, resulting in a social networking behemoth that will soon take over all search and make this whole debate moot.

Microsoft is rubber, Google is glue. And Facebook, which was not present at the search event, is the real sticky honeypot.

Again, be wary of reading too much into any stock’s performance on any given day. But it seems safe to draw at least one conclusion: Investors aren’t freaked out about Demand’s symbiosis with/dependence on Google. Even after a puzzling blog post from the search giant last week.

The post, written by Google engineer Matt Cutts, defended the search engine’s performance against a chorus of criticism. But it acknowledged that Google was paying attention to complaints about “content farms and sites that consist primarily of spammy or low-quality content” clogging its search results.

Lots of people logically assumed that Google/Cutts was talking about Demand, although the post never mentioned the company by name. And if Google, which supplies 28 percent of Demand’s revenue and a big slug of its traffic, has a problem with Demand…

But Demand CEO Richard Rosenblatt insists that Cutts wasn’t talking about his company at all. In fact, he says, Demand and Google are getting along just great, in a relationship that pays out real dividends for both parties. It looks like investors believe him.

I chatted with Rosenblatt about the Google post, and the companies’ relationship, yesterday at Demand’s New York outpost. Here’s an excerpt from our conversation:

Peter Kafka: Do you think that Google post was directed at you in any way?

Richard Rosenblatt: It’s not directed at us in any way.

Did you talk to them about that?

I can’t comment on that.

Okay. But they wrote this post, which talks about content farms, and even though you say they weren’t talking about you, it left a lot of people scratching their heads.

Let’s just say that we know what they’re trying to do. Last year, they put out three major changes. They put out Mayday–that was going specifically after spammers and low-quality content. Our traffic increased when they did that. The reason why is our content is being scraped and stolen, [because we're] the largest content producer. So they’re looking for original, non-duplicated, human-made content. That’s all our content. So if they were targeting us, you’d also see Wikipedia, About.com, Wikihow, every person that makes more than a few dozen articles….Our traffic went up.

Second one: They did something called Caffeine, to increase the [search] index. Our traffic went up.

They then did Google Instant. Our traffic went up.

So the three things [Cutts] talks about in his blog post did not adversely affect us. You can draw your own conclusions.

The post talks about going after spammers and content farms. But when you guys think of content farms, you don’t think that means Demand, right? You’re thinking of people who take my copy or your copy, and cut and paste it, and tweak it enough to fool Google.

He’s talking about duplicate, non-original content. Every single piece of ours is original. Written by somebody. And I understand how that could confuse some people, because of that stupid “content farm” label, which we got tagged with. I don’t know who ever invented it, and who tagged us with it, but that’s not us…We keep getting tagged with “content farm”. [Ahem.] It’s just insulting to our writers. We don’t want our writers to feel like they’re part of a “content farm.”

So can you sum up your relationship with Google today?

This is why our partnership with Google makes sense. 1) We help them fill the gaps in their index, where they don’t have quality content. 2) We’re the largest supplier of all video to YouTube, over two billion views and 3) we’re a large AdSense partner. So our relationship is synergistic, and it’s a great partnership. And it’s a partnership that we’re excited to continue to expand.

Please see this disclosure related to me and Google.

Right now, BoomTown is sitting right behind the very affable Jason Hirschhorn, chief product officer of MySpace, who is here to make one of the many partner announcements with Google at its “search event” in Silicon Valley today.

I also ran right into Twitter’s Biz Stone at the coffee stand. He is also here to talk about the new features Google (GOOG) is adding to its search repertoire, although he is remaining mum until the program starts in five minutes.

It’s about real-time search, of course, given that the partners visiting today are all real-time search folks.

The confab–being held at the Computer History Museum near the Googleplex HQ–is essentially Google’s rejoinder to last week’s event by Microsoft (MSFT), which announced a bunch of new features for its Bing search service, including mapping updates.

Of course, because it is Google, the sound system rocks, the food is better and it is more overproduced than “Dancing With the Stars.”

10:13 am PT: The event is opened by Marissa Mayer, who runs search products and user experience for Google.

And it takes exactly 13 seconds for there to be a classic Silicon Valley buzzword. Modes! Translation: It is how we use the Web.

Mayer is outlining Google’s key components in the future of search. Along with modes, they are media, language and personalization.

“We are a company that likes to launch early and often,” she said, adding that Google has launched 33 search innovations in 67 days.

In other words, take that, Bing. Oh, dear, giant Google just boasted about its innovation cred and is apparently a little worried about weensie Bing.

10:18 am: Mayer welcomes Vic Gundotra, VP of engineering, who will talk about mobile search.

He begins by noting that no one knows where all the new innovations in computing will lead, much as no one got the Gutenberg press way back in the olden days.

Professor Gundotra then launches into a computing history lesson, with stops at Moore’s Law (better, faster, cheaper) and how one understood all the zillions of computing connections that would occur.

The “missing ingredient,” noted Gundotra, is the cloud.

Next, he moves to a demo to show where Google is headed. Gundotra nails a voice query on an Android phone about President Obama at the G8 Summit with the French president. Everyone cheers.

Gundotra now tries to top himself with a Mandarin query for McDonald’s in Beijing. He sticks it.

He then announces support for the voice search on mobile devices for Japan, bringing up a Japanese speaker.

One voice query is a very long one for a favorite restaurant in Tokyo near the Google office there. Does Google find it? Of course Google does.

“Our dreams at Google go way beyond what you just saw,” says Gundotra, who opines on a real-time interpreter on the phone. Of course, he demos the interpreter, which he said will show up sometime in 2010.

It works, again. Natch! These are big-brained dudes here at Google, so don’t mess with them.

10:30 am: Gundotra moves to locations, which he says will be a key element of future versions of Google search. You know, Red Sox comes up in Boston, data appear for nearby stores for digital cameras.

He shows off the “Near Me Now” feature, which is kind of like those many Apple (AAPL) iPhone apps, like Yelp. It explores stuff nearby. It will be available on Google mobile maps for Android right away.

Next, he announces a Google Labs project called Google Goggles, which takes pictures of something and then identifies it. I have seen this kind of thing in a lot of labs at various tech companies.

Gundotra, who is a slick dude at presentations, uses the example of being a wine expert without being one. He scans a wine bottle and then Google quickly shows info on it.

Oooooh, aaaaaah.

Gundotra uses the service to identify a Japanese landmark successfully.

Someday, he predicts, your phone will be a “mouse pointer” to the world.

10:42 am: Back to Mayer, who talks about media relevancy in search. Google Fellow Amit Singhal is the man on deck.

“What we’re going to announce today is one of the most exciting things in my career,” said Singhal, who first launches into a short history of information flow.

Campfires, more Gutenberg! Also some pictures of old Google servers. I feel so educated; plus, Singhal is pretty funny for a supergeek.

Now, he gets to the news: “We are here today to announce Google real-time search.”

The demo is launched and it shows news scrolling as it is produced. “This is the first time ever,” enthuses Singhal.

It looks cool, but reminds me a lot of old tickers that used to be in the newsroom at the Washington Post. You know, the kind of newspaper that Google is often accused of killing off.

Irony alert! I wonder if that will scroll up soon.

The scrolling also includes Twitter updates. One tweet by Googler Matt Cutts about the Google real-time search launch showed up immediately.

The latest results will be available on the search options and in preferences and will also be hyperlocal and mobile on the iPhone and Android.

“Real-time search becomes incredibly powerful, since it shows you exactly what you need in your geography,” said Singhal.

Singhal is a font of news. He also announces that Google Trends is moving out of the labs and will also show real-time results.

He launches into the “how” of how Google did all this. Well, it was really, really hard, said Singhal, because there are a badillion real-time pieces of data out there to analyze and render.

And which company, with its massive computing power, can make this relevant and hand over the info quickly? Three guesses, and the first two don’t count.

Recap: Real-time search, latest search option, update option, mobile real-time search and Google Trends in the real-time world.

“At Google we will not be satisfied,” said Singhal, until Google can get you info at the speed of light.

11:07 am: Just to stick a true fork into anything Microsoft could come up with, Mayer comes back up and announces Google’s Facebook, MySpace and Twitter partnerships as part of the launch of real-time search.

Facebook will be sending in public feeds and MySpace is providing all of them, as is Twitter.

Google now has eyes and ears, says Mayer. When it gets a whole body, get ready to run for your life.

Q&A time!

The first question is about whether Goggles could have facial recognition. Gundotra says Google could do that, but will not until the privacy issues are worked out. Operative thought here: Google is capable of doing this. Eek!

The next question is about advertising opportunities in these new features. Singhal does not really answer, but says businesses will develop.

The next question is about how much content Google is crawling. Answer: About a billion pages a day.

Gundotra adds that the first launch is only available on English-speaking locales. But it will move into other languages next year.

What about spammers taking advantage of real-time search? Oh, says Singhal, they will get a beat-down from Matt Cutts, who is in charge of spam-killing at Google.

Wouldn’t that make a good reality show? “The Spam Hunters!”

About questions on real-time partnerships, Mayer said Google wanted to be comprehensive.

Mayer will not disclose the details of any financial payments for these real-time feeds. Of course, Google is paying up.

And now a question about whether Google will limit development on non-Android phones. “Absolutely not,” says Gundrotra.

At last, a zinger question: Do you feel that Google will be responsible for the death of journalism and doesn’t that make Google a scary black hole of, presumably, evil?

Awkward!

Singhal casts about for an answer, which is mostly about bringing info to users, which is not an answer.

“It’s really about user empowerment,” he says. Uh-oh, we’re doomed!

Mayer jumps in nervously to shoot this meme down and says Google is about facilitation and not decimation.

The PR dude onstage also throws in the boilerplate about Google sending gazillions of clicks all over.

But the point is made: Today Google–which owns universal search–just made its big move in real-time search.

The next question is about the difference between Google’s practice of wanting people off the page and onto the Web and Microsoft Bing’s focus on topic pages of rich information.

Mayer is sticking with quick on and off for Google.

And what about junk information on the silly side that comes with more real-time search, like dead celebs who are not dead, or really untrue information on important issues?

It’s a hard problem, says Singhal, who says Google is working on it.

What about disabling the real-time updates rather than just being able to turn them on and off. Nope, says Singhal. Mayer notes that this may change.

But the truth is: With the big search giant jumping in, real-time search is most definitely here to stay.

This is the second of three interviews with members of the Google (GOOG) team responsible for overseeing search algorithms at the company. The introduction and Part I, an interview with Scott Huffman, appeared yesterday. In today’s installment Google software engineer Matt Cutts talks about search quality and spam. In Part III tomorrow, Google Fellow Amit Singhal will wrap up the series.

Google and the Evolution of Search

1. Human Evaluators — Google Engineering director Scott Huffman
2. Cheating the System — Google software engineer Matt Cutts
3. What’s Next in Search? Much, Much Better Search — Google Fellow Amit Singhal

Part II: Matt Cutts

John Paczkowski: How do you maintain quality in search?

Matt Cutts: Well, broadly, we improve our algorithms and hopefully, every so often, develop some punctuated equilibrium where we create totally new ways to improve our relevance. My contribution… is ensuring that people who try to cheat the system don’t show up higher than they deserve to in our results. We want sites ranking high based on merit, not based on shortcuts.

JP: OK, so how do you do that?

MC: Essentially we look at a wide variety of input. We look at user complaints, for example. We also have a variety of internal metrics we use to track current trends. They help show us what people are using to spam right now. What’s getting past our defenses. And when we detect those things, we write some new algorithms or develop some tool that helps us detect and, hopefully, counteract them. So a large part of what we do is simply spotting trends in spam.

JP: Is there a human evaluation element here as well?

MC: Each team is responsible for general search-quality evaluations, but it’s not like they’re changing rankings or anything like that. That said, there are some policy violations that are pretty egregious. So, for example, if you type in your name and instead of getting All Things Digital, you got a porn site, you would get pretty angry about that. And you might complain to Google. And it would be frustrating if our reply was, “Yeah, well, we think we might have an algorithm that might fix that problem in five or six months, so we’re just going to leave that porn site as the top result for All Things D until we get an algorithm up to help you out.” Obviously, that’s a deeply dissatisfying answer.

So in spam, we are sometimes willing to take manual action on those sorts of policy violations. But Google’s philosophy is that wherever you can use machines and algorithms, it is much better, more robust, more scalable. And so, to the extent that we can, we always want to rely on the computers as our first line of defense.

JP: But you’re willing to remove spam manually until you can find an algorithm to counteract it. Do you think that will always be the case? Will we some day reach a point where human intervention of the sort you just described won’t be necessary or are we headed toward increasing human intervention?

MC: That’s a really fascinating question, but I don’t know the answer. What’s interesting to think about is that page rank, the raw page rank algorithm, actually improves as it ranks more pages. So the more pages you add to it, the easier it is to determine how reputable a particular page is without human intervention.

But as the Web grows in size we also encounter new and different policy violations–hidden text, cloaking. Those are the sorts of things that humans are very good at spotting. You can certainly identify some of them with a computer algorithm, but not all. And so our intent is always to try to make sure that we handle things efficiently with machines and algorithms. But I don’t know that we will ever get there completely.

Google and the Evolution of Search

1. Human Evaluators — Google Engineering director Scott Huffman
2. Cheating the System — Google software engineer Matt Cutts
3. What’s Next in Search? Much, Much Better Search — Google Fellow Amit Singhal