Liz Gannes

Recent Posts by Liz Gannes

Facebook's New Data Center: Smarter, Greener and Doggone It, People Like It

Facebook packed its main Palo Alto cafeteria as full as I’ve ever seen it on Thursday to detail its data center plans, with press, software and hardware partners, and others from the industry in attendance. Here’s the live report:

Mark Zuckerberg up first: calls this the Open Compute Project.

It’s easy to think that the operations is just kind of this cost part of the business, but really products and operations fundamentally are linked, he says.

Zuckerberg is pandering to the infrastructure crowd in attendance by talking up how essential data centers are. Adding capacity improves the Facebook experience by enabling better “type-ahead” features when users search, and helps speed up the site. And these challenges are the same for partners too.

What the mass manufacturers put out wasn’t what we needed for social apps, so we made it more custom, Zuckerberg says. Now Facebook wants to share its improvements with everyone else in the industry so they can improve too. “We’re not the only ones who need the kind of hardware that we’re building out.”

Jonathan Heiliger

Jonathan Heiliger, VP Technical Operations, up next. He points to a rack of servers behind them, says “they’re real Facebook servers, or rather, real Open Compute servers.”

Facebook designed and built its own data center in Prineville, Oregon over the last year and a half, and tailored the servers there to work on its software, Heilinger said. The goal was to be as efficient as possible. The industry average power effective usage is 1.5, average for leased data centers is 1.4-1.6, but Facebook has got it down to 1.07. Across the table from me, a hardware guy mutters, “That’s really good.”

The data center design team is only three people, Heilinger said, because it was built on top of other people’s work, so now the company wants to give back by sharing its specifications and design documents.

The servers Facebook built are 38 percent more energy efficient and cost 24 percent less, according to Heilinger.

Another Facebook data center guy is now digging into the specifics, but I already have a statement from Greenpeace in the inbox about how Facebook’s announcement isn’t good enough.

“It’s commendable that Facebook is working to increase the energy efficiency of its business, and specifically its data centers–an area of neglect for many years. But as the global warming footprint of the IT industry, and Facebook specifically, continues to grow significantly, a focus on energy efficiency alone will only slow the speeding train of unsustainable emissions growth. Efficiency is simply not enough.

“If Facebook wants to be a truly green company, it needs to reduce its greenhouse gas emissions. The way to do that is decouple its growth from its emissions footprint by using clean, renewable energy to power its business instead of dirty coal and dangerous nuclear power.”

Some of the data center design innovation includes the voltage of the electrical distribution system, localized power supply and a ductless evaporative cooling system (no air conditioning at all).

Now Amir Michael is describing and demoing Facebook’s redesigned servers and racks themselves. Facebook motherboards don’t usually need to be expanded, so that plastic and extra size was removed, for example. And servers are mounted on shelves rather than traditional rails.

He tells an endearing story about choosing to use blue LEDs to light the data center rather than green ones even though they were more expensive, concluding that the end result looked “so sweet.”

Now Michael and Heiliger are narrating a video describing their accomplishments and their commitment to openness, complete with uplifting music and camera shots that lovingly pan and linger over racks of servers.

“It’s time to start treating data centers like Fight Club, and really demystify the software and hardware,” said Heiliger.

Om Malik of GigaOM (and my old boss, hey Om!) is brought up to moderate a panel of experts from companies like Zynga, Intel and Dell to talk about how the Open Compute Project affects the industry.

“We commend Facebook for taking this step,” says Zynga infrastructure head Allen Leinwand, who adds he is “definitely considering” using it.

Here’s a link to the YouTube post of that video I described.

And here‘s a site dedicated to the Open Compute Project.

As the testimonials continue at the event, here’s one sent via email: Forrester Analyst Rich Fichera comments,

“The results speak for themselves – a high volume data center with a claimed PUE of 1.07, certainly one of the most efficient large data centers in the world,” writes Fichera. “What’s more valuable is Facebook’s decision to publish its server, rack and power specifications as part of the ‘Open Compute Project,’ making these specifications available to users and vendors in the hopes of creating an ecosystem around these stripped down cost optimized servers and associated infrastructure.”

Back at the event, Rackspace Chairman Graham Weston says, “We’re seeing the emergence of a new stack: data center, server and software working together,” says .

Weston said Rackspace will incorporate Facebook’s innovations in future data centers, in order to benefit from the dramatic decrease in energy costs.

Facebook also sent, via email, a response to Greenpeace’s statement:

From the beginning, we realized that the biggest impact we could have is focusing on efficiency, both for our own operations and the world. The Open Compute Project has the potential to save many times the energy that Facebook will ever use. We sent information on OCP to Greenpeace earlier today and we’ve offered to answer their questions. We hope Greenpeace “friends” the Open Compute Project and we encourage others to do the same.

Jason Waxman of Dell tries to put the announcement in context, “It’s putting something viable on the map that gives a clear direction on efficiency,” he says. Forrest Norrod of Dell adds that the demystification of data centers is “seminal.”

OK, everyone’s looking down at their phones at this point. Time to wrap it up, Facebook!

Please see the disclosure about Facebook in my ethics statement.


Latest Video

View all videos »

Search »

There was a worry before I started this that I was going to burn every bridge I had. But I realize now that there are some bridges that are worth burning.

— Valleywag editor Sam Biddle