Mike Isaac

Recent Posts by Mike Isaac

Twitter Breaks Down How Its Engineering Teams Work


Image: Graham Smith/Flickr

Twitter may not have its work-flow process worked out well in every area of the company. But in its pitch to engineers, it is certainly touting its back-end engineering prowess.

Case in point: Twitter platform engineering VP Raffi Krikorian’s recent talk at an industry conference. The idea was to share the ways Twitter’s organizational structure furthers software development inside the company, making for a more innovative approach to pushing out code.

Krikorian stressed the importance of the engineering org’s compact, five-to-seven-person teams that sit together, are dedicated to specific areas of the site, and can operate autonomously, according to Krikorian’s slideshow pitch (no audio, but the slides are informative). Krikorian stressed how much control the individual groups have over how their teams work — likely a nice way to pitch would-be engineer recruits at the conference.

The biggest overall goal: Making Twitter, a site notorious for its history of unreliability, stay up and functioning 100 percent of the time.

So how is Twitter making that happen? Slowly but surely.

In a separate talk at the conference, Twitter software engineer Jeremy Cloud underscored the three-step rule of keeping Twitter up and running. “Step one: Make the smallest possible change. Step two: Verify that it works, and tweak as necessary. Step three: Return to step one.”

That is to say, stay away from sweeping changes and overhauls that may straight-up break the site (which, as a result of even the smallest code changes, has happened before). Validate your work on isolated “canary servers,” and test with a single-digit percentage of Twitter’s user base before rolling changes out widely.

The lesson in all of this, according to the gospel of Twitter engineering: Incremental changes eventually win. And “failure is always an option.”

(Image courtesy of Graham Smith/Flickr)

Latest Video

View all videos »

Search »

There was a worry before I started this that I was going to burn every bridge I had. But I realize now that there are some bridges that are worth burning.

— Valleywag editor Sam Biddle