How the NSA Could Get So Smart So Fast
Five years ago it would have been unimaginable for a government agency such as the National Security Agency to efficiently parse millions of phone, text and online conversations for keywords that could have warned of an impending terrorist attack. Today, a set of new technologies make it relatively affordable and manageable for it do so.
These technologies can store vastly different types of data in a single database, and can be processed rapidly using inexpensive hardware, without an analyst having to formulate a hypothesis. “They’ve substantially reduced the cost and greatly increased the [government’s] ability to analyze this type of data,” says Tom Davenport, an expert on analytics and a visiting professor at Harvard Business School. The technology needed to outfit data centers to perform these tasks has become “orders of magnitude” less expensive than in the past, he said.