I wasn’t surprised that this data–which shows basically everywhere an iPhone owner has been with the phone in his or her possession since iOS 4 was released–is being collected. But it’s important to make one thing clear before we go any further: There’s no evidence that Apple is collecting any information about where you go. It does collect anonymized information on where iPhones go, and it has a number of legitimate business purposes for doing so.

Nor is any of what Apple is doing some kind of newly discovered secret. In fact, it has been documented for some time. (For more on this, read this excellent post by Alex Levinson, an expert on iOS security.) In fact, Apple has been very clear in its privacy policy about what data it collects, and even highlighted the part about location data in a letter to Congress last year.

Got all that? Okay, let’s proceed.

I found the revelation unsurprising because of stories I’ve written in recent years on the new field of academic research known as “reality mining.” I wrote about it for BusinessWeek in 2008 and interviewed two researchers from MIT, both of whom told me that there is a great deal of value–both commercial value and value to society at large–that can come from gathering data on where people go, and also when they go there and who they go with.

The value comes not in gathering that data about you personally, but in aggregating it, basically mixing it all together with the same data about everyone else, until you have huge databases on the comings and goings of millions of people. It’s helpful for a city to know, for instance, how many cars cross a bridge between the hours of 7 am and 8 am, and how the traffic varies by the day of the week. It’s helpful to see how many people drove to the last New York Yankees game and how many people took the subway, and also how bad the crowd congestion was both on the streets and on the trains.

Getting an accurate picture of exactly how many people are involved is tricky. You can try to do a labor-intensive count or you can estimate, but both are messy and subject to error. A wireless phone is a pretty good sensor because almost everyone will be carrying one and each phone identifies itself to the closest cell tower, so it’s easy to count. The value comes not from knowing who was where at any given time, but how many were there.

Wireless phones already play a big role in tracking traffic congestion. If you use a GPS receiver in your car that gives you live traffic information, those green and red lines that appear on the map are often generated by thousands of cars with wireless phones in them, all of them reporting their location, speed and direction of travel. The company that tracks that information, analyzes it and turns it into something useful is Inrix, and its name can be found stamped on the packaging of a number GPS receivers. When yours pipes up to say “traffic ahead” or says it is changing your route because of congestion, it’s because it is getting a live data feed that is generated in part by information gathered from wireless phones. Are you still so creeped out?

But let’s take it a step further and imagine a case where it actually might be useful and not anonymized. Let’s say you’ve caught a really bad flu but it’s in its early stages, so you’re feeling just a little sick and you go to work. The next day you find out you’ve got this year’s super-flu virus. Would there be a public health benefit in being able to look through a record of where you’ve been? Could there also be a benefit from cross-referencing that with data from other people’s phones to find out how many people–and who–has been within close enough proximity to you during the last 24 hours to maybe catch this flu from you? Data gathered from your phone and others could conceivably help arrest the spread of that super flu by giving authorities an accurate picture of how many people are connected in the branching chains of potential infection.

After a while you start seeing patterns, and these patterns can help solve other problems large and small. Does your town need a traffic light at that intersection based on the number of people who drive through it every day? Does your city need to build another subway line because the existing ones are overwhelmed? Reasonable minds can have different perceptions as to the scale of problems. Real, unimpeachable data can only add clarity to the debates.

MIT’s Sensable City Lab has done some fascinating work in this area. Its most recent project has taken the team to Singapore, and I’ve embedded a video below that shows samples of some of the data they’ve gathered and turned into visualizations. Another older video from a 2008 project, The New York Talk Exchange, showing calls made to and from New York, is just as interesting.

Is there a commercial use for this sort of data? You bet. Advertisers will always pay for the right and the ability to reach you in some new and incrementally intrusive way. But that’s just the way that things go, though more often than not, if you don’t like it, you have the ability to opt out or not participate. But people do choose to participate. Ask the eight million Foursquare users why they like voluntarily giving up their live location data day after day. They have clearly opted in because there’s something about that they like, and it isn’t just claiming the mayorship of the corner tavern. And there are probably scores of other commercial uses for the location data on our phones that I’m not imaginative enough to think of.

My point in all this is really simple. Phones have for about a decade had GPS chipsets in them that can keep track of the phone’s precise coordinates–latitude and longitude plus their position relative to a cellular tower. To anyone who is surprised that this data is being collected and even being used I have only this to say: Well, duh! You better get used to it. As long as there’s value in measuring where we go, the phones we take with us everywhere are going to be the device used to do the measuring.

Yes, there needs to be a clear set of rules of the road, and I think the discussion touched off by this round of coverage will help us get to setting those rules. But the data is so valuable, and the potential for benefits are so great, that no amount of consumer outrage is going to put an end to your phone keeping track of where it is.

(Cool Image borrowed from MIT’s New York Talk Exchange)

In an interview with the Financial Times, O2 CEO Ronan Dunne said the company has seen an 18-fold increase in data traffic since the beginning of the year, most of it driven by super-smartphones like the iPhone. “Where we haven’t met our own high standards then there’s no question, we apologise to customers for that fact,” Dunne said. “But it would be wrong to say O2 has failed its customers en masse.”

Well, that’s one way of looking at it. The thousands of smartphone users who lost data service just a few days ago might see things a bit differently, though. Anyway, O2–like AT&T in the U.S.–is doing its best to contain the public relations damage arising from such outages. Dunne says the carrier has added an additional 200 mobile base stations in London to help it meet data demand and has been working with its infrastructure suppliers on network tweaks to improve data service.

Wise moves. Best to act quickly and get out ahead of issues like these, which can become PR nightmares very quickly. As AT&T recently learned, network congestion issues make great fodder for the advertising campaigns of rivals. Wouldn’t want Vodafone (VOD) running its own version of Verizon’s (VZ) nasty “There’s a Map for That” commercials, now would we?

With usage levels like these and the network degradation and customer satisfaction issues that go along with them, is it reasonable to think that iPhone carriers like AT&T (T) will swap their all-you-can-eat data plans for usage-based pricing? Sacconaghi thinks so.

“iPhone users consume web, email and video data on the mobile network at levels that many believe are adversely affecting other subscribers on those mobile networks. Network congestion in turn is triggering higher capital spending requirements for carriers. Unchecked, the iPhone’s very high usage levels could severely undermine the economic returns of offering the iPhone,” Sacconaghi explains.

“We believe carriers will increasingly have to manage the usage side of the equation as well,” the Bernstein analyst asserts. “Carriers that have not already done so are increasingly likely to adopt usage-based pricing schemes that more fairly match price to usage but which will also inevitably discourage the most profligate kinds of applications.”

Sacconaghi adds: “Today’s combination of a subsidized iPhone with an all-you-can eat data plan can be likened to a carnival where there is a relatively high price of entry, but where all the rides are free. The adoption of usage-based pricing plans is akin to also charging for the Ferris wheel. When the rides are no longer ‘free,’ the value of the entry ticket could decline.”

That’s unfortunate. But in this case, the carnival’s wireless data traffic has increased 50-fold since the introduction of the Ferris wheel and there has not been a commensurate increase in revenue. In other words, AT&T is not getting paid for the level of data traffic it supports. Says Sacconaghi: “Over the…3-year period over which AT&T’s data bandwidth consumption has grown by 50-fold, its data revenues have grown by only 250 percent, resulting in a severe drop in revenue generation per megabyte of data.”

As more adopt unlimited data plans, they become unsustainable from a carrier profitability perspective. So what’s to be done?

Spend even more capital improving your network.

And raise the price of data by adopting usage-based pricing.

That might seem disadvantageous to Apple (AAPL), but as Sacconaghi notes, Apple could use the switch to usage-based pricing to its advantage. It could, for example, introduce a “non-data plan” iPhone that expands the device’s addressable market.

“A non-data plan iPhone might be something like an ‘iPhone Touch,’ a lower cost-of-ownership device (requires voice plan only; utilizes wifi for Internet connectivity) with which to attack the more traditional handset market…while exploiting Apple’s twin unique competitive differentiators: (1) A vibrant App Store with 100,000+ Apps and (2) a large and global iPod community that remains fiercely loyal to the iPod,” the analyst writes. “Such a product would have a price (to customers) that would be similar to a stand-alone iPod Touch today, and have a cost of ownership equal to standard voice-only phones.”

“Contrary to published reports, Apple has not rejected the Google Voice application, and continues to study it.”

So begins Apple’s response to the Federal Communication Commission’s inquiry into its rejection of the app and of its App Store approval process. Seems Google Voice was withheld from the App Store not because of any ill feeling toward Google or a nefarious request from AT&T, but because it too closely mimics the iPhone OS, including management of calls, voicemail and text messages. From Apple’s statement:

Question 1. Why did Apple reject the Google Voice application for iPhone and remove related third-party applications from its App Store?

The application has not been approved because, as submitted for review, it appears to alter the iPhone’s distinctive user experience by replacing the iPhone’s core mobile telephone functionality and Apple user interface with its own user interface for telephone calls, text messaging and voicemail. Apple spent a lot of time and effort developing this distinct and innovative way to seamlessly deliver core functionality of the iPhone. For example, on an iPhone, the “Phone” icon that is always shown at the bottom of the Home Screen launches Apple’s mobile telephone application, providing access to Favorites, Recents, Contacts, a Keypad, and Visual Voicemail. The Google Voice application replaces Apple’s Visual Voicemail by routing calls through a separate Google Voice telephone number that stores any voicemail, preventing voicemail from being stored on the iPhone, i.e., disabling Apple’s Visual Voicemail. Similarly, SMS text messages are managed through the Google hub—replacing the iPhone’s text messaging feature. In addition, the iPhone user’s entire Contacts database is transferred to Google’s servers, and we have yet to obtain any assurances from Google that this data will only be used in appropriate ways. These factors present several new issues and questions to us that we are still pondering at this time.

Question 2. Did Apple act alone, or in consultation with AT&T, in deciding to reject the Google Voice application and related applications? If the latter, please describe the communications between Apple and AT&T in connection with the decision to reject Google Voice. Are there any contractual conditions or non-contractual understandings with AT&T that affected Apple’s decision in this matter?

Apple is acting alone and has not consulted with AT&T about whether or not to approve the Google Voice application. No contractual conditions or non-contractual understandings with AT&T have been a factor in Apple’s decision-making process in this matter.

In a response of its own, AT&T (T) also said this was the case:

1(a). What role, if any, did AT&T play in Apple’s consideration of the Google Voice and related applications?

AT&T had no role in Apple’s consideration of Google Voice or related applications.

1(b). What role, if any, does AT&T play in consideration of iPhone applications generally?

The Apple App Store is owned, operated and controlled by Apple, not AT&T, and Apple makes the decisions regarding the specific applications that are approved for use on the iPhone or included in the Apple App Store. AT&T does not participate in Apple’s day-to-day consideration of specific applications, nor does Apple typically notify AT&T prior to including applications in the App Store. Apple also does not usually advise AT&T after specific applications have been added to the App Store, which reportedly contains more than 65,000 applications. AT&T has had discussions with Apple regarding only a handful of applications that have been submitted to Apple for review where, as described below, there were concerns that the application might create significant network congestion.

And what is Google’s (GOOG) public reply to the whole affair? Nada. The company had more to say to the government, via a response to FCC queries. But in the copy of the document that’s been released for public consumption, the most interesting stuff has been redacted. To wit: “What explanation was given (if any) for Apple’s rejection of the Google Voice application?…Please describe any communications between Google and AT&T or Apple on this topic and a summary of any meetings or discussion.” (Click on text below to enlarge.)

Now that’s a little odd, isn’t it? Why would Google ask the FCC to redact portions of its statement? Says Google: “[Because the redacted] information relates specifically to private business discussions between Apple and Google and, as such, it constitutes commercial data ‘which would customarily be guarded from competitors.’”

That satisfy your curiosity? Doesn’t satisfy mine, either.

(Peter Kafka contributed to this post.)

UPDATE: AT&T may not participate in “Apple’s day-to-day consideration of specific applications,” but its presence is still felt during the approval process. From Apple’s (AAPL) response:

There is a provision in Apple’s agreement with AT&T that obligates Apple not to include functionality in any Apple phone that enables a customer to use AT&T’s cellular network service to originate or terminate a VoIP (voice over Internet Protocol) session without obtaining AT&T’s permission. Apple honors this obligation, in addition to respecting AT&T’s customer terms of service, which, for example, prohibit an AT&T customer from using AT&T’s cellular service to redirect a TV signal to an iPhone. From time to time, AT&T has expressed concerns regarding network efficiency and potential network congestion associated with certain applications, and Apple takes such concerns into consideration.

UPDATE: An interesting nugget from AT&T’s statement:

It is widely recognized by economists and jurists that parties to strategic alliances in competitive markets may enter into contracts to promote and protect their respective business interests and to refrain from taking actions adverse to those interests.12 Consistent with such lawful, economically efficient practices common among parties to strategic alliances, including participants in the mobile wireless marketplace,13 AT&T and Apple agreed that Apple would not take affirmative steps to enable an iPhone to use AT&T’s wireless service (including 2G, 3G and Wi-Fi) to make VoIP calls without first obtaining AT&T’s consent. AT&T and Apple also agreed, however, that if a third party enables an iPhone to make VoIP calls using AT&T’s wireless service, Apple would have no obligation to take action against that third party….AT&T indicated to Apple that it does not object to Apple enabling VoIP applications for the iPhone that use Wi-Fi connectivity (including connectivity at more than 20,000 Wi-Fi hotspots operated by AT&T that may be used by iPhone customers for no additional charge) rather than AT&T’s 2G or 3G wireless data services….we plan to take a fresh look at possibly authorizing VoIP capabilities on the iPhone for use on AT&T’s 3G network.

“Contrary to published reports, Apple has not rejected the Google Voice application, and continues to study it.”

So begins Apple’s response to the Federal Communication Commission’s inquiry into its rejection of the app and of its App Store approval process. Seems Google Voice was withheld from the App Store not because of any ill feeling toward Google or a nefarious request from AT&T, but because it too closely mimics the iPhone OS, including management of calls, voicemail and text messages. From Apple’s statement:

Question 1. Why did Apple reject the Google Voice application for iPhone and remove related third-party applications from its App Store?

The application has not been approved because, as submitted for review, it appears to alter the iPhone’s distinctive user experience by replacing the iPhone’s core mobile telephone functionality and Apple user interface with its own user interface for telephone calls, text messaging and voicemail. Apple spent a lot of time and effort developing this distinct and innovative way to seamlessly deliver core functionality of the iPhone. For example, on an iPhone, the “Phone” icon that is always shown at the bottom of the Home Screen launches Apple’s mobile telephone application, providing access to Favorites, Recents, Contacts, a Keypad, and Visual Voicemail. The Google Voice application replaces Apple’s Visual Voicemail by routing calls through a separate Google Voice telephone number that stores any voicemail, preventing voicemail from being stored on the iPhone, i.e., disabling Apple’s Visual Voicemail. Similarly, SMS text messages are managed through the Google hub—replacing the iPhone’s text messaging feature. In addition, the iPhone user’s entire Contacts database is transferred to Google’s servers, and we have yet to obtain any assurances from Google that this data will only be used in appropriate ways. These factors present several new issues and questions to us that we are still pondering at this time.

Question 2. Did Apple act alone, or in consultation with AT&T, in deciding to reject the Google Voice application and related applications? If the latter, please describe the communications between Apple and AT&T in connection with the decision to reject Google Voice. Are there any contractual conditions or non-contractual understandings with AT&T that affected Apple’s decision in this matter?

Apple is acting alone and has not consulted with AT&T about whether or not to approve the Google Voice application. No contractual conditions or non-contractual understandings with AT&T have been a factor in Apple’s decision-making process in this matter.

In a response of its own, AT&T (T) also said this was the case:

1(a). What role, if any, did AT&T play in Apple’s consideration of the Google Voice and related applications?

AT&T had no role in Apple’s consideration of Google Voice or related applications.

1(b). What role, if any, does AT&T play in consideration of iPhone applications generally?

The Apple App Store is owned, operated and controlled by Apple, not AT&T, and Apple makes the decisions regarding the specific applications that are approved for use on the iPhone or included in the Apple App Store. AT&T does not participate in Apple’s day-to-day consideration of specific applications, nor does Apple typically notify AT&T prior to including applications in the App Store. Apple also does not usually advise AT&T after specific applications have been added to the App Store, which reportedly contains more than 65,000 applications. AT&T has had discussions with Apple regarding only a handful of applications that have been submitted to Apple for review where, as described below, there were concerns that the application might create significant network congestion.

And what is Google’s (GOOG) public reply to the whole affair? Nada. The company had more to say to the government, via a response to FCC queries. But in the copy of the document that’s been released for public consumption, the most interesting stuff has been redacted. To wit: “What explanation was given (if any) for Apple’s rejection of the Google Voice application?…Please describe any communications between Google and AT&T or Apple on this topic and a summary of any meetings or discussion.” (Click on text below to enlarge.)

Now that’s a little odd, isn’t it? Why would Google ask the FCC to redact portions of its statement? Says Google: “[Because the redacted] information relates specifically to private business discussions between Apple and Google and, as such, it constitutes commercial data ‘which would customarily be guarded from competitors.’”

That satisfy your curiosity? Doesn’t satisfy mine, either.

(Peter Kafka contributed to this post.)

UPDATE: AT&T may not participate in “Apple’s day-to-day consideration of specific applications,” but its presence is still felt during the approval process. From Apple’s (AAPL) response:

There is a provision in Apple’s agreement with AT&T that obligates Apple not to include functionality in any Apple phone that enables a customer to use AT&T’s cellular network service to originate or terminate a VoIP (voice over Internet Protocol) session without obtaining AT&T’s permission. Apple honors this obligation, in addition to respecting AT&T’s customer terms of service, which, for example, prohibit an AT&T customer from using AT&T’s cellular service to redirect a TV signal to an iPhone. From time to time, AT&T has expressed concerns regarding network efficiency and potential network congestion associated with certain applications, and Apple takes such concerns into consideration.

UPDATE: An interesting nugget from AT&T’s statement:

It is widely recognized by economists and jurists that parties to strategic alliances in competitive markets may enter into contracts to promote and protect their respective business interests and to refrain from taking actions adverse to those interests.12 Consistent with such lawful, economically efficient practices common among parties to strategic alliances, including participants in the mobile wireless marketplace,13 AT&T and Apple agreed that Apple would not take affirmative steps to enable an iPhone to use AT&T’s wireless service (including 2G, 3G and Wi-Fi) to make VoIP calls without first obtaining AT&T’s consent. AT&T and Apple also agreed, however, that if a third party enables an iPhone to make VoIP calls using AT&T’s wireless service, Apple would have no obligation to take action against that third party….AT&T indicated to Apple that it does not object to Apple enabling VoIP applications for the iPhone that use Wi-Fi connectivity (including connectivity at more than 20,000 Wi-Fi hotspots operated by AT&T that may be used by iPhone customers for no additional charge) rather than AT&T’s 2G or 3G wireless data services….we plan to take a fresh look at possibly authorizing VoIP capabilities on the iPhone for use on AT&T’s 3G network.

We filed this appeal in order to protect our legal rights and to challenge the basis on which the Commission found that Comcast violated federal policy in the absence of pre-existing legally enforceable standards or rules. We continue to recognize that the Commission has jurisdiction over Internet service providers and may regulate them in appropriate circumstances and in accordance with appropriate procedures. However, we are compelled to appeal because we strongly believe that, in this particular case, the Commission’s action was legally inappropriate and its findings were not justified by the record.”

As legally inappropriate and unjustifiable as Comcast (CMCSA) might find it, the cable company will abide by the FCC’s order during the appeal. And it will forge ahead with plans to develop more net neutrality-friendly network management techniques by the end of the year. Said Cohen, “We will follow through on our longstanding commitment to transition to protocol-agnostic network congestion management practices by the end of this year. We also remain committed to bringing our customers a superior Internet experience.”

Superior Internet experience until you hit that 250GB broadband cap, that is.