Samsung Busted Boosting International Galaxy S4 Benchmarks
Powered by a new Exynos 5 Octa system on a chip, the international version of Samsung’s Galaxy S4 boasts some impressive benchmarks — a graphical processing unit that runs at 532 megahertz and a central processing unit that tops out at 1.2GHz. Assuming that you’re testing the device’s performance with certain popular benchmarking software. If you’re not, those specs are liable to be a bit different.
An investigation by AnandTech reveals that Samsung has created within the Exynos 5 Octa-based Galaxy S4 a white list for certain benchmarking apps that allows them — and only them — to take advantage of the full power of the chipset. When the device detects that a white-listed benchmark app is running, it shifts into a high-performance mode not available to “real world” applications.
During normal operation, for example, the S4’s GPU operates at about 480 MHz. But if you run benchmarks on it using a benchmarking tool like GLBenchmark 2.5.1, AnTuTu or Quadrant, that frequency spikes to 532 MHz — a speed the average consumer will never experience.
The S4 exhibits similar behavior with its CPU; it typically operates at about 500 MHz with a maximum speed of about 1.2GHz. Run it through a benchmarking tool and the device automatically switches to that 1.2GHz maximum clock speed. (The same CPU behavior was observed in Qualcomm Snapdragon-powered U.S. versions of the handset.)
In other words, Samsung has optimized the Galaxy S4 to perform better in the lab than it does in the real world, enabling full-speed GPU performance only while running specific benchmarks, and maxing out CPU performance, as well. Dig around inside the device as AnandTech did, and you’ll find a code string called “BenchmarkBooster” apparently intended to produce high and repeatable performance benchmarks.
“These benchmarks appear to have been singled out for special treatment in the power/thermal management software in order to inflate scores,” Kevin Krewell, senior analyst at the Linley Group and a senior editor of Microprocessor Report, told AllThingsD after reviewing AnandTech’s findings. “It does not serve the customer in real-world situations.”
It’s hard not to view this as a sort of benchmark doping. Certainly, no real-world application running on an Exynos 5 Octa-based Galaxy S4 will ever see this kind of GPU performance — not currently, anyway. Now, to be fair, Samsung doesn’t promise a specific GPU clock speed for the S4. So this benchmark optimization that it’s engaging in isn’t quite as untoward as it seems. And one could make the argument that it’s simply intended to showcase the furthest limits of the device’s abilities — though that’s a tenuous stretch.
That said, some consumer electronics publications do use the benchmarking tools for which Samsung is optimizing in their smartphone reviews. And those that do aren’t getting the most accurate portrait of the Galaxy S4’s real-world capabilities.
Said Krewell, “Fundamentally, this shows the futility of using synthetic benchmarks without extensive screening and lab validation. When we get application benchmarks, the consumer will get a better measure of performance, but even those are not immune from being gamed.”
Samsung was not able to comment on the Exynos 5 Octa-based Galaxy S4’s benchmarking optimizations at the time of publication.
UPDATE: Samsung has denied allegations that it designed the international version of its Galaxy S4 smartphone to perform better in benchmark tests than in real-world use. Sort of.