Nearly all Android OEMs found to cheat in benchmarks

By on October 3, 2013, 7:00 AM
google, android, qualcomm, samsung, nvidia, htc, motorola, lg, benchmark, cheating

Yesterday, along with the first wave of Galaxy Note 3 reviews, it was revealed that Samsung once again had optimized their new smartphone for Android benchmarks. Through forcing the device's CPU to run at its maximum clock speed of 2.3 GHz in benchmarks, it scored 20% better than if the CPU was allowed to power gait cores like normal.

AnandTech dug deeper into the issue, testing a number of Android devices to see if OEMs other than Samsung were guilty of optimizing their smartphones for benchmarks. Asus, HTC and LG were caught boosting CPU speeds of their Qualcomm-based devices in a selection of benchmarks, while Samsung was the most aggressive. Out of the seven benchmarks checked, the Note 3 was optimized for six, while Samsung's Intel Atom-based Galaxy Tab 3 10.1 and the Exynos-based Galaxy Note 10.1 (2014) were also found guilty.

Motorola, alongside Google's Nexus devices and Nvidia's Shield, were found not to optimize the CPU for benchmarks. This isn't overly surprising, because the cheating code isn't part of vanilla Android or AOSP, which is essentially what is included on these devices.

The whole process of optimizing CPU clock speeds for running benchmarks is a phenomenal waste of engineering time and money, as in the end it has no real-world effect on performance. It's only making the devices look marginally better in reviews, and now that the media has exposed the trickery of OEMs, it's doubtful whether the effort is worth the negative attention.

Needless to say we'll be looking out for any benchmark cheaters in our upcoming smartphone reviews, making note of where benchmarks may be affected.

Add New Comment

TechSpot Members
Login or sign up for free,
it takes about 30 seconds.
You may also...
Get complete access to the TechSpot community. Join thousands of technology enthusiasts that contribute and share knowledge in our forum. Get a private inbox, upload your own photo gallery and more.