What just happened? Once again, a phone manufacturer has been caught using questionable (i.e., cheating) means to improve benchmark scores. On this occasion, Chinese giant Huawei was found to be engaging in the unethical practice.
AnandTech discovered that Huawei is another company guilty of manipulating benchmarking scores using a technique where the phone increases its clock speed once it detects the software. It’s the same unscrupulous trick used by OnePlus and Meizu in the OnePlus 3, OnePlus 5, and Meizu Pro 6 Plus last year. And in 2013 and 2014, both Samsung and HTC were also found to be using this method.
The publication examined Huawei’s recent Kirin 970-powered Honor Play handset and found there was a huge difference in the various bencmark scores when the BM detection was enabled; this ignores TDP recommendations, increasing a phone’s power draw. In the case of the Honor Play, it jumps from 4.39W to 8.57W.
At IFA, Dr. Wang Chenglu, President of Software at Huawei’s Consumer Business Group, told AnandTech that benchmark scores don’t reflect real-world use and he would prefer a standardized benchmark test that is closer to the user experience—something the company has been working toward for a year alongside its industry partners.
As for the cheating, Dr. Wang said that other Android manufacturers “also mislead with their numbers,” and that it was a “common practice in China.”
“Others do the same testing, get high scores, and Huawei cannot stay silent,” he added.
Huawei has had a controversial year so far. The company’s deals with AT&T and Verizon were blocked by the US government; it asked people to write fake Mate 10 Pro reviews for a chance to become beta testers, and last month it was found misrepresenting DSLR shots as Nova 3 smartphone camera samples.
Huawei has given an official statement on the matter.
Huawei always prioritizes the user experience rather than pursuing high benchmark scores – especially since there isn’t a direct connection between smartphone benchmarks and user experiences. Huawei smartphones use advanced technologies such as AI to optimize the performance of hardware, including the CPU, GPU and NPU.
When someone launches a photography app or plays a graphically-intensive game, Huawei’s intelligent software creates a smooth and stable user experience by applying the full capabilities of the hardware, while simultaneously managing the device’s temperature and power efficiency. For applications that aren’t as power intensive like browsing the web, it will only allocate the resources necessary to deliver the performance that’s needed.
In normal benchmarking scenarios, once Huawei’s software recognizes a benchmarking application, it intelligently adapts to 'Performance Mode' and delivers optimum performance. Huawei is planning to provide users with access to 'Performance Mode' so they can use the maximum power of their device when they need to.
Huawei – as the industry leader – is willing to work with partners to find the best benchmarking standards that can accurately evaluate the user experience.