Huawei found tampering with benchmarks, says it does so to remain competitive
Benchmarking is meant to be used to help manufacturers keep updating their hardware to perform at their best levels and to match what the competition is up to. But it has come out that many manufacturers have found ways of either tampering with the benchmarks or denying access on their devices to the users to the benchmarking apps so that the comparison becomes impossible.
In a surprising statement, an official at the Chinese major Huawei admitted to AnandTech that some amount of tampering of the benchmarking results does take place so that their devices are shown in better light in comparison with the benchmarks.
The report goes on to quote the Huawei officials as claiming that they would want the benchmarks to be reviewed and more realistic standards set for evaluation of performance. The team also said if there is a lower value on one count or feature, the entire device cannot be rated as poor. Huawei is willing to be part of any steps initiated by the industry to review the benchmarking standards and their revision, keeping user experience as the most important factor. These comments came from Dr Wang Chenglu, who is the President of Software at Huawei’s Consumer Business Group.
Wang had also referred to the benchmarking tests done for the gaming experience with devices and they feel these matters were being discussed with other manufacturers so that the tests and results remain as close to the real-life experience as possible rather than being judged against highly theoretical or hypothetical values.
Now that it has come into the public space that the manufacturers of smartphones and other devices are tweaking the benchmarking software, the developers of these benchmarking apps may get down to reviewing their technologies to prevent such a thing from happening. The report alludes to the way smartphone makers claim as the capabilities of say the cameras fitted on their phones. It is becoming increasingly difficult to ascertain if such claims are indeed valid and can be taken on their face value.
Hopefully, the major industry players get together and take the issue to its logical conclusion sooner than later, keeping the interests of the phone users at large in view.