AnTuTu releases an AI benchmarking app, the AI Review

To test the AI capabilities of AnTuTu released its own AI benchmarking tool. It is dubbed as the "AI Review" app.

AnTuTu's AI benchmark app

AI Review

AI Review could be used to measure the AI performance of smartphones. It could be used to compare the AI capability of different platforms. AnTuTu said that they worked with chip manufacturers to make this initiative work.

AI score sample

AnTuTu pointed out the difficulties in measuring AI as there is no current unified standard for AI. Every chip maker has their own understanding and implementation of AI.

Huawei's Hisilicon semiconductor brand handles AI with its own independent NPU aside from the CPU. Qualcomm has Hexagon DSP. Samsung and MediaTek has dedicated NPU and APU chips as well.

Each vendor also has their own SDK for AI. HiSilicon has HiAI, Qualcomm has SNPE, MediaTek has NeuroPilot, and so on. This situation made things more complicated for AnTuTu to measure AI performance.

AnTuTu's method is to divide the benchmark into two sub-categories: Image Classification and Object Recognition.

Image Classification test uses 200 images based on Inception v3 neural network. On the other hand, Object Recognition has 600-frame video based on MobileNet SSD neural network.

The score will be based on both speed and accuracy.  If accuracy is traded off for speed, AnTuTu will penalize the score. It discourages benchmark cheating as well.

The solution of AnTuTu is not yet perfect though. If the chip does not support AI-related algorithms, the benchmark app uses TFLite for benchmarking. AnTuTu themselves warn as it could be unsatisfactory and unreliable.

Samsung has not released its AI SKD yet and HiSilicon is utilizing TFLite for certain functions. This means that Samsung and Huawei devices will get low AI scores until AnTuTu will have a solution to this issue.

It is okay for now, we are sure that AnTuTu will improve its AI benchmarking solution in the near future.

AI Review could make phone makers strive to be more competitive with each other. In the end, consumers will win.

As of writing, the app is only available for Android devices.

To download AI Review, click here.

Sources: AnTuTu, XDA