TF 2.0 is out! Get hands-on practice at TF World, Oct 28-31. Use code TF20 for 20% off select passes. Register now

Performance benchmarks

This document lists TensorFlow Lite performance benchmarks when running well known models on some Android and iOS devices.

These performance benchmark numbers were generated with the Android TFLite benchmark binary and the iOS benchmark app.

Android performance benchmarks

For Android benchmarks, the CPU affinity is set to use big cores on the device to reduce variance (see details).

It assumes that models were download and unzipped to the /data/local/tmp/tflite_models directory. The benchmark binary is built using these instructions and assumed in the /data/local/tmp directory.

To run the benchmark:

adb shell taskset ${CPU_MASK} /data/local/tmp/benchmark_model \
  --num_threads=1 \
  --graph=/data/local/tmp/tflite_models/${GRAPH} \
  --warmup_runs=1 \
  --num_runs=50 \

Here, ${GRAPH} is the name of model and ${CPU_MASK} is the CPU affinity chosen according to the following table:

Pixel 2 f0
Pixel xl 0c
Model Name Device Mean inference time
Mobilenet_1.0_224(float) Pixel 2 123.3 ms
Pixel XL 113.3 ms
Mobilenet_1.0_224 (quant) Pixel 2 65.4 ms
Pixel XL 74.6 ms
NASNet mobile Pixel 2 273.8 ms
Pixel XL 210.8 ms
SqueezeNet Pixel 2 234.0 ms
Pixel XL 158.0 ms
Inception_ResNet_V2 Pixel 2 2846.0 ms
Pixel XL 1973.0 ms
Inception_V4 Pixel 2 3180.0 ms
Pixel XL 2262.0 ms

iOS benchmarks

To run iOS benchmarks, the benchmark app was modified to include the appropriate model and benchmark_params.json was modified to set num_threads to 1.

Model Name Device Mean inference time
Mobilenet_1.0_224(float) iPhone 8 32.2 ms
Mobilenet_1.0_224 (quant) iPhone 8 24.4 ms
NASNet mobile iPhone 8 60.3 ms
SqueezeNet iPhone 8 44.3
Inception_ResNet_V2 iPhone 8 562.4 ms
Inception_V4 iPhone 8 661.0 ms