Interpreter.Options

public static class Interpreter.Options

An options class for controlling runtime interpreter behavior.

Public Constructors

Options ()

Public Methods

Interpreter.Options
addDelegate ( Delegate delegate)
Adds a Delegate to be applied during interpreter creation.
Interpreter.Options
setAllowBufferHandleOutput (boolean allow)
Advanced: Set if buffer handle output is allowed.
Interpreter.Options
setAllowFp16PrecisionForFp32 (boolean allow)
This method was deprecated in API level . Prefer using ERROR(/org.tensorflow.lite.nnapi.NnApiDelegate.Options#setAllowFp16(boolean enable)) .
Interpreter.Options
setCancellable (boolean allow)
Advanced: Set if the interpreter is able to be cancelled.
Interpreter.Options
setNumThreads (int numThreads)
Sets the number of threads to be used for ops that support multi-threading.
Interpreter.Options
setUseNNAPI (boolean useNNAPI)
Sets whether to use NN API (if available) for op execution.
Interpreter.Options
setUseXNNPACK (boolean useXNNPACK)
Experimental: Enable an optimized set of floating point CPU kernels (provided by XNNPACK).

Inherited Methods

Public Constructors

public Options ()

Public Methods

public Interpreter.Options addDelegate ( Delegate delegate)

Adds a Delegate to be applied during interpreter creation.

WARNING: This is an experimental interface that is subject to change.

public Interpreter.Options setAllowBufferHandleOutput (boolean allow)

Advanced: Set if buffer handle output is allowed.

When a Delegate supports hardware acceleration, the interpreter will make the data of output tensors available in the CPU-allocated tensor buffers by default. If the client can consume the buffer handle directly (e.g. reading output from OpenGL texture), it can set this flag to false, avoiding the copy of data to the CPU buffer. The delegate documentation should indicate whether this is supported and how it can be used.

WARNING: This is an experimental interface that is subject to change.

public Interpreter.Options setAllowFp16PrecisionForFp32 (boolean allow)

This method was deprecated in API level .
Prefer using ERROR(/org.tensorflow.lite.nnapi.NnApiDelegate.Options#setAllowFp16(boolean enable)) .

Sets whether to allow float16 precision for FP32 calculation when possible. Defaults to false (disallow).

public Interpreter.Options setCancellable (boolean allow)

Advanced: Set if the interpreter is able to be cancelled.

See Also

public Interpreter.Options setNumThreads (int numThreads)

Sets the number of threads to be used for ops that support multi-threading. Defaults to a platform-dependent value.

public Interpreter.Options setUseNNAPI (boolean useNNAPI)

Sets whether to use NN API (if available) for op execution. Defaults to false (disabled).

public Interpreter.Options setUseXNNPACK (boolean useXNNPACK)

Experimental: Enable an optimized set of floating point CPU kernels (provided by XNNPACK).

Enabling this flag will enable use of a new, highly optimized set of CPU kernels provided via the XNNPACK delegate. Currently, this is restricted to a subset of floating point operations. Eventually, we plan to enable this by default, as it can provide significant peformance benefits for many classes of floating point models. See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/xnnpack/README.md for more details.

Things to keep in mind when enabling this flag:

  • Startup time and resize time may increase.
  • Baseline memory consumption may increase.
  • May be ignored if another delegate (eg NNAPI) have been applied.
  • Quantized models will not see any benefit.

WARNING: This is an experimental interface that is subject to change.