Objective
This document provides examples for the intended usage of SignatureDefs in SavedModel that map to TensorFlow Serving's APIs.
Overview
A SignatureDef defines the signature of a computation supported in a TensorFlow graph. SignatureDefs aim to provide generic support to identify inputs and outputs of a function and can be specified when building a SavedModel.
Background
TF-Exporter
and
SessionBundle
used
Signatures
which are similar in concept but required users to distinguish between named and
default signatures in order for them to be retrieved correctly upon a load. For
those who previously used TF-Exporter/SessionBundle, Signatures
in TF-Exporter
will be replaced by SignatureDefs
in SavedModel.
SignatureDef Structure
A SignatureDef requires specification of:
inputs
as a map of string to TensorInfo.outputs
as a map of string to TensorInfo.method_name
(which corresponds to a supported method name in the loading tool/system).
Note that TensorInfo itself requires specification of name, dtype and tensor shape. While tensor information is already present in the graph, it is useful to explicitly have the TensorInfo defined as part of the SignatureDef since tools can then perform signature validation, etc. without having to read the graph definition.
Related constants and utils
For ease of reuse and sharing across tools and systems, commonly used constants related to SignatureDefs that will be supported in TensorFlow Serving are defined as constants. Specifically:
In addition, SavedModel provides a util to help build a signature-def.
Sample structures
TensorFlow Serving provides high level APIs for performing inference. To enable these APIs, models must include one or more SignatureDefs that define the exact TensorFlow nodes to use for input and output. See below for examples of the specific SignatureDefs that TensorFlow Serving supports for each API.
Note that TensorFlow Serving depends on the keys of each TensorInfo (in the inputs and outputs of the SignatureDef), as well as the method_name of the SignatureDef. The actual contents of the TensorInfo are specific to your graph.
Classification SignatureDef
Classification SignatureDefs support structured calls to TensorFlow Serving's
Classification API. These prescribe that there must be an inputs
Tensor, and
that there are two optional output Tensors: classes
and scores
, at least one
of which must be present.
signature_def: {
key : "my_classification_signature"
value: {
inputs: {
key : "inputs"
value: {
name: "tf_example:0"
dtype: DT_STRING
tensor_shape: ...
}
}
outputs: {
key : "classes"
value: {
name: "index_to_string:0"
dtype: DT_STRING
tensor_shape: ...
}
}
outputs: {
key : "scores"
value: {
name: "TopKV2:0"
dtype: DT_FLOAT
tensor_shape: ...
}
}
method_name: "tensorflow/serving/classify"
}
}
Predict SignatureDef
Predict SignatureDefs support calls to TensorFlow Serving's Predict API. These
signatures allow you to flexibly support arbitrarily many input and output
Tensors. For the example below, the signature my_prediction_signature
has a
single logical input Tensor images
that are mapped to the actual Tensor in
your graph x:0
.
Predict SignatureDefs enable portability across models. This means that you can
swap in different SavedModels, possibly with different underlying Tensor names
(e.g. instead of x:0
perhaps you have a new alternate model with a Tensor
z:0
), while your clients can stay online continuously querying the old and new
versions of this model without client-side changes.
Predict SignatureDefs also allow you to add optional additional Tensors to the
outputs, that you can explicitly query. Let's say that in addition to the output
key below of scores
, you also wanted to fetch a pooling layer for debugging or
other purposes. In that case, you would simply add an additional Tensor with a
key like pool
and appropriate value.
signature_def: {
key : "my_prediction_signature"
value: {
inputs: {
key : "images"
value: {
name: "x:0"
dtype: ...
tensor_shape: ...
}
}
outputs: {
key : "scores"
value: {
name: "y:0"
dtype: ...
tensor_shape: ...
}
}
method_name: "tensorflow/serving/predict"
}
}
Regression SignatureDef
Regression SignatureDefs support structured calls to TensorFlow Serving's
Regression API. These prescribe that there must be exactly one inputs
Tensor,
and one outputs
Tensor.
signature_def: {
key : "my_regression_signature"
value: {
inputs: {
key : "inputs"
value: {
name: "x_input_examples_tensor_0"
dtype: ...
tensor_shape: ...
}
}
outputs: {
key : "outputs"
value: {
name: "y_outputs_0"
dtype: DT_FLOAT
tensor_shape: ...
}
}
method_name: "tensorflow/serving/regress"
}
}