tft.ptransform_analyzer

View source on GitHub

Applies a user-provided PTransform over the whole dataset.

tft.ptransform_analyzer(
    inputs, output_dtypes, output_shapes, ptransform, name=None
)

Note that in order to have asset files copied correctly, any outputs that represent asset filenames must be added to the tf.GraphKeys.ASSET_FILEPATHS collection by the caller.

inputs A list of input Tensors.
output_dtypes The list of TensorFlow dtypes of the output of the analyzer.
output_shapes The list of shapes of the output of the analyzer. Must have the same length as output_dtypes.
ptransform A Beam PTransform that accepts a Beam PCollection where each element is a list of ndarrays. Each element in the list contains a batch of values for the corresponding input tensor of the analyzer. It returns a tuple of PCollection, each containing a single element which is an ndarray.
name (Optional) Similar to a TF op name. Used to define a unique scope for this analyzer, which can be used for debugging info.

A list of output Tensors. These will have dtype and shape as specified by output_dtypes and output_shapes.

ValueError If output_dtypes and output_shapes have different lengths.