Registration is open for TensorFlow Dev Summit 2020 Learn more

tfx.components.ModelValidator

View source on GitHub

Class ModelValidator

A TFX component to validate a newly trained model against a prior model.

Inherits From: BaseComponent

Aliases: tfx.components.model_validator.component.ModelValidator

Used in the tutorials:

The model validator component can be used to check model metrics threshold and validate current model against a previously validated model. If there isn't a prior validated model, model validator will just make sure the threshold passed. Otherwise, ModelValidator compares a newly trained models against a known good model, specifically the last model "blessed" by this component. A model is "blessed" if the exported model's metrics are within predefined thresholds around the prior model's metrics.

Note: This component includes a driver to resolve last blessed model.

Possible causes why model validation fails

Model validation can fail for many reasons, but these are the most common:

  • problems with training data. For example, negative examples are dropped or features are missing.
  • problems with the test or evaluation data. For example, skew exists between the training and evaluation data.
  • changes in data distribution. This indicates the user behavior may have changed over time.
  • problems with the trainer. For example, the trainer was stopped before model is converged or the model is unstable.

Example

  # Performs quality validation of a candidate model (compared to a baseline).
  model_validator = ModelValidator(
      examples=example_gen.outputs['examples'],
      model=trainer.outputs['model'])

__init__

View source

__init__(
    examples,
    model,
    blessing=None,
    instance_name=None
)

Construct a ModelValidator component.

Args:

  • examples: A Channel of 'ExamplesPath' type, usually produced by ExampleGen component. required
  • model: A Channel of 'ModelExportPath' type, usually produced by Trainer component. required
  • blessing: Output channel of 'ModelBlessingPath' that contains the validation result.
  • instance_name: Optional name assigned to this specific instance of ModelValidator. Required only if multiple ModelValidator components are declared in the same pipeline.

Child Classes

class DRIVER_CLASS

class SPEC_CLASS

Properties

component_id

DEPRECATED FUNCTION

component_type

DEPRECATED FUNCTION

downstream_nodes

exec_properties

id

Node id, unique across all TFX nodes in a pipeline.

If instance name is available, node_id will be: . otherwise, node_id will be:

Returns:

node id.

inputs

outputs

type

upstream_nodes

Methods

add_downstream_node

View source

add_downstream_node(downstream_node)

add_upstream_node

View source

add_upstream_node(upstream_node)

from_json_dict

View source

from_json_dict(
    cls,
    dict_data
)

Convert from dictionary data to an object.

to_json_dict

View source

to_json_dict()

Convert from an object to a JSON serializable dictionary.

Class Members

  • EXECUTOR_SPEC