Missed TensorFlow Dev Summit? Check out the video playlist. Watch recordings

tfx.components.ModelValidator

View source on GitHub

DEPRECATED: Please use Evaluator instead.

Inherits From: BaseComponent

tfx.components.ModelValidator(
    examples, model, blessing=None, instance_name=None
)

The model validator component can be used to check model metrics threshold and validate current model against a previously validated model. If there isn't a prior validated model, model validator will just make sure the threshold passed. Otherwise, ModelValidator compares a newly trained models against a known good model, specifically the last model "blessed" by this component. A model is "blessed" if the exported model's metrics are within predefined thresholds around the prior model's metrics.

Note: This component includes a driver to resolve last blessed model.

Possible causes why model validation fails

Model validation can fail for many reasons, but these are the most common:

  • problems with training data. For example, negative examples are dropped or features are missing.
  • problems with the test or evaluation data. For example, skew exists between the training and evaluation data.
  • changes in data distribution. This indicates the user behavior may have changed over time.
  • problems with the trainer. For example, the trainer was stopped before model is converged or the model is unstable.

Example

  # Performs quality validation of a candidate model (compared to a baseline).
  model_validator = ModelValidator(
      examples=example_gen.outputs['examples'],
      model=trainer.outputs['model'])

Args:

  • examples: A Channel of type standard_artifacts.Examples, usually produced by an ExampleGen component. required
  • model: A Channel of type standard_artifacts.Model, usually produced by a Trainer component. required
  • blessing: Output channel of 'ModelBlessingPath' that contains the validation result.
  • instance_name: Optional name assigned to this specific instance of ModelValidator. Required only if multiple ModelValidator components are declared in the same pipeline.

Attributes:

  • component_id: DEPRECATED FUNCTION

  • component_type: DEPRECATED FUNCTION

  • downstream_nodes

  • exec_properties

  • id: Node id, unique across all TFX nodes in a pipeline.

    If instance name is available, node_id will be: . otherwise, node_id will be:

  • inputs

  • outputs

  • type

  • upstream_nodes

Child Classes

class DRIVER_CLASS

class SPEC_CLASS

Methods

add_downstream_node

View source

add_downstream_node(
    downstream_node
)

add_upstream_node

View source

add_upstream_node(
    upstream_node
)

from_json_dict

View source

@classmethod
from_json_dict(
    cls, dict_data
)

Convert from dictionary data to an object.

get_id

View source

@classmethod
get_id(
    cls, instance_name=None
)

Gets the id of a node.

This can be used during pipeline authoring time. For example: from tfx.components import Trainer

resolver = ResolverNode(..., model=Channel( type=Model, producer_component_id=Trainer.get_id('my_trainer')))

Args:

  • instance_name: (Optional) instance name of a node. If given, the instance name will be taken into consideration when generating the id.

Returns:

an id for the node.

to_json_dict

View source

to_json_dict()

Convert from an object to a JSON serializable dictionary.

Class Variables

  • EXECUTOR_SPEC