|View source on GitHub|
A TFX component to validate a newly trained model against a prior model.
Used in the tutorials:
The model validator component can be used to check model metrics threshold and validate current model against a previously validated model. If there isn't a prior validated model, model validator will just make sure the threshold passed. Otherwise, ModelValidator compares a newly trained models against a known good model, specifically the last model "blessed" by this component. A model is "blessed" if the exported model's metrics are within predefined thresholds around the prior model's metrics.
Note: This component includes a driver to resolve last blessed model.
Possible causes why model validation fails
Model validation can fail for many reasons, but these are the most common:
- problems with training data. For example, negative examples are dropped or features are missing.
- problems with the test or evaluation data. For example, skew exists between the training and evaluation data.
- changes in data distribution. This indicates the user behavior may have changed over time.
- problems with the trainer. For example, the trainer was stopped before model is converged or the model is unstable.
# Performs quality validation of a candidate model (compared to a baseline). model_validator = ModelValidator( examples=example_gen.outputs['examples'], model=trainer.outputs['model'])
__init__( examples, model, blessing=None, instance_name=None )
Construct a ModelValidator component.
examples: A Channel of 'ExamplesPath' type, usually produced by ExampleGen component. required
model: A Channel of 'ModelExportPath' type, usually produced by Trainer component. required
blessing: Output channel of 'ModelBlessingPath' that contains the validation result.
instance_name: Optional name assigned to this specific instance of ModelValidator. Required only if multiple ModelValidator components are declared in the same pipeline.
Node id, unique across all TFX nodes in a pipeline.
If instance name is available, node_id will be:
from_json_dict( cls, dict_data )
Convert from dictionary data to an object.
Convert from an object to a JSON serializable dictionary.