A TFX component to evaluate models trained by a TFX Trainer component.
Inherits From: BaseComponent
, BaseNode
tfx.v1.components.Evaluator(
examples: tfx.v1.types.BaseChannel
,
model: Optional[tfx.v1.types.BaseChannel
] = None,
baseline_model: Optional[tfx.v1.types.BaseChannel
] = None,
feature_slicing_spec: Optional[Union[tfx.v1.proto.FeatureSlicingSpec
, tfx.v1.dsl.experimental.RuntimeParameter
]] = None,
fairness_indicator_thresholds: Optional[Union[List[float], tfx.v1.dsl.experimental.RuntimeParameter
]] = None,
example_splits: Optional[List[str]] = None,
eval_config: Optional[tfma.EvalConfig] = None,
schema: Optional[tfx.v1.types.BaseChannel
] = None,
module_file: Optional[str] = None,
module_path: Optional[str] = None
)
Used in the notebooks
Used in the tutorials |
---|
Component outputs
contains:
evaluation
: Channel of typestandard_artifacts.ModelEvaluation
to store the evaluation results.blessing
: Channel of type `standard_artifacts.ModelBlessing' that contains the blessing result.
See the Evaluator guide for more details.
Args | |
---|---|
examples
|
A BaseChannel of type standard_artifacts.Examples , usually
produced by an ExampleGen component. required
|
model
|
A BaseChannel of type standard_artifacts.Model , usually produced
by a Trainer component.
|
baseline_model
|
An optional channel of type 'standard_artifacts.Model' as the baseline model for model diff and model validation purpose. |
feature_slicing_spec
|
Deprecated, please use eval_config instead. Only support estimator. evaluator_pb2.FeatureSlicingSpec instance that describes how Evaluator should slice the data. |
fairness_indicator_thresholds
|
Optional list of float (or RuntimeParameter) threshold values for use with TFMA fairness indicators. Experimental functionality: this interface and functionality may change at any time. to additional documentation for TFMA fairness indicators here. |
example_splits
|
Names of splits on which the metrics are computed. Default behavior (when example_splits is set to None or Empty) is using the 'eval' split. |
eval_config
|
Instance of tfma.EvalConfig containg configuration settings for running the evaluation. This config has options for both estimator and Keras. |
schema
|
A Schema channel to use for TFXIO.
|
module_file
|
A path to python module file containing UDFs for Evaluator customization. This functionality is experimental and may change at any time. The module_file can implement following functions at its top level. def custom_eval_shared_model( eval_saved_model_path, model_name, eval_config, **kwargs, ) -> tfma.EvalSharedModel: def custom_extractors( eval_shared_model, eval_config, tensor_adapter_config, ) -> List[tfma.extractors.Extractor]: |
module_path
|
A python path to the custom module that contains the UDFs. See 'module_file' for the required signature of UDFs. This functionality is experimental and this API may change at any time. Note this can not be set together with module_file. |
Attributes | |
---|---|
outputs
|
Component's output channel dict. |
Methods
with_beam_pipeline_args
with_beam_pipeline_args(
beam_pipeline_args: Iterable[Union[str, placeholder.Placeholder]]
) -> 'BaseBeamComponent'
Add per component Beam pipeline args.
Args | |
---|---|
beam_pipeline_args
|
List of Beam pipeline args to be added to the Beam executor spec. |
Returns | |
---|---|
the same component itself. |