MetricComputation represents one or more metric computations.
tfma.metrics.MetricComputation(
keys: List[tfma.metrics.MetricKey
],
preprocessor: beam.DoFn,
combiner: beam.CombineFn
)
The preprocessor is called with a PCollection of extracts (or list of extracts
if query_key is used) to compute the initial combiner input state which is
then passed to the combiner. This needs to be done in two steps because
slicing happens between the call to the preprocessor and the combiner and this
state may end up in multiple slices so we want the representation to be as
efficient as possible. If the preprocessor is None, then StandardMetricInputs
will be passed.
A MetricComputation is uniquely identified by the combination of the
combiner's name and the keys. Duplicate computations will be removed
automatically.
Attributes |
keys
|
List of metric keys associated with computation. If the keys are
defined as part of the computation then this may be empty in which case
only the combiner name will be used for identifying computation
uniqueness.
|
preprocessor
|
Takes a extracts (or a list of extracts) as input (which
typically will contain labels, predictions, example weights, and
optionally features) and should return the initial state that the combiner
will use as input. The output of a processor should only contain
information needed by the combiner. Note that if a query_key is used the
preprocessor will be passed a list of extracts as input representing the
extracts that matched the query_key. The special FeaturePreprocessor can
be used to add additional features to the default standard metric inputs.
|
combiner
|
Takes preprocessor output as input and outputs a tuple: (slice,
metric results). The metric results should be a dict from MetricKey to
value (float, int, distribution, ...).
|