# tfdv.CombinerStatsGenerator

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/data-validation/blob/v1.16.1/tensorflow_data_validation/statistics/generators/stats_generator.py#L81-L188) |

A StatsGenerator which computes statistics using a combiner function.

    tfdv.CombinerStatsGenerator(
        name: Text, schema: Optional[schema_pb2.Schema] = None
    ) -> None

This class computes statistics using a combiner function. It emits partial
states processing a batch of examples at a time, merges the partial states,
and finally computes the statistics from the merged partial state at the end.

This object mirrors a beam.CombineFn except for the add_input interface, which
is expected to be defined by its sub-classes. Specifically, the generator
must implement the following four methods:

Initializes an accumulator to store the partial state and returns it.
create_accumulator()

Incorporates a batch of input examples (represented as an arrow RecordBatch)
into the current accumulator and returns the updated accumulator.
add_input(accumulator, input_record_batch)

Merge the partial states in the accumulators and returns the accumulator
containing the merged state.
merge_accumulators(accumulators)

Compute statistics from the partial state in the accumulator and
return the result as a DatasetFeatureStatistics proto.
extract_output(accumulator)

<br />

<br />

<br />

| ## Args ||
|---|---|
| `name` | A unique name associated with the statistics generator. |
| `schema` | An optional schema for the dataset. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `name` | <br /> <br /> |
| `schema` | <br /> <br /> |

<br />

## Methods

### `add_input`

[View source](https://github.com/tensorflow/data-validation/blob/v1.16.1/tensorflow_data_validation/statistics/generators/stats_generator.py#L128-L143)

    add_input(
        accumulator: ACCTYPE, input_record_batch: pa.RecordBatch
    ) -> ACCTYPE

Returns result of folding a batch of inputs into accumulator.

<br />

<br />

<br />

| Args ||
|---|---|
| `accumulator` | The current accumulator, which may be modified and returned for efficiency. |
| `input_record_batch` | An Arrow RecordBatch whose columns are features and rows are examples. The columns are of type List or Null (If a feature's value is None across all the examples in the batch, its corresponding column is of Null type). |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| The accumulator after updating the statistics for the batch of inputs. ||

<br />

### `compact`

[View source](https://github.com/tensorflow/data-validation/blob/v1.16.1/tensorflow_data_validation/statistics/generators/stats_generator.py#L161-L173)

    compact(
        accumulator: ACCTYPE
    ) -> ACCTYPE

Returns a compact representation of the accumulator.

This is optionally called before an accumulator is sent across the wire. The
base class is a no-op. This may be overwritten by the derived class.

<br />

<br />

<br />

| Args ||
|---|---|
| `accumulator` | The accumulator to compact. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| The compacted accumulator. By default is an identity. ||

<br />

### `create_accumulator`

[View source](https://github.com/tensorflow/data-validation/blob/v1.16.1/tensorflow_data_validation/statistics/generators/stats_generator.py#L120-L126)

    create_accumulator() -> ACCTYPE

Returns a fresh, empty accumulator.

<br />

<br />

<br />

| Returns ||
|---|---|
| An empty accumulator. ||

<br />

### `extract_output`

[View source](https://github.com/tensorflow/data-validation/blob/v1.16.1/tensorflow_data_validation/statistics/generators/stats_generator.py#L175-L185)

    extract_output(
        accumulator: ACCTYPE
    ) -> statistics_pb2.DatasetFeatureStatistics

Returns result of converting accumulator into the output value.

<br />

<br />

<br />

| Args ||
|---|---|
| `accumulator` | The final accumulator value. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A proto representing the result of this stats generator. ||

<br />

### `merge_accumulators`

[View source](https://github.com/tensorflow/data-validation/blob/v1.16.1/tensorflow_data_validation/statistics/generators/stats_generator.py#L145-L157)

    merge_accumulators(
        accumulators: Iterable[ACCTYPE]
    ) -> ACCTYPE

Merges several accumulators to a single accumulator value.

> [!NOTE]
> **Note:** mutating any element in `accumulators` except for the first is not allowed. The first element may be modified and returned for efficiency.

<br />

<br />

<br />

| Args ||
|---|---|
| `accumulators` | The accumulators to merge. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| The merged accumulator. ||

<br />

### `setup`

[View source](https://github.com/tensorflow/data-validation/blob/v1.16.1/tensorflow_data_validation/statistics/generators/stats_generator.py#L110-L118)

    setup() -> None

Prepares an instance for combining.

Subclasses should put costly initializations here instead of in
**init**(), so that 1) the cost is properly recognized by Beam as
setup cost (per worker) and 2) the cost is not paid at the pipeline
construction time.