# tff.learning.metrics.finalize_then_sample

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/federated/blob/v0.87.0 Version 2.0, January 2004 Licensed under the Apache License, Version 2.0 (the) |

Creates a TFF computation to aggregate metrics via `finalize_then_sample`.

    tff.learning.metrics.finalize_then_sample(
        metric_finalizers: Union[https://www.tensorflow.org/federated/api_docs/python/tff/learning/metrics/MetricFinalizersType, https://www.tensorflow.org/federated/api_docs/python/tff/learning/metrics/FunctionalMetricFinalizersType],
        local_unfinalized_metrics_type: Optional[https://www.tensorflow.org/federated/api_docs/python/tff/types/StructWithPythonType] = None,
        sample_size: int = 100
    ) -> https://www.tensorflow.org/federated/api_docs/python/tff/Computation

The returned federated TFF computation is a polymorphic computation that
accepts unfinalized client metrics, and returns finalized, summed metrics
placed at the server. Invoking the polymorphic computation will initiate
tracing on the argument and will raise a `ValueError` if the keys (i.e.,
metric names) in `metric_finalizers` are not the same as those of the argument
the polymorphic method is invoked on.

> [!NOTE]
> **Note:** invoking this computation outside of a federated context (a method decoratedc with [`tff.federated_computation`](https://www.tensorflow.org/federated/api_docs/python/tff/federated_computation)) will require first wrapping it in a concrete, non-polymorphic [`tff.Computation`](https://www.tensorflow.org/federated/api_docs/python/tff/Computation) with appropriate federated types.

The returned computation is intended to be invoked on the output of
[`tff.learning.models.VariableModel.report_local_unfinalized_metrics()`](https://www.tensorflow.org/federated/api_docs/python/tff/learning/models/VariableModel#report_local_unfinalized_metrics) when
placed at `CLIENTS`. The output is computed by first finalizing each client's
metrics locally, and then collecting metrics from at most `sample_size`
clients at the `SERVER`. If more than `sample_size` clients participating,
then `sample_size` clients are sampled (by reservoir sampling algorithm);
otherwise, all clients' metrics are collected. Sampling is done in a
"per-client" manner, i.e., a client, once sampled, will contribute all its
metrics to the final result.

The collected metrics samples at `SERVER` has the same structure (i.e., same
keys in a dictionary) as the client's local metrics, except that each leaf
node contains a list of scalar metric values, where each value comes from a
sampled client, e.g.,

      sampled_metrics_at_server = {
          'metric_a': [a1, a2, ...],
          'metric_b': [b1, b2, ...],
          ...
      }

where "a1" and "b1" are from the same client (similary for "a2" and "b2" etc).

<br />

<br />

<br />

| ## Example usage ||
|---|---|
| <br /> training_process = tff.learning.algorithms.build_weighted_fed_avg( model_fn=..., ..., metrics_aggregator=finalize_then_sample, ) state = training_process.initialize() for i in range(num_rounds): output = training_process.next(state, client_data_at_round_i) state = output.state sampled_client_metrics = output.metrics['client_work'] <br /> ||

<br />

<br />

<br />

<br />

| ## Args ||
|---|---|
| `metric_finalizers` | Either the result of [`tff.learning.models.VariableModel.metric_finalizers`](https://www.tensorflow.org/federated/api_docs/python/tff/learning/models/VariableModel#metric_finalizers) (an `OrderedDict` of callables) or the [`tff.learning.models.FunctionalModel.finalize_metrics`](https://www.tensorflow.org/federated/api_docs/python/tff/learning/models/FunctionalModel#finalize_metrics) method (a callable that takes an `OrderedDict` argument). If the former, the keys must be the same as the `OrderedDict` returned by [`tff.learning.models.VariableModel.report_local_unfinalized_metrics`](https://www.tensorflow.org/federated/api_docs/python/tff/learning/models/VariableModel#report_local_unfinalized_metrics). If the later, the callable must compute over the same keyspace of the result returned by [`tff.learning.models.FunctionalModel.update_metrics_state`](https://www.tensorflow.org/federated/api_docs/python/tff/learning/models/FunctionalModel#update_metrics_state). |
| `local_unfinalized_metrics_type` | Unused, will be removed from the API in the future. |
| `sample_size` | An integer specifying the number of clients sampled by the reservoir sampling algorithm. Metrics from the sampled clients are collected at the server. If the total number of participating clients are smaller than this value, then all clients' metrics are collected. Default value is 100. |

<br />

<br />

<br />

<br />

| ## Returns ||
|---|---|
| A federated TFF computation that finalizes the unfinalized metrics from `CLIENTS`, samples the clients, and returns the sampled metrics at `SERVER`. ||

<br />

<br />

<br />

<br />

| ## Raises ||
|---|---|
| `TypeError` | If the inputs are of the wrong types. |
| `ValueError` | If `sample_size` is not positive. |

<br />