# tff.learning.algorithms.build_personalization_eval_computation

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/federated/blob/v0.87.0 Version 2.0, January 2004 Licensed under the Apache License, Version 2.0 (the) |

Builds the TFF computation for evaluating personalization strategies.

    tff.learning.algorithms.build_personalization_eval_computation(
        model_fn: Callable[[], https://www.tensorflow.org/federated/api_docs/python/tff/learning/models/VariableModel],
        personalize_fn_dict: Mapping[str, Callable[[], https://www.tensorflow.org/federated/api_docs/python/tff/learning/models/VariableModel]],
        baseline_evaluate_fn: Callable[[variable.VariableModel, tf.data.Dataset], _MetricsType],
        max_num_clients: int = 100
    ) -> https://www.tensorflow.org/federated/api_docs/python/tff/Computation

The returned TFF computation broadcasts model weights from [`tff.SERVER`](https://www.tensorflow.org/federated/api_docs/python/tff#SERVER) to
[`tff.CLIENTS`](https://www.tensorflow.org/federated/api_docs/python/tff#CLIENTS). Each client evaluates the personalization strategies given in
`personalize_fn_dict`. Evaluation metrics from at most `max_num_clients`
participating clients are collected to the server.

> [!NOTE]
> **Note:** The functions in `personalize_fn_dict` and `baseline_evaluate_fn` are expected to take as input *unbatched* datasets, and are responsible for applying batching, if any, to the provided input datasets.

<br />

<br />

<br />

| ## Args ||
|---|---|
| `model_fn` | A no-arg function that returns a [`tff.learning.models.VariableModel`](https://www.tensorflow.org/federated/api_docs/python/tff/learning/models/VariableModel). This method must *not* capture TensorFlow tensors or variables and use them. The model must be constructed entirely from scratch on each invocation, returning the same pre-constructed model each call will result in an error. |
| `personalize_fn_dict` | An `OrderedDict` that maps a `string` (representing a strategy name) to a no-argument function that returns a [`tf.function`](https://www.tensorflow.org/api_docs/python/tf/function). Each [`tf.function`](https://www.tensorflow.org/api_docs/python/tf/function) represents a personalization strategy - it accepts a [`tff.learning.models.VariableModel`](https://www.tensorflow.org/federated/api_docs/python/tff/learning/models/VariableModel) (with weights already initialized to the given model weights when users invoke the returned TFF computation), an unbatched [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) for train, and an unbatched [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) for test, trains a personalized model, and returns the evaluation metrics. The evaluation metrics are represented as an `OrderedDict` (or a nested `OrderedDict`) of `string` metric names to scalar [`tf.Tensor`](https://www.tensorflow.org/api_docs/python/tf/Tensor)s. |
| `baseline_evaluate_fn` | A [`tf.function`](https://www.tensorflow.org/api_docs/python/tf/function) that accepts a [`tff.learning.models.VariableModel`](https://www.tensorflow.org/federated/api_docs/python/tff/learning/models/VariableModel) (with weights already initialized to the provided model weights when users invoke the returned TFF computation), and an unbatched [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset), evaluates the model on the dataset, and returns the evaluation metrics. The evaluation metrics are represented as an `OrderedDict` (or a nested `OrderedDict`) of `string` metric names to scalar [`tf.Tensor`](https://www.tensorflow.org/api_docs/python/tf/Tensor)s. This function is *only* used to compute the baseline metrics of the initial model. |
| `max_num_clients` | A positive `int` specifying the maximum number of clients to collect metrics in a round (default is 100). The clients are sampled without replacement. For each sampled client, all the personalization metrics from this client will be collected. If the number of participating clients in a round is smaller than this value, then metrics from all clients will be collected. |

<br />

<br />

<br />

| ## Returns ||
|---|---|
| A federated [`tff.Computation`](https://www.tensorflow.org/federated/api_docs/python/tff/Computation) with the functional type signature `(<model_weights@SERVER, input@CLIENTS> -> personalization_metrics@SERVER)`: <br /> - `model_weights` is a [`tff.learning.models.ModelWeights`](https://www.tensorflow.org/federated/api_docs/python/tff/learning/models/ModelWeights). - Each client's input is an `OrderedDict` of two required keys `train_data` and `test_data`; each key is mapped to an unbatched [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset). - `personalization_metrics` is an `OrderedDict` that maps a key 'baseline_metrics' to the evaluation metrics of the initial model (computed by `baseline_evaluate_fn`), and maps keys (strategy names) in `personalize_fn_dict` to the evaluation metrics of the corresponding personalization strategies. - > [!NOTE] > **Note:** only metrics from at most `max_num_clients` participating clients (sampled without replacement) are collected to the SERVER. All collected metrics are stored in a single `OrderedDict` (`personalization_metrics` shown above), where each metric is mapped to a list of scalars (each scalar comes from one client). Metric values at the same position, e.g., metric_1\[i\], metric_2\[i\]..., all come from the same client. ||

<br />

<br />

<br />

<br />

| ## Raises ||
|---|---|
| `TypeError` | If arguments are of the wrong types. |
| `ValueError` | If `baseline_metrics` is used as a key in `personalize_fn_dict`. |
| `ValueError` | If `max_num_clients` is not positive. |

<br />