# tf_agents.bandits.policies.ranking_policy.RankingPolicy

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/bandits/policies/ranking_policy.py#L158-L269) |

A class implementing ranking policies in TF Agents.

Inherits From: [`TFPolicy`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/policies/TFPolicy)

    tf_agents.bandits.policies.ranking_policy.RankingPolicy(
        num_items: int,
        num_slots: int,
        time_step_spec: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types#TimeStep,
        network: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types#Network,
        item_sampler: tfd.Distribution,
        penalty_mixture_coefficient: float = 1.0,
        logits_temperature: float = 1.0,
        name: Optional[Text] = None
    )

The ranking policy needs at initialization the number of items per round to
rank, a scorer network, and a score penalizer function. This function should
ensure that similar items don't all get high scores and thus a diverse set of
items is recommended.

In the case the number of items to rank varies from iteration to iteration,
the observation contains a `num_actions` value, that specifies the number of
items available. Note that in this case it can happen that the number of
ranked items is less than the number of slots. Thus, consumers of the output
of `policy.action` should always use the `num_actions` value to determine what
part of the output is the action ranking.

If `num_actions` field is not used, the policy is always presented with
`num_items` many items, which should be greater than or equal to `num_slots`.

<br />

<br />

<br />

| ## Args ||
|---|---|
| `num_items` | The number of items the policy can choose from, to be slotted. |
| `num_slots` | The number of recommendation slots presented to the user, i.e., chosen by the policy. |
| `time_step_spec` | The time step spec. |
| `network` | The network that estimates scores of items, given global and item features. |
| `item_sampler` | A distibution that takes scores and item features, and samples an ordered list of `num_slots` items. Similarity penalization can be implemented within this sampler. |
| `penalty_mixture_coefficient` | A parameter responsible for the balance between selecting high scoring items and enforcing diverisity. |
| `logits_temperature` | The "temperature" parameter for sampling. All the logits will be divided by this float value. This value must be positive. |
| `name` | The name of this policy instance. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `action_spec` | Describes the TensorSpecs of the Tensors expected by `step(action)`. <br /> `action` can be a single Tensor, or a nested dict, list or tuple of Tensors. |
| `collect_data_spec` | Describes the Tensors written when using this policy with an environment. |
| `emit_log_probability` | Whether this policy instance emits log probabilities or not. |
| `info_spec` | Describes the Tensors emitted as info by `action` and `distribution`. <br /> `info` can be an empty tuple, a single Tensor, or a nested dict, list or tuple of Tensors. |
| `num_slots` | <br /> <br /> |
| `observation_and_action_constraint_splitter` | <br /> <br /> |
| `policy_state_spec` | Describes the Tensors expected by `step(_, policy_state)`. <br /> `policy_state` can be an empty tuple, a single Tensor, or a nested dict, list or tuple of Tensors. |
| `policy_step_spec` | Describes the output of `action()`. |
| `time_step_spec` | Describes the `TimeStep` tensors returned by `step()`. |
| `trajectory_spec` | Describes the Tensors written when using this policy with an environment. |
| `validate_args` | Whether `action` \& `distribution` validate input and output args. |

<br />

## Methods

### `action`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/policies/tf_policy.py#L276-L381)

    action(
        time_step: https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep,
        policy_state: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedTensor = (),
        seed: Optional[types.Seed] = None
    ) -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/PolicyStep

Generates next action given the time_step and policy_state.

<br />

<br />

<br />

| Args ||
|---|---|
| `time_step` | A `TimeStep` tuple corresponding to `time_step_spec()`. |
| `policy_state` | A Tensor, or a nested dict, list or tuple of Tensors representing the previous policy_state. |
| `seed` | Seed to use if action performs sampling (optional). |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A `PolicyStep` named tuple containing: `action`: An action Tensor matching the `action_spec`. `state`: A policy state tensor to be fed into the next call to action. `info`: Optional side information such as action log probabilities. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `RuntimeError` | If subclass **init** didn't call super().**init** . ValueError or TypeError: If `validate_args is True` and inputs or outputs do not match `time_step_spec`, `policy_state_spec`, or `policy_step_spec`. |

<br />

### `distribution`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/policies/tf_policy.py#L383-L441)

    distribution(
        time_step: https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep,
        policy_state: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedTensor = ()
    ) -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/PolicyStep

Generates the distribution over next actions given the time_step.

<br />

<br />

<br />

| Args ||
|---|---|
| `time_step` | A `TimeStep` tuple corresponding to `time_step_spec()`. |
| `policy_state` | A Tensor, or a nested dict, list or tuple of Tensors representing the previous policy_state. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A `PolicyStep` named tuple containing: <br /> `action`: A tf.distribution capturing the distribution of next actions. `state`: A policy state tensor for the next call to distribution. `info`: Optional side information such as action log probabilities. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| ValueError or TypeError: If `validate_args is True` and inputs or outputs do not match `time_step_spec`, `policy_state_spec`, or `policy_step_spec`. ||

<br />

### `get_initial_state`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/policies/tf_policy.py#L242-L255)

    get_initial_state(
        batch_size: Optional[types.Int]
    ) -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedTensor

Returns an initial state usable by the policy.

<br />

<br />

<br />

| Args ||
|---|---|
| `batch_size` | Tensor or constant: size of the batch dimension. Can be None in which case no dimensions gets added. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A nested object of type `policy_state` containing properly initialized Tensors. ||

<br />

### `update`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/policies/tf_policy.py#L443-L475)

    update(
        policy,
        tau: float = 1.0,
        tau_non_trainable: Optional[float] = None,
        sort_variables_by_name: bool = False
    ) -> tf.Operation

Update the current policy with another policy.

This would include copying the variables from the other policy.

<br />

<br />

<br />

| Args ||
|---|---|
| `policy` | Another policy it can update from. |
| `tau` | A float scalar in \[0, 1\]. When tau is 1.0 (the default), we do a hard update. This is used for trainable variables. |
| `tau_non_trainable` | A float scalar in \[0, 1\] for non_trainable variables. If None, will copy from tau. |
| `sort_variables_by_name` | A bool, when True would sort the variables by name before doing the update. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| An TF op to do the update. ||

<br />