# tf_agents.bandits.policies.boltzmann_reward_prediction_policy.BoltzmannRewardPredictionPolicy

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/bandits/policies/boltzmann_reward_prediction_policy.py#L37-L206) |

Class to build Reward Prediction Policies with Boltzmann exploration.

Inherits From: [`RewardPredictionBasePolicy`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/bandits/policies/reward_prediction_base_policy/RewardPredictionBasePolicy), [`TFPolicy`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/policies/TFPolicy)

    tf_agents.bandits.policies.boltzmann_reward_prediction_policy.BoltzmannRewardPredictionPolicy(
        time_step_spec: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types#TimeStep,
        action_spec: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedTensorSpec,
        reward_network: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types#Network,
        temperature: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/FloatOrReturningFloat = 1.0,
        boltzmann_gumbel_exploration_constant: Optional[types.Float] = None,
        observation_and_action_constraint_splitter: Optional[types.Splitter] = None,
        accepts_per_arm_features: bool = False,
        constraints: Iterable[https://www.tensorflow.org/agents/api_docs/python/tf_agents/bandits/policies/constraints/BaseConstraint] = (),
        emit_policy_info: Tuple[Text, ...] = (),
        num_samples_list: Sequence[tf.Variable] = (),
        name: Optional[Text] = None
    )

<br />

<br />

<br />

| ## Args ||
|---|---|
| `time_step_spec` | A `TimeStep` spec of the expected time_steps. |
| `action_spec` | A nest of BoundedTensorSpec representing the actions. |
| `reward_network` | An instance of a `tf_agents.network.Network`, callable via `network(observation, step_type) -> (output, final_state)`. |
| `temperature` | float or callable that returns a float. The temperature used in the Boltzmann exploration. |
| `boltzmann_gumbel_exploration_constant` | optional positive float. When provided, the policy implements Neural Bandit with Boltzmann-Gumbel exploration from the paper: N. Cesa-Bianchi et al., "Boltzmann Exploration Done Right", NIPS 2017. |
| `observation_and_action_constraint_splitter` | A function used for masking valid/invalid actions with each state of the environment. The function takes in a full observation and returns a tuple consisting of 1) the part of the observation intended as input to the network and 2) the mask. The mask should be a 0-1 `Tensor` of shape `[batch_size, num_actions]`. This function should also work with a `TensorSpec` as input, and should output `TensorSpec` objects for the observation and mask. |
| `accepts_per_arm_features` | (bool) Whether the policy accepts per-arm features. |
| `constraints` | iterable of constraints objects that are instances of `tf_agents.bandits.agents.BaseConstraint`. |
| `emit_policy_info` | (tuple of strings) what side information we want to get as part of the policy info. Allowed values can be found in `policy_utilities.PolicyInfo`. |
| `num_samples_list` | list or tuple of tf.Variable's. Used only in Boltzmann-Gumbel exploration. Otherwise, empty. |
| `name` | The name of this policy. All variables in this module will fall under that name. Defaults to the class name. |

<br />

<br />

<br />

<br />

| ## Raises ||
|---|---|
| `NotImplementedError` | If `action_spec` contains more than one `BoundedTensorSpec` or the `BoundedTensorSpec` is not valid. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `accepts_per_arm_features` | <br /> <br /> |
| `action_spec` | Describes the TensorSpecs of the Tensors expected by `step(action)`. <br /> `action` can be a single Tensor, or a nested dict, list or tuple of Tensors. |
| `collect_data_spec` | Describes the Tensors written when using this policy with an environment. |
| `emit_log_probability` | Whether this policy instance emits log probabilities or not. |
| `info_spec` | Describes the Tensors emitted as info by `action` and `distribution`. <br /> `info` can be an empty tuple, a single Tensor, or a nested dict, list or tuple of Tensors. |
| `observation_and_action_constraint_splitter` | <br /> <br /> |
| `policy_state_spec` | Describes the Tensors expected by `step(_, policy_state)`. <br /> `policy_state` can be an empty tuple, a single Tensor, or a nested dict, list or tuple of Tensors. |
| `policy_step_spec` | Describes the output of `action()`. |
| `time_step_spec` | Describes the `TimeStep` tensors returned by `step()`. |
| `trajectory_spec` | Describes the Tensors written when using this policy with an environment. |
| `validate_args` | Whether `action` \& `distribution` validate input and output args. |

<br />

## Methods

### `action`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/policies/tf_policy.py#L276-L381)

    action(
        time_step: https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep,
        policy_state: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedTensor = (),
        seed: Optional[types.Seed] = None
    ) -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/PolicyStep

Generates next action given the time_step and policy_state.

<br />

<br />

<br />

| Args ||
|---|---|
| `time_step` | A `TimeStep` tuple corresponding to `time_step_spec()`. |
| `policy_state` | A Tensor, or a nested dict, list or tuple of Tensors representing the previous policy_state. |
| `seed` | Seed to use if action performs sampling (optional). |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A `PolicyStep` named tuple containing: `action`: An action Tensor matching the `action_spec`. `state`: A policy state tensor to be fed into the next call to action. `info`: Optional side information such as action log probabilities. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `RuntimeError` | If subclass **init** didn't call super().**init** . ValueError or TypeError: If `validate_args is True` and inputs or outputs do not match `time_step_spec`, `policy_state_spec`, or `policy_step_spec`. |

<br />

### `distribution`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/policies/tf_policy.py#L383-L441)

    distribution(
        time_step: https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep,
        policy_state: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedTensor = ()
    ) -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/PolicyStep

Generates the distribution over next actions given the time_step.

<br />

<br />

<br />

| Args ||
|---|---|
| `time_step` | A `TimeStep` tuple corresponding to `time_step_spec()`. |
| `policy_state` | A Tensor, or a nested dict, list or tuple of Tensors representing the previous policy_state. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A `PolicyStep` named tuple containing: <br /> `action`: A tf.distribution capturing the distribution of next actions. `state`: A policy state tensor for the next call to distribution. `info`: Optional side information such as action log probabilities. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| ValueError or TypeError: If `validate_args is True` and inputs or outputs do not match `time_step_spec`, `policy_state_spec`, or `policy_step_spec`. ||

<br />

### `get_initial_state`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/policies/tf_policy.py#L242-L255)

    get_initial_state(
        batch_size: Optional[types.Int]
    ) -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedTensor

Returns an initial state usable by the policy.

<br />

<br />

<br />

| Args ||
|---|---|
| `batch_size` | Tensor or constant: size of the batch dimension. Can be None in which case no dimensions gets added. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A nested object of type `policy_state` containing properly initialized Tensors. ||

<br />

### `update`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/policies/tf_policy.py#L443-L475)

    update(
        policy,
        tau: float = 1.0,
        tau_non_trainable: Optional[float] = None,
        sort_variables_by_name: bool = False
    ) -> tf.Operation

Update the current policy with another policy.

This would include copying the variables from the other policy.

<br />

<br />

<br />

| Args ||
|---|---|
| `policy` | Another policy it can update from. |
| `tau` | A float scalar in \[0, 1\]. When tau is 1.0 (the default), we do a hard update. This is used for trainable variables. |
| `tau_non_trainable` | A float scalar in \[0, 1\] for non_trainable variables. If None, will copy from tau. |
| `sort_variables_by_name` | A bool, when True would sort the variables by name before doing the update. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| An TF op to do the update. ||

<br />