# tf_agents.bandits.environments.ranking_environment.RankingPyEnvironment

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/bandits/environments/ranking_environment.py#L78-L314) |

Stationary Stochastic Bandit environment with per-arm features.

Inherits From: [`BanditPyEnvironment`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/bandits/environments/bandit_py_environment/BanditPyEnvironment), [`PyEnvironment`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/environments/PyEnvironment)

    tf_agents.bandits.environments.ranking_environment.RankingPyEnvironment(
        global_sampling_fn: Callable[[], https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/Array],
        item_sampling_fn: Callable[[], https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/Array],
        num_items: int,
        num_slots: int,
        scores_weight_matrix: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/Float,
        feedback_model: int = FeedbackModel.CASCADING,
        click_model: int = ClickModel.GHOST_ACTIONS,
        real_cascade: bool = False,
        distance_threshold: Optional[float] = None,
        batch_size: Optional[int] = 1,
        name: Optional[Text] = 'ranking_environment'
    )

### Used in the notebooks

| Used in the tutorials |
|---|
| - [Tutorial on Ranking in TF-Agents](https://www.tensorflow.org/agents/tutorials/ranking_tutorial) |

<br />

<br />

<br />

| ## Args ||
|---|---|
| `global_sampling_fn` | A function that outputs a random 1d array or list of ints or floats. This output is the global context. Its shape and type must be consistent across calls. |
| `item_sampling_fn` | A function that outputs a random 1 array or list of ints or floats (same type as the output of `global_context_sampling_fn`). This output is the per-arm context. Its shape must be consistent across calls. |
| `num_items` | (int) the number of items in every sample. |
| `num_slots` | (int) the number of items recommended in every round. |
| `scores_weight_matrix` | A tensor of shape `[item_dim, global_dim]`. The score of an item is calculated as `global * M * item + noise`. |
| `feedback_model` | The type of feedback model. Currently only implemented is -- `cascading`: the feedback is a tuple `(k, v)`, where `k` is the index of the chosen item, and `v` is the value of the choice. |
| `click_model` | The way the environment models that diversity is desired. -- `ghost_actions`: For every dimension of the item space, a unit vector is added to the list of available items. If one of these unit-vector items gets selected, it results in a `no-click`. -- `distance_based`: Inner-product scores are calculated, and if none of the scores exceed a given parameter, no item is clicked. |
| `real_cascade` | If true, the user is actually modeled as a sequentional decision maker, clicking on the first relevant item. |
| `distance_threshold` | (float) In case the diversity model is distance based, this threshold governs if the user actually clicked on any of the items. |
| `batch_size` | The batch size. |
| `name` | The name of this environment instance. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `batch_size` | The batch size of the environment. |
| `name` | <br /> <br /> |

<br />

## Methods

### `action_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/bandits/environments/bandit_py_environment.py#L88-L89)

    action_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the actions that should be provided to `step()`.

May use a subclass of `ArraySpec` that specifies additional properties such
as min and max bounds on the values.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `batched`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/bandits/environments/ranking_environment.py#L195-L196)

    batched() -> bool

Whether the environment is batched or not.

If the environment supports batched observations and actions, then overwrite
this property to True.

A batched environment takes in a batched set of actions and returns a
batched set of observations. This means for all numpy arrays in the input
and output nested structures, the first dimension is the batch size.

When batched, the left-most dimension is not part of the action_spec
or the observation_spec and corresponds to the batch dimension.

When batched and handle_auto_reset, it checks `np.all(steps.is_last())`.

<br />

<br />

<br />

| Returns ||
|---|---|
| A boolean indicating whether the environment is batched or not. ||

<br />

### `close`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L241-L261)

    close() -> None

Frees any resources used by the environment.

Implement this method for an environment backed by an external process.

This method be used directly

    env = Env(...)
    # Use env.
    env.close()

or via a context manager

    with Env(...) as env:
      # Use env.

### `current_time_step`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L179-L183)

    current_time_step() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Returns the current timestep.

### `discount_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L153-L164)

    discount_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the discount that are returned by `step()`.

Override this method to define an environment that uses non-standard
discount values, for example an environment with array-valued discounts.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `get_info`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L297-L306)

    get_info() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArray

Returns the environment info returned on the last step.

<br />

<br />

<br />

| Returns ||
|---|---|
| Info returned by last call to step(). None by default. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `NotImplementedError` | If the environment does not use info. |

<br />

### `get_state`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L308-L330)

    get_state() -> Any

Returns the `state` of the environment.

The `state` contains everything required to restore the environment to the
current configuration. This can contain e.g.

- The current time_step.
- The number of steps taken in the environment (for finite horizon MDPs).
- Hidden state (for POMDPs).

Callers should not assume anything about the contents or format of the
returned `state`. It should be treated as a token that can be passed back to
`set_state()` later.

Note that the returned `state` handle should not be modified by the
environment later on, and ensuring this (e.g. using copy.deepcopy) is the
responsibility of the environment.

<br />

<br />

<br />

| Returns ||
|---|---|
| `state` | The current state of the environment. |

<br />

### `observation_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/bandits/environments/bandit_py_environment.py#L91-L92)

    observation_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the observations provided by the environment.

May use a subclass of `ArraySpec` that specifies additional properties such
as min and max bounds on the values.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `render`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L271-L286)

    render(
        mode: Text = 'rgb_array'
    ) -> Optional[types.NestedArray]

Renders the environment.

<br />

<br />

<br />

| Args ||
|---|---|
| `mode` | One of \['rgb_array', 'human'\]. Renders to an numpy array, or brings up a window where the environment can be visualized. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| An ndarray of shape \[width, height, 3\] denoting an RGB image if mode is `rgb_array`. Otherwise return nothing and render directly to a display window. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `NotImplementedError` | If the environment does not support rendering. |

<br />

### `reset`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L185-L201)

    reset() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Starts a new sequence and returns the first `TimeStep` of this sequence.

> [!NOTE]
> **Note:** Subclasses cannot override this directly. Subclasses implement _reset() which will be called by this method. The output of _reset() will be cached and made available through current_time_step().

<br />

<br />

<br />

| Returns ||
|---|---|
| A `TimeStep` namedtuple containing: step_type: A `StepType` of `FIRST`. reward: 0.0, indicating the reward. discount: 1.0, indicating the discount. observation: A NumPy array, or a nested dict, list or tuple of arrays corresponding to `observation_spec()`. ||

<br />

### `reward_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/bandits/environments/bandit_py_environment.py#L94-L95)

    reward_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the rewards that are returned by `step()`.

Override this method to define an environment that uses non-standard reward
values, for example an environment with array-valued rewards.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `seed`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L288-L295)

    seed(
        seed: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/Seed
    ) -> Any

Seeds the environment.

<br />

<br />

<br />

| Args ||
|---|---|
| `seed` | Value to use as seed for the environment. |

<br />

### `set_state`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L332-L342)

    set_state(
        state: Any
    ) -> None

Restores the environment to a given `state`.

See definition of `state` in the documentation for get_state().

<br />

<br />

<br />

| Args ||
|---|---|
| `state` | A state to restore the environment to. |

<br />

### `should_reset`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L106-L118)

    should_reset(
        current_time_step: https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep
    ) -> bool

Whether the Environmet should reset given the current timestep.

By default it only resets when all time_steps are `LAST`.

<br />

<br />

<br />

| Args ||
|---|---|
| `current_time_step` | The current `TimeStep`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A bool indicating whether the Environment should reset or not. ||

<br />

### `step`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L203-L239)

    step(
        action: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArray
    ) -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Updates the environment according to the action and returns a `TimeStep`.

If the environment returned a `TimeStep` with [`StepType.LAST`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/StepType#LAST) at the
previous step the implementation of `_step` in the environment should call
`reset` to start a new sequence and ignore `action`.

This method will start a new sequence if called after the environment
has been constructed and `reset` has not been called. In this case
`action` will be ignored.

If `should_reset(current_time_step)` is True, then this method will `reset`
by itself. In this case `action` will be ignored.

> [!NOTE]
> **Note:** Subclasses cannot override this directly. Subclasses implement _step() which will be called by this method. The output of _step() will be cached and made available through current_time_step().

<br />

<br />

<br />

| Args ||
|---|---|
| `action` | A NumPy array, or a nested dict, list or tuple of arrays corresponding to `action_spec()`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A `TimeStep` namedtuple containing: step_type: A `StepType` value. reward: A NumPy array, reward value for this timestep. discount: A NumPy array, discount in the range \[0, 1\]. observation: A NumPy array, or a nested dict, list or tuple of arrays corresponding to `observation_spec()`. ||

<br />

### `time_step_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L166-L177)

    time_step_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Describes the `TimeStep` fields returned by `step()`.

Override this method to define an environment that uses non-standard values
for any of the items returned by `step()`. For example, an environment with
array-valued rewards.

<br />

<br />

<br />

| Returns ||
|---|---|
| A `TimeStep` namedtuple containing (possibly nested) `ArraySpec`s defining the step_type, reward, discount, and observation structure. ||

<br />

### `__enter__`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L263-L265)

    __enter__()

Allows the environment to be used in a with-statement context.

### `__exit__`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L267-L269)

    __exit__(
        unused_exception_type, unused_exc_value, unused_traceback
    )

Allows the environment to be used in a with-statement context.