# tf_agents.bandits.environments.movielens_per_arm_py_environment.MovieLensPerArmPyEnvironment

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/bandits/environments/movielens_per_arm_py_environment.py#L35-L185) |

Implements the per-arm version of the MovieLens Bandit environment.

Inherits From: [`BanditPyEnvironment`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/bandits/environments/bandit_py_environment/BanditPyEnvironment), [`PyEnvironment`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/environments/PyEnvironment)

    tf_agents.bandits.environments.movielens_per_arm_py_environment.MovieLensPerArmPyEnvironment(
        data_dir: Text,
        rank_k: int,
        batch_size: int = 1,
        num_actions: int = 50,
        csv_delimiter=',',
        name: Optional[Text] = 'movielens_per_arm'
    )

This environment implements the MovieLens 100K dataset, available at:
<https://www.kaggle.com/prajitdatta/movielens-100k-dataset>

This dataset contains 100K ratings from 943 users on 1682 items.
This csv list of:
user id \| item id \| rating \| timestamp.
This environment computes a low-rank matrix factorization (using SVD) of the
data matrix `A`, such that: `A ~= U * Sigma * V^T`.

The environment uses the rows of `U` as global (or user) features, and the
rows of `V` as per-arm (or movie) features.

The reward of recommending movie `v` to user `u` is `u * Sigma * v^T`.

<br />

<br />

<br />

| ## Args ||
|---|---|
| `data_dir` | (string) Directory where the data lies (in text form). |
| `rank_k` | (int) Which rank to use in the matrix factorization. This will also be the feature dimension of both the user and the movie features. |
| `batch_size` | (int) Number of observations generated per call. |
| `num_actions` | (int) How many movies to choose from per round. |
| `csv_delimiter` | (string) The delimiter to use in loading the data csv file. |
| `name` | (string) The name of this environment instance. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `batch_size` | The batch size of the environment. |
| `batched` | Whether the environment is batched or not. <br /> If the environment supports batched observations and actions, then overwrite this property to True. A batched environment takes in a batched set of actions and returns a batched set of observations. This means for all numpy arrays in the input and output nested structures, the first dimension is the batch size. When batched, the left-most dimension is not part of the action_spec or the observation_spec and corresponds to the batch dimension. When batched and handle_auto_reset, it checks `np.all(steps.is_last())`. |
| `name` | <br /> <br /> |

<br />

## Methods

### `action_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/bandits/environments/bandit_py_environment.py#L88-L89)

    action_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the actions that should be provided to `step()`.

May use a subclass of `ArraySpec` that specifies additional properties such
as min and max bounds on the values.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `close`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L241-L261)

    close() -> None

Frees any resources used by the environment.

Implement this method for an environment backed by an external process.

This method be used directly

    env = Env(...)
    # Use env.
    env.close()

or via a context manager

    with Env(...) as env:
      # Use env.

### `compute_optimal_action`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/bandits/environments/movielens_per_arm_py_environment.py#L181-L182)

    compute_optimal_action()

### `compute_optimal_reward`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/bandits/environments/movielens_per_arm_py_environment.py#L184-L185)

    compute_optimal_reward()

### `current_time_step`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L179-L183)

    current_time_step() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Returns the current timestep.

### `discount_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L153-L164)

    discount_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the discount that are returned by `step()`.

Override this method to define an environment that uses non-standard
discount values, for example an environment with array-valued discounts.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `get_info`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L297-L306)

    get_info() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArray

Returns the environment info returned on the last step.

<br />

<br />

<br />

| Returns ||
|---|---|
| Info returned by last call to step(). None by default. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `NotImplementedError` | If the environment does not use info. |

<br />

### `get_state`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L308-L330)

    get_state() -> Any

Returns the `state` of the environment.

The `state` contains everything required to restore the environment to the
current configuration. This can contain e.g.

- The current time_step.
- The number of steps taken in the environment (for finite horizon MDPs).
- Hidden state (for POMDPs).

Callers should not assume anything about the contents or format of the
returned `state`. It should be treated as a token that can be passed back to
`set_state()` later.

Note that the returned `state` handle should not be modified by the
environment later on, and ensuring this (e.g. using copy.deepcopy) is the
responsibility of the environment.

<br />

<br />

<br />

| Returns ||
|---|---|
| `state` | The current state of the environment. |

<br />

### `observation_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/bandits/environments/bandit_py_environment.py#L91-L92)

    observation_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the observations provided by the environment.

May use a subclass of `ArraySpec` that specifies additional properties such
as min and max bounds on the values.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `render`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L271-L286)

    render(
        mode: Text = 'rgb_array'
    ) -> Optional[types.NestedArray]

Renders the environment.

<br />

<br />

<br />

| Args ||
|---|---|
| `mode` | One of \['rgb_array', 'human'\]. Renders to an numpy array, or brings up a window where the environment can be visualized. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| An ndarray of shape \[width, height, 3\] denoting an RGB image if mode is `rgb_array`. Otherwise return nothing and render directly to a display window. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `NotImplementedError` | If the environment does not support rendering. |

<br />

### `reset`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L185-L201)

    reset() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Starts a new sequence and returns the first `TimeStep` of this sequence.

> [!NOTE]
> **Note:** Subclasses cannot override this directly. Subclasses implement _reset() which will be called by this method. The output of _reset() will be cached and made available through current_time_step().

<br />

<br />

<br />

| Returns ||
|---|---|
| A `TimeStep` namedtuple containing: step_type: A `StepType` of `FIRST`. reward: 0.0, indicating the reward. discount: 1.0, indicating the discount. observation: A NumPy array, or a nested dict, list or tuple of arrays corresponding to `observation_spec()`. ||

<br />

### `reward_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/bandits/environments/bandit_py_environment.py#L94-L95)

    reward_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the rewards that are returned by `step()`.

Override this method to define an environment that uses non-standard reward
values, for example an environment with array-valued rewards.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `seed`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L288-L295)

    seed(
        seed: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/Seed
    ) -> Any

Seeds the environment.

<br />

<br />

<br />

| Args ||
|---|---|
| `seed` | Value to use as seed for the environment. |

<br />

### `set_state`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L332-L342)

    set_state(
        state: Any
    ) -> None

Restores the environment to a given `state`.

See definition of `state` in the documentation for get_state().

<br />

<br />

<br />

| Args ||
|---|---|
| `state` | A state to restore the environment to. |

<br />

### `should_reset`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L106-L118)

    should_reset(
        current_time_step: https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep
    ) -> bool

Whether the Environmet should reset given the current timestep.

By default it only resets when all time_steps are `LAST`.

<br />

<br />

<br />

| Args ||
|---|---|
| `current_time_step` | The current `TimeStep`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A bool indicating whether the Environment should reset or not. ||

<br />

### `step`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L203-L239)

    step(
        action: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArray
    ) -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Updates the environment according to the action and returns a `TimeStep`.

If the environment returned a `TimeStep` with [`StepType.LAST`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/StepType#LAST) at the
previous step the implementation of `_step` in the environment should call
`reset` to start a new sequence and ignore `action`.

This method will start a new sequence if called after the environment
has been constructed and `reset` has not been called. In this case
`action` will be ignored.

If `should_reset(current_time_step)` is True, then this method will `reset`
by itself. In this case `action` will be ignored.

> [!NOTE]
> **Note:** Subclasses cannot override this directly. Subclasses implement _step() which will be called by this method. The output of _step() will be cached and made available through current_time_step().

<br />

<br />

<br />

| Args ||
|---|---|
| `action` | A NumPy array, or a nested dict, list or tuple of arrays corresponding to `action_spec()`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A `TimeStep` namedtuple containing: step_type: A `StepType` value. reward: A NumPy array, reward value for this timestep. discount: A NumPy array, discount in the range \[0, 1\]. observation: A NumPy array, or a nested dict, list or tuple of arrays corresponding to `observation_spec()`. ||

<br />

### `time_step_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L166-L177)

    time_step_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Describes the `TimeStep` fields returned by `step()`.

Override this method to define an environment that uses non-standard values
for any of the items returned by `step()`. For example, an environment with
array-valued rewards.

<br />

<br />

<br />

| Returns ||
|---|---|
| A `TimeStep` namedtuple containing (possibly nested) `ArraySpec`s defining the step_type, reward, discount, and observation structure. ||

<br />

### `__enter__`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L263-L265)

    __enter__()

Allows the environment to be used in a with-statement context.

### `__exit__`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L267-L269)

    __exit__(
        unused_exception_type, unused_exc_value, unused_traceback
    )

Allows the environment to be used in a with-statement context.