# tf_agents.environments.PyEnvironment

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L37-L362) |

Abstract base class for Python RL environments.

#### View aliases


**Main aliases**

[`tf_agents.environments.py_environment.PyEnvironment`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/environments/PyEnvironment)

<br />

    tf_agents.environments.PyEnvironment(
        handle_auto_reset: bool = False
    )

### Used in the notebooks

| Used in the tutorials |
|---|
| - [Environments](https://www.tensorflow.org/agents/tutorials/2_environments_tutorial) - [Tutorial on Multi Armed Bandits in TF-Agents](https://www.tensorflow.org/agents/tutorials/bandits_tutorial) |

Observations and valid actions are described with `ArraySpec`s, defined in
the `specs` module.

If the environment can run multiple steps at the same time and take a batched
set of actions and return a batched set of observations, it should overwrite
the property batched to True.

Environments are assumed to auto reset once they reach the end of the episode,
if prefer that the base class handle auto_reset set `handle_auto_reset=True`.

<br />

<br />

<br />

| ## Args ||
|---|---|
| `handle_auto_reset` | When `True` the base class will handle auto_reset of the Environment. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `batch_size` | The batch size of the environment. |
| `batched` | Whether the environment is batched or not. <br /> If the environment supports batched observations and actions, then overwrite this property to True. A batched environment takes in a batched set of actions and returns a batched set of observations. This means for all numpy arrays in the input and output nested structures, the first dimension is the batch size. When batched, the left-most dimension is not part of the action_spec or the observation_spec and corresponds to the batch dimension. When batched and handle_auto_reset, it checks `np.all(steps.is_last())`. |

<br />

## Methods

### `action_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L131-L140)

    @abc.abstractmethod
    action_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the actions that should be provided to `step()`.

May use a subclass of `ArraySpec` that specifies additional properties such
as min and max bounds on the values.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `close`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L241-L261)

    close() -> None

Frees any resources used by the environment.

Implement this method for an environment backed by an external process.

This method be used directly

    env = Env(...)
    # Use env.
    env.close()

or via a context manager

    with Env(...) as env:
      # Use env.

### `current_time_step`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L179-L183)

    current_time_step() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Returns the current timestep.

### `discount_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L153-L164)

    discount_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the discount that are returned by `step()`.

Override this method to define an environment that uses non-standard
discount values, for example an environment with array-valued discounts.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `get_info`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L297-L306)

    get_info() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArray

Returns the environment info returned on the last step.

<br />

<br />

<br />

| Returns ||
|---|---|
| Info returned by last call to step(). None by default. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `NotImplementedError` | If the environment does not use info. |

<br />

### `get_state`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L308-L330)

    get_state() -> Any

Returns the `state` of the environment.

The `state` contains everything required to restore the environment to the
current configuration. This can contain e.g.

- The current time_step.
- The number of steps taken in the environment (for finite horizon MDPs).
- Hidden state (for POMDPs).

Callers should not assume anything about the contents or format of the
returned `state`. It should be treated as a token that can be passed back to
`set_state()` later.

Note that the returned `state` handle should not be modified by the
environment later on, and ensuring this (e.g. using copy.deepcopy) is the
responsibility of the environment.

<br />

<br />

<br />

| Returns ||
|---|---|
| `state` | The current state of the environment. |

<br />

### `observation_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L120-L129)

    @abc.abstractmethod
    observation_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the observations provided by the environment.

May use a subclass of `ArraySpec` that specifies additional properties such
as min and max bounds on the values.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `render`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L271-L286)

    render(
        mode: Text = 'rgb_array'
    ) -> Optional[types.NestedArray]

Renders the environment.

<br />

<br />

<br />

| Args ||
|---|---|
| `mode` | One of \['rgb_array', 'human'\]. Renders to an numpy array, or brings up a window where the environment can be visualized. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| An ndarray of shape \[width, height, 3\] denoting an RGB image if mode is `rgb_array`. Otherwise return nothing and render directly to a display window. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `NotImplementedError` | If the environment does not support rendering. |

<br />

### `reset`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L185-L201)

    reset() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Starts a new sequence and returns the first `TimeStep` of this sequence.

> [!NOTE]
> **Note:** Subclasses cannot override this directly. Subclasses implement _reset() which will be called by this method. The output of _reset() will be cached and made available through current_time_step().

<br />

<br />

<br />

| Returns ||
|---|---|
| A `TimeStep` namedtuple containing: step_type: A `StepType` of `FIRST`. reward: 0.0, indicating the reward. discount: 1.0, indicating the discount. observation: A NumPy array, or a nested dict, list or tuple of arrays corresponding to `observation_spec()`. ||

<br />

### `reward_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L142-L151)

    reward_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the rewards that are returned by `step()`.

Override this method to define an environment that uses non-standard reward
values, for example an environment with array-valued rewards.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `seed`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L288-L295)

    seed(
        seed: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/Seed
    ) -> Any

Seeds the environment.

<br />

<br />

<br />

| Args ||
|---|---|
| `seed` | Value to use as seed for the environment. |

<br />

### `set_state`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L332-L342)

    set_state(
        state: Any
    ) -> None

Restores the environment to a given `state`.

See definition of `state` in the documentation for get_state().

<br />

<br />

<br />

| Args ||
|---|---|
| `state` | A state to restore the environment to. |

<br />

### `should_reset`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L106-L118)

    should_reset(
        current_time_step: https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep
    ) -> bool

Whether the Environmet should reset given the current timestep.

By default it only resets when all time_steps are `LAST`.

<br />

<br />

<br />

| Args ||
|---|---|
| `current_time_step` | The current `TimeStep`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A bool indicating whether the Environment should reset or not. ||

<br />

### `step`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L203-L239)

    step(
        action: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArray
    ) -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Updates the environment according to the action and returns a `TimeStep`.

If the environment returned a `TimeStep` with [`StepType.LAST`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/StepType#LAST) at the
previous step the implementation of `_step` in the environment should call
`reset` to start a new sequence and ignore `action`.

This method will start a new sequence if called after the environment
has been constructed and `reset` has not been called. In this case
`action` will be ignored.

If `should_reset(current_time_step)` is True, then this method will `reset`
by itself. In this case `action` will be ignored.

> [!NOTE]
> **Note:** Subclasses cannot override this directly. Subclasses implement _step() which will be called by this method. The output of _step() will be cached and made available through current_time_step().

<br />

<br />

<br />

| Args ||
|---|---|
| `action` | A NumPy array, or a nested dict, list or tuple of arrays corresponding to `action_spec()`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A `TimeStep` namedtuple containing: step_type: A `StepType` value. reward: A NumPy array, reward value for this timestep. discount: A NumPy array, discount in the range \[0, 1\]. observation: A NumPy array, or a nested dict, list or tuple of arrays corresponding to `observation_spec()`. ||

<br />

### `time_step_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L166-L177)

    time_step_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Describes the `TimeStep` fields returned by `step()`.

Override this method to define an environment that uses non-standard values
for any of the items returned by `step()`. For example, an environment with
array-valued rewards.

<br />

<br />

<br />

| Returns ||
|---|---|
| A `TimeStep` namedtuple containing (possibly nested) `ArraySpec`s defining the step_type, reward, discount, and observation structure. ||

<br />

### `__enter__`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L263-L265)

    __enter__()

Allows the environment to be used in a with-statement context.

### `__exit__`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L267-L269)

    __exit__(
        unused_exception_type, unused_exc_value, unused_traceback
    )

Allows the environment to be used in a with-statement context.