# tf_agents.environments.ParallelPyEnvironment

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/parallel_py_environment.py#L46-L227) |

Batch together environments and simulate them in external processes.

Inherits From: [`PyEnvironment`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/environments/PyEnvironment)

#### View aliases


**Main aliases**

[`tf_agents.environments.parallel_py_environment.ParallelPyEnvironment`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/environments/ParallelPyEnvironment)

<br />

    tf_agents.environments.ParallelPyEnvironment(
        env_constructors: Sequence[https://www.tensorflow.org/agents/api_docs/python/tf_agents/environments/parallel_py_environment/EnvConstructor],
        start_serially: bool = True,
        blocking: bool = False,
        flatten: bool = False
    )

The environments are created in external processes by calling the provided
callables. This can be an environment class, or a function creating the
environment and potentially wrapping it. The returned environment should not
access global variables.

<br />

<br />

<br />

| ## Args ||
|---|---|
| `env_constructors` | List of callables that create environments. |
| `start_serially` | Whether to start environments serially or in parallel. |
| `blocking` | Whether to step environments one after another. |
| `flatten` | Boolean, whether to use flatten action and time_steps during communication to reduce overhead. |

<br />

<br />

<br />

<br />

| ## Raises ||
|---|---|
| `ValueError` | If the action or observation specs don't match. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `batch_size` | The batch size of the environment. |
| `batched` | Whether the environment is batched or not. <br /> If the environment supports batched observations and actions, then overwrite this property to True. A batched environment takes in a batched set of actions and returns a batched set of observations. This means for all numpy arrays in the input and output nested structures, the first dimension is the batch size. When batched, the left-most dimension is not part of the action_spec or the observation_spec and corresponds to the batch dimension. When batched and handle_auto_reset, it checks `np.all(steps.is_last())`. |
| `envs` | <br /> <br /> |

<br />

## Methods

### `action_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/parallel_py_environment.py#L127-L128)

    action_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the actions that should be provided to `step()`.

May use a subclass of `ArraySpec` that specifies additional properties such
as min and max bounds on the values.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `close`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/parallel_py_environment.py#L165-L170)

    close() -> None

Close all external process.

### `current_time_step`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L179-L183)

    current_time_step() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Returns the current timestep.

### `discount_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L153-L164)

    discount_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the discount that are returned by `step()`.

Override this method to define an environment that uses non-standard
discount values, for example an environment with array-valued discounts.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `get_info`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L297-L306)

    get_info() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArray

Returns the environment info returned on the last step.

<br />

<br />

<br />

| Returns ||
|---|---|
| Info returned by last call to step(). None by default. ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `NotImplementedError` | If the environment does not use info. |

<br />

### `get_state`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L308-L330)

    get_state() -> Any

Returns the `state` of the environment.

The `state` contains everything required to restore the environment to the
current configuration. This can contain e.g.

- The current time_step.
- The number of steps taken in the environment (for finite horizon MDPs).
- Hidden state (for POMDPs).

Callers should not assume anything about the contents or format of the
returned `state`. It should be treated as a token that can be passed back to
`set_state()` later.

Note that the returned `state` handle should not be modified by the
environment later on, and ensuring this (e.g. using copy.deepcopy) is the
responsibility of the environment.

<br />

<br />

<br />

| Returns ||
|---|---|
| `state` | The current state of the environment. |

<br />

### `observation_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/parallel_py_environment.py#L124-L125)

    observation_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the observations provided by the environment.

May use a subclass of `ArraySpec` that specifies additional properties such
as min and max bounds on the values.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `render`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/parallel_py_environment.py#L206-L227)

    render(
        mode: Text = 'rgb_array'
    ) -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArray

Renders the environment.

<br />

<br />

<br />

| Args ||
|---|---|
| `mode` | Rendering mode. Currently only 'rgb_array' is supported because this is a batched environment. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| An ndarray of shape \[batch_size, width, height, 3\] denoting RGB images (for mode=`rgb_array`). ||

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `NotImplementedError` | If the environment does not support rendering, or any other mode than `rgb_array` is given. |

<br />

### `reset`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L185-L201)

    reset() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Starts a new sequence and returns the first `TimeStep` of this sequence.

> [!NOTE]
> **Note:** Subclasses cannot override this directly. Subclasses implement _reset() which will be called by this method. The output of _reset() will be cached and made available through current_time_step().

<br />

<br />

<br />

| Returns ||
|---|---|
| A `TimeStep` namedtuple containing: step_type: A `StepType` of `FIRST`. reward: 0.0, indicating the reward. discount: 1.0, indicating the discount. observation: A NumPy array, or a nested dict, list or tuple of arrays corresponding to `observation_spec()`. ||

<br />

### `reward_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L142-L151)

    reward_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArraySpec

Defines the rewards that are returned by `step()`.

Override this method to define an environment that uses non-standard reward
values, for example an environment with array-valued rewards.

<br />

<br />

<br />

| Returns ||
|---|---|
| An `ArraySpec`, or a nested dict, list or tuple of `ArraySpec`s. ||

<br />

### `seed`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/parallel_py_environment.py#L195-L204)

    seed(
        seeds: Sequence[https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/Seed]
    ) -> Sequence[Any]

Seeds the parallel environments.

### `set_state`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L332-L342)

    set_state(
        state: Any
    ) -> None

Restores the environment to a given `state`.

See definition of `state` in the documentation for get_state().

<br />

<br />

<br />

| Args ||
|---|---|
| `state` | A state to restore the environment to. |

<br />

### `should_reset`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L106-L118)

    should_reset(
        current_time_step: https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep
    ) -> bool

Whether the Environmet should reset given the current timestep.

By default it only resets when all time_steps are `LAST`.

<br />

<br />

<br />

| Args ||
|---|---|
| `current_time_step` | The current `TimeStep`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A bool indicating whether the Environment should reset or not. ||

<br />

### `start`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/parallel_py_environment.py#L102-L110)

    start() -> None

### `step`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L203-L239)

    step(
        action: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types/NestedArray
    ) -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Updates the environment according to the action and returns a `TimeStep`.

If the environment returned a `TimeStep` with [`StepType.LAST`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/StepType#LAST) at the
previous step the implementation of `_step` in the environment should call
`reset` to start a new sequence and ignore `action`.

This method will start a new sequence if called after the environment
has been constructed and `reset` has not been called. In this case
`action` will be ignored.

If `should_reset(current_time_step)` is True, then this method will `reset`
by itself. In this case `action` will be ignored.

> [!NOTE]
> **Note:** Subclasses cannot override this directly. Subclasses implement _step() which will be called by this method. The output of _step() will be cached and made available through current_time_step().

<br />

<br />

<br />

| Args ||
|---|---|
| `action` | A NumPy array, or a nested dict, list or tuple of arrays corresponding to `action_spec()`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A `TimeStep` namedtuple containing: step_type: A `StepType` value. reward: A NumPy array, reward value for this timestep. discount: A NumPy array, discount in the range \[0, 1\]. observation: A NumPy array, or a nested dict, list or tuple of arrays corresponding to `observation_spec()`. ||

<br />

### `time_step_spec`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/parallel_py_environment.py#L130-L131)

    time_step_spec() -> https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/TimeStep

Describes the `TimeStep` fields returned by `step()`.

Override this method to define an environment that uses non-standard values
for any of the items returned by `step()`. For example, an environment with
array-valued rewards.

<br />

<br />

<br />

| Returns ||
|---|---|
| A `TimeStep` namedtuple containing (possibly nested) `ArraySpec`s defining the step_type, reward, discount, and observation structure. ||

<br />

### `__enter__`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L263-L265)

    __enter__()

Allows the environment to be used in a with-statement context.

### `__exit__`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/environments/py_environment.py#L267-L269)

    __exit__(
        unused_exception_type, unused_exc_value, unused_traceback
    )

Allows the environment to be used in a with-statement context.