# tf_agents.replay_buffers.ReverbAddTrajectoryObserver

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/replay_buffers/reverb_utils.py#L270-L506) |

Stateful observer for writing fixed length trajectories to Reverb.

#### View aliases


**Main aliases**

[`tf_agents.replay_buffers.reverb_utils.ReverbAddTrajectoryObserver`](https://www.tensorflow.org/agents/api_docs/python/tf_agents/replay_buffers/ReverbAddTrajectoryObserver)

<br />

    tf_agents.replay_buffers.ReverbAddTrajectoryObserver(
        py_client: https://www.tensorflow.org/agents/api_docs/python/tf_agents/typing/types#ReverbClient,
        table_name: Union[Text, Sequence[Text]],
        sequence_length: int,
        stride_length: int = 1,
        priority: Union[float, int] = 1,
        pad_end_of_episodes: bool = False,
        tile_end_of_episodes: bool = False
    )

### Used in the notebooks

| Used in the tutorials |
|---|
| - [Train a Deep Q Network with TF-Agents](https://www.tensorflow.org/agents/tutorials/1_dqn_tutorial) - [SAC minitaur with the Actor-Learner API](https://www.tensorflow.org/agents/tutorials/7_SAC_minitaur_tutorial) |

This observer should be called at every environment step. It does not support
batched trajectories.

Steps are cached until `sequence_length` steps are gathered. At which point an
item is created. From there on a new item is created every `stride_length`
observer calls.

If an episode terminates before enough steps are cached, the data is discarded
unless `pad_end_of_episodes` is set.

<br />

<br />

<br />

| ## Args ||
|---|---|
| `py_client` | Python client for the reverb replay server. |
| `table_name` | The table name(s) where samples will be written to. |
| `sequence_length` | The sequence_length used to write to the given table. |
| `stride_length` | The integer stride for the sliding window for overlapping sequences. The default value of `1` creates an item for every window. Using `L = sequence_length` this means items are created for times `{0, 1, .., L-1}, {1, 2, .., L}, ...`. In contrast, `stride_length = L` will create an item only for disjoint windows `{0, 1, ..., L-1}, {L, ..., 2 * L - 1}, ...`. |
| `priority` | Initial priority for new samples in the RB. |
| `pad_end_of_episodes` | At the end of an episode, the cache is dropped by default. When `pad_end_of_episodes = True`, the cache gets padded with boundary steps (last-\>first) with `0` values everywhere and padded items of `sequence_length` are written to Reverb. |
| `tile_end_of_episodes` | If `pad_end_of_episodes` is True then, the last padded item starts with a boundary step from the episode. When this option is True the following items will be generated: F, M, L, P M, L, P, P L, P, P, P If False, only a single one will be generated: F, M, L, P For training recurrent models on environments where required information is only available at the start of the episode it is useful to set `tile_end_of_episodes=False` and the `sequence_length` to be the length of the longest episode. |

<br />

<br />

<br />

<br />

| ## Raises ||
|---|---|
| `ValueError` | If `tile_end_of_episodes` is set without `pad_end_of_episodes`. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `py_client` | <br /> <br /> |

<br />

## Methods

### `close`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/replay_buffers/reverb_utils.py#L497-L506)

    close() -> None

Closes the writer of the observer.

> [!NOTE]
> **Note:** Using the observer after it is closed (and not reopened) is not supported.

### `flush`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/replay_buffers/reverb_utils.py#L419-L427)

    flush()

Ensures that items are pushed to the service.

> [!NOTE]
> **Note:** The items are not always immediately pushed. This method is often needed when `rate_limiter_timeout_ms` is set for the replay buffer. By calling this method before the `learner.run()`, we ensure that there is enough data to be consumed.

### `get_table_signature`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/replay_buffers/reverb_utils.py#L366-L369)

    get_table_signature()

### `open`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/replay_buffers/reverb_utils.py#L489-L495)

    open() -> None

Open the writer of the observer.

### `reset`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/replay_buffers/reverb_utils.py#L442-L487)

    reset(
        write_cached_steps: bool = True
    ) -> None

Resets the state of the observer.

> [!NOTE]
> **Note:** Reset should be called only after all collection has finished in a standard workflow. No need to manually call reset between episodes.

<br />

<br />

<br />

| Args ||
|---|---|
| `write_cached_steps` | boolean flag indicating whether we want to write the cached trajectory. When this argument is True, the function attempts to write the cached data before resetting (optionally with padding). Otherwise, the cached data gets dropped. |

<br />

### `__call__`

[View source](https://github.com/tensorflow/agents/blob/v0.19.0/tf_agents/replay_buffers/reverb_utils.py#L371-L394)

    __call__(
        trajectory: https://www.tensorflow.org/agents/api_docs/python/tf_agents/trajectories/Trajectory
    ) -> None

Writes the trajectory into the underlying replay buffer.

Allows trajectory to be a flattened trajectory. No batch dimension allowed.

<br />

<br />

<br />

| Args ||
|---|---|
| `trajectory` | The trajectory to be written which could be (possibly nested) trajectory object or a flattened version of a trajectory. It assumes there is *no* batch dimension. |

<br />