tf_agents.bandits.environments.bandit_tf_environment.BanditTFEnvironment

Base class for bandit environments implemented in TensorFlow.

Inherits From: TFEnvironment

Subclasses should implement the _apply_action and _observe methods.

Example usage with eager mode:

  # reset() creates the initial time_step and resets the environment.
  time_step = environment.reset()
  for _ in tf.range(num_steps):
    action_step = policy.action(time_step)
    time_step = environment.step(action_step.action)

Example usage with graph mode:

  # current_time_step() creates the initial TimeStep.
  time_step = environment.current_time_step()
  action_step = policy.action(time_step)
  # Apply the action and return the new TimeStep.
  next_time_step = environment.step(action_step.action)

  sess.run([time_step, action_step, next_time_step])

time_step_spec A TimeStep namedtuple containing TensorSpecs defining the tensors returned by step() (step_type, reward, discount, and observation).
action_spec A nest of BoundedTensorSpec representing the actions of the environment.
batch_size The batch size expected for the actions and observations.
name The name of this environment instance.

batch_size

batched

name

Methods

action_spec

View source

Describes the specs of the Tensors expected by step(action).

action can be a single Tensor, or a nested dict, list or tuple of Tensors.

Returns
An single TensorSpec, or a nested dict, list or tuple of TensorSpec objects, which describe the shape and dtype of each Tensor expected by step().

current_time_step

View source

Returns the current TimeStep.

Returns
A TimeStep namedtuple containing: step_type: A StepType value. reward: Reward at this time_step. discount: A discount in the range [0, 1]. observation: A Tensor, or a nested dict, list or tuple of Tensors corresponding to observation_spec().

observation_spec

View source

Defines the TensorSpec of observations provided by the environment.

Returns
A TensorSpec, or a nested dict, list or tuple of TensorSpec objects, which describe the observation.

render

View source

Renders a frame from the environment.

Raises
NotImplementedError If the environment does not support rendering.

reset

View source

Resets the environment and returns the current time_step.

Returns
A TimeStep namedtuple containing: step_type: A StepType value. reward: Reward at this time_step. discount: A discount in the range [0, 1]. observation: A Tensor, or a nested dict, list or tuple of Tensors corresponding to observation_spec().

reward_spec

View source

Defines the TensorSpec of rewards provided by the environment.

Returns
A TensorSpec, or a nested dict, list or tuple of TensorSpec objects, which describe the reward.

step

View source

Steps the environment according to the action.

If the environment returned a TimeStep with StepType.LAST at the previous step, this call to step should reset the environment (note that it is expected that whoever defines this method, calls reset in this case), start a new sequence and action will be ignored.

This method will also start a new sequence if called after the environment has been constructed and reset() has not been called. In this case action will be ignored.

Expected sequences look like:

time_step -> action -> next_time_step

The action should depend on the previous time_step for correctness.

Args
action A Tensor, or a nested dict, list or tuple of Tensors corresponding to action_spec().

Returns
A TimeStep namedtuple containing: step_type: A StepType value. reward: Reward at this time_step. discount: A discount in the range [0, 1]. observation: A Tensor, or a nested dict, list or tuple of Tensors corresponding to observation_spec().

time_step_spec

View source

Describes the TimeStep specs of Tensors returned by step().

Returns
A TimeStep namedtuple containing TensorSpec objects defining the Tensors returned by step(), i.e. (step_type, reward, discount, observation).