![]() |
High-level configurations for Runtime.
Inherits From: Config
, ParamsDict
tfm.core.base_task.RuntimeConfig(
default_params: dataclasses.InitVar[Optional[Mapping[str, Any]]] = None,
restrictions: dataclasses.InitVar[Optional[List[str]]] = None,
distribution_strategy: str = 'mirrored',
enable_xla: bool = False,
gpu_thread_mode: Optional[str] = None,
dataset_num_private_threads: Optional[int] = None,
per_gpu_thread_count: int = 0,
tpu: Optional[str] = None,
num_gpus: int = 0,
worker_hosts: Optional[str] = None,
task_index: int = -1,
all_reduce_alg: Optional[str] = None,
num_packs: int = 1,
mixed_precision_dtype: Optional[str] = None,
loss_scale: Optional[Union[str, float]] = None,
run_eagerly: bool = False,
batchnorm_spatial_persistent: bool = False,
tpu_enable_xla_dynamic_padder: Optional[bool] = None,
num_cores_per_replica: int = 1,
default_shard_dim: int = -1
)
These include parameters that are not directly related to the experiment, e.g. directories, accelerator type, etc.
Attributes | |
---|---|
distribution_strategy
|
e.g. 'mirrored', 'tpu', etc. |
enable_xla
|
Whether or not to enable XLA. |
per_gpu_thread_count
|
thread count per GPU. |
gpu_thread_mode
|
Whether and how the GPU device uses its own threadpool. |
dataset_num_private_threads
|
Number of threads for a private threadpool created for all datasets computation. |
tpu
|
The address of the TPU to use, if any. |
num_gpus
|
The number of GPUs to use, if any. |
worker_hosts
|
comma-separated list of worker ip:port pairs for running multi-worker models with DistributionStrategy. |
task_index
|
If multi-worker training, the task index of this worker. |
all_reduce_alg
|
Defines the algorithm for performing all-reduce. |
num_packs
|
Sets num_packs in the cross device ops used in
MirroredStrategy. For details, see tf.distribute.NcclAllReduce.
|
mixed_precision_dtype
|
dtype of mixed precision policy. It can be 'float32', 'float16', or 'bfloat16'. |
loss_scale
|
The type of loss scale, or 'float' value. This is used when setting the mixed precision policy. |
run_eagerly
|
Whether or not to run the experiment eagerly. |
batchnorm_spatial_persistent
|
Whether or not to enable the spatial persistent mode for CuDNN batch norm kernel for improved GPU performance. |
BUILDER
|
|
default_params
|
Dataclass field |
restrictions
|
Dataclass field |
tpu_enable_xla_dynamic_padder
|
Dataclass field |
num_cores_per_replica
|
Dataclass field |
default_shard_dim
|
Dataclass field |
Methods
as_dict
as_dict()
Returns a dict representation of params_dict.ParamsDict.
For the nested params_dict.ParamsDict, a nested dict will be returned.
from_args
@classmethod
from_args( *args, **kwargs )
Builds a config from the given list of arguments.
from_json
@classmethod
from_json( file_path: str )
Wrapper for from_yaml
.
from_yaml
@classmethod
from_yaml( file_path: str )
get
get(
key, value=None
)
Accesses through built-in dictionary get method.
lock
lock()
Makes the ParamsDict immutable.
model_parallelism
model_parallelism()
override
override(
override_params, is_strict=True
)
Override the ParamsDict with a set of given params.
Args | |
---|---|
override_params
|
a dict or a ParamsDict specifying the parameters to be overridden. |
is_strict
|
a boolean specifying whether override is strict or not. If
True, keys in override_params must be present in the ParamsDict. If
False, keys in override_params can be different from what is currently
defined in the ParamsDict. In this case, the ParamsDict will be extended
to include the new keys.
|
replace
replace(
**kwargs
)
Overrides/returns a unlocked copy with the current config unchanged.
validate
validate()
Validate the parameters consistency based on the restrictions.
This method validates the internal consistency using the pre-defined list of restrictions. A restriction is defined as a string which specfiies a binary operation. The supported binary operations are {'==', '!=', '<', '<=', '>', '>='}. Note that the meaning of these operators are consistent with the underlying Python immplementation. Users should make sure the define restrictions on their type make sense.
For example, for a ParamsDict like the following
a:
a1: 1
a2: 2
b:
bb:
bb1: 10
bb2: 20
ccc:
a1: 1
a3: 3
one can define two restrictions like this ['a.a1 == b.ccc.a1', 'a.a2 <= b.bb.bb2']
What it enforces are:
- a.a1 = 1 == b.ccc.a1 = 1
- a.a2 = 2 <= b.bb.bb2 = 20
Raises | |
---|---|
KeyError
|
if any of the following happens (1) any of parameters in any of restrictions is not defined in ParamsDict, (2) any inconsistency violating the restriction is found. |
ValueError
|
if the restriction defined in the string is not supported. |
__contains__
__contains__(
key
)
Implements the membership test operator.
__eq__
__eq__(
other
)