Creates a dataset that shards the input dataset.
Creates a dataset that shards the input dataset by num_workers, returning a sharded dataset for the index-th worker. This attempts to automatically shard a dataset by examining the Dataset graph and inserting a shard op before the inputs to a reader Dataset (e.g. CSVDataset, TFRecordDataset).
This dataset will throw a NotFound error if we cannot shard the dataset automatically.
Nested Classes
| class | AutoShardDataset.Options |
Optional attributes for
AutoShardDataset
|
|
Public Methods
| Output <Object> |
asOutput
()
Returns the symbolic handle of a tensor.
|
| static AutoShardDataset.Options |
autoShardPolicy
(Long autoShardPolicy)
|
| static AutoShardDataset | |
| Output <?> |
handle
()
|
| static AutoShardDataset.Options |
numReplicas
(Long numReplicas)
|
Inherited Methods
Public Methods
public Output <Object> asOutput ()
Returns the symbolic handle of a tensor.
Inputs to TensorFlow operations are outputs of another TensorFlow operation. This method is used to obtain a symbolic handle that represents the computation of the input.
public static AutoShardDataset create ( Scope scope, Operand <?> inputDataset, Operand <Long> numWorkers, Operand <Long> index, List<Class<?>> outputTypes, List< Shape > outputShapes, Options... options)
Factory method to create a class wrapping a new AutoShardDataset operation.
Parameters
| scope | current scope |
|---|---|
| inputDataset | A variant tensor representing the input dataset. |
| numWorkers | A scalar representing the number of workers to distribute this dataset across. |
| index | A scalar representing the index of the current worker out of num_workers. |
| options | carries optional attributes values |
Returns
- a new instance of AutoShardDataset