![]() |
![]() |
See the Variables Guide.
Inherits From: Variable
tf.Variable(
initial_value=None, trainable=None, collections=None, validate_shape=True,
caching_device=None, name=None, variable_def=None, dtype=None,
expected_shape=None, import_scope=None, constraint=None, use_resource=None,
synchronization=tf.VariableSynchronization.AUTO,
aggregation=tf.VariableAggregation.NONE, shape=None
)
A variable maintains state in the graph across calls to run()
. You add a
variable to the graph by constructing an instance of the class Variable
.
The Variable()
constructor requires an initial value for the variable,
which can be a Tensor
of any type and shape. The initial value defines the
type and shape of the variable. After construction, the type and shape of
the variable are fixed. The value can be changed using one of the assign
methods.
If you want to change the shape of a variable later you have to use an
assign
Op with validate_shape=False
.
Just like any Tensor
, variables created with Variable()
can be used as
inputs for other Ops in the graph. Additionally, all the operators
overloaded for the Tensor
class are carried over to variables, so you can
also add nodes to the graph by just doing arithmetic on variables.
import tensorflow as tf
# Create a variable.
w = tf.Variable(<initial-value>, name=<optional-name>)
# Use the variable in the graph like any Tensor.
y = tf.matmul(w, ...another variable or tensor...)
# The overloaded operators are available too.
z = tf.sigmoid(w + y)
# Assign a new value to the variable with `assign()` or a related method.
w.assign(w + 1.0)
w.assign_add(1.0)
When you launch the graph, variables have to be explicitly initialized before
you can run Ops that use their value. You can initialize a variable by
running its initializer op, restoring the variable from a save file, or
simply running an assign
Op that assigns a value to the variable. In fact,
the variable initializer op is just an assign
Op that assigns the
variable's initial value to the variable itself.
# Launch the graph in a session.
with tf.compat.v1.Session() as sess:
# Run the variable initializer.
sess.run(w.initializer)
# ...you now can run ops that use the value of 'w'...
The most common initialization pattern is to use the convenience function
global_variables_initializer()
to add an Op to the graph that initializes
all the variables. You then run that Op after launching the graph.
# Add an Op to initialize global variables.
init_op = tf.compat.v1.global_variables_initializer()
# Launch the graph in a session.
with tf.compat.v1.Session() as sess:
# Run the Op that initializes global variables.
sess.run(init_op)
# ...you can now run any Op that uses variable values...
If you need to create a variable with an initial value dependent on another
variable, use the other variable's initialized_value()
. This ensures that
variables are initialized in the right order.
All variables are automatically collected in the graph where they are
created. By default, the constructor adds the new variable to the graph
collection GraphKeys.GLOBAL_VARIABLES
. The convenience function
global_variables()
returns the contents of that collection.
When building a machine learning model it is often convenient to distinguish
between variables holding the trainable model parameters and other variables
such as a global step
variable used to count training steps. To make this
easier, the variable constructor supports a trainable=<bool>
parameter. If
True
, the new variable is also added to the graph collection
GraphKeys.TRAINABLE_VARIABLES
. The convenience function
trainable_variables()
returns the contents of this collection. The
various Optimizer
classes use this collection as the default list of
variables to optimize.
v = tf.Variable(True)
tf.cond(v, lambda: v.assign(False), my_false_fn) # Note: this is broken.
Here, adding use_resource=True
when constructing the variable will
fix any nondeterminism issues:
v = tf.Variable(True, use_resource=True)
tf.cond(v, lambda: v.assign(False), my_false_fn)
To use the replacement for variables which does not have these issues:
- Add
use_resource=True
when constructingtf.Variable
; - Call
tf.compat.v1.get_variable_scope().set_use_resource(True)
inside atf.compat.v1.variable_scope
before thetf.compat.v1.get_variable()
call.
Args | |
---|---|
initial_value
|
A Tensor , or Python object convertible to a Tensor ,
which is the initial value for the Variable. The initial value must have
a shape specified unless validate_shape is set to False. Can also be a
callable with no argument that returns the initial value when called. In
that case, dtype must be specified. (Note that initializer functions
from init_ops.py must first be bound to a shape before being used here.)
|
trainable
|
If True , also adds the variable to the graph collection
GraphKeys.TRAINABLE_VARIABLES . This collection is used as the default
list of variables to use by the Optimizer classes. Defaults to True ,
unless synchronization is set to ON_READ , in which case it defaults
to False .
|
collections
|
List of graph collections keys. The new variable is added to
these collections. Defaults to [GraphKeys.GLOBAL_VARIABLES] .
|
validate_shape
|
If False , allows the variable to be initialized with a
value of unknown shape. If True , the default, the shape of
initial_value must be known.
|
caching_device
|
Optional device string describing where the Variable
should be cached for reading. Defaults to the Variable's device. If not
None , caches on another device. Typical use is to cache on the device
where the Ops using the Variable reside, to deduplicate copying through
Switch and other conditional statements.
|
name
|
Optional name for the variable. Defaults to 'Variable' and gets
uniquified automatically.
|
variable_def
|
VariableDef protocol buffer. If not None , recreates the
Variable object with its contents, referencing the variable's nodes in
the graph, which must already exist. The graph is not changed.
variable_def and the other arguments are mutually exclusive.
|
dtype
|
If set, initial_value will be converted to the given type. If
None , either the datatype will be kept (if initial_value is a
Tensor), or convert_to_tensor will decide.
|
expected_shape
|
A TensorShape. If set, initial_value is expected to have this shape. |
import_scope
|
Optional string . Name scope to add to the Variable. Only
used when initializing from protocol buffer.
|
constraint
|
An optional projection function to be applied to the variable
after being updated by an Optimizer (e.g. used to implement norm
constraints or value constraints for layer weights). The function must
take as input the unprojected Tensor representing the value of the
variable and return the Tensor for the projected value (which must have
the same shape). Constraints are not safe to use when doing asynchronous
distributed training.
|
use_resource
|
whether to use resource variables. |
synchronization
|
Indicates when a distributed a variable will be
aggregated. Accepted values are constants defined in the class
tf.VariableSynchronization . By default the synchronization is set to
AUTO and the current DistributionStrategy chooses when to
synchronize.
|
aggregation
|
Indicates how a distributed variable will be aggregated.
Accepted values are constants defined in the class
tf.VariableAggregation .
|
shape
|
(optional) The shape of this variable. If None, the shape of
initial_value will be used. When setting this argument to
tf.TensorShape(None) (representing an unspecified shape), the variable
can be assigned with values of different shapes.
|
Raises | |
---|---|
ValueError
|
If both variable_def and initial_value are specified.
|
ValueError
|
If the initial value is not specified, or does not have a
shape and validate_shape is True .
|
RuntimeError
|
If eager execution is enabled. |
Attributes | |
---|---|
aggregation
|
|
constraint
|
Returns the constraint function associated with this variable. |
device
|
The device of this variable. |
dtype
|
The DType of this variable.
|
graph
|
The Graph of this variable.
|
initial_value
|
Returns the Tensor used as the initial value for the variable.
Note that this is different from |
initializer
|
The initializer operation for this variable. |
name
|
The name of this variable. |
op
|
The Operation of this variable.
|
shape
|
The TensorShape of this variable.
|
synchronization
|
|
trainable
|
Child Classes
Methods
assign
assign(
value, use_locking=False, name=None, read_value=True
)
Assigns a new value to the variable.
This is essentially a shortcut for assign(self, value)
.
Args | |
---|---|
value
|
A Tensor . The new value for this variable.
|
use_locking
|
If True , use locking during the assignment.
|
name
|
The name of the operation to be created |
read_value
|
if True, will return something which evaluates to the new value of the variable; if False will return the assign op. |
Returns | |
---|---|
A Tensor that will hold the new value of this variable after
the assignment has completed.
|
assign_add
assign_add(
delta, use_locking=False, name=None, read_value=True
)
Adds a value to this variable.
This is essentially a shortcut for assign_add(self, delta)
.
Args | |
---|---|
delta
|
A Tensor . The value to add to this variable.
|
use_locking
|
If True , use locking during the operation.
|
name
|
The name of the operation to be created |
read_value
|
if True, will return something which evaluates to the new value of the variable; if False will return the assign op. |
Returns | |
---|---|
A Tensor that will hold the new value of this variable after
the addition has completed.
|
assign_sub
assign_sub(
delta, use_locking=False, name=None, read_value=True
)
Subtracts a value from this variable.
This is essentially a shortcut for assign_sub(self, delta)
.
Args | |
---|---|
delta
|
A Tensor . The value to subtract from this variable.
|
use_locking
|
If True , use locking during the operation.
|
name
|
The name of the operation to be created |
read_value
|
if True, will return something which evaluates to the new value of the variable; if False will return the assign op. |
Returns | |
---|---|
A Tensor that will hold the new value of this variable after
the subtraction has completed.
|
batch_scatter_update
batch_scatter_update(
sparse_delta, use_locking=False, name=None
)
Assigns tf.IndexedSlices
to this variable batch-wise.
Analogous to batch_gather
. This assumes that this variable and the
sparse_delta IndexedSlices have a series of leading dimensions that are the
same for all of them, and the updates are performed on the last dimension of
indices. In other words, the dimensions should be the following:
num_prefix_dims = sparse_delta.indices.ndims - 1
batch_dim = num_prefix_dims + 1
sparse_delta.updates.shape = sparse_delta.indices.shape + var.shape[
batch_dim:]
where
sparse_delta.updates.shape[:num_prefix_dims]
== sparse_delta.indices.shape[:num_prefix_dims]
== var.shape[:num_prefix_dims]
And the operation performed can be expressed as:
var[i_1, ..., i_n,
sparse_delta.indices[i_1, ..., i_n, j]] = sparse_delta.updates[
i_1, ..., i_n, j]
When sparse_delta.indices is a 1D tensor, this operation is equivalent to
scatter_update
.
To avoid this operation one can looping over the first ndims
of the
variable and using scatter_update
on the subtensors that result of slicing
the first dimension. This is a valid option for ndims = 1
, but less
efficient than this implementation.
Args | |
---|---|
sparse_delta
|
tf.IndexedSlices to be assigned to this variable.
|
use_locking
|
If True , use locking during the operation.
|
name
|
the name of the operation. |
Returns | |
---|---|
A Tensor that will hold the new value of this variable after
the scattered assignment has completed.
|
Raises | |
---|---|
TypeError
|
if sparse_delta is not an IndexedSlices .
|
count_up_to
count_up_to(
limit
)
Increments this variable until it reaches limit
. (deprecated)
When that Op is run it tries to increment the variable by 1
. If
incrementing the variable would bring it above limit
then the Op raises
the exception OutOfRangeError
.
If no error is raised, the Op outputs the value of the variable before the increment.
This is essentially a shortcut for count_up_to(self, limit)
.
Args | |
---|---|
limit
|
value at which incrementing the variable raises an error. |
Returns | |
---|---|
A Tensor that will hold the variable value before the increment. If no
other Op modifies this variable, the values produced will all be
distinct.
|
eval
eval(
session=None
)
In a session, computes and returns the value of this variable.
This is not a graph construction method, it does not add ops to the graph.
This convenience method requires a session where the graph
containing this variable has been launched. If no session is
passed, the default session is used. See tf.compat.v1.Session
for more
information on launching a graph and on sessions.
v = tf.Variable([1, 2])
init = tf.compat.v1.global_variables_initializer()
with tf.compat.v1.Session() as sess:
sess.run(init)
# Usage passing the session explicitly.
print(v.eval(sess))
# Usage with the default session. The 'with' block
# above makes 'sess' the default session.
print(v.eval())
Args | |
---|---|
session
|
The session to use to evaluate this variable. If none, the default session is used. |
Returns | |
---|---|
A numpy ndarray with a copy of the value of this variable.
|
experimental_ref
experimental_ref()
Returns a hashable reference object to this Variable.
The primary usecase for this API is to put variables in a set/dictionary.
We can't put variables in a set/dictionary as variable.__hash__()
is no
longer available starting Tensorflow 2.0.
import tensorflow as tf
x = tf.Variable(5)
y = tf.Variable(10)
z = tf.Variable(10)
# The followings will raise an exception starting 2.0
# TypeError: Variable is unhashable if Variable equality is enabled.
variable_set = {x, y, z}
variable_dict = {x: 'five', y: 'ten'}
Instead, we can use variable.experimental_ref()
.
variable_set = {x.experimental_ref(),
y.experimental_ref(),
z.experimental_ref()}
print(x.experimental_ref() in variable_set)
==> True
variable_dict = {x.experimental_ref(): 'five',
y.experimental_ref(): 'ten',
z.experimental_ref(): 'ten'}
print(variable_dict[y.experimental_ref()])
==> ten
Also, the reference object provides .deref()
function that returns the
original Variable.
x = tf.Variable(5)
print(x.experimental_ref().deref())
==> <tf.Variable 'Variable:0' shape=() dtype=int32, numpy=5>
from_proto
@staticmethod
from_proto( variable_def, import_scope=None )
Returns a Variable
object created from variable_def
.
gather_nd
gather_nd(
indices, name=None
)
Gather slices from params
into a Tensor with shape specified by indices
.
See tf.gather_nd for details.
Args | |
---|---|
indices
|
A Tensor . Must be one of the following types: int32 , int64 .
Index tensor.
|
name
|
A name for the operation (optional). |
Returns | |
---|---|
A Tensor . Has the same type as params .
|
get_shape
get_shape()
Alias of Variable.shape
.
initialized_value
initialized_value()
Returns the value of the initialized variable. (deprecated)
You should use this instead of the variable itself to initialize another variable with a value that depends on the value of this variable.
# Initialize 'v' with a random tensor.
v = tf.Variable(tf.random.truncated_normal([10, 40]))
# Use `initialized_value` to guarantee that `v` has been
# initialized before its value is used to initialize `w`.
# The random values are picked only once.
w = tf.Variable(v.initialized_value() * 2.0)
Returns | |
---|---|
A Tensor holding the value of this variable after its initializer
has run.
|
load
load(
value, session=None
)
Load new value into this variable. (deprecated)
Writes new value to variable's memory. Doesn't add ops to the graph.
This convenience method requires a session where the graph
containing this variable has been launched. If no session is
passed, the default session is used. See tf.compat.v1.Session
for more
information on launching a graph and on sessions.
v = tf.Variable([1, 2])
init = tf.compat.v1.global_variables_initializer()
with tf.compat.v1.Session() as sess:
sess.run(init)
# Usage passing the session explicitly.
v.load([2, 3], sess)
print(v.eval(sess)) # prints [2 3]
# Usage with the default session. The 'with' block
# above makes 'sess' the default session.
v.load([3, 4], sess)
print(v.eval()) # prints [3 4]
Args | |
---|---|
value
|
New variable value |
session
|
The session to use to evaluate this variable. If none, the default session is used. |
Raises | |
---|---|
ValueError
|
Session is not passed and no default session |
read_value
read_value()
Returns the value of this variable, read in the current context.
Can be different from value() if it's on another device, with control dependencies, etc.
Returns | |
---|---|
A Tensor containing the value of the variable.
|
scatter_add
scatter_add(
sparse_delta, use_locking=False, name=None
)
Adds tf.IndexedSlices
to this variable.
Args | |
---|---|
sparse_delta
|
tf.IndexedSlices to be added to this variable.
|
use_locking
|
If True , use locking during the operation.
|
name
|
the name of the operation. |
Returns | |
---|---|
A Tensor that will hold the new value of this variable after
the scattered addition has completed.
|
Raises | |
---|---|
TypeError
|
if sparse_delta is not an IndexedSlices .
|
scatter_div
scatter_div(
sparse_delta, use_locking=False, name=None
)
Divide this variable by tf.IndexedSlices
.
Args | |
---|---|
sparse_delta
|
tf.IndexedSlices to divide this variable by.
|
use_locking
|
If True , use locking during the operation.
|
name
|
the name of the operation. |
Returns | |
---|---|
A Tensor that will hold the new value of this variable after
the scattered division has completed.
|
Raises | |
---|---|
TypeError
|
if sparse_delta is not an IndexedSlices .
|
scatter_max
scatter_max(
sparse_delta, use_locking=False, name=None
)
Updates this variable with the max of tf.IndexedSlices
and itself.
Args | |
---|---|
sparse_delta
|
tf.IndexedSlices to use as an argument of max with this
variable.
|
use_locking
|
If True , use locking during the operation.
|
name
|
the name of the operation. |
Returns | |
---|---|
A Tensor that will hold the new value of this variable after
the scattered maximization has completed.
|
Raises | |
---|---|
TypeError
|
if sparse_delta is not an IndexedSlices .
|
scatter_min
scatter_min(
sparse_delta, use_locking=False, name=None
)
Updates this variable with the min of tf.IndexedSlices
and itself.
Args | |
---|---|
sparse_delta
|
tf.IndexedSlices to use as an argument of min with this
variable.
|
use_locking
|
If True , use locking during the operation.
|
name
|
the name of the operation. |
Returns | |
---|---|
A Tensor that will hold the new value of this variable after
the scattered minimization has completed.
|
Raises | |
---|---|
TypeError
|
if sparse_delta is not an IndexedSlices .
|
scatter_mul
scatter_mul(
sparse_delta, use_locking=False, name=None
)
Multiply this variable by tf.IndexedSlices
.
Args | |
---|---|
sparse_delta
|
tf.IndexedSlices to multiply this variable by.
|
use_locking
|
If True , use locking during the operation.
|
name
|
the name of the operation. |
Returns | |
---|---|
A Tensor that will hold the new value of this variable after
the scattered multiplication has completed.
|
Raises | |
---|---|
TypeError
|
if sparse_delta is not an IndexedSlices .
|
scatter_nd_add
scatter_nd_add(
indices, updates, name=None
)
Applies sparse addition to individual values or slices in a Variable.
The Variable has rank P
and indices
is a Tensor
of rank Q
.
indices
must be integer tensor, containing indices into self.
It must be shape [d_0, ..., d_{Q-2}, K]
where 0 < K <= P
.
The innermost dimension of indices
(with length K
) corresponds to
indices into elements (if K = P
) or slices (if K < P
) along the K
th
dimension of self.
updates
is Tensor
of rank Q-1+P-K
with shape:
[d_0, ..., d_{Q-2}, self.shape[K], ..., self.shape[P-1]].
For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 elements. In Python, that update would look like this:
v = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8])
indices = tf.constant([[4], [3], [1] ,[7]])
updates = tf.constant([9, 10, 11, 12])
add = v.scatter_nd_add(indices, updates)
with tf.compat.v1.Session() as sess:
print sess.run(add)
The resulting update to v would look like this:
[1, 13, 3, 14, 14, 6, 7, 20]
See tf.scatter_nd
for more details about how to make updates to
slices.
Args | |
---|---|
indices
|
The indices to be used in the operation. |
updates
|
The values to be used in the operation. |
name
|
the name of the operation. |
Returns | |
---|---|
A Tensor that will hold the new value of this variable after
the scattered addition has completed.
|
scatter_nd_sub
scatter_nd_sub(
indices, updates, name=None
)
Applies sparse subtraction to individual values or slices in a Variable.
Assuming the variable has rank P
and indices
is a Tensor
of rank Q
.
indices
must be integer tensor, containing indices into self.
It must be shape [d_0, ..., d_{Q-2}, K]
where 0 < K <= P
.
The innermost dimension of indices
(with length K
) corresponds to
indices into elements (if K = P
) or slices (if K < P
) along the K
th
dimension of self.
updates
is Tensor
of rank Q-1+P-K
with shape:
[d_0, ..., d_{Q-2}, self.shape[K], ..., self.shape[P-1]].
For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 elements. In Python, that update would look like this:
v = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8])
indices = tf.constant([[4], [3], [1] ,[7]])
updates = tf.constant([9, 10, 11, 12])
op = v.scatter_nd_sub(indices, updates)
with tf.compat.v1.Session() as sess:
print sess.run(op)
The resulting update to v would look like this:
[1, -9, 3, -6, -6, 6, 7, -4]
See tf.scatter_nd
for more details about how to make updates to
slices.
Args | |
---|---|
indices
|
The indices to be used in the operation. |
updates
|
The values to be used in the operation. |
name
|
the name of the operation. |
Returns | |
---|---|
A Tensor that will hold the new value of this variable after
the scattered subtraction has completed.
|
scatter_nd_update
scatter_nd_update(
indices, updates, name=None
)
Applies sparse assignment to individual values or slices in a Variable.
The Variable has rank P
and indices
is a Tensor
of rank Q
.
indices
must be integer tensor, containing indices into self.
It must be shape [d_0, ..., d_{Q-2}, K]
where 0 < K <= P
.
The innermost dimension of indices
(with length K
) corresponds to
indices into elements (if K = P
) or slices (if K < P
) along the K
th
dimension of self.
updates
is Tensor
of rank Q-1+P-K
with shape:
[d_0, ..., d_{Q-2}, self.shape[K], ..., self.shape[P-1]].
For example, say we want to add 4 scattered elements to a rank-1 tensor to 8 elements. In Python, that update would look like this:
v = tf.Variable([1, 2, 3, 4, 5, 6, 7, 8])
indices = tf.constant([[4], [3], [1] ,[7]])
updates = tf.constant([9, 10, 11, 12])
op = v.scatter_nd_assign(indices, updates)
with tf.compat.v1.Session() as sess:
print sess.run(op)
The resulting update to v would look like this:
[1, 11, 3, 10, 9, 6, 7, 12]
See tf.scatter_nd
for more details about how to make updates to
slices.
Args | |
---|---|
indices
|
The indices to be used in the operation. |
updates
|
The values to be used in the operation. |
name
|
the name of the operation. |
Returns | |
---|---|
A Tensor that will hold the new value of this variable after
the scattered assignment has completed.
|
scatter_sub
scatter_sub(
sparse_delta, use_locking=False, name=None
)
Subtracts tf.IndexedSlices
from this variable.
Args | |
---|---|
sparse_delta
|
tf.IndexedSlices to be subtracted from this variable.
|
use_locking
|
If True , use locking during the operation.
|
name
|
the name of the operation. |
Returns | |
---|---|
A Tensor that will hold the new value of this variable after
the scattered subtraction has completed.
|
Raises | |
---|---|
TypeError
|
if sparse_delta is not an IndexedSlices .
|
scatter_update
scatter_update(
sparse_delta, use_locking=False, name=None
)
Assigns tf.IndexedSlices
to this variable.
Args | |
---|---|
spars |