此页面由 Cloud Translation API 翻译。
Switch to English

迁移TensorFlow 1码TensorFlow 2

查看上TensorFlow.org 在谷歌Colab运行 GitHub上查看源代码 下载笔记本

这对于文档低水平TensorFlow的API的用户。如果您使用的是高层次的API( tf.keras )有可能需要采取完全让你的代码很少或根本没有采取行动TensorFlow 2.0兼容:

它仍然是可能的运行1.X代码,未经修饰的( 除了contrib ),在TensorFlow 2.0:

 import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
 

然而,这并不让你把许多在TensorFlow 2.0所做的改进的优势。本指南将帮助你升级你的代码,使之更简单,更高性能,更易于维护。

自动转换脚本

第一步,试图实施本文档中描述的改变之前,是尝试运行升级脚本

这会在你的代码升级到2.0 TensorFlow做一个初始趟次。但它不能使你的代码习惯到2.0。您的代码仍然可以利用tf.compat.v1端点访问占位符,会话,收藏等的1.x风格的功能。

顶级的行为改变

如果你的代码中TensorFlow 2.0适用于使用tf.compat.v1.disable_v2_behavior()仍有可能需要地址的全球行为的变化。主要的变化是:

  • 急于执行, v1.enable_eager_execution()隐含使用任何代码tf.Graph将失败。请务必在包裹这个代码with tf.Graph().as_default()上下文。

  • 资源变量, v1.enable_resource_variables()有些代码可能会依赖于非确定性的行为通过TF引用变量启用。资源变量被锁定,同时被写入,因此提供了更直观的一致性保证。

    • 这可能会在边缘的情况下改变自己的行为。
    • 这会造成额外的拷贝,可以有更高的内存使用情况。
    • 这可以通过将被禁用use_resource=Falsetf.Variable构造。
  • 张量的形状, v1.enable_v2_tensorshape() :TF 2.0简化张量的形状的行为。取而代之的t.shape[0].value ,你可以说t.shape[0]这些变化要小,而且是有意义的解决这些问题的时候了。见TensorShape的例子。

  • 控制流动, v1.enable_control_flow_v2()该TF 2.0控制流程实现已经被简化,并且因此产生不同的曲线图表示。请文件中的错误存在任何问题。

使代码2.0原生

该指南将通过TensorFlow 1.x的代码转换为TensorFlow 2.0的几个例子行走。这些变化将让性能优化和简化API调用的代码中占据优势。

在每种情况下,该模式是:

1.更换v1.Session.run电话

每个v1.Session.run呼应该由Python的功能所取代。

  • feed_dictv1.placeholder小号成为函数的参数。
  • fetches成为函数的返回值。
  • 在转换过程中急于执行允许与标准Python的工具,如方便调试pdb

在此之后添加tf.function装饰使它在图形有效地运行。见签名指南以获得更多关于如何工作的。

注意:

2.使用Python对象,以跟踪变量和损失

所有基于名称的变量跟踪在TF 2.0极力劝阻。使用Python对象,以跟踪变量。

使用tf.Variable代替v1.get_variable

每个v1.variable_scope应转换为Python对象。通常,这将是一个:

如果你需要的变量汇总表(样tf.Graph.get_collection(tf.GraphKeys.VARIABLES)使用.variables.trainable_variables的属性LayerModel对象。

这些LayerModel类实现的是消除对全球收藏需要一些其他的属性。他们.losses属性可以使用替代tf.GraphKeys.LOSSES集合。

keras指南了解详情。

3.升级你的训练循环

使用最高级别的API,为您的使用情况下工作。身高tf.keras.Model.fit在建设自己的训练循环。

这些高级功能管理了很多低级别的细节,如果你编写自己的训练循环,可能是很容易错过。例如,他们会自动收集正规化的损失,并设置training=True调用模型时的说法。

4.升级您的数据输入管道

使用tf.data数据的输入数据集。这些对象是有效的,富有表现力,并与tensorflow很好地集成。

他们可以直接传递给tf.keras.Model.fit方法。

 model.fit(dataset, epochs=5)
 

它们可以遍历的直接标准Python:

 for example_batch, label_batch in dataset:
    break
 

5.迁移关闭compat.v1符号

tf.compat.v1模块包含完整的TensorFlow 1.x的API,其原有的语义。

TF2升级脚本将符号转换为其2.0当量,如果这种转换是安全的,也就是说,如果能确定的2.0版本的行为是完全等价的(例如,它会重命名v1.arg_maxtf.argmax ,因为那些是相同的功能)。

升级脚本用一段代码完成后,很可能有不少提到的compat.v1 。值得一经历的代码和手动转换这些到2.0当量(它应该在日志中提到的,如果有一个)。

转换模式

建立

 import tensorflow as tf


import tensorflow_datasets as tfds
 

低级别的变量和操作执行

低级别的API的例子包括:

之前转换

下面是这些模式可能看起来像在代码中使用TensorFlow 1.x的

 in_a = tf.placeholder(dtype=tf.float32, shape=(2))
in_b = tf.placeholder(dtype=tf.float32, shape=(2))

def forward(x):
  with tf.variable_scope("matmul", reuse=tf.AUTO_REUSE):
    W = tf.get_variable("W", initializer=tf.ones(shape=(2,2)),
                        regularizer=tf.contrib.layers.l2_regularizer(0.04))
    b = tf.get_variable("b", initializer=tf.zeros(shape=(2)))
    return W * x + b

out_a = forward(in_a)
out_b = forward(in_b)

reg_loss=tf.losses.get_regularization_loss(scope="matmul")

with tf.Session() as sess:
  sess.run(tf.global_variables_initializer())
  outs = sess.run([out_a, out_b, reg_loss],
                feed_dict={in_a: [1, 0], in_b: [0, 1]})

 

转换后

在转换后的代码:

  • 变量是Python的对象。
  • forward功能仍然定义的计算。
  • Session.run调用替换调用forward
  • 可选tf.function装饰可以添加性能。
  • 在正则化是手动计算,不参照全球征集。
  • 没有会议或占位符。
 W = tf.Variable(tf.ones(shape=(2,2)), name="W")
b = tf.Variable(tf.zeros(shape=(2)), name="b")

@tf.function
def forward(x):
  return W * x + b

out_a = forward([1,0])
print(out_a)
 
tf.Tensor(
[[1. 0.]
 [1. 0.]], shape=(2, 2), dtype=float32)

 out_b = forward([0,1])

regularizer = tf.keras.regularizers.l2(0.04)
reg_loss=regularizer(W)
 

基于模型tf.layers

所述v1.layers模块用于包含层功能上依赖v1.variable_scope以限定和重复使用的变量。

之前转换

 def model(x, training, scope='model'):
  with tf.variable_scope(scope, reuse=tf.AUTO_REUSE):
    x = tf.layers.conv2d(x, 32, 3, activation=tf.nn.relu,
          kernel_regularizer=tf.contrib.layers.l2_regularizer(0.04))
    x = tf.layers.max_pooling2d(x, (2, 2), 1)
    x = tf.layers.flatten(x)
    x = tf.layers.dropout(x, 0.1, training=training)
    x = tf.layers.dense(x, 64, activation=tf.nn.relu)
    x = tf.layers.batch_normalization(x, training=training)
    x = tf.layers.dense(x, 10)
    return x

train_out = model(train_data, training=True)
test_out = model(test_data, training=False)
 

转换后

大多数参数保持不变。但是请注意区别:

  • training参数传递给每一层由模型在运行时。
  • 第一个参数为原始model函数(输入x )是走了。这是因为对象层分开调用模型构建模型。

还要注意的是:

  • 如果您使用的初始化的regularizers tf.contrib ,这些比别人有更多的参数变化。
  • 该代码不再写入集合,所以像功能v1.losses.get_regularization_loss将不再返回这些值,可能破坏你的训练循环。
 model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, 3, activation='relu',
                           kernel_regularizer=tf.keras.regularizers.l2(0.04),
                           input_shape=(28, 28, 1)),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dropout(0.1),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.Dense(10)
])

train_data = tf.ones(shape=(1, 28, 28, 1))
test_data = tf.ones(shape=(1, 28, 28, 1))
 
 train_out = model(train_data, training=True)
print(train_out)
 
tf.Tensor([[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]], shape=(1, 10), dtype=float32)

 test_out = model(test_data, training=False)
print(test_out)
 
tf.Tensor(
[[-0.11456077 -0.3126101   0.13154565 -0.50197905 -0.02416557  0.36460522
  -0.24887308 -0.37784547  0.05524942  0.01696768]], shape=(1, 10), dtype=float32)

 # Here are all the trainable variables.
len(model.trainable_variables)
 
8
 # Here is the regularization loss.
model.losses
 
[<tf.Tensor: shape=(), dtype=float32, numpy=0.077528305>]

混合变量和v1.layers

通常现有的代码混合级别较低的TF 1.x的变量和操作与更高级别的v1.layers

之前转换

 def model(x, training, scope='model'):
  with tf.variable_scope(scope, reuse=tf.AUTO_REUSE):
    W = tf.get_variable(
      "W", dtype=tf.float32,
      initializer=tf.ones(shape=x.shape),
      regularizer=tf.contrib.layers.l2_regularizer(0.04),
      trainable=True)
    if training:
      x = x + W
    else:
      x = x + W * 0.5
    x = tf.layers.conv2d(x, 32, 3, activation=tf.nn.relu)
    x = tf.layers.max_pooling2d(x, (2, 2), 1)
    x = tf.layers.flatten(x)
    return x

train_out = model(train_data, training=True)
test_out = model(test_data, training=False)
 

转换后

要转换此代码,请映射层来层作为在前面的例子中的图案。

一般的模式是:

  • 在收集层参数__init__
  • 内建变量build
  • 执行在计算call ,并返回结果。

v1.variable_scope是基于自己的层。所以把它改写为tf.keras.layers.Layer 。见导向的细节。

 # Create a custom layer for part of the model
class CustomLayer(tf.keras.layers.Layer):
  def __init__(self, *args, **kwargs):
    super(CustomLayer, self).__init__(*args, **kwargs)

  def build(self, input_shape):
    self.w = self.add_weight(
        shape=input_shape[1:],
        dtype=tf.float32,
        initializer=tf.keras.initializers.ones(),
        regularizer=tf.keras.regularizers.l2(0.02),
        trainable=True)

  # Call method will sometimes get used in graph mode,
  # training will get turned into a tensor
  @tf.function
  def call(self, inputs, training=None):
    if training:
      return inputs + self.w
    else:
      return inputs + self.w * 0.5
 
 custom_layer = CustomLayer()
print(custom_layer([1]).numpy())
print(custom_layer([1], training=True).numpy())
 
[1.5]
[2.]

 train_data = tf.ones(shape=(1, 28, 28, 1))
test_data = tf.ones(shape=(1, 28, 28, 1))

# Build the model including the custom layer
model = tf.keras.Sequential([
    CustomLayer(input_shape=(28, 28, 1)),
    tf.keras.layers.Conv2D(32, 3, activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
])

train_out = model(train_data, training=True)
test_out = model(test_data, training=False)

 

需要注意以下几点:

  • 子类Keras模型&层需要在两个V1曲线图(没有自动控制的依赖关系),并在急切模式下运行

  • 不要忘了接受training的参数来call

    • 有时它是一个tf.Tensor
    • 有时,它是一个Python布尔值。
  • 创建构造函数或模型变量Model.build使用self.add_weight()

  • 不要让tf.Tensors在你的对象。

    • 他们可能会创建无论是在tf.function或渴望背景下,这些张量表现不同。
    • 使用tf.Variable S对于国家,他们总是从两种情况下可用
    • tf.Tensors仅用于中间值。

在斯利姆和contrib.layers的注意事项

大量的旧TensorFlow的1.x代码使用超薄库,它被打包与TensorFlow 1.x中为tf.contrib.layers 。作为一个contrib模块,这是TensorFlow 2.0不再可用,甚至在tf.compat.v1 。使用超薄到TF 2.0转换码被多个参与不是转换使用库v1.layers 。事实上,它可能是有意义的你苗条代码转换为v1.layers ,然后再转换为Keras。

  • 删除arg_scopes ,所有ARGS需要明确
  • 如果你使用它们,分裂normalizer_fnactivation_fn到自己的图层
  • 可分离CONV层映射到一个或多个不同的层Keras(深度方向,逐点,和可分离Keras层)
  • 斯利姆和v1.layers有不同的名称ARG和默认值
  • 一些ARGS有不同的尺度
  • 如果使用超薄预先训练模型,从尝试Keras的预traimed模型tf.keras.applicationsTF集线器的TF2 SavedModels从原来的修身代码出口。

有些tf.contrib层可能没有被移动到核心TensorFlow但反而被移到了TF的附加组件包

训练

有很多方法,以饲料数据到tf.keras模型。他们将接受Python生成和numpy的数组作为输入。

到馈送数据到一个模型中的推荐的方法是使用tf.data包,其中包含高性能类用于操纵数据的集合。

如果你还在使用tf.queue ,这些现在只支持如数据结构,而不是输入管道。

使用数据集

所述TensorFlow数据集包( tfds )包含用于加载预定义的数据集公用事业tf.data.Dataset对象。

对于这个例子,加载MNISTdataset,使用tfds

 datasets, info = tfds.load(name='mnist', with_info=True, as_supervised=True)
mnist_train, mnist_test = datasets['train'], datasets['test']
 
Downloading and preparing dataset mnist/3.0.1 (download: 11.06 MiB, generated: 21.00 MiB, total: 32.06 MiB) to /home/kbuilder/tensorflow_datasets/mnist/3.0.1...

Warning:absl:Dataset mnist is hosted on GCS. It will automatically be downloaded to your
local data directory. If you'd instead prefer to read directly from our public
GCS bucket (recommended if you're running on GCP), you can instead pass
`try_gcs=True` to `tfds.load` or set `data_dir=gs://tfds-data/datasets`.


Dataset mnist downloaded and prepared to /home/kbuilder/tensorflow_datasets/mnist/3.0.1. Subsequent calls will reuse this data.

然后准备训练数据:

  • 重新缩放的各图像。
  • 洗牌的例子的顺序。
  • 图像和标签的收集批次。
 BUFFER_SIZE = 10 # Use a much larger value for real code.
BATCH_SIZE = 64
NUM_EPOCHS = 5


def scale(image, label):
  image = tf.cast(image, tf.float32)
  image /= 255

  return image, label
 

为了使这个示例短,修剪数据集只返回5批:

 train_data = mnist_train.map(scale).shuffle(BUFFER_SIZE).batch(BATCH_SIZE)
test_data = mnist_test.map(scale).batch(BATCH_SIZE)

STEPS_PER_EPOCH = 5

train_data = train_data.take(STEPS_PER_EPOCH)
test_data = test_data.take(STEPS_PER_EPOCH)
 
 image_batch, label_batch = next(iter(train_data))
 

使用Keras培训圈

如果你不需要你的训练过程的低电平控制,利用Keras内置的fitevaluatepredict方法的建议。这些方法提供一个统一的接口来训练模型而不管执行(顺序,功能性的,或子类)。

这些方法的优点包括:

  • 他们接受numpy的阵列,Python生成和, tf.data.Datasets
  • 他们自动应用正规化,并激活损失。
  • 它们支持tf.distribute 多设备的培训
  • 他们支持任意可以调用的损失和指标。
  • 他们支持回调像tf.keras.callbacks.TensorBoard和自定义回调。
  • 他们是高性能,使用自动TensorFlow图。

这里是训练用的模型的实例Dataset 。 (有关该如何工作的详情请参阅教程 。)

 model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, 3, activation='relu',
                           kernel_regularizer=tf.keras.regularizers.l2(0.02),
                           input_shape=(28, 28, 1)),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dropout(0.1),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.Dense(10)
])

# Model is the full model w/o custom layers
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

model.fit(train_data, epochs=NUM_EPOCHS)
loss, acc = model.evaluate(test_data)

print("Loss {}, Accuracy {}".format(loss, acc))
 
Epoch 1/5
5/5 [==============================] - 0s 6ms/step - loss: 1.5323 - accuracy: 0.5063
Epoch 2/5
5/5 [==============================] - 0s 6ms/step - loss: 0.4105 - accuracy: 0.9219
Epoch 3/5
5/5 [==============================] - 0s 7ms/step - loss: 0.2495 - accuracy: 0.9531
Epoch 4/5
5/5 [==============================] - 0s 6ms/step - loss: 0.1806 - accuracy: 0.9875
Epoch 5/5
5/5 [==============================] - 0s 6ms/step - loss: 0.1416 - accuracy: 0.9937
5/5 [==============================] - 0s 4ms/step - loss: 1.5655 - accuracy: 0.6469
Loss 1.565544605255127, Accuracy 0.6468750238418579

写自己的循环

如果Keras模型的训练步骤为你工作,但你需要更多的控制范围之外的那一步,可以考虑使用tf.keras.Model.train_on_batch方法,在你自己的数据迭代循环。

请记住:很多事情可以作为一个实现tf.keras.callbacks.Callback

该方法具有许多的上一节中提到的方法的优点,但给人的外环的用户控制。

您还可以使用tf.keras.Model.test_on_batchtf.keras.Model.evaluate训练期间检查的性能。

继续训练上面的模型:

 # Model is the full model w/o custom layers
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

for epoch in range(NUM_EPOCHS):
  #Reset the metric accumulators
  model.reset_metrics()

  for image_batch, label_batch in train_data:
    result = model.train_on_batch(image_batch, label_batch)
    metrics_names = model.metrics_names
    print("train: ",
          "{}: {:.3f}".format(metrics_names[0], result[0]),
          "{}: {:.3f}".format(metrics_names[1], result[1]))
  for image_batch, label_batch in test_data:
    result = model.test_on_batch(image_batch, label_batch,
                                 # return accumulated metrics
                                 reset_metrics=False)
  metrics_names = model.metrics_names
  print("\neval: ",
        "{}: {:.3f}".format(metrics_names[0], result[0]),
        "{}: {:.3f}".format(metrics_names[1], result[1]))
 
train:  loss: 0.122 accuracy: 0.984
train:  loss: 0.133 accuracy: 0.984
train:  loss: 0.164 accuracy: 0.969
train:  loss: 0.167 accuracy: 0.969
train:  loss: 0.161 accuracy: 0.984

eval:  loss: 1.583 accuracy: 0.669
train:  loss: 0.074 accuracy: 1.000
train:  loss: 0.090 accuracy: 1.000
train:  loss: 0.089 accuracy: 1.000
train:  loss: 0.095 accuracy: 1.000
train:  loss: 0.090 accuracy: 1.000

eval:  loss: 1.567 accuracy: 0.747
train:  loss: 0.065 accuracy: 1.000
train:  loss: 0.068 accuracy: 1.000
train:  loss: 0.056 accuracy: 1.000
train:  loss: 0.069 accuracy: 1.000
train:  loss: 0.067 accuracy: 1.000

eval:  loss: 1.545 accuracy: 0.772
train:  loss: 0.053 accuracy: 1.000
train:  loss: 0.063 accuracy: 0.984
train:  loss: 0.050 accuracy: 1.000
train:  loss: 0.051 accuracy: 1.000
train:  loss: 0.049 accuracy: 1.000

eval:  loss: 1.520 accuracy: 0.778
train:  loss: 0.049 accuracy: 1.000
train:  loss: 0.046 accuracy: 1.000
train:  loss: 0.044 accuracy: 1.000
train:  loss: 0.045 accuracy: 1.000
train:  loss: 0.044 accuracy: 1.000

eval:  loss: 1.494 accuracy: 0.791

定制培训的步骤

如果您需要更多的灵活性和控制,您可以通过实现自己的训练循环有它。有三个步骤:

  1. 遍历一个Python发电机或tf.data.Dataset获得的例子批次。
  2. 使用tf.GradientTape收集梯度。
  3. 使用的一个tf.keras.optimizers重量更新应用到模型的变量。

记得:

  • 始终包括training的参数call子类层和模型的方法。
  • 请确保调用与模型training正确的参数设置。
  • 根据使用情况,直到模型上批量数据的运行可能不存在模型变量。
  • 您需要手动处理事情像模型正规化损失。

注意简化相对于第一版:

  • 没有必要运行变量初始化。变量初始化的创作。
  • 有没有必要加手动控制的依赖。即使在tf.function操作充当渴望模式。
 model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, 3, activation='relu',
                           kernel_regularizer=tf.keras.regularizers.l2(0.02),
                           input_shape=(28, 28, 1)),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dropout(0.1),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.Dense(10)
])

optimizer = tf.keras.optimizers.Adam(0.001)
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

@tf.function
def train_step(inputs, labels):
  with tf.GradientTape() as tape:
    predictions = model(inputs, training=True)
    regularization_loss=tf.math.add_n(model.losses)
    pred_loss=loss_fn(labels, predictions)
    total_loss=pred_loss + regularization_loss

  gradients = tape.gradient(total_loss, model.trainable_variables)
  optimizer.apply_gradients(zip(gradients, model.trainable_variables))

for epoch in range(NUM_EPOCHS):
  for inputs, labels in train_data:
    train_step(inputs, labels)
  print("Finished epoch", epoch)

 
Finished epoch 0
Finished epoch 1
Finished epoch 2
Finished epoch 3
Finished epoch 4

新型指标和损失

在TensorFlow 2.0,度量和损失是对象。既热切而在这些工作tf.function秒。

甲损失目的是调用,和预期(y_true,y_pred)作为参数:

 cce = tf.keras.losses.CategoricalCrossentropy(from_logits=True)
cce([[1, 0]], [[-1.0,3.0]]).numpy()
 
4.01815

度量对象有以下方法:

对象本身是可调用。调用更新状态,新的观察结果,与update_state ,并返回度量的新的结果。

您不必手动初始化度量的变量,因为TensorFlow 2.0具有自动控制的依赖,你不需要那些担心无论是。

下面采用的是公制的代码,以跟踪定制培训环路内观察到的平均损失。

 # Create the metrics
loss_metric = tf.keras.metrics.Mean(name='train_loss')
accuracy_metric = tf.keras.metrics.SparseCategoricalAccuracy(name='train_accuracy')

@tf.function
def train_step(inputs, labels):
  with tf.GradientTape() as tape:
    predictions = model(inputs, training=True)
    regularization_loss=tf.math.add_n(model.losses)
    pred_loss=loss_fn(labels, predictions)
    total_loss=pred_loss + regularization_loss

  gradients = tape.gradient(total_loss, model.trainable_variables)
  optimizer.apply_gradients(zip(gradients, model.trainable_variables))
  # Update the metrics
  loss_metric.update_state(total_loss)
  accuracy_metric.update_state(labels, predictions)


for epoch in range(NUM_EPOCHS):
  # Reset the metrics
  loss_metric.reset_states()
  accuracy_metric.reset_states()

  for inputs, labels in train_data:
    train_step(inputs, labels)
  # Get the metric results
  mean_loss=loss_metric.result()
  mean_accuracy = accuracy_metric.result()

  print('Epoch: ', epoch)
  print('  loss:     {:.3f}'.format(mean_loss))
  print('  accuracy: {:.3f}'.format(mean_accuracy))

 
Epoch:  0
  loss:     0.175
  accuracy: 0.994
Epoch:  1
  loss:     0.149
  accuracy: 0.991
Epoch:  2
  loss:     0.133
  accuracy: 0.991
Epoch:  3
  loss:     0.113
  accuracy: 0.997
Epoch:  4
  loss:     0.101
  accuracy: 0.997

Keras指标名称

在TensorFlow 2.0 keras模型有关操作的指标名称更加一致。

现在,当你通过在指标列表中选择一个字符串,该字符串完全相同作为度量的name 。这些名字都在返回的历史对象可见model.fit ,并在日志传递给keras.callbacks 。设为您在指标列表中传递的字符串。

 model.compile(
    optimizer = tf.keras.optimizers.Adam(0.001),
    loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    metrics = ['acc', 'accuracy', tf.keras.metrics.SparseCategoricalAccuracy(name="my_accuracy")])
history = model.fit(train_data)
 
5/5 [==============================] - 0s 6ms/step - loss: 0.1076 - acc: 0.9969 - accuracy: 0.9969 - my_accuracy: 0.9969

 history.history.keys()
 
dict_keys(['loss', 'acc', 'accuracy', 'my_accuracy'])

这不同于以前的版本,其中通过metrics=["accuracy"]将导致dict_keys(['loss', 'acc'])

Keras优化

在优化v1.train ,像v1.train.AdamOptimizerv1.train.GradientDescentOptimizer ,具有当量tf.keras.optimizers

转换v1.trainkeras.optimizers

这里有东西转换您的优化时要牢记:

对于一些新的默认tf.keras.optimizers

有毫无变化optimizers.SGDoptimizers.Adam ,或optimizers.RMSprop

下面的默认学习速率已更改:

TensorBoard

TensorFlow 2包括到显著变化tf.summary用于写入摘要数据在可视化TensorBoard API。对于一般的介绍新tf.summary ,有几个可用的教程是使用TF 2 API。这包括TensorBoard TF 2迁移指南

保存和载入

兼容性检查站

TensorFlow 2.0用途基于对象的检查点

旧式基于名字的检查站可以被载入,如果你小心。代码转换过程可导致变量名的变化,但也有替代方法。

最简单的方法是排队在检查点名称的新模式的名称:

  • 变量仍然都有一个name参数可以设置。
  • Keras车型也采取了name参数作为其设置为前缀的变量。
  • v1.name_scope功能可用于设置变量的名称前缀。这是非常不同tf.variable_scope 。它仅影响名称,不跟踪变量和重用。

如果没有为您的使用情况的工作,尝试v1.train.init_from_checkpoint功能。它需要一个assignment_map参数,它规定从旧名称到新名称的映射。

所述TensorFlow估算储存库包括转换工具 ,以用于预制估计检查站升级从TensorFlow 1.X至2.0。它可作为如何建立一个类似的用例的工具的例子。

保存的模型兼容性

有对保存的模型没有显著兼容性问题。

  • TensorFlow 1.x的saved_models在TensorFlow 2.x的工作
  • TensorFlow 2.X saved_models在TensorFlow工作1.x中,如果所有的OPS支持。

一个Graph.pb或Graph.pbtxt

还有就是原始升级没有简单的方法Graph.pb文件TensorFlow 2.0。最好的办法是升级生成该文件的代码。

但是,如果你有一个“冻结图”(一tf.Graph其中的变量已变成常数),那么就可以将其转换为一个concrete_function使用v1.wrap_function

 def wrap_frozen_graph(graph_def, inputs, outputs):
  def _imports_graph_def():
    tf.compat.v1.import_graph_def(graph_def, name="")
  wrapped_import = tf.compat.v1.wrap_function(_imports_graph_def, [])
  import_graph = wrapped_import.graph
  return wrapped_import.prune(
      tf.nest.map_structure(import_graph.as_graph_element, inputs),
      tf.nest.map_structure(import_graph.as_graph_element, outputs))
 

例如,这里是盗梦空间V1一个frozed图,2016年:

 path = tf.keras.utils.get_file(
    'inception_v1_2016_08_28_frozen.pb',
    'http://storage.googleapis.com/download.tensorflow.org/models/inception_v1_2016_08_28_frozen.pb.tar.gz',
    untar=True)
 
Downloading data from http://storage.googleapis.com/download.tensorflow.org/models/inception_v1_2016_08_28_frozen.pb.tar.gz
24698880/24695710 [==============================] - 1s 0us/step

加载tf.GraphDef

 graph_def = tf.compat.v1.GraphDef()
loaded = graph_def.ParseFromString(open(path,'rb').read())
 

裹成一个concrete_function

 inception_func = wrap_frozen_graph(
    graph_def, inputs='input:0',
    outputs='InceptionV1/InceptionV1/Mixed_3b/Branch_1/Conv2d_0a_1x1/Relu:0')
 

它传递一个张量作为输入:

 input_img = tf.ones([1,224,224,3], dtype=tf.float32)
inception_func(input_img).shape
 
TensorShape([1, 28, 28, 96])

估计

有估计培训

估计在TensorFlow 2.0的支持。

当您使用估计,您可以使用input_fn() tf.estimator.TrainSpectf.estimator.EvalSpec从TensorFlow 1.x的

下面是使用一个例子input_fn与火车和评估的规格。

创建input_fn和火车/ EVAL规格

 # Define the estimator's input_fn
def input_fn():
  datasets, info = tfds.load(name='mnist', with_info=True, as_supervised=True)
  mnist_train, mnist_test = datasets['train'], datasets['test']

  BUFFER_SIZE = 10000
  BATCH_SIZE = 64

  def scale(image, label):
    image = tf.cast(image, tf.float32)
    image /= 255

    return image, label[..., tf.newaxis]

  train_data = mnist_train.map(scale).shuffle(BUFFER_SIZE).batch(BATCH_SIZE)
  return train_data.repeat()

# Define train & eval specs
train_spec = tf.estimator.TrainSpec(input_fn=input_fn,
                                    max_steps=STEPS_PER_EPOCH * NUM_EPOCHS)
eval_spec = tf.estimator.EvalSpec(input_fn=input_fn,
                                  steps=STEPS_PER_EPOCH)

 

使用Keras模型定义

有在如何构建你的估计在2.0 TensorFlow一些差异。

我们建议您使用Keras,然后使用定义的模型tf.keras.estimator.model_to_estimator工具来把你的模型转换成一个估计。下面显示的代码如何创建和训练的估计时,使用该工具。

 def make_model():
  return tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, 3, activation='relu',
                           kernel_regularizer=tf.keras.regularizers.l2(0.02),
                           input_shape=(28, 28, 1)),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dropout(0.1),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.Dense(10)
  ])
 
 model = make_model()

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

estimator = tf.keras.estimator.model_to_estimator(
  keras_model = model
)

tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)
 
INFO:tensorflow:Using default config.

INFO:tensorflow:Using default config.

Warning:tensorflow:Using temporary folder as model directory: /tmp/tmpb3_a632k

Warning:tensorflow:Using temporary folder as model directory: /tmp/tmpb3_a632k

INFO:tensorflow:Using the Keras model provided.

INFO:tensorflow:Using the Keras model provided.

Warning:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py:1666: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.

Warning:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py:1666: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.

INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpb3_a632k', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}

INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpb3_a632k', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}

INFO:tensorflow:Not using Distribute Coordinator.

INFO:tensorflow:Not using Distribute Coordinator.

INFO:tensorflow:Running training and evaluation locally (non-distributed).

INFO:tensorflow:Running training and evaluation locally (non-distributed).

INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps None or save_checkpoints_secs 600.

INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps None or save_checkpoints_secs 600.

Warning:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.

Warning:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.

INFO:tensorflow:Calling model_fn.

INFO:tensorflow:Calling model_fn.

INFO:tensorflow:Done calling model_fn.

INFO:tensorflow:Done calling model_fn.

INFO:tensorflow:Warm-starting with WarmStartSettings: WarmStartSettings(ckpt_to_initialize_from='/tmp/tmpb3_a632k/keras/keras_model.ckpt', vars_to_warm_start='.*', var_name_to_vocab_info={}, var_name_to_prev_var_name={})

INFO:tensorflow:Warm-starting with WarmStartSettings: WarmStartSettings(ckpt_to_initialize_from='/tmp/tmpb3_a632k/keras/keras_model.ckpt', vars_to_warm_start='.*', var_name_to_vocab_info={}, var_name_to_prev_var_name={})

INFO:tensorflow:Warm-starting from: /tmp/tmpb3_a632k/keras/keras_model.ckpt

INFO:tensorflow:Warm-starting from: /tmp/tmpb3_a632k/keras/keras_model.ckpt

INFO:tensorflow:Warm-starting variables only in TRAINABLE_VARIABLES.

INFO:tensorflow:Warm-starting variables only in TRAINABLE_VARIABLES.

INFO:tensorflow:Warm-started 8 variables.

INFO:tensorflow:Warm-started 8 variables.

INFO:tensorflow:Create CheckpointSaverHook.

INFO:tensorflow:Create CheckpointSaverHook.

INFO:tensorflow:Graph was finalized.

INFO:tensorflow:Graph was finalized.

INFO:tensorflow:Running local_init_op.

INFO:tensorflow:Running local_init_op.

INFO:tensorflow:Done running local_init_op.

INFO:tensorflow:Done running local_init_op.

INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...

INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...

INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpb3_a632k/model.ckpt.

INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpb3_a632k/model.ckpt.

INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...

INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...

INFO:tensorflow:loss = 2.270717, step = 0

INFO:tensorflow:loss = 2.270717, step = 0

INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 25...

INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 25...

INFO:tensorflow:Saving checkpoints for 25 into /tmp/tmpb3_a632k/model.ckpt.

INFO:tensorflow:Saving checkpoints for 25 into /tmp/tmpb3_a632k/model.ckpt.

INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 25...

INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 25...

INFO:tensorflow:Calling model_fn.

INFO:tensorflow:Calling model_fn.

INFO:tensorflow:Done calling model_fn.

INFO:tensorflow:Done calling model_fn.

INFO:tensorflow:Starting evaluation at 2020-07-23T01:32:32Z

INFO:tensorflow:Starting evaluation at 2020-07-23T01:32:32Z

INFO:tensorflow:Graph was finalized.

INFO:tensorflow:Graph was finalized.

INFO:tensorflow:Restoring parameters from /tmp/tmpb3_a632k/model.ckpt-25

INFO:tensorflow:Restoring parameters from /tmp/tmpb3_a632k/model.ckpt-25

INFO:tensorflow:Running local_init_op.

INFO:tensorflow:Running local_init_op.

INFO:tensorflow:Done running local_init_op.

INFO:tensorflow:Done running local_init_op.

INFO:tensorflow:Evaluation [1/5]

INFO:tensorflow:Evaluation [1/5]

INFO:tensorflow:Evaluation [2/5]

INFO:tensorflow:Evaluation [2/5]

INFO:tensorflow:Evaluation [3/5]

INFO:tensorflow:Evaluation [3/5]

INFO:tensorflow:Evaluation [4/5]

INFO:tensorflow:Evaluation [4/5]

INFO:tensorflow:Evaluation [5/5]

INFO:tensorflow:Evaluation [5/5]

INFO:tensorflow:Inference Time : 1.01619s

INFO:tensorflow:Inference Time : 1.01619s

INFO:tensorflow:Finished evaluation at 2020-07-23-01:32:33

INFO:tensorflow:Finished evaluation at 2020-07-23-01:32:33

INFO:tensorflow:Saving dict for global step 25: accuracy = 0.671875, global_step = 25, loss = 1.5162958

INFO:tensorflow:Saving dict for global step 25: accuracy = 0.671875, global_step = 25, loss = 1.5162958

INFO:tensorflow:Saving 'checkpoint_path' summary for global step 25: /tmp/tmpb3_a632k/model.ckpt-25

INFO:tensorflow:Saving 'checkpoint_path' summary for global step 25: /tmp/tmpb3_a632k/model.ckpt-25

INFO:tensorflow:Loss for final step: 0.39261085.

INFO:tensorflow:Loss for final step: 0.39261085.

({'accuracy': 0.671875, 'loss': 1.5162958, 'global_step': 25}, [])

使用自定义model_fn

如果您有现有的自定义估计model_fn ,你需要维护,您可以将您的model_fn使用Keras模型。

然而,出于兼容性考虑,自定义model_fn仍将在1.x中风格的图形模式下运行。这意味着没有急于执行和没有自动控制的依赖关系。

定制model_fn最小的变化

为了使您的自定义model_fn工作TF 2.0,如果你喜欢最小的改变现有的代码, tf.compat.v1符号,如optimizersmetrics都可以使用。

在自定义使用Keras车型model_fn类似于定制的训练循环使用它:

  • 设置training阶段适当的基础上, mode的说法。
  • 明确地传递模型的trainable_variables的优化。

但也有重要的区别,相对于一个自定义的循环

  • 而不是使用Model.losses ,使用提取的损失Model.get_losses_for
  • 使用提取模型的更新Model.get_updates_for

下面的代码创建从定制的估计model_fn ,说明了所有这些问题。

 def my_model_fn(features, labels, mode):
  model = make_model()

  optimizer = tf.compat.v1.train.AdamOptimizer()
  loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

  training = (mode == tf.estimator.ModeKeys.TRAIN)
  predictions = model(features, training=training)

  if mode == tf.estimator.ModeKeys.PREDICT:
    return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions)

  reg_losses = model.get_losses_for(None) + model.get_losses_for(features)
  total_loss=loss_fn(labels, predictions) + tf.math.add_n(reg_losses)

  accuracy = tf.compat.v1.metrics.accuracy(labels=labels,
                                           predictions=tf.math.argmax(predictions, axis=1),
                                           name='acc_op')

  update_ops = model.get_updates_for(None) + model.get_updates_for(features)
  minimize_op = optimizer.minimize(
      total_loss,
      var_list=model.trainable_variables,
      global_step=tf.compat.v1.train.get_or_create_global_step())
  train_op = tf.group(minimize_op, update_ops)

  return tf.estimator.EstimatorSpec(
    mode=mode,
    predictions=predictions,
    loss=total_loss,
    train_op=train_op, eval_metric_ops={'accuracy': accuracy})

# Create the Estimator & Train
estimator = tf.estimator.Estimator(model_fn=my_model_fn)
tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)
 
INFO:tensorflow:Using default config.

INFO:tensorflow:Using default config.

Warning:tensorflow:Using temporary folder as model directory: /tmp/tmpjb2yik9a

Warning:tensorflow:Using temporary folder as model directory: /tmp/tmpjb2yik9a

INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpjb2yik9a', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}

INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpjb2yik9a', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}

INFO:tensorflow:Not using Distribute Coordinator.

INFO:tensorflow:Not using Distribute Coordinator.

INFO:tensorflow:Running training and evaluation locally (non-distributed).

INFO:tensorflow:Running training and evaluation locally (non-distributed).

INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps None or save_checkpoints_secs 600.

INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps None or save_checkpoints_secs 600.

INFO:tensorflow:Calling model_fn.

INFO:tensorflow:Calling model_fn.

INFO:tensorflow:Done calling model_fn.

INFO:tensorflow:Done calling model_fn.

INFO:tensorflow:Create CheckpointSaverHook.

INFO:tensorflow:Create CheckpointSaverHook.

INFO:tensorflow:Graph was finalized.

INFO:tensorflow:Graph was finalized.

INFO:tensorflow:Running local_init_op.

INFO:tensorflow:Running local_init_op.

INFO:tensorflow:Done running local_init_op.

INFO:tensorflow:Done running local_init_op.

INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...

INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...

INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpjb2yik9a/model.ckpt.

INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpjb2yik9a/model.ckpt.

INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...

INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...

INFO:tensorflow:loss = 2.6231384, step = 0

INFO:tensorflow:loss = 2.6231384, step = 0

INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 25...

INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 25...

INFO:tensorflow:Saving checkpoints for 25 into /tmp/tmpjb2yik9a/model.ckpt.

INFO:tensorflow:Saving checkpoints for 25 into /tmp/tmpjb2yik9a/model.ckpt.

INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 25...

INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 25...

INFO:tensorflow:Calling model_fn.

INFO:tensorflow:Calling model_fn.

INFO:tensorflow:Done calling model_fn.

INFO:tensorflow:Done calling model_fn.

INFO:tensorflow:Starting evaluation at 2020-07-23T01:32:36Z

INFO:tensorflow:Starting evaluation at 2020-07-23T01:32:36Z

INFO:tensorflow:Graph was finalized.

INFO:tensorflow:Graph was finalized.

INFO:tensorflow:Restoring parameters from /tmp/tmpjb2yik9a/model.ckpt-25

INFO:tensorflow:Restoring parameters from /tmp/tmpjb2yik9a/model.ckpt-25

INFO:tensorflow:Running local_init_op.

INFO:tensorflow:Running local_init_op.

INFO:tensorflow:Done running local_init_op.

INFO:tensorflow:Done running local_init_op.

INFO:tensorflow:Evaluation [1/5]

INFO:tensorflow:Evaluation [1/5]

INFO:tensorflow:Evaluation [2/5]

INFO:tensorflow:Evaluation [2/5]

INFO:tensorflow:Evaluation [3/5]

INFO:tensorflow:Evaluation [3/5]

INFO:tensorflow:Evaluation [4/5]

INFO:tensorflow:Evaluation [4/5]

INFO:tensorflow:Evaluation [5/5]

INFO:tensorflow:Evaluation [5/5]

INFO:tensorflow:Inference Time : 1.09900s

INFO:tensorflow:Inference Time : 1.09900s

INFO:tensorflow:Finished evaluation at 2020-07-23-01:32:37

INFO:tensorflow:Finished evaluation at 2020-07-23-01:32:37

INFO:tensorflow:Saving dict for global step 25: accuracy = 0.621875, global_step = 25, loss = 1.6634324

INFO:tensorflow:Saving dict for global step 25: accuracy = 0.621875, global_step = 25, loss = 1.6634324

INFO:tensorflow:Saving 'checkpoint_path' summary for global step 25: /tmp/tmpjb2yik9a/model.ckpt-25

INFO:tensorflow:Saving 'checkpoint_path' summary for global step 25: /tmp/tmpjb2yik9a/model.ckpt-25

INFO:tensorflow:Loss for final step: 0.56862533.

INFO:tensorflow:Loss for final step: 0.56862533.

({'accuracy': 0.621875, 'loss': 1.6634324, 'global_step': 25}, [])

定制model_fn与TF 2.0符号

如果你想摆脱所有TF 1.x的符号和升级您的自定义model_fn以天然TF 2.0,您需要更新优化和标准来tf.keras.optimizerstf.keras.metrics

在自定义model_fn ,除了上述的变化 ,更多的升级需要做:

对于上面的例子my_model_fn ,具有2.0的符号迁移代码被示为:

 def my_model_fn(features, labels, mode):
  model = make_model()

  training = (mode == tf.estimator.ModeKeys.TRAIN)
  loss_obj = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
  predictions = model(features, training=training)

  # Get both the unconditional losses (the None part)
  # and the input-conditional losses (the features part).
  reg_losses = model.get_losses_for(None) + model.get_losses_for(features)
  total_loss=loss_obj(labels, predictions) + tf.math.add_n(reg_losses)

  # Upgrade to tf.keras.metrics.
  accuracy_obj = tf.keras.metrics.Accuracy(name='acc_obj')
  accuracy = accuracy_obj.update_state(
      y_true=labels, y_pred=tf.math.argmax(predictions, axis=1))

  train_op = None
  if training:
    # Upgrade to tf.keras.optimizers.
    optimizer = tf.keras.optimizers.Adam()
    # Manually assign tf.compat.v1.global_step variable to optimizer.iterations
    # to make tf.compat.v1.train.global_step increased correctly.
    # This assignment is a must for any `tf.train.SessionRunHook` specified in
    # estimator, as SessionRunHooks rely on global step.
    optimizer.iterations = tf.compat.v1.train.get_or_create_global_step()
    # Get both the unconditional updates (the None part)
    # and the input-conditional updates (the features part).
    update_ops = model.get_updates_for(None) + model.get_updates_for(features)
    # Compute the minimize_op.
    minimize_op = optimizer.get_updates(
        total_loss,
        model.trainable_variables)[0]
    train_op = tf.group(minimize_op, *update_ops)

  return tf.estimator.EstimatorSpec(
    mode=mode,
    predictions=predictions,
    loss=total_loss,
    train_op=train_op,
    eval_metric_ops={'Accuracy': accuracy_obj})

# Create the Estimator & Train.
estimator = tf.estimator.Estimator(model_fn=my_model_fn)
tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)
 
INFO:tensorflow:Using default config.

INFO:tensorflow:Using default config.

Warning:tensorflow:Using temporary folder as model directory: /tmp/tmppkb1q1hq

Warning:tensorflow:Using temporary folder as model directory: /tmp/tmppkb1q1hq

INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmppkb1q1hq', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}

INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmppkb1q1hq', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}

INFO:tensorflow:Not using Distribute Coordinator.

INFO:tensorflow:Not using Distribute Coordinator.

INFO:tensorflow:Running training and evaluation locally (non-distributed).

INFO:tensorflow:Running training and evaluation locally (non-distributed).

INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps None or save_checkpoints_secs 600.

INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps None or save_checkpoints_secs 600.

INFO:tensorflow:Calling model_fn.

INFO:tensorflow:Calling model_fn.

INFO:tensorflow:Done calling model_fn.

INFO:tensorflow:Done calling model_fn.

INFO:tensorflow:Create CheckpointSaverHook.

INFO:tensorflow:Create CheckpointSaverHook.

INFO:tensorflow:Graph was finalized.

INFO:tensorflow:Graph was finalized.

INFO:tensorflow:Running local_init_op.

INFO:tensorflow:Running local_init_op.

INFO:tensorflow:Done running local_init_op.

INFO:tensorflow:Done running local_init_op.

INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...

INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...

INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmppkb1q1hq/model.ckpt.

INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmppkb1q1hq/model.ckpt.

INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...

INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...

INFO:tensorflow:loss = 2.6947184, step = 0

INFO:tensorflow:loss = 2.6947184, step = 0

INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 25...

INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 25...

INFO:tensorflow:Saving checkpoints for 25 into /tmp/tmppkb1q1hq/model.ckpt.

INFO:tensorflow:Saving checkpoints for 25 into /tmp/tmppkb1q1hq/model.ckpt.

INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 25...

INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 25...

INFO:tensorflow:Calling model_fn.

INFO:tensorflow:Calling model_fn.

INFO:tensorflow:Done calling model_fn.

INFO:tensorflow:Done calling model_fn.

INFO:tensorflow:Starting evaluation at 2020-07-23T01:32:40Z

INFO:tensorflow:Starting evaluation at 2020-07-23T01:32:40Z

INFO:tensorflow:Graph was finalized.

INFO:tensorflow:Graph was finalized.

INFO:tensorflow:Restoring parameters from /tmp/tmppkb1q1hq/model.ckpt-25

INFO:tensorflow:Restoring parameters from /tmp/tmppkb1q1hq/model.ckpt-25

INFO:tensorflow:Running local_init_op.

INFO:tensorflow:Running local_init_op.

INFO:tensorflow:Done running local_init_op.

INFO:tensorflow:Done running local_init_op.

INFO:tensorflow:Evaluation [1/5]

INFO:tensorflow:Evaluation [1/5]

INFO:tensorflow:Evaluation [2/5]

INFO:tensorflow:Evaluation [2/5]

INFO:tensorflow:Evaluation [3/5]

INFO:tensorflow:Evaluation [3/5]

INFO:tensorflow:Evaluation [4/5]

INFO:tensorflow:Evaluation [4/5]

INFO:tensorflow:Evaluation [5/5]

INFO:tensorflow:Evaluation [5/5]

INFO:tensorflow:Inference Time : 1.05743s

INFO:tensorflow:Inference Time : 1.05743s

INFO:tensorflow:Finished evaluation at 2020-07-23-01:32:41

INFO:tensorflow:Finished evaluation at 2020-07-23-01:32:41

INFO:tensorflow:Saving dict for global step 25: Accuracy = 0.784375, global_step = 25, loss = 1.4717665

INFO:tensorflow:Saving dict for global step 25: Accuracy = 0.784375, global_step = 25, loss = 1.4717665

INFO:tensorflow:Saving 'checkpoint_path' summary for global step 25: /tmp/tmppkb1q1hq/model.ckpt-25

INFO:tensorflow:Saving 'checkpoint_path' summary for global step 25: /tmp/tmppkb1q1hq/model.ckpt-25

INFO:tensorflow:Loss for final step: 0.5751184.

INFO:tensorflow:Loss for final step: 0.5751184.

({'Accuracy': 0.784375, 'loss': 1.4717665, 'global_step': 25}, [])

预制估计

预制估计在家庭tf.estimator.DNN*tf.estimator.Linear*tf.estimator.DNNLinearCombined*的TensorFlow 2.0 API中仍然支持,然而,一些参数已经改变:

  1. input_layer_partitioner :在2.0中已删除。
  2. loss_reduction :更新后tf.keras.losses.Reduction而不是tf.compat.v1.losses.Reduction 。它的默认值也改为tf.keras.losses.Reduction.SUM_OVER_BATCH_SIZEtf.compat.v1.losses.Reduction.SUM
  3. optimizerdnn_optimizerlinear_optimizer :此ARG已经更新到tf.keras.optimizers代替tf.compat.v1.train.Optimizer

要迁移上述变化:

  1. 无需迁移input_layer_partitioner因为Distribution Strategy将在TF 2.0自动处理它。
  2. 对于loss_reduction ,检查tf.keras.losses.Reduction为支持的选项。
  3. 对于optimizer指定参数时,如果你没有在通过optimizerdnn_optimizerlinear_optimizer阿根廷,或者如果指定optimizer ARG作为一个string在你的代码,你不需要改变任何东西。 tf.keras.optimizers默认情况下使用。否则,你需要从更新tf.compat.v1.train.Optimizer到其对应的tf.keras.optimizers

检查点转换器

到迁移keras.optimizers将打破使用TF 1.x中保存的检查点,作为tf.keras.optimizers生成一组不同的变量被保存在检查点。要迁移到TF 2.0后,使旧的校验可重复使用,请尝试检查点转换工具

 curl -O https://raw.githubusercontent.com/tensorflow/estimator/master/tensorflow_estimator/python/estimator/tools/checkpoint_converter.py
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 15157  100 15157    0     0  63684      0 --:--:-- --:--:-- --:--:-- 63684

该工具具有内置的帮助:

 python checkpoint_converter.py -h
usage: checkpoint_converter.py [-h]
                               {dnn,linear,combined} source_checkpoint
                               source_graph target_checkpoint

positional arguments:
  {dnn,linear,combined}
                        The type of estimator to be converted. So far, the
                        checkpoint converter only supports Canned Estimator.
                        So the allowed types include linear, dnn and combined.
  source_checkpoint     Path to source checkpoint file to be read in.
  source_graph          Path to source graph file to be read in.
  target_checkpoint     Path to checkpoint file to be written out.

optional arguments:
  -h, --help            show this help message and exit

TensorShape

这个类简化为保持int S,而不是tf.compat.v1.Dimension对象。因此,有没有需要调用.value()得到一个int

个人tf.compat.v1.Dimension对象仍然可以访问tf.TensorShape.dims

以下表明TensorFlow 1.x和TensorFlow 2.0之间的差异。

 # Create a shape and choose an index
i = 0
shape = tf.TensorShape([16, None, 256])
shape
 
TensorShape([16, None, 256])

如果您在TF 1.x中有这样的:

 value = shape[i].value
 

然后,为此在TF 2.0:

 value = shape[i]
value
 
16

如果您在TF 1.x中有这样的:

 for dim in shape:
    value = dim.value
    print(value)
 

然后,为此在TF 2.0:

 for value in shape:
  print(value)
 
16
None
256

如果你有这样的TF 1.x中(或用于其他任何尺寸的方法):

 dim = shape[i]
dim.assert_is_compatible_with(other_dim)
 

然后,为此在TF 2.0:

 other_dim = 16
Dimension = tf.compat.v1.Dimension

if shape.rank is None:
  dim = Dimension(None)
else:
  dim = shape.dims[i]
dim.is_compatible_with(other_dim) # or any other dimension method
 
True
 shape = tf.TensorShape(None)

if shape:
  dim = shape.dims[i]
  dim.is_compatible_with(other_dim) # or any other dimension method
 

一个的布尔值tf.TensorShapeTrue ,如果排名是已知的, False否则。

 print(bool(tf.TensorShape([])))      # Scalar
print(bool(tf.TensorShape([0])))     # 0-length vector
print(bool(tf.TensorShape([1])))     # 1-length vector
print(bool(tf.TensorShape([None])))  # Unknown-length vector
print(bool(tf.TensorShape([1, 10, 100])))       # 3D tensor
print(bool(tf.TensorShape([None, None, None]))) # 3D tensor with no known dimensions
print()
print(bool(tf.TensorShape(None)))  # A tensor with unknown rank.
 
True
True
True
True
True
True

False

其他变更

结论

整个过程是:

  1. 运行升级脚本。
  2. 删除的contrib符号。
  3. 切换的模式,以面向对象的风格(Keras)。
  4. 使用tf.kerastf.estimator训练和评估循环您可以在其中。
  5. 否则,使用自定义的循环,但一定要避免会话与类别。

这需要一点的工作转换代码惯用TensorFlow 2.0,但每一个变化的结果:

  • 更少的行代码。
  • 更高的清晰度和简洁性。
  • 简化调试工作。