Help protect the Great Barrier Reef with TensorFlow on Kaggle

# 模块、层和模型简介

• 一个在张量上进行某些计算的函数（前向传递
• 一些可以更新以响应训练的变量

## 设置

``````import tensorflow as tf
from datetime import datetime

``````

## 在 TensorFlow 中定义模型和层

``````class SimpleModule(tf.Module):
def __init__(self, name=None):
super().__init__(name=name)
self.a_variable = tf.Variable(5.0, name="train_me")
self.non_trainable_variable = tf.Variable(5.0, trainable=False, name="do_not_train_me")
def __call__(self, x):
return self.a_variable * x + self.non_trainable_variable

simple_module = SimpleModule(name="simple")

simple_module(tf.constant(5.0))
``````
```<tf.Tensor: shape=(), dtype=float32, numpy=30.0>
```

`__call__` 并无特殊之处，只是其行为与 Python 可调用对象类似；您可以使用任何函数来调用模型。

``````# All trainable variables
print("trainable variables:", simple_module.trainable_variables)
# Every variable
print("all variables:", simple_module.variables)
``````
```trainable variables: (<tf.Variable 'train_me:0' shape=() dtype=float32, numpy=5.0>,)
all variables: (<tf.Variable 'train_me:0' shape=() dtype=float32, numpy=5.0>, <tf.Variable 'do_not_train_me:0' shape=() dtype=float32, numpy=5.0>)
```

``````class Dense(tf.Module):
def __init__(self, in_features, out_features, name=None):
super().__init__(name=name)
self.w = tf.Variable(
tf.random.normal([in_features, out_features]), name='w')
self.b = tf.Variable(tf.zeros([out_features]), name='b')
def __call__(self, x):
y = tf.matmul(x, self.w) + self.b
return tf.nn.relu(y)
``````

``````class SequentialModule(tf.Module):
def __init__(self, name=None):
super().__init__(name=name)

self.dense_1 = Dense(in_features=3, out_features=3)
self.dense_2 = Dense(in_features=3, out_features=2)

def __call__(self, x):
x = self.dense_1(x)
return self.dense_2(x)

# You have made a model!
my_model = SequentialModule(name="the_model")

# Call it, with random results
print("Model results:", my_model(tf.constant([[2.0, 2.0, 2.0]])))
``````
```Model results: tf.Tensor([[4.758574 0.      ]], shape=(1, 2), dtype=float32)
```

`tf.Module` 实例将以递归方式自动收集分配给它的任何 `tf.Variable``tf.Module` 实例。这样，您可以使用单个模型实例管理 `tf.Module` 的集合，并保存和加载整个模型。

``````print("Submodules:", my_model.submodules)
``````
```Submodules: (<__main__.Dense object at 0x7f1c39e5ca20>, <__main__.Dense object at 0x7f1c39e5ca90>)
```
``````for var in my_model.variables:
print(var, "\n")
``````
```<tf.Variable 'b:0' shape=(3,) dtype=float32, numpy=array([0., 0., 0.], dtype=float32)>

<tf.Variable 'w:0' shape=(3, 3) dtype=float32, numpy=
array([[-0.3808288 , -0.31930852, -0.8358304 ],
[ 1.2048029 ,  0.30923682,  0.40288115],
[ 1.5672269 , -1.160634  , -0.44552004]], dtype=float32)>

<tf.Variable 'b:0' shape=(2,) dtype=float32, numpy=array([0., 0.], dtype=float32)>

<tf.Variable 'w:0' shape=(3, 2) dtype=float32, numpy=
array([[ 0.9950175 , -1.077783  ],
[-2.0974526 ,  0.92524624],
[ 0.9254379 , -1.3660892 ]], dtype=float32)>
```

### 等待创建变量

``````class FlexibleDenseModule(tf.Module):
# Note: No need for `in+features`
def __init__(self, out_features, name=None):
super().__init__(name=name)
self.is_built = False
self.out_features = out_features

def __call__(self, x):
# Create variables on first call.
if not self.is_built:
self.w = tf.Variable(
tf.random.normal([x.shape[-1], self.out_features]), name='w')
self.b = tf.Variable(tf.zeros([self.out_features]), name='b')
self.is_built = True

y = tf.matmul(x, self.w) + self.b
return tf.nn.relu(y)
``````
``````# Used in a module
class MySequentialModule(tf.Module):
def __init__(self, name=None):
super().__init__(name=name)

self.dense_1 = FlexibleDenseModule(out_features=3)
self.dense_2 = FlexibleDenseModule(out_features=2)

def __call__(self, x):
x = self.dense_1(x)
return self.dense_2(x)

my_model = MySequentialModule(name="the_model")
print("Model results:", my_model(tf.constant([[2.0, 2.0, 2.0]])))
``````
```Model results: tf.Tensor([[0. 0.]], shape=(1, 2), dtype=float32)
```

## 保存权重

``````chkp_path = "my_checkpoint"
checkpoint = tf.train.Checkpoint(model=my_model)
checkpoint.write(chkp_path)
checkpoint.write(chkp_path)
``````
```'my_checkpoint'
```

````ls my_checkpoint*`
```
```my_checkpoint.data-00000-of-00001  my_checkpoint.index
```

``````tf.train.list_variables(chkp_path)
``````
```[('_CHECKPOINTABLE_OBJECT_GRAPH', []),
('model/dense_1/b/.ATTRIBUTES/VARIABLE_VALUE', [3]),
('model/dense_1/w/.ATTRIBUTES/VARIABLE_VALUE', [3, 3]),
('model/dense_2/b/.ATTRIBUTES/VARIABLE_VALUE', [2]),
('model/dense_2/w/.ATTRIBUTES/VARIABLE_VALUE', [3, 2])]
```

``````new_model = MySequentialModule()
new_checkpoint = tf.train.Checkpoint(model=new_model)
new_checkpoint.restore("my_checkpoint")

# Should be the same result as above
new_model(tf.constant([[2.0, 2.0, 2.0]]))
``````
```<tf.Tensor: shape=(1, 2), dtype=float32, numpy=array([[0., 0.]], dtype=float32)>
```

## 保存函数

TensorFlow 可以在不使用原始 Python 对象的情况下运行模型，如 TensorFlow ServingTensorFlow Lite 中所见，甚至当您从 TensorFlow Hub 下载经过训练的模型时也是如此。

TensorFlow 需要了解如何执行 Python 中描述的计算，但不需要原始代码。为此，您可以创建一个计算图，如上一篇指南中所述。

``````class MySequentialModule(tf.Module):
def __init__(self, name=None):
super().__init__(name=name)

self.dense_1 = Dense(in_features=3, out_features=3)
self.dense_2 = Dense(in_features=3, out_features=2)

@tf.function
def __call__(self, x):
x = self.dense_1(x)
return self.dense_2(x)

# You have made a model with a graph!
my_model = MySequentialModule(name="the_model")
``````

``````print(my_model([[2.0, 2.0, 2.0]]))
print(my_model([[[2.0, 2.0, 2.0], [2.0, 2.0, 2.0]]]))
``````
```tf.Tensor([[0.        0.8042159]], shape=(1, 2), dtype=float32)
tf.Tensor(
[[[0.        0.8042159]
[0.        0.8042159]]], shape=(1, 2, 2), dtype=float32)
```

``````# Set up logging.
stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
logdir = "logs/func/%s" % stamp
writer = tf.summary.create_file_writer(logdir)

# Create a new model to get a fresh trace
# Otherwise the summary will not see the graph.
new_model = MySequentialModule()

# Bracket the function call with
# tf.summary.trace_on() and tf.summary.trace_export().
tf.summary.trace_on(graph=True, profiler=True)
# Call only one tf.function when tracing.
z = print(new_model(tf.constant([[2.0, 2.0, 2.0]])))
with writer.as_default():
tf.summary.trace_export(
name="my_func_trace",
step=0,
profiler_outdir=logdir)
``````
```WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py:1297: start (from tensorflow.python.eager.profiler) is deprecated and will be removed after 2020-07-01.
Instructions for updating:
tf.Tensor([[0. 0.]], shape=(1, 2), dtype=float32)
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py:1353: stop (from tensorflow.python.eager.profiler) is deprecated and will be removed after 2020-07-01.
Instructions for updating:
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py:1353: save (from tensorflow.python.eager.profiler) is deprecated and will be removed after 2020-07-01.
Instructions for updating:
`tf.python.eager.profiler` has deprecated, use `tf.profiler` instead.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/eager/profiler.py:151: maybe_create_event_file (from tensorflow.python.eager.profiler) is deprecated and will be removed after 2020-07-01.
Instructions for updating:
`tf.python.eager.profiler` has deprecated, use `tf.profiler` instead.
```

``````#docs_infra: no_execute
%tensorboard --logdir logs/func
``````

### 创建 `SavedModel`

``````tf.saved_model.save(my_model, "the_saved_model")
``````
```INFO:tensorflow:Assets written to: the_saved_model/assets
```
````# Inspect the in the directory`
`ls -l the_saved_model`
```
```total 24
drwxr-sr-x 2 kbuilder kokoro  4096 Feb 11 19:07 assets
-rw-rw-r-- 1 kbuilder kokoro 14140 Feb 11 19:07 saved_model.pb
drwxr-sr-x 2 kbuilder kokoro  4096 Feb 11 19:07 variables
```
````# The variables/ directory contains a checkpoint of the variables`
`ls -l the_saved_model/variables`
```
```total 8
-rw-rw-r-- 1 kbuilder kokoro 408 Feb 11 19:07 variables.data-00000-of-00001
-rw-rw-r-- 1 kbuilder kokoro 356 Feb 11 19:07 variables.index
```

`saved_model.pb` 文件是一个描述函数式 `tf.Graph`协议缓冲区

``````new_model = tf.saved_model.load("the_saved_model")
``````

``````isinstance(new_model, SequentialModule)
``````
```False
```

``````print(my_model([[2.0, 2.0, 2.0]]))
print(my_model([[[2.0, 2.0, 2.0], [2.0, 2.0, 2.0]]]))
``````
```tf.Tensor([[0.        0.8042159]], shape=(1, 2), dtype=float32)
tf.Tensor(
[[[0.        0.8042159]
[0.        0.8042159]]], shape=(1, 2, 2), dtype=float32)
```

## Keras 模型和层

### Keras 层

`tf.keras.layers.Layer` 是所有 Keras 层的基类，它继承自 `tf.Module`

``````class MyDense(tf.keras.layers.Layer):
# Adding **kwargs to support base Keras layer arguemnts
def __init__(self, in_features, out_features, **kwargs):
super().__init__(**kwargs)

# This will soon move to the build step; see below
self.w = tf.Variable(
tf.random.normal([in_features, out_features]), name='w')
self.b = tf.Variable(tf.zeros([out_features]), name='b')
def call(self, x):
y = tf.matmul(x, self.w) + self.b
return tf.nn.relu(y)

simple_layer = MyDense(name="simple", in_features=3, out_features=3)
``````

Keras 层有自己的 `__call__`，它会进行下一部分中所述的某些簿记，然后调用 `call()`。您应当不会看到功能上的任何变化。

``````simple_layer([[2.0, 2.0, 2.0]])
``````
```<tf.Tensor: shape=(1, 3), dtype=float32, numpy=array([[0.       , 1.6455138, 0.       ]], dtype=float32)>
```

### `build` 步骤

Keras 层具有额外的生命周期步骤，可让您在定义层时获得更高的灵活性。这是在 `build()` 函数中定义的。

`build` 仅被调用一次，而且是使用输入的形状调用的。它通常用于创建变量（权重）。

``````class FlexibleDense(tf.keras.layers.Layer):
# Note the added `**kwargs`, as Keras supports many arguments
def __init__(self, out_features, **kwargs):
super().__init__(**kwargs)
self.out_features = out_features

def build(self, input_shape):  # Create the state of the layer (weights)
self.w = tf.Variable(
tf.random.normal([input_shape[-1], self.out_features]), name='w')
self.b = tf.Variable(tf.zeros([self.out_features]), name='b')

def call(self, inputs):  # Defines the computation from inputs to outputs
return tf.matmul(inputs, self.w) + self.b

# Create the instance of the layer
flexible_dense = FlexibleDense(out_features=3)
``````

``````flexible_dense.variables
``````
```[]
```

``````# Call it, with predictably random results
print("Model results:", flexible_dense(tf.constant([[2.0, 2.0, 2.0], [3.0, 3.0, 3.0]])))
``````
```Model results: tf.Tensor(
[[ 2.5384042  -4.430261   -0.39644408]
[ 3.8076067  -6.6453915  -0.594666  ]], shape=(2, 3), dtype=float32)
```
``````flexible_dense.variables
``````
```[<tf.Variable 'flexible_dense/w:0' shape=(3, 3) dtype=float32, numpy=
array([[-0.22142771, -0.58348554,  0.07171401],
[ 0.4220801 , -1.597373  , -0.8835892 ],
[ 1.0685498 , -0.03427194,  0.6136532 ]], dtype=float32)>,
<tf.Variable 'flexible_dense/b:0' shape=(3,) dtype=float32, numpy=array([0., 0., 0.], dtype=float32)>]
```

``````try:
print("Model results:", flexible_dense(tf.constant([[2.0, 2.0, 2.0, 2.0]])))
except tf.errors.InvalidArgumentError as e:
print("Failed:", e)
``````
```Failed: Matrix size-incompatible: In[0]: [1,4], In[1]: [3,3] [Op:MatMul]
```

Keras 层具有许多额外的功能，包括：

• 可选损失
• 对指标的支持
• 对可选 `training` 参数的内置支持，用于区分训练和推断用途
• `get_config``from_config` 方法，允许您准确存储配置以在 Python 中克隆模型

### Keras 模型

``````class MySequentialModel(tf.keras.Model):
def __init__(self, name=None, **kwargs):
super().__init__(**kwargs)

self.dense_1 = FlexibleDense(out_features=3)
self.dense_2 = FlexibleDense(out_features=2)
def call(self, x):
x = self.dense_1(x)
return self.dense_2(x)

# You have made a Keras model!
my_sequential_model = MySequentialModel(name="the_model")

# Call it on a tensor, with random results
print("Model results:", my_sequential_model(tf.constant([[2.0, 2.0, 2.0]])))
``````
```Model results: tf.Tensor([[-12.241047    2.6145513]], shape=(1, 2), dtype=float32)
```

``````my_sequential_model.variables
``````
```[<tf.Variable 'my_sequential_model/flexible_dense_1/w:0' shape=(3, 3) dtype=float32, numpy=
array([[ 2.0393457 , -0.26025367, -0.45268586],
[-0.29897013,  0.18454784, -0.586858  ],
[ 1.5166968 ,  0.3722855 , -0.200079  ]], dtype=float32)>,
<tf.Variable 'my_sequential_model/flexible_dense_1/b:0' shape=(3,) dtype=float32, numpy=array([0., 0., 0.], dtype=float32)>,
<tf.Variable 'my_sequential_model/flexible_dense_2/w:0' shape=(3, 2) dtype=float32, numpy=
array([[-2.5438178 ,  0.2093776 ],
[ 0.63271874, -0.33846512],
[-1.5950202 , -0.5854196 ]], dtype=float32)>,
<tf.Variable 'my_sequential_model/flexible_dense_2/b:0' shape=(2,) dtype=float32, numpy=array([0., 0.], dtype=float32)>]
```
``````my_sequential_model.submodules
``````
```(<__main__.FlexibleDense at 0x7f1ccc738dd8>,
<__main__.FlexibleDense at 0x7f1ccc738588>)
```

``````inputs = tf.keras.Input(shape=[3,])

x = FlexibleDense(3)(inputs)
x = FlexibleDense(2)(x)

my_functional_model = tf.keras.Model(inputs=inputs, outputs=x)

my_functional_model.summary()
``````
```Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         [(None, 3)]               0
_________________________________________________________________
flexible_dense_3 (FlexibleDe (None, 3)                 12
_________________________________________________________________
flexible_dense_4 (FlexibleDe (None, 2)                 8
=================================================================
Total params: 20
Trainable params: 20
Non-trainable params: 0
_________________________________________________________________
```
``````my_functional_model(tf.constant([[2.0, 2.0, 2.0]]))
``````
```<tf.Tensor: shape=(1, 2), dtype=float32, numpy=array([[-17.974905 ,   2.8019438]], dtype=float32)>
```

## 保存 Keras 模型

Keras 模型也可以使用 `tf.saved_models.save()` 保存，因为它们是模块。但是，Keras 模型具有更方便的方法和其他功能。

``````my_sequential_model.save("exname_of_file")
``````
```INFO:tensorflow:Assets written to: exname_of_file/assets
```

``````reconstructed_model = tf.keras.models.load_model("exname_of_file")
``````
```WARNING:tensorflow:No training configuration found in save file, so the model was *not* compiled. Compile it manually.
```

Keras `SavedModels` 还可以保存指标、损失和优化器状态。

``````reconstructed_model(tf.constant([[2.0, 2.0, 2.0]]))
``````
```<tf.Tensor: shape=(1, 2), dtype=float32, numpy=array([[-12.241047 ,   2.6145513]], dtype=float32)>
```

# 后续步骤

`tf.module` 上构建的高级 API 的另一个示例是 DeepMind 的 Sonnet，其网站上有详细介绍。

[{ "type": "thumb-down", "id": "missingTheInformationINeed", "label":"没有我需要的信息" },{ "type": "thumb-down", "id": "tooComplicatedTooManySteps", "label":"太复杂/步骤太多" },{ "type": "thumb-down", "id": "outOfDate", "label":"内容需要更新" },{ "type": "thumb-down", "id": "translationIssue", "label":"翻译问题" },{ "type": "thumb-down", "id": "samplesCodeIssue", "label":"Samples / code issue" },{ "type": "thumb-down", "id": "otherDown", "label":"其他" }]
[{ "type": "thumb-up", "id": "easyToUnderstand", "label":"易于理解" },{ "type": "thumb-up", "id": "solvedMyProblem", "label":"解决了我的问题" },{ "type": "thumb-up", "id": "otherUp", "label":"其他" }]