ML Community Day is November 9! Join us for updates from TensorFlow, JAX, and more Learn more

如何使用 TF-Hub 构建简单的文本分类器

注:本教程使用已弃用的 TensorFlow 1 功能。有关完成此任务的新方式,请参阅 TensorFlow 2 版本

在 Google Colab 中运行 在 GitHub 上查看源代码 查看 TF Hub 模型

TF-Hub 是一个共享打包在可重用资源(尤其是预训练的模块)中的机器学习专业知识的平台。本教程分为两个主要部分。

入门:使用 TF-Hub 训练文本分类器

我们将使用 TF-Hub 文本嵌入向量模块训练具有合理基线准确率的简单情感分类器。然后,我们将分析预测结果以确保模型合理,并提出改进措施以提高准确率。

高级:迁移学习分析

在本部分中,我们将使用各种 TF-Hub 模块来比较它们对 Estimator 准确率的影响,并展示迁移学习的优势和缺陷。

可选前提条件

设置

# Install TF-Hub.
pip install seaborn

有关安装 Tensorflow 的更多详细信息,请访问 https://tensorflow.google.cn/install/

from absl import logging

import tensorflow as tf
import tensorflow_hub as hub
import matplotlib.pyplot as plt
import numpy as np
import os
import pandas as pd
import re
import seaborn as sns

开始

数据

我们将尝试解决 Large Movie Review Dataset v1.0 任务(Mass 等人,2011 年)。数据集由 IMDB 电影评论组成,这些评论使用从 1 到 10 的正数标记。任务是将评论标记为负面正面

# Load all files from a directory in a DataFrame.
def load_directory_data(directory):
  data = {}
  data["sentence"] = []
  data["sentiment"] = []
  for file_path in os.listdir(directory):
    with tf.io.gfile.GFile(os.path.join(directory, file_path), "r") as f:
      data["sentence"].append(f.read())
      data["sentiment"].append(re.match("\d+_(\d+)\.txt", file_path).group(1))
  return pd.DataFrame.from_dict(data)

# Merge positive and negative examples, add a polarity column and shuffle.
def load_dataset(directory):
  pos_df = load_directory_data(os.path.join(directory, "pos"))
  neg_df = load_directory_data(os.path.join(directory, "neg"))
  pos_df["polarity"] = 1
  neg_df["polarity"] = 0
  return pd.concat([pos_df, neg_df]).sample(frac=1).reset_index(drop=True)

# Download and process the dataset files.
def download_and_load_datasets(force_download=False):
  dataset = tf.keras.utils.get_file(
      fname="aclImdb.tar.gz", 
      origin="http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz", 
      extract=True)

  train_df = load_dataset(os.path.join(os.path.dirname(dataset), 
                                       "aclImdb", "train"))
  test_df = load_dataset(os.path.join(os.path.dirname(dataset), 
                                      "aclImdb", "test"))

  return train_df, test_df

# Reduce logging output.
logging.set_verbosity(logging.ERROR)

train_df, test_df = download_and_load_datasets()
train_df.head()
Downloading data from http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz
84131840/84125825 [==============================] - 7s 0us/step
84140032/84125825 [==============================] - 7s 0us/step

模型

输入函数

Estimator 框架提供了封装 Pandas 数据帧的输入函数

# Training input on the whole training set with no limit on training epochs.
train_input_fn = tf.compat.v1.estimator.inputs.pandas_input_fn(
    train_df, train_df["polarity"], num_epochs=None, shuffle=True)

# Prediction on the whole training set.
predict_train_input_fn = tf.compat.v1.estimator.inputs.pandas_input_fn(
    train_df, train_df["polarity"], shuffle=False)
# Prediction on the test set.
predict_test_input_fn = tf.compat.v1.estimator.inputs.pandas_input_fn(
    test_df, test_df["polarity"], shuffle=False)
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/util/lazy_loader.py:63: The name tf.estimator.inputs is deprecated. Please use tf.compat.v1.estimator.inputs instead.

WARNING:tensorflow:From /tmp/ipykernel_32734/2827882428.py:2: The name tf.estimator.inputs.pandas_input_fn is deprecated. Please use tf.compat.v1.estimator.inputs.pandas_input_fn instead.

特征列

TF-Hub 提供了一个特征列,此列在给定的文本特征上应用模块,并进一步传递模块的输出。在本教程中,我们将使用 nnlm-en-dim128 模块。对于本教程而言,最重要的事实如下:

  • 模块将字符串的一维张量中的一批句子作为输入。
  • 模块负责句子的预处理(例如,移除标点符号和在空格处拆分)。
  • 模块可以使用任何输入(例如,nnlm-en-dim128 将词汇中不存在的单词散列到约 20000 个桶中)。
embedded_text_feature_column = hub.text_embedding_column(
    key="sentence", 
    module_spec="https://tfhub.dev/google/nnlm-en-dim128/1")

Estimator

要实现分类,我们可以使用 DNN 分类器(请注意本教程结尾处有关标签函数的不同建模的补充说明)。

estimator = tf.estimator.DNNClassifier(
    hidden_units=[500, 100],
    feature_columns=[embedded_text_feature_column],
    n_classes=2,
    optimizer=tf.keras.optimizers.Adagrad(lr=0.003))
INFO:tensorflow:Using default config.
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
  "The `lr` argument is deprecated, use `learning_rate` instead.")
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpe3tgujsw
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpe3tgujsw
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpe3tgujsw', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpe3tgujsw', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}

训练

以合理的步骤数训练 Estimator。

# Training for 5,000 steps means 640,000 training examples with the default
# batch size. This is roughly equivalent to 25 epochs since the training dataset
# contains 25,000 examples.
estimator.train(input_fn=train_input_fn, steps=5000);
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/inputs/queues/feeding_queue_runner.py:65: QueueRunner.__init__ (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.
2021-08-13 20:40:16.266892: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:40:16.276144: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:40:16.277149: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/inputs/queues/feeding_queue_runner.py:65: QueueRunner.__init__ (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/inputs/queues/feeding_functions.py:491: add_queue_runner (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/inputs/queues/feeding_functions.py:491: add_queue_runner (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2021-08-13 20:40:16.553392: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/keras/optimizer_v2/adagrad.py:84: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/keras/optimizer_v2/adagrad.py:84: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
2021-08-13 20:40:17.001556: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-08-13 20:40:17.002360: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:40:17.003633: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:40:17.004702: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:40:17.617724: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:40:17.618921: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:40:17.620007: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:40:17.620986: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/monitored_session.py:907: start_queue_runners (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/monitored_session.py:907: start_queue_runners (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpe3tgujsw/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpe3tgujsw/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 0.687513, step = 0
INFO:tensorflow:loss = 0.687513, step = 0
INFO:tensorflow:global_step/sec: 107.11
INFO:tensorflow:global_step/sec: 107.11
INFO:tensorflow:loss = 0.6733377, step = 100 (0.935 sec)
INFO:tensorflow:loss = 0.6733377, step = 100 (0.935 sec)
INFO:tensorflow:global_step/sec: 116.201
INFO:tensorflow:global_step/sec: 116.201
INFO:tensorflow:loss = 0.6642936, step = 200 (0.861 sec)
INFO:tensorflow:loss = 0.6642936, step = 200 (0.861 sec)
INFO:tensorflow:global_step/sec: 119.491
INFO:tensorflow:global_step/sec: 119.491
INFO:tensorflow:loss = 0.6674034, step = 300 (0.837 sec)
INFO:tensorflow:loss = 0.6674034, step = 300 (0.837 sec)
INFO:tensorflow:global_step/sec: 120.868
INFO:tensorflow:global_step/sec: 120.868
INFO:tensorflow:loss = 0.6313045, step = 400 (0.827 sec)
INFO:tensorflow:loss = 0.6313045, step = 400 (0.827 sec)
INFO:tensorflow:global_step/sec: 120.609
INFO:tensorflow:global_step/sec: 120.609
INFO:tensorflow:loss = 0.6554498, step = 500 (0.829 sec)
INFO:tensorflow:loss = 0.6554498, step = 500 (0.829 sec)
INFO:tensorflow:global_step/sec: 123.863
INFO:tensorflow:global_step/sec: 123.863
INFO:tensorflow:loss = 0.62276417, step = 600 (0.808 sec)
INFO:tensorflow:loss = 0.62276417, step = 600 (0.808 sec)
INFO:tensorflow:global_step/sec: 126.951
INFO:tensorflow:global_step/sec: 126.951
INFO:tensorflow:loss = 0.6320714, step = 700 (0.787 sec)
INFO:tensorflow:loss = 0.6320714, step = 700 (0.787 sec)
INFO:tensorflow:global_step/sec: 132.579
INFO:tensorflow:global_step/sec: 132.579
INFO:tensorflow:loss = 0.6080027, step = 800 (0.755 sec)
INFO:tensorflow:loss = 0.6080027, step = 800 (0.755 sec)
INFO:tensorflow:global_step/sec: 124.868
INFO:tensorflow:global_step/sec: 124.868
INFO:tensorflow:loss = 0.59382325, step = 900 (0.800 sec)
INFO:tensorflow:loss = 0.59382325, step = 900 (0.800 sec)
INFO:tensorflow:global_step/sec: 125.014
INFO:tensorflow:global_step/sec: 125.014
INFO:tensorflow:loss = 0.61341786, step = 1000 (0.800 sec)
INFO:tensorflow:loss = 0.61341786, step = 1000 (0.800 sec)
INFO:tensorflow:global_step/sec: 123.892
INFO:tensorflow:global_step/sec: 123.892
INFO:tensorflow:loss = 0.5677572, step = 1100 (0.807 sec)
INFO:tensorflow:loss = 0.5677572, step = 1100 (0.807 sec)
INFO:tensorflow:global_step/sec: 123.708
INFO:tensorflow:global_step/sec: 123.708
INFO:tensorflow:loss = 0.548428, step = 1200 (0.808 sec)
INFO:tensorflow:loss = 0.548428, step = 1200 (0.808 sec)
INFO:tensorflow:global_step/sec: 126.699
INFO:tensorflow:global_step/sec: 126.699
INFO:tensorflow:loss = 0.5608907, step = 1300 (0.789 sec)
INFO:tensorflow:loss = 0.5608907, step = 1300 (0.789 sec)
INFO:tensorflow:global_step/sec: 131.366
INFO:tensorflow:global_step/sec: 131.366
INFO:tensorflow:loss = 0.53517467, step = 1400 (0.761 sec)
INFO:tensorflow:loss = 0.53517467, step = 1400 (0.761 sec)
INFO:tensorflow:global_step/sec: 128.917
INFO:tensorflow:global_step/sec: 128.917
INFO:tensorflow:loss = 0.48058185, step = 1500 (0.776 sec)
INFO:tensorflow:loss = 0.48058185, step = 1500 (0.776 sec)
INFO:tensorflow:global_step/sec: 130.529
INFO:tensorflow:global_step/sec: 130.529
INFO:tensorflow:loss = 0.4949787, step = 1600 (0.766 sec)
INFO:tensorflow:loss = 0.4949787, step = 1600 (0.766 sec)
INFO:tensorflow:global_step/sec: 132.762
INFO:tensorflow:global_step/sec: 132.762
INFO:tensorflow:loss = 0.5509753, step = 1700 (0.753 sec)
INFO:tensorflow:loss = 0.5509753, step = 1700 (0.753 sec)
INFO:tensorflow:global_step/sec: 131.669
INFO:tensorflow:global_step/sec: 131.669
INFO:tensorflow:loss = 0.49355528, step = 1800 (0.759 sec)
INFO:tensorflow:loss = 0.49355528, step = 1800 (0.759 sec)
INFO:tensorflow:global_step/sec: 130.105
INFO:tensorflow:global_step/sec: 130.105
INFO:tensorflow:loss = 0.47266114, step = 1900 (0.769 sec)
INFO:tensorflow:loss = 0.47266114, step = 1900 (0.769 sec)
INFO:tensorflow:global_step/sec: 129.402
INFO:tensorflow:global_step/sec: 129.402
INFO:tensorflow:loss = 0.5123067, step = 2000 (0.773 sec)
INFO:tensorflow:loss = 0.5123067, step = 2000 (0.773 sec)
INFO:tensorflow:global_step/sec: 128.92
INFO:tensorflow:global_step/sec: 128.92
INFO:tensorflow:loss = 0.45954973, step = 2100 (0.775 sec)
INFO:tensorflow:loss = 0.45954973, step = 2100 (0.775 sec)
INFO:tensorflow:global_step/sec: 136.499
INFO:tensorflow:global_step/sec: 136.499
INFO:tensorflow:loss = 0.49801522, step = 2200 (0.733 sec)
INFO:tensorflow:loss = 0.49801522, step = 2200 (0.733 sec)
INFO:tensorflow:global_step/sec: 126.258
INFO:tensorflow:global_step/sec: 126.258
INFO:tensorflow:loss = 0.4576664, step = 2300 (0.792 sec)
INFO:tensorflow:loss = 0.4576664, step = 2300 (0.792 sec)
INFO:tensorflow:global_step/sec: 130.495
INFO:tensorflow:global_step/sec: 130.495
INFO:tensorflow:loss = 0.45581916, step = 2400 (0.766 sec)
INFO:tensorflow:loss = 0.45581916, step = 2400 (0.766 sec)
INFO:tensorflow:global_step/sec: 126.424
INFO:tensorflow:global_step/sec: 126.424
INFO:tensorflow:loss = 0.48384622, step = 2500 (0.791 sec)
INFO:tensorflow:loss = 0.48384622, step = 2500 (0.791 sec)
INFO:tensorflow:global_step/sec: 126.648
INFO:tensorflow:global_step/sec: 126.648
INFO:tensorflow:loss = 0.5069208, step = 2600 (0.790 sec)
INFO:tensorflow:loss = 0.5069208, step = 2600 (0.790 sec)
INFO:tensorflow:global_step/sec: 127.332
INFO:tensorflow:global_step/sec: 127.332
INFO:tensorflow:loss = 0.44651443, step = 2700 (0.785 sec)
INFO:tensorflow:loss = 0.44651443, step = 2700 (0.785 sec)
INFO:tensorflow:global_step/sec: 129.317
INFO:tensorflow:global_step/sec: 129.317
INFO:tensorflow:loss = 0.48448595, step = 2800 (0.773 sec)
INFO:tensorflow:loss = 0.48448595, step = 2800 (0.773 sec)
INFO:tensorflow:global_step/sec: 126.6
INFO:tensorflow:global_step/sec: 126.6
INFO:tensorflow:loss = 0.48869008, step = 2900 (0.790 sec)
INFO:tensorflow:loss = 0.48869008, step = 2900 (0.790 sec)
INFO:tensorflow:global_step/sec: 126.959
INFO:tensorflow:global_step/sec: 126.959
INFO:tensorflow:loss = 0.53256905, step = 3000 (0.788 sec)
INFO:tensorflow:loss = 0.53256905, step = 3000 (0.788 sec)
INFO:tensorflow:global_step/sec: 125.017
INFO:tensorflow:global_step/sec: 125.017
INFO:tensorflow:loss = 0.47483972, step = 3100 (0.800 sec)
INFO:tensorflow:loss = 0.47483972, step = 3100 (0.800 sec)
INFO:tensorflow:global_step/sec: 125.995
INFO:tensorflow:global_step/sec: 125.995
INFO:tensorflow:loss = 0.5209049, step = 3200 (0.794 sec)
INFO:tensorflow:loss = 0.5209049, step = 3200 (0.794 sec)
INFO:tensorflow:global_step/sec: 128.15
INFO:tensorflow:global_step/sec: 128.15
INFO:tensorflow:loss = 0.53780186, step = 3300 (0.780 sec)
INFO:tensorflow:loss = 0.53780186, step = 3300 (0.780 sec)
INFO:tensorflow:global_step/sec: 129.148
INFO:tensorflow:global_step/sec: 129.148
INFO:tensorflow:loss = 0.4846049, step = 3400 (0.775 sec)
INFO:tensorflow:loss = 0.4846049, step = 3400 (0.775 sec)
INFO:tensorflow:global_step/sec: 130.737
INFO:tensorflow:global_step/sec: 130.737
INFO:tensorflow:loss = 0.4741645, step = 3500 (0.765 sec)
INFO:tensorflow:loss = 0.4741645, step = 3500 (0.765 sec)
INFO:tensorflow:global_step/sec: 130.97
INFO:tensorflow:global_step/sec: 130.97
INFO:tensorflow:loss = 0.5254193, step = 3600 (0.763 sec)
INFO:tensorflow:loss = 0.5254193, step = 3600 (0.763 sec)
INFO:tensorflow:global_step/sec: 131.407
INFO:tensorflow:global_step/sec: 131.407
INFO:tensorflow:loss = 0.47172904, step = 3700 (0.761 sec)
INFO:tensorflow:loss = 0.47172904, step = 3700 (0.761 sec)
INFO:tensorflow:global_step/sec: 133.013
INFO:tensorflow:global_step/sec: 133.013
INFO:tensorflow:loss = 0.4585212, step = 3800 (0.752 sec)
INFO:tensorflow:loss = 0.4585212, step = 3800 (0.752 sec)
INFO:tensorflow:global_step/sec: 128.897
INFO:tensorflow:global_step/sec: 128.897
INFO:tensorflow:loss = 0.43348682, step = 3900 (0.776 sec)
INFO:tensorflow:loss = 0.43348682, step = 3900 (0.776 sec)
INFO:tensorflow:global_step/sec: 130.915
INFO:tensorflow:global_step/sec: 130.915
INFO:tensorflow:loss = 0.44302213, step = 4000 (0.764 sec)
INFO:tensorflow:loss = 0.44302213, step = 4000 (0.764 sec)
INFO:tensorflow:global_step/sec: 129.915
INFO:tensorflow:global_step/sec: 129.915
INFO:tensorflow:loss = 0.43885598, step = 4100 (0.770 sec)
INFO:tensorflow:loss = 0.43885598, step = 4100 (0.770 sec)
INFO:tensorflow:global_step/sec: 126.966
INFO:tensorflow:global_step/sec: 126.966
INFO:tensorflow:loss = 0.43448776, step = 4200 (0.788 sec)
INFO:tensorflow:loss = 0.43448776, step = 4200 (0.788 sec)
INFO:tensorflow:global_step/sec: 123.314
INFO:tensorflow:global_step/sec: 123.314
INFO:tensorflow:loss = 0.54825383, step = 4300 (0.811 sec)
INFO:tensorflow:loss = 0.54825383, step = 4300 (0.811 sec)
INFO:tensorflow:global_step/sec: 124.105
INFO:tensorflow:global_step/sec: 124.105
INFO:tensorflow:loss = 0.4153015, step = 4400 (0.805 sec)
INFO:tensorflow:loss = 0.4153015, step = 4400 (0.805 sec)
INFO:tensorflow:global_step/sec: 125.483
INFO:tensorflow:global_step/sec: 125.483
INFO:tensorflow:loss = 0.43536562, step = 4500 (0.797 sec)
INFO:tensorflow:loss = 0.43536562, step = 4500 (0.797 sec)
INFO:tensorflow:global_step/sec: 125.797
INFO:tensorflow:global_step/sec: 125.797
INFO:tensorflow:loss = 0.3577639, step = 4600 (0.795 sec)
INFO:tensorflow:loss = 0.3577639, step = 4600 (0.795 sec)
INFO:tensorflow:global_step/sec: 128.293
INFO:tensorflow:global_step/sec: 128.293
INFO:tensorflow:loss = 0.5486526, step = 4700 (0.780 sec)
INFO:tensorflow:loss = 0.5486526, step = 4700 (0.780 sec)
INFO:tensorflow:global_step/sec: 129.408
INFO:tensorflow:global_step/sec: 129.408
INFO:tensorflow:loss = 0.42326072, step = 4800 (0.773 sec)
INFO:tensorflow:loss = 0.42326072, step = 4800 (0.773 sec)
INFO:tensorflow:global_step/sec: 129.509
INFO:tensorflow:global_step/sec: 129.509
INFO:tensorflow:loss = 0.4116273, step = 4900 (0.772 sec)
INFO:tensorflow:loss = 0.4116273, step = 4900 (0.772 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 5000...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 5000...
INFO:tensorflow:Saving checkpoints for 5000 into /tmp/tmpe3tgujsw/model.ckpt.
INFO:tensorflow:Saving checkpoints for 5000 into /tmp/tmpe3tgujsw/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 5000...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 5000...
INFO:tensorflow:Loss for final step: 0.4856371.
INFO:tensorflow:Loss for final step: 0.4856371.

预测

为训练集和测试集运行预测。

train_eval_result = estimator.evaluate(input_fn=predict_train_input_fn)
test_eval_result = estimator.evaluate(input_fn=predict_test_input_fn)

print("Training set accuracy: {accuracy}".format(**train_eval_result))
print("Test set accuracy: {accuracy}".format(**test_eval_result))
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2021-08-13 20:41:00.741021: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-08-13T20:41:01
INFO:tensorflow:Starting evaluation at 2021-08-13T20:41:01
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpe3tgujsw/model.ckpt-5000
2021-08-13 20:41:01.672050: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:01.672753: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:01.673237: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:01.673764: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:01.674210: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:01.674574: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Restoring parameters from /tmp/tmpe3tgujsw/model.ckpt-5000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Inference Time : 3.06648s
INFO:tensorflow:Inference Time : 3.06648s
INFO:tensorflow:Finished evaluation at 2021-08-13-20:41:04
INFO:tensorflow:Finished evaluation at 2021-08-13-20:41:04
INFO:tensorflow:Saving dict for global step 5000: accuracy = 0.79512, accuracy_baseline = 0.5, auc = 0.87581825, auc_precision_recall = 0.8763145, average_loss = 0.44353428, global_step = 5000, label/mean = 0.5, loss = 0.4435647, precision = 0.79711664, prediction/mean = 0.5018455, recall = 0.79176
INFO:tensorflow:Saving dict for global step 5000: accuracy = 0.79512, accuracy_baseline = 0.5, auc = 0.87581825, auc_precision_recall = 0.8763145, average_loss = 0.44353428, global_step = 5000, label/mean = 0.5, loss = 0.4435647, precision = 0.79711664, prediction/mean = 0.5018455, recall = 0.79176
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 5000: /tmp/tmpe3tgujsw/model.ckpt-5000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 5000: /tmp/tmpe3tgujsw/model.ckpt-5000
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
2021-08-13 20:41:04.868621: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-08-13T20:41:05
INFO:tensorflow:Starting evaluation at 2021-08-13T20:41:05
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpe3tgujsw/model.ckpt-5000
2021-08-13 20:41:05.784773: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:05.785332: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:05.785708: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:05.786138: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:05.786467: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:05.786766: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Restoring parameters from /tmp/tmpe3tgujsw/model.ckpt-5000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Inference Time : 2.93511s
INFO:tensorflow:Inference Time : 2.93511s
INFO:tensorflow:Finished evaluation at 2021-08-13-20:41:08
INFO:tensorflow:Finished evaluation at 2021-08-13-20:41:08
INFO:tensorflow:Saving dict for global step 5000: accuracy = 0.78776, accuracy_baseline = 0.5, auc = 0.8701355, auc_precision_recall = 0.8724754, average_loss = 0.4517151, global_step = 5000, label/mean = 0.5, loss = 0.45183375, precision = 0.7947394, prediction/mean = 0.49761578, recall = 0.77592
INFO:tensorflow:Saving dict for global step 5000: accuracy = 0.78776, accuracy_baseline = 0.5, auc = 0.8701355, auc_precision_recall = 0.8724754, average_loss = 0.4517151, global_step = 5000, label/mean = 0.5, loss = 0.45183375, precision = 0.7947394, prediction/mean = 0.49761578, recall = 0.77592
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 5000: /tmp/tmpe3tgujsw/model.ckpt-5000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 5000: /tmp/tmpe3tgujsw/model.ckpt-5000
Training set accuracy: 0.7951200008392334
Test set accuracy: 0.7877600193023682

混淆矩阵

我们可以目视检查混淆矩阵,以了解错误分类的分布。

def get_predictions(estimator, input_fn):
  return [x["class_ids"][0] for x in estimator.predict(input_fn=input_fn)]

LABELS = [
    "negative", "positive"
]

# Create a confusion matrix on training data.
cm = tf.math.confusion_matrix(train_df["polarity"], 
                              get_predictions(estimator, predict_train_input_fn))

# Normalize the confusion matrix so that each row sums to 1.
cm = tf.cast(cm, dtype=tf.float32)
cm = cm / tf.math.reduce_sum(cm, axis=1)[:, np.newaxis]

sns.heatmap(cm, annot=True, xticklabels=LABELS, yticklabels=LABELS);
plt.xlabel("Predicted");
plt.ylabel("True");
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2021-08-13 20:41:08.696052: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpe3tgujsw/model.ckpt-5000
2021-08-13 20:41:08.908444: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:08.909055: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:08.909509: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:08.910024: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:08.910454: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:08.910817: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Restoring parameters from /tmp/tmpe3tgujsw/model.ckpt-5000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
2021-08-13 20:41:11.121044: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:11.121677: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:11.122049: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:11.122461: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:11.122785: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:11.123077: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0

png

进一步改进

  1. 情感回归:我们使用分类器将每个样本分配给一个极性类。但实际上,我们还有另一个分类特征 - 情感。在这里,类实际上表示一个比例,并且基础值(正/负)可以很好地映射到连续范围内。我们可以通过计算回归(DNN 回归器)而不是分类(DNN 分类器)来利用此属性。
  2. 较大的模块:对于本教程而言,我们使用了较小的模块来限制内存使用。有些模块具有更大的词汇和更大的嵌入向量空间,可以提供更多的准确率点。
  3. 参数调节:我们可以通过调节元参数(例如学习率或步骤数)来提高准确率,尤其是在使用不同模块的情况下。如果我们想获得任何合理的结果,那么验证集非常重要,因为这样可以轻松建立一个模型来学习预测训练数据,而无需很好地泛化到测试集。
  4. 更复杂的模型:我们使用了一个通过嵌入每个单词并随后将其与平均值相结合来计算句子嵌入向量的模块。此外,也可以使用序贯模块(例如 Universal Sentence Encoder 模块)来更好地捕获句子的性质。或者,使用两个或多个 TF-Hub 模块的集合。
  5. 正则化:为了避免过拟合,我们可以尝试使用执行某种正则化的优化器,例如近端 Adagrad 优化器

高级:迁移学习分析

迁移学习可以节省训练资源,即使基于小数据集训练也可以实现良好的模型泛化。在这一部分中,我们将通过使用两个不同的 TF-Hub 模块进行训练来演示这一点:

  • nnlm-en-dim128 - 预训练的文本嵌入向量模块;
  • random-nnlm-en-dim128 - 文本嵌入向量模块,其词汇和网络与 nnlm-en-dim128 相同,但权重只是随机初始化的,从未基于真实数据进行训练。

在以下两种模式下训练:

  • 仅训练分类器(即冻结模块),以及
  • 将分类器与模块一起训练

我们运行一些训练和评估来查看使用各种模块如何影响准确率。

def train_and_evaluate_with_module(hub_module, train_module=False):
  embedded_text_feature_column = hub.text_embedding_column(
      key="sentence", module_spec=hub_module, trainable=train_module)

  estimator = tf.estimator.DNNClassifier(
      hidden_units=[500, 100],
      feature_columns=[embedded_text_feature_column],
      n_classes=2,
      optimizer=tf.keras.optimizers.Adagrad(learning_rate=0.003))

  estimator.train(input_fn=train_input_fn, steps=1000)

  train_eval_result = estimator.evaluate(input_fn=predict_train_input_fn)
  test_eval_result = estimator.evaluate(input_fn=predict_test_input_fn)

  training_set_accuracy = train_eval_result["accuracy"]
  test_set_accuracy = test_eval_result["accuracy"]

  return {
      "Training accuracy": training_set_accuracy,
      "Test accuracy": test_set_accuracy
  }


results = {}
results["nnlm-en-dim128"] = train_and_evaluate_with_module(
    "https://tfhub.dev/google/nnlm-en-dim128/1")
results["nnlm-en-dim128-with-module-training"] = train_and_evaluate_with_module(
    "https://tfhub.dev/google/nnlm-en-dim128/1", True)
results["random-nnlm-en-dim128"] = train_and_evaluate_with_module(
    "https://tfhub.dev/google/random-nnlm-en-dim128/1")
results["random-nnlm-en-dim128-with-module-training"] = train_and_evaluate_with_module(
    "https://tfhub.dev/google/random-nnlm-en-dim128/1", True)
INFO:tensorflow:Using default config.
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpf9le4a8r
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpf9le4a8r
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpf9le4a8r', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpf9le4a8r', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2021-08-13 20:41:11.560531: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
2021-08-13 20:41:11.964496: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:11.965024: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:11.965340: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:11.965801: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:11.966179: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:11.966495: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpf9le4a8r/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpf9le4a8r/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 0.699322, step = 0
INFO:tensorflow:loss = 0.699322, step = 0
INFO:tensorflow:global_step/sec: 121.363
INFO:tensorflow:global_step/sec: 121.363
INFO:tensorflow:loss = 0.68454915, step = 100 (0.826 sec)
INFO:tensorflow:loss = 0.68454915, step = 100 (0.826 sec)
INFO:tensorflow:global_step/sec: 126.323
INFO:tensorflow:global_step/sec: 126.323
INFO:tensorflow:loss = 0.6725577, step = 200 (0.792 sec)
INFO:tensorflow:loss = 0.6725577, step = 200 (0.792 sec)
INFO:tensorflow:global_step/sec: 130.085
INFO:tensorflow:global_step/sec: 130.085
INFO:tensorflow:loss = 0.6617315, step = 300 (0.769 sec)
INFO:tensorflow:loss = 0.6617315, step = 300 (0.769 sec)
INFO:tensorflow:global_step/sec: 126.969
INFO:tensorflow:global_step/sec: 126.969
INFO:tensorflow:loss = 0.6477518, step = 400 (0.788 sec)
INFO:tensorflow:loss = 0.6477518, step = 400 (0.788 sec)
INFO:tensorflow:global_step/sec: 126.931
INFO:tensorflow:global_step/sec: 126.931
INFO:tensorflow:loss = 0.64095956, step = 500 (0.788 sec)
INFO:tensorflow:loss = 0.64095956, step = 500 (0.788 sec)
INFO:tensorflow:global_step/sec: 126.836
INFO:tensorflow:global_step/sec: 126.836
INFO:tensorflow:loss = 0.63748664, step = 600 (0.789 sec)
INFO:tensorflow:loss = 0.63748664, step = 600 (0.789 sec)
INFO:tensorflow:global_step/sec: 130.362
INFO:tensorflow:global_step/sec: 130.362
INFO:tensorflow:loss = 0.6191129, step = 700 (0.767 sec)
INFO:tensorflow:loss = 0.6191129, step = 700 (0.767 sec)
INFO:tensorflow:global_step/sec: 127.225
INFO:tensorflow:global_step/sec: 127.225
INFO:tensorflow:loss = 0.6288119, step = 800 (0.786 sec)
INFO:tensorflow:loss = 0.6288119, step = 800 (0.786 sec)
INFO:tensorflow:global_step/sec: 126.582
INFO:tensorflow:global_step/sec: 126.582
INFO:tensorflow:loss = 0.60983485, step = 900 (0.790 sec)
INFO:tensorflow:loss = 0.60983485, step = 900 (0.790 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1000...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1000...
INFO:tensorflow:Saving checkpoints for 1000 into /tmp/tmpf9le4a8r/model.ckpt.
INFO:tensorflow:Saving checkpoints for 1000 into /tmp/tmpf9le4a8r/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1000...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1000...
INFO:tensorflow:Loss for final step: 0.5810377.
INFO:tensorflow:Loss for final step: 0.5810377.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2021-08-13 20:41:22.590203: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-08-13T20:41:23
INFO:tensorflow:Starting evaluation at 2021-08-13T20:41:23
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpf9le4a8r/model.ckpt-1000
2021-08-13 20:41:23.447086: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:23.447635: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:23.447970: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:23.448380: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:23.448718: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:23.449008: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Restoring parameters from /tmp/tmpf9le4a8r/model.ckpt-1000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Inference Time : 2.96935s
INFO:tensorflow:Inference Time : 2.96935s
INFO:tensorflow:Finished evaluation at 2021-08-13-20:41:26
INFO:tensorflow:Finished evaluation at 2021-08-13-20:41:26
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.7332, accuracy_baseline = 0.5, auc = 0.8060533, auc_precision_recall = 0.8051846, average_loss = 0.5826626, global_step = 1000, label/mean = 0.5, loss = 0.5825906, precision = 0.7437699, prediction/mean = 0.49652255, recall = 0.71152
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.7332, accuracy_baseline = 0.5, auc = 0.8060533, auc_precision_recall = 0.8051846, average_loss = 0.5826626, global_step = 1000, label/mean = 0.5, loss = 0.5825906, precision = 0.7437699, prediction/mean = 0.49652255, recall = 0.71152
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmpf9le4a8r/model.ckpt-1000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmpf9le4a8r/model.ckpt-1000
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2021-08-13 20:41:26.561893: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-08-13T20:41:27
INFO:tensorflow:Starting evaluation at 2021-08-13T20:41:27
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpf9le4a8r/model.ckpt-1000
2021-08-13 20:41:27.443614: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:27.444199: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:27.444632: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:27.445103: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:27.445529: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:27.445898: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Restoring parameters from /tmp/tmpf9le4a8r/model.ckpt-1000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Inference Time : 2.87107s
INFO:tensorflow:Inference Time : 2.87107s
INFO:tensorflow:Finished evaluation at 2021-08-13-20:41:30
INFO:tensorflow:Finished evaluation at 2021-08-13-20:41:30
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.72248, accuracy_baseline = 0.5, auc = 0.7976883, auc_precision_recall = 0.79631793, average_loss = 0.5873021, global_step = 1000, label/mean = 0.5, loss = 0.5873377, precision = 0.73773295, prediction/mean = 0.4935559, recall = 0.6904
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.72248, accuracy_baseline = 0.5, auc = 0.7976883, auc_precision_recall = 0.79631793, average_loss = 0.5873021, global_step = 1000, label/mean = 0.5, loss = 0.5873377, precision = 0.73773295, prediction/mean = 0.4935559, recall = 0.6904
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmpf9le4a8r/model.ckpt-1000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmpf9le4a8r/model.ckpt-1000
INFO:tensorflow:Using default config.
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmphkcp2fqp
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmphkcp2fqp
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmphkcp2fqp', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmphkcp2fqp', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2021-08-13 20:41:30.295174: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
2021-08-13 20:41:30.703796: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:30.704298: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:30.704632: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:30.705039: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:30.705372: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:30.705690: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmphkcp2fqp/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmphkcp2fqp/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 0.69634485, step = 0
INFO:tensorflow:loss = 0.69634485, step = 0
INFO:tensorflow:global_step/sec: 116.842
INFO:tensorflow:global_step/sec: 116.842
INFO:tensorflow:loss = 0.68452764, step = 100 (0.858 sec)
INFO:tensorflow:loss = 0.68452764, step = 100 (0.858 sec)
INFO:tensorflow:global_step/sec: 123.232
INFO:tensorflow:global_step/sec: 123.232
INFO:tensorflow:loss = 0.6717119, step = 200 (0.811 sec)
INFO:tensorflow:loss = 0.6717119, step = 200 (0.811 sec)
INFO:tensorflow:global_step/sec: 124.142
INFO:tensorflow:global_step/sec: 124.142
INFO:tensorflow:loss = 0.6545883, step = 300 (0.806 sec)
INFO:tensorflow:loss = 0.6545883, step = 300 (0.806 sec)
INFO:tensorflow:global_step/sec: 126.136
INFO:tensorflow:global_step/sec: 126.136
INFO:tensorflow:loss = 0.65944564, step = 400 (0.793 sec)
INFO:tensorflow:loss = 0.65944564, step = 400 (0.793 sec)
INFO:tensorflow:global_step/sec: 127.896
INFO:tensorflow:global_step/sec: 127.896
INFO:tensorflow:loss = 0.6310526, step = 500 (0.782 sec)
INFO:tensorflow:loss = 0.6310526, step = 500 (0.782 sec)
INFO:tensorflow:global_step/sec: 126.276
INFO:tensorflow:global_step/sec: 126.276
INFO:tensorflow:loss = 0.6136538, step = 600 (0.792 sec)
INFO:tensorflow:loss = 0.6136538, step = 600 (0.792 sec)
INFO:tensorflow:global_step/sec: 124.834
INFO:tensorflow:global_step/sec: 124.834
INFO:tensorflow:loss = 0.630298, step = 700 (0.801 sec)
INFO:tensorflow:loss = 0.630298, step = 700 (0.801 sec)
INFO:tensorflow:global_step/sec: 126.795
INFO:tensorflow:global_step/sec: 126.795
INFO:tensorflow:loss = 0.61833024, step = 800 (0.789 sec)
INFO:tensorflow:loss = 0.61833024, step = 800 (0.789 sec)
INFO:tensorflow:global_step/sec: 128.185
INFO:tensorflow:global_step/sec: 128.185
INFO:tensorflow:loss = 0.5946367, step = 900 (0.780 sec)
INFO:tensorflow:loss = 0.5946367, step = 900 (0.780 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1000...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1000...
INFO:tensorflow:Saving checkpoints for 1000 into /tmp/tmphkcp2fqp/model.ckpt.
INFO:tensorflow:Saving checkpoints for 1000 into /tmp/tmphkcp2fqp/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1000...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1000...
INFO:tensorflow:Loss for final step: 0.5756554.
INFO:tensorflow:Loss for final step: 0.5756554.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
2021-08-13 20:41:41.440876: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-08-13T20:41:42
INFO:tensorflow:Starting evaluation at 2021-08-13T20:41:42
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmphkcp2fqp/model.ckpt-1000
2021-08-13 20:41:42.535243: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:42.535765: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:42.536097: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:42.536507: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:42.536825: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:42.537126: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Restoring parameters from /tmp/tmphkcp2fqp/model.ckpt-1000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Inference Time : 3.04638s
INFO:tensorflow:Inference Time : 3.04638s
INFO:tensorflow:Finished evaluation at 2021-08-13-20:41:45
INFO:tensorflow:Finished evaluation at 2021-08-13-20:41:45
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.73196, accuracy_baseline = 0.5, auc = 0.8071692, auc_precision_recall = 0.80879474, average_loss = 0.5813461, global_step = 1000, label/mean = 0.5, loss = 0.581275, precision = 0.75307673, prediction/mean = 0.48986855, recall = 0.69024
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.73196, accuracy_baseline = 0.5, auc = 0.8071692, auc_precision_recall = 0.80879474, average_loss = 0.5813461, global_step = 1000, label/mean = 0.5, loss = 0.581275, precision = 0.75307673, prediction/mean = 0.48986855, recall = 0.69024
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmphkcp2fqp/model.ckpt-1000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmphkcp2fqp/model.ckpt-1000
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
2021-08-13 20:41:45.718601: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-08-13T20:41:46
INFO:tensorflow:Starting evaluation at 2021-08-13T20:41:46
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmphkcp2fqp/model.ckpt-1000
2021-08-13 20:41:46.637898: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:46.638426: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:46.638763: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:46.639185: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:46.639533: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:46.639822: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Restoring parameters from /tmp/tmphkcp2fqp/model.ckpt-1000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Inference Time : 3.08011s
INFO:tensorflow:Inference Time : 3.08011s
INFO:tensorflow:Finished evaluation at 2021-08-13-20:41:49
INFO:tensorflow:Finished evaluation at 2021-08-13-20:41:49
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.72404, accuracy_baseline = 0.5, auc = 0.79833513, auc_precision_recall = 0.79977196, average_loss = 0.5863028, global_step = 1000, label/mean = 0.5, loss = 0.58635324, precision = 0.7493101, prediction/mean = 0.48684335, recall = 0.67336
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.72404, accuracy_baseline = 0.5, auc = 0.79833513, auc_precision_recall = 0.79977196, average_loss = 0.5863028, global_step = 1000, label/mean = 0.5, loss = 0.58635324, precision = 0.7493101, prediction/mean = 0.48684335, recall = 0.67336
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmphkcp2fqp/model.ckpt-1000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmphkcp2fqp/model.ckpt-1000
INFO:tensorflow:Using default config.
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpf348so1i
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpf348so1i
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpf348so1i', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpf348so1i', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2021-08-13 20:41:57.196413: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
2021-08-13 20:41:57.578718: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:57.579294: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:57.579640: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:57.580068: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:57.580417: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:41:57.580716: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpf348so1i/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpf348so1i/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 0.7094606, step = 0
INFO:tensorflow:loss = 0.7094606, step = 0
INFO:tensorflow:global_step/sec: 116.195
INFO:tensorflow:global_step/sec: 116.195
INFO:tensorflow:loss = 0.68820524, step = 100 (0.862 sec)
INFO:tensorflow:loss = 0.68820524, step = 100 (0.862 sec)
INFO:tensorflow:global_step/sec: 129.514
INFO:tensorflow:global_step/sec: 129.514
INFO:tensorflow:loss = 0.6394646, step = 200 (0.772 sec)
INFO:tensorflow:loss = 0.6394646, step = 200 (0.772 sec)
INFO:tensorflow:global_step/sec: 130.2
INFO:tensorflow:global_step/sec: 130.2
INFO:tensorflow:loss = 0.6234229, step = 300 (0.768 sec)
INFO:tensorflow:loss = 0.6234229, step = 300 (0.768 sec)
INFO:tensorflow:global_step/sec: 127.69
INFO:tensorflow:global_step/sec: 127.69
INFO:tensorflow:loss = 0.62564087, step = 400 (0.784 sec)
INFO:tensorflow:loss = 0.62564087, step = 400 (0.784 sec)
INFO:tensorflow:global_step/sec: 133.624
INFO:tensorflow:global_step/sec: 133.624
INFO:tensorflow:loss = 0.57936853, step = 500 (0.748 sec)
INFO:tensorflow:loss = 0.57936853, step = 500 (0.748 sec)
INFO:tensorflow:global_step/sec: 131.507
INFO:tensorflow:global_step/sec: 131.507
INFO:tensorflow:loss = 0.5679269, step = 600 (0.760 sec)
INFO:tensorflow:loss = 0.5679269, step = 600 (0.760 sec)
INFO:tensorflow:global_step/sec: 134.151
INFO:tensorflow:global_step/sec: 134.151
INFO:tensorflow:loss = 0.60456765, step = 700 (0.745 sec)
INFO:tensorflow:loss = 0.60456765, step = 700 (0.745 sec)
INFO:tensorflow:global_step/sec: 128.369
INFO:tensorflow:global_step/sec: 128.369
INFO:tensorflow:loss = 0.6112944, step = 800 (0.779 sec)
INFO:tensorflow:loss = 0.6112944, step = 800 (0.779 sec)
INFO:tensorflow:global_step/sec: 130.013
INFO:tensorflow:global_step/sec: 130.013
INFO:tensorflow:loss = 0.5807582, step = 900 (0.769 sec)
INFO:tensorflow:loss = 0.5807582, step = 900 (0.769 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1000...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1000...
INFO:tensorflow:Saving checkpoints for 1000 into /tmp/tmpf348so1i/model.ckpt.
INFO:tensorflow:Saving checkpoints for 1000 into /tmp/tmpf348so1i/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1000...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1000...
INFO:tensorflow:Loss for final step: 0.5828902.
INFO:tensorflow:Loss for final step: 0.5828902.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2021-08-13 20:42:08.197525: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-08-13T20:42:09
INFO:tensorflow:Starting evaluation at 2021-08-13T20:42:09
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpf348so1i/model.ckpt-1000
2021-08-13 20:42:09.070954: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:09.071642: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:09.072180: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:09.072742: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:09.073191: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:09.073480: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Restoring parameters from /tmp/tmpf348so1i/model.ckpt-1000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Inference Time : 2.87427s
INFO:tensorflow:Inference Time : 2.87427s
INFO:tensorflow:Finished evaluation at 2021-08-13-20:42:11
INFO:tensorflow:Finished evaluation at 2021-08-13-20:42:11
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.67724, accuracy_baseline = 0.5, auc = 0.7452364, auc_precision_recall = 0.73742837, average_loss = 0.59576946, global_step = 1000, label/mean = 0.5, loss = 0.5955765, precision = 0.68863344, prediction/mean = 0.48327833, recall = 0.64704
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.67724, accuracy_baseline = 0.5, auc = 0.7452364, auc_precision_recall = 0.73742837, average_loss = 0.59576946, global_step = 1000, label/mean = 0.5, loss = 0.5955765, precision = 0.68863344, prediction/mean = 0.48327833, recall = 0.64704
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmpf348so1i/model.ckpt-1000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmpf348so1i/model.ckpt-1000
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2021-08-13 20:42:12.308404: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-08-13T20:42:13
INFO:tensorflow:Starting evaluation at 2021-08-13T20:42:13
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpf348so1i/model.ckpt-1000
2021-08-13 20:42:13.195439: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:13.196114: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:13.196668: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:13.197259: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:13.197804: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:13.198141: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Restoring parameters from /tmp/tmpf348so1i/model.ckpt-1000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Inference Time : 2.98069s
INFO:tensorflow:Inference Time : 2.98069s
INFO:tensorflow:Finished evaluation at 2021-08-13-20:42:16
INFO:tensorflow:Finished evaluation at 2021-08-13-20:42:16
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.66584, accuracy_baseline = 0.5, auc = 0.72593313, auc_precision_recall = 0.7166992, average_loss = 0.6109817, global_step = 1000, label/mean = 0.5, loss = 0.61089945, precision = 0.6774829, prediction/mean = 0.48421034, recall = 0.63304
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.66584, accuracy_baseline = 0.5, auc = 0.72593313, auc_precision_recall = 0.7166992, average_loss = 0.6109817, global_step = 1000, label/mean = 0.5, loss = 0.61089945, precision = 0.6774829, prediction/mean = 0.48421034, recall = 0.63304
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmpf348so1i/model.ckpt-1000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmpf348so1i/model.ckpt-1000
INFO:tensorflow:Using default config.
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpahw9qrg7
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpahw9qrg7
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpahw9qrg7', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpahw9qrg7', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2021-08-13 20:42:16.156173: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
2021-08-13 20:42:16.548917: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:16.549444: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:16.549794: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:16.550184: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:16.550549: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:16.550853: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpahw9qrg7/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpahw9qrg7/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 0.81187314, step = 0
INFO:tensorflow:loss = 0.81187314, step = 0
INFO:tensorflow:global_step/sec: 120.882
INFO:tensorflow:global_step/sec: 120.882
INFO:tensorflow:loss = 0.692839, step = 100 (0.829 sec)
INFO:tensorflow:loss = 0.692839, step = 100 (0.829 sec)
INFO:tensorflow:global_step/sec: 127.888
INFO:tensorflow:global_step/sec: 127.888
INFO:tensorflow:loss = 0.6833136, step = 200 (0.782 sec)
INFO:tensorflow:loss = 0.6833136, step = 200 (0.782 sec)
INFO:tensorflow:global_step/sec: 129.595
INFO:tensorflow:global_step/sec: 129.595
INFO:tensorflow:loss = 0.5580744, step = 300 (0.771 sec)
INFO:tensorflow:loss = 0.5580744, step = 300 (0.771 sec)
INFO:tensorflow:global_step/sec: 127.788
INFO:tensorflow:global_step/sec: 127.788
INFO:tensorflow:loss = 0.6532499, step = 400 (0.783 sec)
INFO:tensorflow:loss = 0.6532499, step = 400 (0.783 sec)
INFO:tensorflow:global_step/sec: 130.369
INFO:tensorflow:global_step/sec: 130.369
INFO:tensorflow:loss = 0.6274036, step = 500 (0.767 sec)
INFO:tensorflow:loss = 0.6274036, step = 500 (0.767 sec)
INFO:tensorflow:global_step/sec: 131.942
INFO:tensorflow:global_step/sec: 131.942
INFO:tensorflow:loss = 0.6237198, step = 600 (0.758 sec)
INFO:tensorflow:loss = 0.6237198, step = 600 (0.758 sec)
INFO:tensorflow:global_step/sec: 129.141
INFO:tensorflow:global_step/sec: 129.141
INFO:tensorflow:loss = 0.6329469, step = 700 (0.775 sec)
INFO:tensorflow:loss = 0.6329469, step = 700 (0.775 sec)
INFO:tensorflow:global_step/sec: 122.98
INFO:tensorflow:global_step/sec: 122.98
INFO:tensorflow:loss = 0.6036942, step = 800 (0.813 sec)
INFO:tensorflow:loss = 0.6036942, step = 800 (0.813 sec)
INFO:tensorflow:global_step/sec: 124.563
INFO:tensorflow:global_step/sec: 124.563
INFO:tensorflow:loss = 0.59442043, step = 900 (0.802 sec)
INFO:tensorflow:loss = 0.59442043, step = 900 (0.802 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1000...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1000...
INFO:tensorflow:Saving checkpoints for 1000 into /tmp/tmpahw9qrg7/model.ckpt.
INFO:tensorflow:Saving checkpoints for 1000 into /tmp/tmpahw9qrg7/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1000...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1000...
INFO:tensorflow:Loss for final step: 0.5860566.
INFO:tensorflow:Loss for final step: 0.5860566.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2021-08-13 20:42:27.182320: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-08-13T20:42:28
INFO:tensorflow:Starting evaluation at 2021-08-13T20:42:28
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpahw9qrg7/model.ckpt-1000
2021-08-13 20:42:28.102255: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:28.102959: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:28.103435: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:28.103904: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:28.104332: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:28.104648: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Restoring parameters from /tmp/tmpahw9qrg7/model.ckpt-1000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Inference Time : 3.01503s
INFO:tensorflow:Inference Time : 3.01503s
INFO:tensorflow:Finished evaluation at 2021-08-13-20:42:31
INFO:tensorflow:Finished evaluation at 2021-08-13-20:42:31
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.67748, accuracy_baseline = 0.5, auc = 0.74443555, auc_precision_recall = 0.7340101, average_loss = 0.5974762, global_step = 1000, label/mean = 0.5, loss = 0.5971627, precision = 0.6931314, prediction/mean = 0.47900605, recall = 0.63696
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.67748, accuracy_baseline = 0.5, auc = 0.74443555, auc_precision_recall = 0.7340101, average_loss = 0.5974762, global_step = 1000, label/mean = 0.5, loss = 0.5971627, precision = 0.6931314, prediction/mean = 0.47900605, recall = 0.63696
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmpahw9qrg7/model.ckpt-1000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmpahw9qrg7/model.ckpt-1000
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
2021-08-13 20:42:31.248494: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-08-13T20:42:32
INFO:tensorflow:Starting evaluation at 2021-08-13T20:42:32
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpahw9qrg7/model.ckpt-1000
2021-08-13 20:42:32.153707: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:32.154247: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:32.154587: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:32.154986: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:32.155348: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:32.155639: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Restoring parameters from /tmp/tmpahw9qrg7/model.ckpt-1000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Inference Time : 3.09068s
INFO:tensorflow:Inference Time : 3.09068s
INFO:tensorflow:Finished evaluation at 2021-08-13-20:42:35
INFO:tensorflow:Finished evaluation at 2021-08-13-20:42:35
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.66584, accuracy_baseline = 0.5, auc = 0.7268084, auc_precision_recall = 0.71749336, average_loss = 0.611086, global_step = 1000, label/mean = 0.5, loss = 0.6109853, precision = 0.6804178, prediction/mean = 0.4796192, recall = 0.62544
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.66584, accuracy_baseline = 0.5, auc = 0.7268084, auc_precision_recall = 0.71749336, average_loss = 0.611086, global_step = 1000, label/mean = 0.5, loss = 0.6109853, precision = 0.6804178, prediction/mean = 0.4796192, recall = 0.62544
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmpahw9qrg7/model.ckpt-1000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: /tmp/tmpahw9qrg7/model.ckpt-1000

我们来看看结果。

pd.DataFrame.from_dict(results, orient="index")

我们已经看到了一些模式,但首先我们应当建立测试集的基线准确率 - 通过仅输出最具代表性的类的标签可以实现的下限:

estimator.evaluate(input_fn=predict_test_input_fn)["accuracy_baseline"]
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2021-08-13 20:42:35.237979: W tensorflow/core/common_runtime/graph_constructor.cc:1511] Importing a graph with a lower producer version 26 into an existing graph with producer version 808. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-08-13T20:42:36
INFO:tensorflow:Starting evaluation at 2021-08-13T20:42:36
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpe3tgujsw/model.ckpt-5000
2021-08-13 20:42:36.436512: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:36.437181: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:36.437787: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:36.438395: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:36.438940: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-13 20:42:36.439247: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
INFO:tensorflow:Restoring parameters from /tmp/tmpe3tgujsw/model.ckpt-5000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Inference Time : 2.84018s
INFO:tensorflow:Inference Time : 2.84018s
INFO:tensorflow:Finished evaluation at 2021-08-13-20:42:39
INFO:tensorflow:Finished evaluation at 2021-08-13-20:42:39
INFO:tensorflow:Saving dict for global step 5000: accuracy = 0.78776, accuracy_baseline = 0.5, auc = 0.8701355, auc_precision_recall = 0.8724754, average_loss = 0.4517151, global_step = 5000, label/mean = 0.5, loss = 0.45183375, precision = 0.7947394, prediction/mean = 0.49761578, recall = 0.77592
INFO:tensorflow:Saving dict for global step 5000: accuracy = 0.78776, accuracy_baseline = 0.5, auc = 0.8701355, auc_precision_recall = 0.8724754, average_loss = 0.4517151, global_step = 5000, label/mean = 0.5, loss = 0.45183375, precision = 0.7947394, prediction/mean = 0.49761578, recall = 0.77592
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 5000: /tmp/tmpe3tgujsw/model.ckpt-5000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 5000: /tmp/tmpe3tgujsw/model.ckpt-5000
0.5

分配最具代表性的类将使我们的准确率达到 50%。这里有几件事需要注意:

  1. 也许会令人惊讶,但此时仍然可以在固定的随机嵌入向量上学习模型。原因是,即使字典中的每个单词都映射到随机向量,Estimator 也可以仅仅使用其全连接层来分隔空间。
  2. 与仅训练分类器相比,允许使用随机嵌入向量训练模块有助于提高训练和测试的准确率。
  3. 使用预训练的嵌入向量训练模块也有助于提高这两个准确率。不过,此时需要注意在训练集上的过拟合。即使进行正则化,训练一个预训练的模块也可能存在风险,因为嵌入向量权重不再表示基于各种数据训练的语言模型,相反,它们会收敛到新数据集的理想表示。