Treten Sie der SIG TFX-Addons-Community bei und helfen Sie mit, TFX noch besser zu machen! SIG TFX-Addons beitreten

Graphbasiertes neuronales strukturiertes Lernen in TFX

Dieses Tutorial beschreibt , aus dem grafischen Darstellung Regularisierung Neural Structured Learning Framework und zeigt einen End-to-End - Workflow für die Stimmung Einstufung in einer TFX - Pipeline.

Überblick

Dieses Notebook stuft Film Bewertungen als positiv oder negativ , den Text der Überprüfung mit. Dies ist ein Beispiel für binäre Klassifizierung, eine wichtige und breit anwendbare Art des maschinellen Lernens Problem.

Wir werden die Verwendung der Graphenregularisierung in diesem Notebook demonstrieren, indem wir einen Graphen aus der gegebenen Eingabe erstellen. Das allgemeine Rezept zum Erstellen eines graphenregulierten Modells unter Verwendung des Neural Structured Learning (NSL)-Frameworks, wenn die Eingabe keinen expliziten Graphen enthält, lautet wie folgt:

  1. Erstellen Sie Einbettungen für jedes Textbeispiel in der Eingabe. Dies kann mit vortrainiert Modelle wie getan werden word2vec , Swivel , BERT usw.
  2. Erstellen Sie einen Graphen basierend auf diesen Einbettungen, indem Sie eine Ähnlichkeitsmetrik wie den 'L2'-Abstand, den 'Cosinus'-Abstand usw. verwenden. Knoten im Diagramm entsprechen Abtastwerten und Kanten im Diagramm entsprechen Ähnlichkeiten zwischen Abtastwertpaaren.
  3. Generieren Sie Trainingsdaten aus dem oben synthetisierten Diagramm und den Beispielfunktionen. Die resultierenden Trainingsdaten enthalten zusätzlich zu den ursprünglichen Knotenmerkmalen Nachbarmerkmale.
  4. Erstellen Sie mithilfe von Estimators ein neuronales Netzwerk als Basismodell.
  5. Wickeln Sie das Basismodell mit der add_graph_regularization Wrapper - Funktion, die von dem NSL Rahmen vorgesehen ist, ein neues Diagramm Estimator Modell zu erstellen. Dieses neue Modell wird einen Graph-Regularisierungsverlust als Regularisierungsterm in sein Trainingsziel aufnehmen.
  6. Trainieren und bewerten Sie das Graph Estimator-Modell.

In diesem Tutorial integrieren wir den obigen Workflow in eine TFX-Pipeline mit mehreren benutzerdefinierten TFX-Komponenten sowie einer benutzerdefinierten grafikregulierten Trainerkomponente.

Unten ist das Schema für unsere TFX-Pipeline. Orangefarbene Kästchen stehen für Standard-TFX-Komponenten und rosa Kästchen für benutzerdefinierte TFX-Komponenten.

TFX-Pipeline

Upgrade-Pip

Um zu vermeiden, dass Pip in einem System aktualisiert wird, wenn es lokal ausgeführt wird, stellen Sie sicher, dass wir in Colab ausgeführt werden. Lokale Systeme können natürlich separat nachgerüstet werden.

try:
  import colab
  !pip install --upgrade pip
except:
  pass

Erforderliche Pakete installieren

!pip install -q -U \
  tfx==1.0.0 \
  neural-structured-learning \
  tensorflow-hub \
  tensorflow-datasets

Hast du die Laufzeit neu gestartet?

Wenn Sie Google Colab verwenden, müssen Sie beim ersten Ausführen der obigen Zelle die Laufzeit neu starten (Laufzeit > Laufzeit neu starten ...). Dies liegt an der Art und Weise, wie Colab Pakete lädt.

Abhängigkeiten und Importe

import apache_beam as beam
import gzip as gzip_lib
import numpy as np
import os
import pprint
import shutil
import tempfile
import urllib
import uuid
pp = pprint.PrettyPrinter()

import tensorflow as tf
import neural_structured_learning as nsl

import tfx
from tfx.components.evaluator.component import Evaluator
from tfx.components.example_gen.import_example_gen.component import ImportExampleGen
from tfx.components.example_validator.component import ExampleValidator
from tfx.components.model_validator.component import ModelValidator
from tfx.components.pusher.component import Pusher
from tfx.components.schema_gen.component import SchemaGen
from tfx.components.statistics_gen.component import StatisticsGen
from tfx.components.trainer import executor as trainer_executor
from tfx.components.trainer.component import Trainer
from tfx.components.transform.component import Transform
from tfx.dsl.components.base import executor_spec
from tfx.orchestration.experimental.interactive.interactive_context import InteractiveContext
from tfx.proto import evaluator_pb2
from tfx.proto import example_gen_pb2
from tfx.proto import pusher_pb2
from tfx.proto import trainer_pb2

from tfx.types import artifact
from tfx.types import artifact_utils
from tfx.types import channel
from tfx.types import standard_artifacts
from tfx.types.standard_artifacts import Examples

from tfx.dsl.component.experimental.annotations import InputArtifact
from tfx.dsl.component.experimental.annotations import OutputArtifact
from tfx.dsl.component.experimental.annotations import Parameter
from tfx.dsl.component.experimental.decorators import component

from tensorflow_metadata.proto.v0 import anomalies_pb2
from tensorflow_metadata.proto.v0 import schema_pb2
from tensorflow_metadata.proto.v0 import statistics_pb2

import tensorflow_data_validation as tfdv
import tensorflow_transform as tft
import tensorflow_model_analysis as tfma
import tensorflow_hub as hub
import tensorflow_datasets as tfds

print("TF Version: ", tf.__version__)
print("Eager mode: ", tf.executing_eagerly())
print(
    "GPU is",
    "available" if tf.config.list_physical_devices("GPU") else "NOT AVAILABLE")
print("NSL Version: ", nsl.__version__)
print("TFX Version: ", tfx.__version__)
print("TFDV version: ", tfdv.__version__)
print("TFT version: ", tft.__version__)
print("TFMA version: ", tfma.__version__)
print("Hub version: ", hub.__version__)
print("Beam version: ", beam.__version__)
2021-07-27 09:11:37.702558: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
TF Version:  2.5.0
Eager mode:  True
GPU is available
NSL Version:  1.3.1
TFX Version:  1.0.0
TFDV version:  1.0.0
TFT version:  1.0.0
TFMA version:  0.31.0
Hub version:  0.12.0
Beam version:  2.31.0
2021-07-27 09:11:40.751607: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcuda.so.1
2021-07-27 09:11:41.797940: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:11:41.798952: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
pciBusID: 0000:00:05.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2021-07-27 09:11:41.798988: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2021-07-27 09:11:41.802699: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.11
2021-07-27 09:11:41.802801: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.11
2021-07-27 09:11:41.803991: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcufft.so.10
2021-07-27 09:11:41.804324: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcurand.so.10
2021-07-27 09:11:41.805464: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusolver.so.11
2021-07-27 09:11:41.806455: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusparse.so.11
2021-07-27 09:11:41.806646: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
2021-07-27 09:11:41.806779: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:11:41.807827: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:11:41.808771: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0

IMDB-Datensatz

Der IMDB - Datensatz enthält den Text von 50.000 Filmberichte aus dem Internet Movie Database . Diese verteilen sich auf 25.000 Bewertungen für Schulungen und 25.000 Bewertungen für Tests. Die Trainings- und Testsätze werden ausgeglichen, das heißt , sie eine gleiche Anzahl von positiven und negativen Bewertungen enthalten. Darüber hinaus gibt es 50.000 zusätzliche unbeschriftete Filmkritiken.

Vorverarbeiteten IMDB-Datensatz herunterladen

Der folgende Code lädt das IMDB-Dataset mit TFDS herunter (oder verwendet eine zwischengespeicherte Kopie, wenn es bereits heruntergeladen wurde). Um dieses Notebook zu beschleunigen, verwenden wir nur 10.000 beschriftete Rezensionen und 10.000 nicht beschriftete Rezensionen für Schulungen und 10.000 Testberichte zur Bewertung.

train_set, eval_set = tfds.load(
    "imdb_reviews:1.0.0",
    split=["train[:10000]+unsupervised[:10000]", "test[:10000]"],
    shuffle_files=False)
2021-07-27 09:11:46.214651: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-07-27 09:11:46.215481: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:11:46.216555: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
pciBusID: 0000:00:05.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2021-07-27 09:11:46.216736: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:11:46.217742: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:11:46.218645: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-07-27 09:11:46.218771: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2021-07-27 09:11:46.859431: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-07-27 09:11:46.859472: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264]      0 
2021-07-27 09:11:46.859481: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0:   N 
2021-07-27 09:11:46.859773: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:11:46.860851: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:11:46.861877: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:11:46.862883: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14646 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0)

Schauen wir uns ein paar Rezensionen aus dem Trainingsset an:

for tfrecord in train_set.take(4):
  print("Review: {}".format(tfrecord["text"].numpy().decode("utf-8")[:300]))
  print("Label: {}\n".format(tfrecord["label"].numpy()))
2021-07-27 09:11:46.943783: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
2021-07-27 09:11:46.944421: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 2000185000 Hz
Review: This was an absolutely terrible movie. Don't be lured in by Christopher Walken or Michael Ironside. Both are great actors, but this must simply be their worst role in history. Even their great acting could not redeem this movie's ridiculous storyline. This movie is an early nineties US propaganda pi
Label: 0

Review: I have been known to fall asleep during films, but this is usually due to a combination of things including, really tired, being warm and comfortable on the sette and having just eaten a lot. However on this occasion I fell asleep because the film was rubbish. The plot development was constant. Cons
Label: 0

Review: Mann photographs the Alberta Rocky Mountains in a superb fashion, and Jimmy Stewart and Walter Brennan give enjoyable performances as they always seem to do. <br /><br />But come on Hollywood - a Mountie telling the people of Dawson City, Yukon to elect themselves a marshal (yes a marshal!) and to e
Label: 0

Review: This is the kind of film for a snowy Sunday afternoon when the rest of the world can go ahead with its own business as you descend into a big arm-chair and mellow for a couple of hours. Wonderful performances from Cher and Nicolas Cage (as always) gently row the plot along. There are no rapids to cr
Label: 1
def _dict_to_example(instance):
  """Decoded CSV to tf example."""
  feature = {}
  for key, value in instance.items():
    if value is None:
      feature[key] = tf.train.Feature()
    elif value.dtype == np.integer:
      feature[key] = tf.train.Feature(
          int64_list=tf.train.Int64List(value=value.tolist()))
    elif value.dtype == np.float32:
      feature[key] = tf.train.Feature(
          float_list=tf.train.FloatList(value=value.tolist()))
    else:
      feature[key] = tf.train.Feature(
          bytes_list=tf.train.BytesList(value=value.tolist()))
  return tf.train.Example(features=tf.train.Features(feature=feature))


examples_path = tempfile.mkdtemp(prefix="tfx-data")
train_path = os.path.join(examples_path, "train.tfrecord")
eval_path = os.path.join(examples_path, "eval.tfrecord")

for path, dataset in [(train_path, train_set), (eval_path, eval_set)]:
  with tf.io.TFRecordWriter(path) as writer:
    for example in dataset:
      writer.write(
          _dict_to_example({
              "label": np.array([example["label"].numpy()]),
              "text": np.array([example["text"].numpy()]),
          }).SerializeToString())
/home/kbuilder/.local/lib/python3.7/site-packages/ipykernel_launcher.py:7: DeprecationWarning: Converting `np.integer` or `np.signedinteger` to a dtype is deprecated. The current result is `np.dtype(np.int_)` which is not strictly correct. Note that the result depends on the system. To ensure stable results use may want to use `np.int64` or `np.int32`.
  import sys

TFX-Komponenten interaktiv ausführen

In den Zellen , die Sie folgen TFX Komponenten konstruieren und jedes interaktiv innerhalb des InteractiveContext laufen zu erhalten ExecutionResult Objekte. Dies spiegelt den Prozess eines Orchestrators, der Komponenten in einer TFX-DAG ausführt, basierend darauf, wann die Abhängigkeiten für jede Komponente erfüllt sind.

context = InteractiveContext()
WARNING:absl:InteractiveContext pipeline_root argument not provided: using temporary directory /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw as root for pipeline outputs.
WARNING:absl:InteractiveContext metadata_connection_config not provided: using SQLite ML Metadata database at /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/metadata.sqlite.

Die ExampleGen-Komponente

In jedem ML-Entwicklungsprozess besteht der erste Schritt beim Starten der Codeentwicklung darin, die Trainings- und Testdatensätze aufzunehmen. Die ExampleGen Komponente bringt Daten in die TFX - Pipeline.

Erstellen Sie eine ExampleGen-Komponente und führen Sie sie aus.

input_config = example_gen_pb2.Input(splits=[
    example_gen_pb2.Input.Split(name='train', pattern='train.tfrecord'),
    example_gen_pb2.Input.Split(name='eval', pattern='eval.tfrecord')
])

example_gen = ImportExampleGen(input_base=examples_path, input_config=input_config)

context.run(example_gen, enable_cache=True)
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING:apache_beam.io.tfrecordio:Couldn't find python-snappy so the implementation of _TFRecordUtil._masked_crc32c is not as fast as it could be.
for artifact in example_gen.outputs['examples'].get():
  print(artifact)

print('\nexample_gen.outputs is a {}'.format(type(example_gen.outputs)))
print(example_gen.outputs)

print(example_gen.outputs['examples'].get()[0].split_names)
Artifact(artifact: id: 1
type_id: 5
uri: "/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/ImportExampleGen/examples/1"
properties {
  key: "split_names"
  value {
    string_value: "[\"train\", \"eval\"]"
  }
}
custom_properties {
  key: "input_fingerprint"
  value {
    string_value: "split:train,num_files:1,total_bytes:27706811,xor_checksum:1627377115,sum_checksum:1627377115\nsplit:eval,num_files:1,total_bytes:13374744,xor_checksum:1627377119,sum_checksum:1627377119"
  }
}
custom_properties {
  key: "payload_format"
  value {
    string_value: "FORMAT_TF_EXAMPLE"
  }
}
custom_properties {
  key: "span"
  value {
    int_value: 0
  }
}
custom_properties {
  key: "state"
  value {
    string_value: "published"
  }
}
custom_properties {
  key: "tfx_version"
  value {
    string_value: "1.0.0"
  }
}
state: LIVE
, artifact_type: id: 5
name: "Examples"
properties {
  key: "span"
  value: INT
}
properties {
  key: "split_names"
  value: STRING
}
properties {
  key: "version"
  value: INT
}
)

example_gen.outputs is a <class 'dict'>
{'examples': Channel(
    type_name: Examples
    artifacts: [Artifact(artifact: id: 1
type_id: 5
uri: "/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/ImportExampleGen/examples/1"
properties {
  key: "split_names"
  value {
    string_value: "[\"train\", \"eval\"]"
  }
}
custom_properties {
  key: "input_fingerprint"
  value {
    string_value: "split:train,num_files:1,total_bytes:27706811,xor_checksum:1627377115,sum_checksum:1627377115\nsplit:eval,num_files:1,total_bytes:13374744,xor_checksum:1627377119,sum_checksum:1627377119"
  }
}
custom_properties {
  key: "payload_format"
  value {
    string_value: "FORMAT_TF_EXAMPLE"
  }
}
custom_properties {
  key: "span"
  value {
    int_value: 0
  }
}
custom_properties {
  key: "state"
  value {
    string_value: "published"
  }
}
custom_properties {
  key: "tfx_version"
  value {
    string_value: "1.0.0"
  }
}
state: LIVE
, artifact_type: id: 5
name: "Examples"
properties {
  key: "span"
  value: INT
}
properties {
  key: "split_names"
  value: STRING
}
properties {
  key: "version"
  value: INT
}
)]
    additional_properties: {}
    additional_custom_properties: {}
)}
["train", "eval"]

Die Ausgaben der Komponente umfassen 2 Artefakte:

  • die Schulungsbeispiele (10.000 gekennzeichnete Bewertungen + 10.000 nicht gekennzeichnete Bewertungen)
  • die Bewertungsbeispiele (10.000 gekennzeichnete Bewertungen)

Die benutzerdefinierte IdentifyExamples-Komponente

Um NSL zu verwenden, benötigen wir für jede Instanz eine eindeutige ID. Wir erstellen eine benutzerdefinierte Komponente, die allen Instanzen über alle Splits hinweg eine solche eindeutige ID hinzufügt. Wir nutzen Apache Breite der Lage sein , leicht zu große Datensätze zu skalieren , wenn nötig.

def make_example_with_unique_id(example, id_feature_name):
  """Adds a unique ID to the given `tf.train.Example` proto.

  This function uses Python's 'uuid' module to generate a universally unique
  identifier for each example.

  Args:
    example: An instance of a `tf.train.Example` proto.
    id_feature_name: The name of the feature in the resulting `tf.train.Example`
      that will contain the unique identifier.

  Returns:
    A new `tf.train.Example` proto that includes a unique identifier as an
    additional feature.
  """
  result = tf.train.Example()
  result.CopyFrom(example)
  unique_id = uuid.uuid4()
  result.features.feature.get_or_create(
      id_feature_name).bytes_list.MergeFrom(
          tf.train.BytesList(value=[str(unique_id).encode('utf-8')]))
  return result


@component
def IdentifyExamples(orig_examples: InputArtifact[Examples],
                     identified_examples: OutputArtifact[Examples],
                     id_feature_name: Parameter[str],
                     component_name: Parameter[str]) -> None:

  # Get a list of the splits in input_data
  splits_list = artifact_utils.decode_split_names(
      split_names=orig_examples.split_names)
  # For completeness, encode the splits names and payload_format.
  # We could also just use input_data.split_names.
  identified_examples.split_names = artifact_utils.encode_split_names(
      splits=splits_list)
  # TODO(b/168616829): Remove populating payload_format after tfx 0.25.0.
  identified_examples.set_string_custom_property(
      "payload_format",
      orig_examples.get_string_custom_property("payload_format"))


  for split in splits_list:
    input_dir = artifact_utils.get_split_uri([orig_examples], split)
    output_dir = artifact_utils.get_split_uri([identified_examples], split)
    os.mkdir(output_dir)
    with beam.Pipeline() as pipeline:
      (pipeline
       | 'ReadExamples' >> beam.io.ReadFromTFRecord(
           os.path.join(input_dir, '*'),
           coder=beam.coders.coders.ProtoCoder(tf.train.Example))
       | 'AddUniqueId' >> beam.Map(make_example_with_unique_id, id_feature_name)
       | 'WriteIdentifiedExamples' >> beam.io.WriteToTFRecord(
           file_path_prefix=os.path.join(output_dir, 'data_tfrecord'),
           coder=beam.coders.coders.ProtoCoder(tf.train.Example),
           file_name_suffix='.gz'))

  return
identify_examples = IdentifyExamples(
    orig_examples=example_gen.outputs['examples'],
    component_name=u'IdentifyExamples',
    id_feature_name=u'id')
context.run(identify_examples, enable_cache=False)
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.

Die StatisticsGen-Komponente

Die StatisticsGen Komponente berechnet deskriptiven Statistiken für Ihre Daten - Set. Die generierten Statistiken können zur Überprüfung visualisiert und beispielsweise zur Validierung und zum Ableiten eines Schemas verwendet werden.

Erstellen Sie eine StatisticsGen-Komponente und führen Sie sie aus.

# Computes statistics over data for visualization and example validation.
statistics_gen = StatisticsGen(
    examples=identify_examples.outputs["identified_examples"])
context.run(statistics_gen, enable_cache=True)
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.

Die SchemaGen-Komponente

Die SchemaGen Komponente generiert ein Schema für Ihre Daten auf der Grundlage der Statistiken von StatisticsGen. Es versucht, die Datentypen jedes Ihrer Features und die zulässigen Wertebereiche für kategoriale Features abzuleiten.

Erstellen Sie eine SchemaGen-Komponente und führen Sie sie aus.

# Generates schema based on statistics files.
schema_gen = SchemaGen(
    statistics=statistics_gen.outputs['statistics'], infer_feature_shape=False)
context.run(schema_gen, enable_cache=True)
2021-07-27 09:12:23.230961: W ml_metadata/metadata_store/rdbms_metadata_access_object.cc:623] No property is defined for the Type

Das erzeugte Artefakt ist nur ein schema.pbtxt eine Textdarstellung eines enthält schema_pb2.Schema protobuf:

train_uri = schema_gen.outputs['schema'].get()[0].uri
schema_filename = os.path.join(train_uri, 'schema.pbtxt')
schema = tfx.utils.io_utils.parse_pbtxt_file(
    file_name=schema_filename, message=schema_pb2.Schema())

Es kann mit visualisiert wird tfdv.display_schema() (wir diese im Detail in einem nachfolgenden Labor aussehen werden):

tfdv.display_schema(schema)

Die ExampleValidator-Komponente

Die ExampleValidator führt Anomalieerkennung auf der Grundlage der Statistiken aus StatisticsGen und das Schema von SchemaGen. Es sucht nach Problemen wie fehlenden Werten, Werten des falschen Typs oder kategorialen Werten außerhalb des zulässigen Wertebereichs.

Erstellen Sie eine ExampleValidator-Komponente und führen Sie sie aus.

# Performs anomaly detection based on statistics and data schema.
validate_stats = ExampleValidator(
    statistics=statistics_gen.outputs['statistics'],
    schema=schema_gen.outputs['schema'])
context.run(validate_stats, enable_cache=False)

Die SynthesizeGraph-Komponente

Die Graphkonstruktion beinhaltet das Erstellen von Einbettungen für Textproben und die anschließende Verwendung einer Ähnlichkeitsfunktion, um die Einbettungen zu vergleichen.

Wir werden vortrainierter Swivel Einbettungen verwenden Einbettungen in der erstellen tf.train.Example Format für jede Probe in der Eingabe. Wir speichern die resultierenden Einbettungen im TFRecord Format zusammen mit der ID der Probe. Dies ist wichtig und ermöglicht es uns später, Beispieleinbettungen mit entsprechenden Knoten im Diagramm abzugleichen.

Sobald wir die Probeneinbettungen haben, werden wir sie verwenden, um einen Ähnlichkeitsgraphen zu erstellen, dh Knoten in diesem Graphen entsprechen Proben und Kanten in diesem Graphen entsprechen der Ähnlichkeit zwischen Knotenpaaren.

Neural Structured Learning bietet eine Bibliothek zum Erstellen von Graphen, um einen Graphen basierend auf Beispieleinbettungen zu erstellen. Es verwendet Kosinusähnlichkeit als Ähnlichkeitsmaß Einbettungen und Build Kanten zwischen ihnen zu vergleichen. Es ermöglicht uns auch, einen Ähnlichkeitsschwellenwert anzugeben, der verwendet werden kann, um unterschiedliche Kanten aus dem endgültigen Graphen zu verwerfen. Im folgenden Beispiel erhalten wir mit 0,99 als Ähnlichkeitsschwelle einen Graphen mit 111.066 bidirektionalen Kanten.

swivel_url = 'https://tfhub.dev/google/tf2-preview/gnews-swivel-20dim/1'
hub_layer = hub.KerasLayer(swivel_url, input_shape=[], dtype=tf.string)


def _bytes_feature(value):
  """Returns a bytes_list from a string / byte."""
  return tf.train.Feature(bytes_list=tf.train.BytesList(value=value))


def _float_feature(value):
  """Returns a float_list from a float / double."""
  return tf.train.Feature(float_list=tf.train.FloatList(value=value))


def create_embedding_example(example):
  """Create tf.Example containing the sample's embedding and its ID."""
  sentence_embedding = hub_layer(tf.sparse.to_dense(example['text']))

  # Flatten the sentence embedding back to 1-D.
  sentence_embedding = tf.reshape(sentence_embedding, shape=[-1])

  feature_dict = {
      'id': _bytes_feature(tf.sparse.to_dense(example['id']).numpy()),
      'embedding': _float_feature(sentence_embedding.numpy().tolist())
  }

  return tf.train.Example(features=tf.train.Features(feature=feature_dict))


def create_dataset(uri):
  tfrecord_filenames = [os.path.join(uri, name) for name in os.listdir(uri)]
  return tf.data.TFRecordDataset(tfrecord_filenames, compression_type='GZIP')


def create_embeddings(train_path, output_path):
  dataset = create_dataset(train_path)
  embeddings_path = os.path.join(output_path, 'embeddings.tfr')

  feature_map = {
      'label': tf.io.FixedLenFeature([], tf.int64),
      'id': tf.io.VarLenFeature(tf.string),
      'text': tf.io.VarLenFeature(tf.string)
  }

  with tf.io.TFRecordWriter(embeddings_path) as writer:
    for tfrecord in dataset:
      tensor_dict = tf.io.parse_single_example(tfrecord, feature_map)
      embedding_example = create_embedding_example(tensor_dict)
      writer.write(embedding_example.SerializeToString())


def build_graph(output_path, similarity_threshold):
  embeddings_path = os.path.join(output_path, 'embeddings.tfr')
  graph_path = os.path.join(output_path, 'graph.tsv')
  graph_builder_config = nsl.configs.GraphBuilderConfig(
      similarity_threshold=similarity_threshold,
      lsh_splits=32,
      lsh_rounds=15,
      random_seed=12345)
  nsl.tools.build_graph_from_config([embeddings_path], graph_path,
                                    graph_builder_config)
"""Custom Artifact type"""


class SynthesizedGraph(tfx.types.artifact.Artifact):
  """Output artifact of the SynthesizeGraph component"""
  TYPE_NAME = 'SynthesizedGraphPath'
  PROPERTIES = {
      'span': standard_artifacts.SPAN_PROPERTY,
      'split_names': standard_artifacts.SPLIT_NAMES_PROPERTY,
  }


@component
def SynthesizeGraph(identified_examples: InputArtifact[Examples],
                    synthesized_graph: OutputArtifact[SynthesizedGraph],
                    similarity_threshold: Parameter[float],
                    component_name: Parameter[str]) -> None:

  # Get a list of the splits in input_data
  splits_list = artifact_utils.decode_split_names(
      split_names=identified_examples.split_names)

  # We build a graph only based on the 'Split-train' split which includes both
  # labeled and unlabeled examples.
  train_input_examples_uri = os.path.join(identified_examples.uri,
                                          'Split-train')
  output_graph_uri = os.path.join(synthesized_graph.uri, 'Split-train')
  os.mkdir(output_graph_uri)

  print('Creating embeddings...')
  create_embeddings(train_input_examples_uri, output_graph_uri)

  print('Synthesizing graph...')
  build_graph(output_graph_uri, similarity_threshold)

  synthesized_graph.split_names = artifact_utils.encode_split_names(
      splits=['Split-train'])

  return
synthesize_graph = SynthesizeGraph(
    identified_examples=identify_examples.outputs['identified_examples'],
    component_name=u'SynthesizeGraph',
    similarity_threshold=0.99)
context.run(synthesize_graph, enable_cache=False)
Creating embeddings...
Synthesizing graph...
train_uri = synthesize_graph.outputs["synthesized_graph"].get()[0].uri
os.listdir(train_uri)
['Split-train']
graph_path = os.path.join(train_uri, "Split-train", "graph.tsv")
print("node 1\t\t\t\t\tnode 2\t\t\t\t\tsimilarity")
!head {graph_path}
print("...")
!tail {graph_path}
node 1                  node 2                  similarity
748802aa-a483-4340-9a65-0590df997dd2    baef7e7f-c89f-4f19-8a85-0fa2aeb7f2b8    0.992471
baef7e7f-c89f-4f19-8a85-0fa2aeb7f2b8    748802aa-a483-4340-9a65-0590df997dd2    0.992471
98f1b9c8-9f2c-4777-a6a5-d0be466a93e5    4d9a030d-f894-4df1-8e5b-f0fd1611573b    0.992586
4d9a030d-f894-4df1-8e5b-f0fd1611573b    98f1b9c8-9f2c-4777-a6a5-d0be466a93e5    0.992586
98f1b9c8-9f2c-4777-a6a5-d0be466a93e5    32d8f5bf-6ba5-47d0-82ff-4cad66b02874    0.992823
32d8f5bf-6ba5-47d0-82ff-4cad66b02874    98f1b9c8-9f2c-4777-a6a5-d0be466a93e5    0.992823
98f1b9c8-9f2c-4777-a6a5-d0be466a93e5    fba6a0bf-4113-4979-aea4-159103f914bb    0.992505
fba6a0bf-4113-4979-aea4-159103f914bb    98f1b9c8-9f2c-4777-a6a5-d0be466a93e5    0.992505
1c1e8101-e62a-45c6-a60a-6a5f9ec83774    a37527b1-bb96-49c4-b582-c3f5b86ada9e    0.990020
a37527b1-bb96-49c4-b582-c3f5b86ada9e    1c1e8101-e62a-45c6-a60a-6a5f9ec83774    0.990020
...
5043eaf4-aa1f-4475-864a-2d3c87e83f40    f0bc8fcd-9068-4c5c-adad-1e9b001cffaf    0.990002
f0bc8fcd-9068-4c5c-adad-1e9b001cffaf    5043eaf4-aa1f-4475-864a-2d3c87e83f40    0.990002
f388287c-14d1-4123-8c3d-379188d7ceee    8d6646d3-121f-4be6-8a0f-62a6377eabf5    0.991046
8d6646d3-121f-4be6-8a0f-62a6377eabf5    f388287c-14d1-4123-8c3d-379188d7ceee    0.991046
bf4b25ae-54bd-4b59-938d-a055c382668a    dc710c10-0470-4fd0-8be1-0e51ddfc3105    0.991198
dc710c10-0470-4fd0-8be1-0e51ddfc3105    bf4b25ae-54bd-4b59-938d-a055c382668a    0.991198
f97a9b2e-73ad-4074-b068-a6ca270adf98    41be7cc4-e186-4f65-97c8-c7d3d6fe3603    0.990260
41be7cc4-e186-4f65-97c8-c7d3d6fe3603    f97a9b2e-73ad-4074-b068-a6ca270adf98    0.990260
451f159a-793f-4a7e-b8e4-8d86327ef824    1b367595-b6fa-4eb1-ae55-33094054e1dc    0.991317
1b367595-b6fa-4eb1-ae55-33094054e1dc    451f159a-793f-4a7e-b8e4-8d86327ef824    0.991317
wc -l {graph_path}
222132 /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/SynthesizeGraph/synthesized_graph/6/Split-train/graph.tsv

Die Transform-Komponente

Die Transform Komponente führt eine Datentransformation und Feature - Engineering. Die Ergebnisse umfassen ein TensorFlow-Eingabediagramm, das sowohl während des Trainings verwendet wird als auch dazu dient, die Daten vor dem Training oder der Inferenz vorzuverarbeiten. Dieses Diagramm wird Teil des SavedModel, das das Ergebnis des Modelltrainings ist. Da für Training und Bereitstellung derselbe Eingabegraph verwendet wird, ist die Vorverarbeitung immer gleich und muss nur einmal geschrieben werden.

Die Transform-Komponente erfordert aufgrund der willkürlichen Komplexität des Feature-Engineerings, das Sie für die Daten und/oder das Modell, mit dem Sie arbeiten, möglicherweise benötigen, mehr Code als viele andere Komponenten. Es erfordert, dass Codedateien verfügbar sind, die die erforderliche Verarbeitung definieren.

Jedes Beispiel enthält die folgenden drei Funktionen:

  1. ID: Die Knoten - ID der Probe.
  2. text_xf: Eine int64 Liste Wort - IDs enthalten.
  3. label_xf: A Singletons int64 die Zielklasse der Überprüfung identifizieren: 0 = negativ ist , 1 = positiv.

Definieren wir ein Modul enthält , das preprocessing_fn() Funktion , die wir den passieren Transform Komponente:

_transform_module_file = 'imdb_transform.py'
%%writefile {_transform_module_file}

import tensorflow as tf

import tensorflow_transform as tft

SEQUENCE_LENGTH = 100
VOCAB_SIZE = 10000
OOV_SIZE = 100

def tokenize_reviews(reviews, sequence_length=SEQUENCE_LENGTH):
  reviews = tf.strings.lower(reviews)
  reviews = tf.strings.regex_replace(reviews, r" '| '|^'|'$", " ")
  reviews = tf.strings.regex_replace(reviews, "[^a-z' ]", " ")
  tokens = tf.strings.split(reviews)[:, :sequence_length]
  start_tokens = tf.fill([tf.shape(reviews)[0], 1], "<START>")
  end_tokens = tf.fill([tf.shape(reviews)[0], 1], "<END>")
  tokens = tf.concat([start_tokens, tokens, end_tokens], axis=1)
  tokens = tokens[:, :sequence_length]
  tokens = tokens.to_tensor(default_value="<PAD>")
  pad = sequence_length - tf.shape(tokens)[1]
  tokens = tf.pad(tokens, [[0, 0], [0, pad]], constant_values="<PAD>")
  return tf.reshape(tokens, [-1, sequence_length])

def preprocessing_fn(inputs):
  """tf.transform's callback function for preprocessing inputs.

  Args:
    inputs: map from feature keys to raw not-yet-transformed features.

  Returns:
    Map from string feature key to transformed feature operations.
  """
  outputs = {}
  outputs["id"] = inputs["id"]
  tokens = tokenize_reviews(_fill_in_missing(inputs["text"], ''))
  outputs["text_xf"] = tft.compute_and_apply_vocabulary(
      tokens,
      top_k=VOCAB_SIZE,
      num_oov_buckets=OOV_SIZE)
  outputs["label_xf"] = _fill_in_missing(inputs["label"], -1)
  return outputs

def _fill_in_missing(x, default_value):
  """Replace missing values in a SparseTensor.

  Fills in missing values of `x` with the default_value.

  Args:
    x: A `SparseTensor` of rank 2.  Its dense shape should have size at most 1
      in the second dimension.
    default_value: the value with which to replace the missing values.

  Returns:
    A rank 1 tensor where missing values of `x` have been filled in.
  """
  if not isinstance(x, tf.sparse.SparseTensor):
    return x
  return tf.squeeze(
      tf.sparse.to_dense(
          tf.SparseTensor(x.indices, x.values, [x.dense_shape[0], 1]),
          default_value),
      axis=1)
Writing imdb_transform.py

Erstellen und Ausführen der Transform Komponente, die sich auf die Dateien , die oben erstellt wurden.

# Performs transformations and feature engineering in training and serving.
transform = Transform(
    examples=identify_examples.outputs['identified_examples'],
    schema=schema_gen.outputs['schema'],
    module_file=_transform_module_file)
context.run(transform, enable_cache=True)
running bdist_wheel
running build
running build_py
creating build
creating build/lib
copying imdb_transform.py -> build/lib
installing to /tmp/tmp8qwk9f8j
running install
running install_lib
copying build/lib/imdb_transform.py -> /tmp/tmp8qwk9f8j
running install_egg_info
running egg_info
creating tfx_user_code_Transform.egg-info
writing tfx_user_code_Transform.egg-info/PKG-INFO
writing dependency_links to tfx_user_code_Transform.egg-info/dependency_links.txt
writing top-level names to tfx_user_code_Transform.egg-info/top_level.txt
writing manifest file 'tfx_user_code_Transform.egg-info/SOURCES.txt'
reading manifest file 'tfx_user_code_Transform.egg-info/SOURCES.txt'
writing manifest file 'tfx_user_code_Transform.egg-info/SOURCES.txt'
Copying tfx_user_code_Transform.egg-info to /tmp/tmp8qwk9f8j/tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50-py3.7.egg-info
running install_scripts
creating /tmp/tmp8qwk9f8j/tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50.dist-info/WHEEL
creating '/tmp/tmpau8udk0q/tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50-py3-none-any.whl' and adding '/tmp/tmp8qwk9f8j' to it
adding 'imdb_transform.py'
adding 'tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50.dist-info/METADATA'
adding 'tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50.dist-info/WHEEL'
adding 'tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50.dist-info/top_level.txt'
adding 'tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50.dist-info/RECORD'
removing /tmp/tmp8qwk9f8j
2021-07-27 09:14:08.178895: W ml_metadata/metadata_store/rdbms_metadata_access_object.cc:623] No property is defined for the Type
2021-07-27 09:14:08.183573: W ml_metadata/metadata_store/rdbms_metadata_access_object.cc:623] No property is defined for the Type
Processing /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/_wheels/tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50-py3-none-any.whl
Installing collected packages: tfx-user-code-Transform
Successfully installed tfx-user-code-Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50
WARNING: You are using pip version 21.1.3; however, version 21.2.1 is available.
You should consider upgrading via the '/tmpfs/src/tf_docs_env/bin/python -m pip install --upgrade pip' command.
Processing /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/_wheels/tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50-py3-none-any.whl
Installing collected packages: tfx-user-code-Transform
Successfully installed tfx-user-code-Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50
WARNING: You are using pip version 21.1.3; however, version 21.2.1 is available.
You should consider upgrading via the '/tmpfs/src/tf_docs_env/bin/python -m pip install --upgrade pip' command.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_transform/tf_utils.py:266: Tensor.experimental_ref (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use ref() instead.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_transform/tf_utils.py:266: Tensor.experimental_ref (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use ref() instead.
WARNING:tensorflow:Tables initialized inside a tf.function will be re-initialized on every invocation of the function. This re-initialization can have significant impact on performance. Consider lifting them out of the graph context using `tf.init_scope`.
WARNING:tensorflow:Tables initialized inside a tf.function will be re-initialized on every invocation of the function. This re-initialization can have significant impact on performance. Consider lifting them out of the graph context using `tf.init_scope`.
WARNING:tensorflow:Tables initialized inside a tf.function will be re-initialized on every invocation of the function. This re-initialization can have significant impact on performance. Consider lifting them out of the graph context using `tf.init_scope`.
WARNING:tensorflow:Tables initialized inside a tf.function will be re-initialized on every invocation of the function. This re-initialization can have significant impact on performance. Consider lifting them out of the graph context using `tf.init_scope`.
WARNING:tensorflow:Tables initialized inside a tf.function will be re-initialized on every invocation of the function. This re-initialization can have significant impact on performance. Consider lifting them out of the graph context using `tf.init_scope`.
WARNING:tensorflow:Tables initialized inside a tf.function will be re-initialized on every invocation of the function. This re-initialization can have significant impact on performance. Consider lifting them out of the graph context using `tf.init_scope`.
Processing /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/_wheels/tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50-py3-none-any.whl
WARNING: You are using pip version 21.1.3; however, version 21.2.1 is available.
You should consider upgrading via the '/tmpfs/src/tf_docs_env/bin/python -m pip install --upgrade pip' command.
WARNING:root:This output type hint will be ignored and not used for type-checking purposes. Typically, output type hints for a PTransform are single (or nested) types wrapped by a PCollection, PDone, or None. Got: Tuple[Dict[str, Union[NoneType, _Dataset]], Union[Dict[str, Dict[str, PCollection]], NoneType]] instead.
Installing collected packages: tfx-user-code-Transform
Successfully installed tfx-user-code-Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50
WARNING:tensorflow:Tables initialized inside a tf.function will be re-initialized on every invocation of the function. This re-initialization can have significant impact on performance. Consider lifting them out of the graph context using `tf.init_scope`.
WARNING:tensorflow:Tables initialized inside a tf.function will be re-initialized on every invocation of the function. This re-initialization can have significant impact on performance. Consider lifting them out of the graph context using `tf.init_scope`.
WARNING:tensorflow:Tables initialized inside a tf.function will be re-initialized on every invocation of the function. This re-initialization can have significant impact on performance. Consider lifting them out of the graph context using `tf.init_scope`.
WARNING:tensorflow:Tables initialized inside a tf.function will be re-initialized on every invocation of the function. This re-initialization can have significant impact on performance. Consider lifting them out of the graph context using `tf.init_scope`.
WARNING:root:This output type hint will be ignored and not used for type-checking purposes. Typically, output type hints for a PTransform are single (or nested) types wrapped by a PCollection, PDone, or None. Got: Tuple[Dict[str, Union[NoneType, _Dataset]], Union[Dict[str, Dict[str, PCollection]], NoneType]] instead.
WARNING:apache_beam.typehints.typehints:Ignoring send_type hint: <class 'NoneType'>
WARNING:apache_beam.typehints.typehints:Ignoring return_type hint: <class 'NoneType'>
WARNING:apache_beam.typehints.typehints:Ignoring send_type hint: <class 'NoneType'>
WARNING:apache_beam.typehints.typehints:Ignoring return_type hint: <class 'NoneType'>
WARNING:apache_beam.typehints.typehints:Ignoring send_type hint: <class 'NoneType'>
WARNING:apache_beam.typehints.typehints:Ignoring return_type hint: <class 'NoneType'>
WARNING:apache_beam.typehints.typehints:Ignoring send_type hint: <class 'NoneType'>
WARNING:apache_beam.typehints.typehints:Ignoring return_type hint: <class 'NoneType'>
WARNING:apache_beam.typehints.typehints:Ignoring send_type hint: <class 'NoneType'>
WARNING:apache_beam.typehints.typehints:Ignoring return_type hint: <class 'NoneType'>
WARNING:apache_beam.typehints.typehints:Ignoring send_type hint: <class 'NoneType'>
WARNING:apache_beam.typehints.typehints:Ignoring return_type hint: <class 'NoneType'>
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/transform_graph/7/.temp_path/tftransform_tmp/0d8b617c4b39446d8dac05f6f2bf9a1d/assets
2021-07-27 09:14:18.985192: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/transform_graph/7/.temp_path/tftransform_tmp/0d8b617c4b39446d8dac05f6f2bf9a1d/assets
INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/transform_graph/7/.temp_path/tftransform_tmp/d5b928df215f42d5bb12b72e3bfb0457/assets
INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/transform_graph/7/.temp_path/tftransform_tmp/d5b928df215f42d5bb12b72e3bfb0457/assets

Die Transform - Komponente hat 2 Typen von Ausgängen:

  • transform_graph ist der Graph, der die Vorverarbeitung Operationen durchführen kann (Dieser Graph wird in der bedienenden und Bewertungsmodelle enthalten sein).
  • transformed_examples stellt die vorverarbeiteten Ausbildung und Bewertungsdaten.
transform.outputs
{'transform_graph': Channel(
     type_name: TransformGraph
     artifacts: [Artifact(artifact: id: 7
 type_id: 16
 uri: "/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/transform_graph/7"
 custom_properties {
   key: "name"
   value {
     string_value: "transform_graph"
   }
 }
 custom_properties {
   key: "producer_component"
   value {
     string_value: "Transform"
   }
 }
 custom_properties {
   key: "state"
   value {
     string_value: "published"
   }
 }
 custom_properties {
   key: "tfx_version"
   value {
     string_value: "1.0.0"
   }
 }
 state: LIVE
 , artifact_type: id: 16
 name: "TransformGraph"
 )]
     additional_properties: {}
     additional_custom_properties: {}
 ),
 'transformed_examples': Channel(
     type_name: Examples
     artifacts: [Artifact(artifact: id: 8
 type_id: 5
 uri: "/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/transformed_examples/7"
 properties {
   key: "split_names"
   value {
     string_value: "[\"train\", \"eval\"]"
   }
 }
 custom_properties {
   key: "name"
   value {
     string_value: "transformed_examples"
   }
 }
 custom_properties {
   key: "producer_component"
   value {
     string_value: "Transform"
   }
 }
 custom_properties {
   key: "state"
   value {
     string_value: "published"
   }
 }
 custom_properties {
   key: "tfx_version"
   value {
     string_value: "1.0.0"
   }
 }
 state: LIVE
 , artifact_type: id: 5
 name: "Examples"
 properties {
   key: "span"
   value: INT
 }
 properties {
   key: "split_names"
   value: STRING
 }
 properties {
   key: "version"
   value: INT
 }
 )]
     additional_properties: {}
     additional_custom_properties: {}
 ),
 'updated_analyzer_cache': Channel(
     type_name: TransformCache
     artifacts: [Artifact(artifact: id: 9
 type_id: 17
 uri: "/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/updated_analyzer_cache/7"
 custom_properties {
   key: "name"
   value {
     string_value: "updated_analyzer_cache"
   }
 }
 custom_properties {
   key: "producer_component"
   value {
     string_value: "Transform"
   }
 }
 custom_properties {
   key: "state"
   value {
     string_value: "published"
   }
 }
 custom_properties {
   key: "tfx_version"
   value {
     string_value: "1.0.0"
   }
 }
 state: LIVE
 , artifact_type: id: 17
 name: "TransformCache"
 )]
     additional_properties: {}
     additional_custom_properties: {}
 ),
 'pre_transform_schema': Channel(
     type_name: Schema
     artifacts: [Artifact(artifact: id: 10
 type_id: 10
 uri: "/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/pre_transform_schema/7"
 custom_properties {
   key: "name"
   value {
     string_value: "pre_transform_schema"
   }
 }
 custom_properties {
   key: "producer_component"
   value {
     string_value: "Transform"
   }
 }
 custom_properties {
   key: "state"
   value {
     string_value: "published"
   }
 }
 custom_properties {
   key: "tfx_version"
   value {
     string_value: "1.0.0"
   }
 }
 state: LIVE
 , artifact_type: id: 10
 name: "Schema"
 )]
     additional_properties: {}
     additional_custom_properties: {}
 ),
 'pre_transform_stats': Channel(
     type_name: ExampleStatistics
     artifacts: [Artifact(artifact: id: 11
 type_id: 8
 uri: "/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/pre_transform_stats/7"
 custom_properties {
   key: "name"
   value {
     string_value: "pre_transform_stats"
   }
 }
 custom_properties {
   key: "producer_component"
   value {
     string_value: "Transform"
   }
 }
 custom_properties {
   key: "state"
   value {
     string_value: "published"
   }
 }
 custom_properties {
   key: "tfx_version"
   value {
     string_value: "1.0.0"
   }
 }
 state: LIVE
 , artifact_type: id: 8
 name: "ExampleStatistics"
 properties {
   key: "span"
   value: INT
 }
 properties {
   key: "split_names"
   value: STRING
 }
 )]
     additional_properties: {}
     additional_custom_properties: {}
 ),
 'post_transform_schema': Channel(
     type_name: Schema
     artifacts: [Artifact(artifact: id: 12
 type_id: 10
 uri: "/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/post_transform_schema/7"
 custom_properties {
   key: "name"
   value {
     string_value: "post_transform_schema"
   }
 }
 custom_properties {
   key: "producer_component"
   value {
     string_value: "Transform"
   }
 }
 custom_properties {
   key: "state"
   value {
     string_value: "published"
   }
 }
 custom_properties {
   key: "tfx_version"
   value {
     string_value: "1.0.0"
   }
 }
 state: LIVE
 , artifact_type: id: 10
 name: "Schema"
 )]
     additional_properties: {}
     additional_custom_properties: {}
 ),
 'post_transform_stats': Channel(
     type_name: ExampleStatistics
     artifacts: [Artifact(artifact: id: 13
 type_id: 8
 uri: "/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/post_transform_stats/7"
 custom_properties {
   key: "name"
   value {
     string_value: "post_transform_stats"
   }
 }
 custom_properties {
   key: "producer_component"
   value {
     string_value: "Transform"
   }
 }
 custom_properties {
   key: "state"
   value {
     string_value: "published"
   }
 }
 custom_properties {
   key: "tfx_version"
   value {
     string_value: "1.0.0"
   }
 }
 state: LIVE
 , artifact_type: id: 8
 name: "ExampleStatistics"
 properties {
   key: "span"
   value: INT
 }
 properties {
   key: "split_names"
   value: STRING
 }
 )]
     additional_properties: {}
     additional_custom_properties: {}
 ),
 'post_transform_anomalies': Channel(
     type_name: ExampleAnomalies
     artifacts: [Artifact(artifact: id: 14
 type_id: 12
 uri: "/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/post_transform_anomalies/7"
 custom_properties {
   key: "name"
   value {
     string_value: "post_transform_anomalies"
   }
 }
 custom_properties {
   key: "producer_component"
   value {
     string_value: "Transform"
   }
 }
 custom_properties {
   key: "state"
   value {
     string_value: "published"
   }
 }
 custom_properties {
   key: "tfx_version"
   value {
     string_value: "1.0.0"
   }
 }
 state: LIVE
 , artifact_type: id: 12
 name: "ExampleAnomalies"
 properties {
   key: "span"
   value: INT
 }
 properties {
   key: "split_names"
   value: STRING
 }
 )]
     additional_properties: {}
     additional_custom_properties: {}
 )}

Werfen Sie einen Blick auf die transform_graph Artefakt: es auf ein Verzeichnis , das 3 Verzeichnisse:

train_uri = transform.outputs['transform_graph'].get()[0].uri
os.listdir(train_uri)
['transform_fn', 'transformed_metadata', 'metadata']

Das transform_fn Unterverzeichnis enthält die eigentliche Vorverarbeitung Graph. Das metadata - Unterverzeichnis enthält das Schema der ursprünglichen Daten. Das transformed_metadata Unterverzeichnis enthält das Schema der vorverarbeiteten Daten.

Sehen Sie sich einige der transformierten Beispiele an und überprüfen Sie, ob sie tatsächlich wie beabsichtigt verarbeitet werden.

def pprint_examples(artifact, n_examples=3):
  print("artifact:", artifact)
  uri = os.path.join(artifact.uri, "Split-train")
  print("uri:", uri)
  tfrecord_filenames = [os.path.join(uri, name) for name in os.listdir(uri)]
  print("tfrecord_filenames:", tfrecord_filenames)
  dataset = tf.data.TFRecordDataset(tfrecord_filenames, compression_type="GZIP")
  for tfrecord in dataset.take(n_examples):
    serialized_example = tfrecord.numpy()
    example = tf.train.Example.FromString(serialized_example)
    pp.pprint(example)
pprint_examples(transform.outputs['transformed_examples'].get()[0])
artifact: Artifact(artifact: id: 8
type_id: 5
uri: "/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/transformed_examples/7"
properties {
  key: "split_names"
  value {
    string_value: "[\"train\", \"eval\"]"
  }
}
custom_properties {
  key: "name"
  value {
    string_value: "transformed_examples"
  }
}
custom_properties {
  key: "producer_component"
  value {
    string_value: "Transform"
  }
}
custom_properties {
  key: "state"
  value {
    string_value: "published"
  }
}
custom_properties {
  key: "tfx_version"
  value {
    string_value: "1.0.0"
  }
}
state: LIVE
, artifact_type: id: 5
name: "Examples"
properties {
  key: "span"
  value: INT
}
properties {
  key: "split_names"
  value: STRING
}
properties {
  key: "version"
  value: INT
}
)
uri: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/transformed_examples/7/Split-train
tfrecord_filenames: ['/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Transform/transformed_examples/7/Split-train/transformed_examples-00000-of-00001.gz']
features {
  feature {
    key: "id"
    value {
      bytes_list {
        value: "5ea2f9ef-e6b5-4ba1-b278-1d08079ca6c3"
      }
    }
  }
  feature {
    key: "label_xf"
    value {
      int64_list {
        value: 0
      }
    }
  }
  feature {
    key: "text_xf"
    value {
      int64_list {
        value: 13
        value: 8
        value: 14
        value: 32
        value: 338
        value: 310
        value: 15
        value: 95
        value: 27
        value: 10001
        value: 9
        value: 31
        value: 1173
        value: 3153
        value: 43
        value: 495
        value: 10060
        value: 214
        value: 26
        value: 71
        value: 142
        value: 19
        value: 8
        value: 204
        value: 339
        value: 27
        value: 74
        value: 181
        value: 238
        value: 9
        value: 440
        value: 67
        value: 74
        value: 71
        value: 94
        value: 100
        value: 22
        value: 5442
        value: 8
        value: 1573
        value: 607
        value: 530
        value: 8
        value: 15
        value: 6
        value: 32
        value: 378
        value: 6292
        value: 207
        value: 2276
        value: 388
        value: 0
        value: 84
        value: 1023
        value: 154
        value: 65
        value: 155
        value: 52
        value: 0
        value: 10080
        value: 7871
        value: 65
        value: 250
        value: 74
        value: 3202
        value: 20
        value: 10000
        value: 3720
        value: 10020
        value: 10008
        value: 1282
        value: 3862
        value: 3
        value: 53
        value: 3952
        value: 110
        value: 1879
        value: 17
        value: 3153
        value: 14
        value: 166
        value: 19
        value: 2
        value: 1023
        value: 1007
        value: 9405
        value: 9
        value: 2
        value: 15
        value: 12
        value: 14
        value: 4504
        value: 4
        value: 109
        value: 158
        value: 1202
        value: 7
        value: 174
        value: 505
        value: 12
      }
    }
  }
}

features {
  feature {
    key: "id"
    value {
      bytes_list {
        value: "2532952c-2853-413e-8df6-e6d54a82da9d"
      }
    }
  }
  feature {
    key: "label_xf"
    value {
      int64_list {
        value: 0
      }
    }
  }
  feature {
    key: "text_xf"
    value {
      int64_list {
        value: 13
        value: 7
        value: 23
        value: 75
        value: 494
        value: 5
        value: 748
        value: 2155
        value: 307
        value: 91
        value: 19
        value: 8
        value: 6
        value: 499
        value: 763
        value: 5
        value: 2
        value: 1690
        value: 4
        value: 200
        value: 593
        value: 57
        value: 1244
        value: 120
        value: 2364
        value: 3
        value: 4407
        value: 21
        value: 0
        value: 10081
        value: 3
        value: 263
        value: 42
        value: 6947
        value: 2
        value: 169
        value: 185
        value: 21
        value: 8
        value: 5143
        value: 7
        value: 1339
        value: 2155
        value: 81
        value: 0
        value: 18
        value: 14
        value: 1468
        value: 0
        value: 86
        value: 986
        value: 14
        value: 2259
        value: 1790
        value: 562
        value: 3
        value: 284
        value: 200
        value: 401
        value: 5
        value: 668
        value: 19
        value: 17
        value: 58
        value: 1934
        value: 4
        value: 45
        value: 14
        value: 4212
        value: 113
        value: 43
        value: 135
        value: 7
        value: 753
        value: 7
        value: 224
        value: 23
        value: 1155
        value: 179
        value: 4
        value: 0
        value: 18
        value: 19
        value: 7
        value: 191
        value: 0
        value: 2047
        value: 4
        value: 10
        value: 3
        value: 283
        value: 42
        value: 401
        value: 5
        value: 668
        value: 4
        value: 90
        value: 234
        value: 10023
        value: 227
      }
    }
  }
}

features {
  feature {
    key: "id"
    value {
      bytes_list {
        value: "49970ec3-f99f-46a1-8bbe-8d153c1caa1a"
      }
    }
  }
  feature {
    key: "label_xf"
    value {
      int64_list {
        value: 0
      }
    }
  }
  feature {
    key: "text_xf"
    value {
      int64_list {
        value: 13
        value: 4577
        value: 7158
        value: 0
        value: 10047
        value: 3778
        value: 3346
        value: 9
        value: 2
        value: 758
        value: 1915
        value: 3
        value: 2280
        value: 1511
        value: 3
        value: 2003
        value: 10020
        value: 225
        value: 786
        value: 382
        value: 16
        value: 39
        value: 203
        value: 361
        value: 5
        value: 93
        value: 11
        value: 11
        value: 19
        value: 220
        value: 21
        value: 341
        value: 2
        value: 10000
        value: 966
        value: 0
        value: 77
        value: 4
        value: 6677
        value: 464
        value: 10071
        value: 5
        value: 10042
        value: 630
        value: 2
        value: 10044
        value: 404
        value: 2
        value: 10044
        value: 3
        value: 5
        value: 10008
        value: 0
        value: 1259
        value: 630
        value: 106
        value: 10042
        value: 6721
        value: 10
        value: 49
        value: 21
        value: 0
        value: 2071
        value: 20
        value: 1292
        value: 4
        value: 0
        value: 431
        value: 11
        value: 11
        value: 166
        value: 67
        value: 2342
        value: 5815
        value: 12
        value: 575
        value: 21
        value: 0
        value: 1691
        value: 537
        value: 4
        value: 0
        value: 3605
        value: 307
        value: 0
        value: 10054
        value: 1563
        value: 3115
        value: 467
        value: 4577
        value: 3
        value: 1069
        value: 1158
        value: 5
        value: 23
        value: 4279
        value: 6677
        value: 464
        value: 20
        value: 10004
      }
    }
  }
}

Die GraphAugmentation-Komponente

Da wir über die Beispielfunktionen und den synthetisierten Graphen verfügen, können wir die erweiterten Trainingsdaten für neuronales strukturiertes Lernen generieren. Das NSL-Framework bietet eine Bibliothek zum Kombinieren des Diagramms und der Beispielfunktionen, um die endgültigen Trainingsdaten für die Diagrammregularisierung zu erzeugen. Die resultierenden Trainingsdaten enthalten ursprüngliche Mustermerkmale sowie Merkmale ihrer entsprechenden Nachbarn.

In diesem Tutorial betrachten wir ungerichtete Kanten und verwenden maximal 3 Nachbarn pro Stichprobe, um Trainingsdaten mit Graphennachbarn zu erweitern.

def split_train_and_unsup(input_uri):
  'Separate the labeled and unlabeled instances.'

  tmp_dir = tempfile.mkdtemp(prefix='tfx-data')
  tfrecord_filenames = [
      os.path.join(input_uri, filename) for filename in os.listdir(input_uri)
  ]
  train_path = os.path.join(tmp_dir, 'train.tfrecord')
  unsup_path = os.path.join(tmp_dir, 'unsup.tfrecord')
  with tf.io.TFRecordWriter(train_path) as train_writer, \
       tf.io.TFRecordWriter(unsup_path) as unsup_writer:
    for tfrecord in tf.data.TFRecordDataset(
        tfrecord_filenames, compression_type='GZIP'):
      example = tf.train.Example()
      example.ParseFromString(tfrecord.numpy())
      if ('label_xf' not in example.features.feature or
          example.features.feature['label_xf'].int64_list.value[0] == -1):
        writer = unsup_writer
      else:
        writer = train_writer
      writer.write(tfrecord.numpy())
  return train_path, unsup_path


def gzip(filepath):
  with open(filepath, 'rb') as f_in:
    with gzip_lib.open(filepath + '.gz', 'wb') as f_out:
      shutil.copyfileobj(f_in, f_out)
  os.remove(filepath)


def copy_tfrecords(input_uri, output_uri):
  for filename in os.listdir(input_uri):
    input_filename = os.path.join(input_uri, filename)
    output_filename = os.path.join(output_uri, filename)
    shutil.copyfile(input_filename, output_filename)


@component
def GraphAugmentation(identified_examples: InputArtifact[Examples],
                      synthesized_graph: InputArtifact[SynthesizedGraph],
                      augmented_examples: OutputArtifact[Examples],
                      num_neighbors: Parameter[int],
                      component_name: Parameter[str]) -> None:

  # Get a list of the splits in input_data
  splits_list = artifact_utils.decode_split_names(
      split_names=identified_examples.split_names)

  train_input_uri = os.path.join(identified_examples.uri, 'Split-train')
  eval_input_uri = os.path.join(identified_examples.uri, 'Split-eval')
  train_graph_uri = os.path.join(synthesized_graph.uri, 'Split-train')
  train_output_uri = os.path.join(augmented_examples.uri, 'Split-train')
  eval_output_uri = os.path.join(augmented_examples.uri, 'Split-eval')

  os.mkdir(train_output_uri)
  os.mkdir(eval_output_uri)

  # Separate the labeled and unlabeled examples from the 'Split-train' split.
  train_path, unsup_path = split_train_and_unsup(train_input_uri)

  output_path = os.path.join(train_output_uri, 'nsl_train_data.tfr')
  pack_nbrs_args = dict(
      labeled_examples_path=train_path,
      unlabeled_examples_path=unsup_path,
      graph_path=os.path.join(train_graph_uri, 'graph.tsv'),
      output_training_data_path=output_path,
      add_undirected_edges=True,
      max_nbrs=num_neighbors)
  print('nsl.tools.pack_nbrs arguments:', pack_nbrs_args)
  nsl.tools.pack_nbrs(**pack_nbrs_args)

  # Downstream components expect gzip'ed TFRecords.
  gzip(output_path)

  # The test examples are left untouched and are simply copied over.
  copy_tfrecords(eval_input_uri, eval_output_uri)

  augmented_examples.split_names = identified_examples.split_names

  return
# Augments training data with graph neighbors.
graph_augmentation = GraphAugmentation(
    identified_examples=transform.outputs['transformed_examples'],
    synthesized_graph=synthesize_graph.outputs['synthesized_graph'],
    component_name=u'GraphAugmentation',
    num_neighbors=3)
context.run(graph_augmentation, enable_cache=False)
nsl.tools.pack_nbrs arguments: {'labeled_examples_path': '/tmp/tfx-datao1kluw5r/train.tfrecord', 'unlabeled_examples_path': '/tmp/tfx-datao1kluw5r/unsup.tfrecord', 'graph_path': '/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/SynthesizeGraph/synthesized_graph/6/Split-train/graph.tsv', 'output_training_data_path': '/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/GraphAugmentation/augmented_examples/8/Split-train/nsl_train_data.tfr', 'add_undirected_edges': True, 'max_nbrs': 3}
pprint_examples(graph_augmentation.outputs['augmented_examples'].get()[0], 6)
artifact: Artifact(artifact: id: 15
type_id: 5
uri: "/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/GraphAugmentation/augmented_examples/8"
properties {
  key: "split_names"
  value {
    string_value: "[\"train\", \"eval\"]"
  }
}
custom_properties {
  key: "name"
  value {
    string_value: "augmented_examples"
  }
}
custom_properties {
  key: "producer_component"
  value {
    string_value: "GraphAugmentation"
  }
}
custom_properties {
  key: "state"
  value {
    string_value: "published"
  }
}
custom_properties {
  key: "tfx_version"
  value {
    string_value: "1.0.0"
  }
}
state: LIVE
, artifact_type: id: 5
name: "Examples"
properties {
  key: "span"
  value: INT
}
properties {
  key: "split_names"
  value: STRING
}
properties {
  key: "version"
  value: INT
}
)
uri: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/GraphAugmentation/augmented_examples/8/Split-train
tfrecord_filenames: ['/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/GraphAugmentation/augmented_examples/8/Split-train/nsl_train_data.tfr.gz']
features {
  feature {
    key: "NL_num_nbrs"
    value {
      int64_list {
        value: 0
      }
    }
  }
  feature {
    key: "id"
    value {
      bytes_list {
        value: "5ea2f9ef-e6b5-4ba1-b278-1d08079ca6c3"
      }
    }
  }
  feature {
    key: "label_xf"
    value {
      int64_list {
        value: 0
      }
    }
  }
  feature {
    key: "text_xf"
    value {
      int64_list {
        value: 13
        value: 8
        value: 14
        value: 32
        value: 338
        value: 310
        value: 15
        value: 95
        value: 27
        value: 10001
        value: 9
        value: 31
        value: 1173
        value: 3153
        value: 43
        value: 495
        value: 10060
        value: 214
        value: 26
        value: 71
        value: 142
        value: 19
        value: 8
        value: 204
        value: 339
        value: 27
        value: 74
        value: 181
        value: 238
        value: 9
        value: 440
        value: 67
        value: 74
        value: 71
        value: 94
        value: 100
        value: 22
        value: 5442
        value: 8
        value: 1573
        value: 607
        value: 530
        value: 8
        value: 15
        value: 6
        value: 32
        value: 378
        value: 6292
        value: 207
        value: 2276
        value: 388
        value: 0
        value: 84
        value: 1023
        value: 154
        value: 65
        value: 155
        value: 52
        value: 0
        value: 10080
        value: 7871
        value: 65
        value: 250
        value: 74
        value: 3202
        value: 20
        value: 10000
        value: 3720
        value: 10020
        value: 10008
        value: 1282
        value: 3862
        value: 3
        value: 53
        value: 3952
        value: 110
        value: 1879
        value: 17
        value: 3153
        value: 14
        value: 166
        value: 19
        value: 2
        value: 1023
        value: 1007
        value: 9405
        value: 9
        value: 2
        value: 15
        value: 12
        value: 14
        value: 4504
        value: 4
        value: 109
        value: 158
        value: 1202
        value: 7
        value: 174
        value: 505
        value: 12
      }
    }
  }
}

features {
  feature {
    key: "NL_num_nbrs"
    value {
      int64_list {
        value: 0
      }
    }
  }
  feature {
    key: "id"
    value {
      bytes_list {
        value: "2532952c-2853-413e-8df6-e6d54a82da9d"
      }
    }
  }
  feature {
    key: "label_xf"
    value {
      int64_list {
        value: 0
      }
    }
  }
  feature {
    key: "text_xf"
    value {
      int64_list {
        value: 13
        value: 7
        value: 23
        value: 75
        value: 494
        value: 5
        value: 748
        value: 2155
        value: 307
        value: 91
        value: 19
        value: 8
        value: 6
        value: 499
        value: 763
        value: 5
        value: 2
        value: 1690
        value: 4
        value: 200
        value: 593
        value: 57
        value: 1244
        value: 120
        value: 2364
        value: 3
        value: 4407
        value: 21
        value: 0
        value: 10081
        value: 3
        value: 263
        value: 42
        value: 6947
        value: 2
        value: 169
        value: 185
        value: 21
        value: 8
        value: 5143
        value: 7
        value: 1339
        value: 2155
        value: 81
        value: 0
        value: 18
        value: 14
        value: 1468
        value: 0
        value: 86
        value: 986
        value: 14
        value: 2259
        value: 1790
        value: 562
        value: 3
        value: 284
        value: 200
        value: 401
        value: 5
        value: 668
        value: 19
        value: 17
        value: 58
        value: 1934
        value: 4
        value: 45
        value: 14
        value: 4212
        value: 113
        value: 43
        value: 135
        value: 7
        value: 753
        value: 7
        value: 224
        value: 23
        value: 1155
        value: 179
        value: 4
        value: 0
        value: 18
        value: 19
        value: 7
        value: 191
        value: 0
        value: 2047
        value: 4
        value: 10
        value: 3
        value: 283
        value: 42
        value: 401
        value: 5
        value: 668
        value: 4
        value: 90
        value: 234
        value: 10023
        value: 227
      }
    }
  }
}

features {
  feature {
    key: "NL_num_nbrs"
    value {
      int64_list {
        value: 0
      }
    }
  }
  feature {
    key: "id"
    value {
      bytes_list {
        value: "49970ec3-f99f-46a1-8bbe-8d153c1caa1a"
      }
    }
  }
  feature {
    key: "label_xf"
    value {
      int64_list {
        value: 0
      }
    }
  }
  feature {
    key: "text_xf"
    value {
      int64_list {
        value: 13
        value: 4577
        value: 7158
        value: 0
        value: 10047
        value: 3778
        value: 3346
        value: 9
        value: 2
        value: 758
        value: 1915
        value: 3
        value: 2280
        value: 1511
        value: 3
        value: 2003
        value: 10020
        value: 225
        value: 786
        value: 382
        value: 16
        value: 39
        value: 203
        value: 361
        value: 5
        value: 93
        value: 11
        value: 11
        value: 19
        value: 220
        value: 21
        value: 341
        value: 2
        value: 10000
        value: 966
        value: 0
        value: 77
        value: 4
        value: 6677
        value: 464
        value: 10071
        value: 5
        value: 10042
        value: 630
        value: 2
        value: 10044
        value: 404
        value: 2
        value: 10044
        value: 3
        value: 5
        value: 10008
        value: 0
        value: 1259
        value: 630
        value: 106
        value: 10042
        value: 6721
        value: 10
        value: 49
        value: 21
        value: 0
        value: 2071
        value: 20
        value: 1292
        value: 4
        value: 0
        value: 431
        value: 11
        value: 11
        value: 166
        value: 67
        value: 2342
        value: 5815
        value: 12
        value: 575
        value: 21
        value: 0
        value: 1691
        value: 537
        value: 4
        value: 0
        value: 3605
        value: 307
        value: 0
        value: 10054
        value: 1563
        value: 3115
        value: 467
        value: 4577
        value: 3
        value: 1069
        value: 1158
        value: 5
        value: 23
        value: 4279
        value: 6677
        value: 464
        value: 20
        value: 10004
      }
    }
  }
}

features {
  feature {
    key: "NL_num_nbrs"
    value {
      int64_list {
        value: 0
      }
    }
  }
  feature {
    key: "id"
    value {
      bytes_list {
        value: "fef861ab-db08-4c97-9157-b590790f3ef3"
      }
    }
  }
  feature {
    key: "label_xf"
    value {
      int64_list {
        value: 1
      }
    }
  }
  feature {
    key: "text_xf"
    value {
      int64_list {
        value: 13
        value: 8
        value: 6
        value: 0
        value: 251
        value: 4
        value: 18
        value: 20
        value: 2
        value: 6783
        value: 2295
        value: 2338
        value: 52
        value: 0
        value: 468
        value: 4
        value: 0
        value: 189
        value: 73
        value: 153
        value: 1294
        value: 17
        value: 90
        value: 234
        value: 935
        value: 16
        value: 25
        value: 10024
        value: 92
        value: 2
        value: 192
        value: 4218
        value: 3317
        value: 3
        value: 10098
        value: 20
        value: 2
        value: 356
        value: 4
        value: 565
        value: 334
        value: 382
        value: 36
        value: 6989
        value: 3
        value: 6065
        value: 2510
        value: 16
        value: 203
        value: 7264
        value: 2849
        value: 0
        value: 86
        value: 346
        value: 50
        value: 26
        value: 58
        value: 10020
        value: 5
        value: 1464
        value: 58
        value: 2081
        value: 2969
        value: 42
        value: 2
        value: 2364
        value: 3
        value: 1402
        value: 10062
        value: 138
        value: 147
        value: 614
        value: 115
        value: 29
        value: 90
        value: 105
        value: 2
        value: 223
        value: 18
        value: 9
        value: 160
        value: 324
        value: 3
        value: 24
        value: 12
        value: 1252
        value: 0
        value: 2142
        value: 10
        value: 1832
        value: 111
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
      }
    }
  }
}

features {
  feature {
    key: "NL_num_nbrs"
    value {
      int64_list {
        value: 0
      }
    }
  }
  feature {
    key: "id"
    value {
      bytes_list {
        value: "897af22a-ad05-4cca-96df-3f76808f51b3"
      }
    }
  }
  feature {
    key: "label_xf"
    value {
      int64_list {
        value: 1
      }
    }
  }
  feature {
    key: "text_xf"
    value {
      int64_list {
        value: 13
        value: 16
        value: 423
        value: 23
        value: 1367
        value: 30
        value: 0
        value: 363
        value: 12
        value: 153
        value: 3174
        value: 9
        value: 8
        value: 18
        value: 26
        value: 667
        value: 338
        value: 1372
        value: 0
        value: 86
        value: 46
        value: 9200
        value: 282
        value: 0
        value: 10091
        value: 4
        value: 0
        value: 694
        value: 10028
        value: 52
        value: 362
        value: 26
        value: 202
        value: 39
        value: 216
        value: 5
        value: 27
        value: 5822
        value: 19
        value: 52
        value: 58
        value: 362
        value: 26
        value: 202
        value: 39
        value: 474
        value: 0
        value: 10029
        value: 4
        value: 2
        value: 243
        value: 143
        value: 386
        value: 3
        value: 0
        value: 386
        value: 579
        value: 2
        value: 132
        value: 57
        value: 725
        value: 88
        value: 140
        value: 30
        value: 27
        value: 33
        value: 1359
        value: 29
        value: 8
        value: 567
        value: 35
        value: 106
        value: 230
        value: 60
        value: 0
        value: 3041
        value: 5
        value: 7879
        value: 28
        value: 281
        value: 110
        value: 111
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
        value: 1
      }
    }
  }
}

features {
  feature {
    key: "NL_num_nbrs"
    value {
      int64_list {
        value: 0
      }
    }
  }
  feature {
    key: "id"
    value {
      bytes_list {
        value: "cb42b672-39d3-4e7a-9521-a4610bf7cfde"
      }
    }
  }
  feature {
    key: "label_xf"
    value {
      int64_list {
        value: 1
      }
    }
  }
  feature {
    key: "text_xf"
    value {
      int64_list {
        value: 13
        value: 8
        value: 6
        value: 2
        value: 18
        value: 69
        value: 140
        value: 27
        value: 83
        value: 31
        value: 1877
        value: 905
        value: 9
        value: 10057
        value: 31
        value: 43
        value: 2115
        value: 36
        value: 32
        value: 2057
        value: 6133
        value: 10
        value: 6
        value: 32
        value: 2474
        value: 1614
        value: 3
        value: 2707
        value: 990
        value: 4
        value: 10067
        value: 9
        value: 2
        value: 1532
        value: 242
        value: 90
        value: 3757
        value: 3
        value: 90
        value: 10026
        value: 0
        value: 242
        value: 6
        value: 260
        value: 31
        value: 24
        value: 4
        value: 0
        value: 84
        value: 497
        value: 177
        value: 1151
        value: 777
        value: 9
        value: 397
        value: 552
        value: 7726
        value: 10051
        value: 34
        value: 14
        value: 379
        value: 33
        value: 1829
        value: 9
        value: 123
        value: 0
        value: 916
        value: 10028
        value: 7
        value: 64
        value: 571
        value: 12
        value: 8
        value: 18
        value: 27
        value: 687
        value: 9
        value: 30
        value: 5609
        value: 16
        value: 25
        value: 99
        value: 117
        value: 66
        value: 2
        value: 130
        value: 21
        value: 8
        value: 842
        value: 7726
        value: 10051
        value: 6
        value: 338
        value: 1107
        value: 3
        value: 24
        value: 10020
        value: 29
        value: 53
        value: 1476
      }
    }
  }
}

Die Trainerkomponente

Die Trainer Komponente Züge Modelle mit TensorFlow.

Erstellen eines Python Modul eine enthaltenden trainer_fn - Funktion, die einen Schätzer zurückgeben muss. Wenn Sie ein Keras Modell bevorzugen erstellt haben , können Sie dies tun und wandelt es dann in einen Schätzer mit keras.model_to_estimator() .

# Setup paths.
_trainer_module_file = 'imdb_trainer.py'
%%writefile {_trainer_module_file}

import neural_structured_learning as nsl

import tensorflow as tf

import tensorflow_model_analysis as tfma
import tensorflow_transform as tft
from tensorflow_transform.tf_metadata import schema_utils


NBR_FEATURE_PREFIX = 'NL_nbr_'
NBR_WEIGHT_SUFFIX = '_weight'
LABEL_KEY = 'label'
ID_FEATURE_KEY = 'id'

def _transformed_name(key):
  return key + '_xf'


def _transformed_names(keys):
  return [_transformed_name(key) for key in keys]


# Hyperparameters:
#
# We will use an instance of `HParams` to inclue various hyperparameters and
# constants used for training and evaluation. We briefly describe each of them
# below:
#
# -   max_seq_length: This is the maximum number of words considered from each
#                     movie review in this example.
# -   vocab_size: This is the size of the vocabulary considered for this
#                 example.
# -   oov_size: This is the out-of-vocabulary size considered for this example.
# -   distance_type: This is the distance metric used to regularize the sample
#                    with its neighbors.
# -   graph_regularization_multiplier: This controls the relative weight of the
#                                      graph regularization term in the overall
#                                      loss function.
# -   num_neighbors: The number of neighbors used for graph regularization. This
#                    value has to be less than or equal to the `num_neighbors`
#                    argument used above in the GraphAugmentation component when
#                    invoking `nsl.tools.pack_nbrs`.
# -   num_fc_units: The number of units in the fully connected layer of the
#                   neural network.
class HParams(object):
  """Hyperparameters used for training."""
  def __init__(self):
    ### dataset parameters
    # The following 3 values should match those defined in the Transform
    # Component.
    self.max_seq_length = 100
    self.vocab_size = 10000
    self.oov_size = 100
    ### Neural Graph Learning parameters
    self.distance_type = nsl.configs.DistanceType.L2
    self.graph_regularization_multiplier = 0.1
    # The following value has to be at most the value of 'num_neighbors' used
    # in the GraphAugmentation component.
    self.num_neighbors = 1
    ### Model Architecture
    self.num_embedding_dims = 16
    self.num_fc_units = 64

HPARAMS = HParams()


def optimizer_fn():
  """Returns an instance of `tf.Optimizer`."""
  return tf.compat.v1.train.RMSPropOptimizer(
    learning_rate=0.0001, decay=1e-6)


def build_train_op(loss, global_step):
  """Builds a train op to optimize the given loss using gradient descent."""
  with tf.name_scope('train'):
    optimizer = optimizer_fn()
    train_op = optimizer.minimize(loss=loss, global_step=global_step)
  return train_op


# Building the model:
#
# A neural network is created by stacking layers—this requires two main
# architectural decisions:
# * How many layers to use in the model?
# * How many *hidden units* to use for each layer?
#
# In this example, the input data consists of an array of word-indices. The
# labels to predict are either 0 or 1. We will use a feed-forward neural network
# as our base model in this tutorial.
def feed_forward_model(features, is_training, reuse=tf.compat.v1.AUTO_REUSE):
  """Builds a simple 2 layer feed forward neural network.

  The layers are effectively stacked sequentially to build the classifier. The
  first layer is an Embedding layer, which takes the integer-encoded vocabulary
  and looks up the embedding vector for each word-index. These vectors are
  learned as the model trains. The vectors add a dimension to the output array.
  The resulting dimensions are: (batch, sequence, embedding). Next is a global
  average pooling 1D layer, which reduces the dimensionality of its inputs from
  3D to 2D. This fixed-length output vector is piped through a fully-connected
  (Dense) layer with 16 hidden units. The last layer is densely connected with a
  single output node. Using the sigmoid activation function, this value is a
  float between 0 and 1, representing a probability, or confidence level.

  Args:
    features: A dictionary containing batch features returned from the
      `input_fn`, that include sample features, corresponding neighbor features,
      and neighbor weights.
    is_training: a Python Boolean value or a Boolean scalar Tensor, indicating
      whether to apply dropout.
    reuse: a Python Boolean value for reusing variable scope.

  Returns:
    logits: Tensor of shape [batch_size, 1].
    representations: Tensor of shape [batch_size, _] for graph regularization.
      This is the representation of each example at the graph regularization
      layer.
  """

  with tf.compat.v1.variable_scope('ff', reuse=reuse):
    inputs = features[_transformed_name('text')]
    embeddings = tf.compat.v1.get_variable(
        'embeddings',
        shape=[
            HPARAMS.vocab_size + HPARAMS.oov_size, HPARAMS.num_embedding_dims
        ])
    embedding_layer = tf.nn.embedding_lookup(embeddings, inputs)

    pooling_layer = tf.compat.v1.layers.AveragePooling1D(
        pool_size=HPARAMS.max_seq_length, strides=HPARAMS.max_seq_length)(
            embedding_layer)
    # Shape of pooling_layer is now [batch_size, 1, HPARAMS.num_embedding_dims]
    pooling_layer = tf.reshape(pooling_layer, [-1, HPARAMS.num_embedding_dims])

    dense_layer = tf.compat.v1.layers.Dense(
        16, activation='relu')(
            pooling_layer)

    output_layer = tf.compat.v1.layers.Dense(
        1, activation='sigmoid')(
            dense_layer)

    # Graph regularization will be done on the penultimate (dense) layer
    # because the output layer is a single floating point number.
    return output_layer, dense_layer


# A note on hidden units:
#
# The above model has two intermediate or "hidden" layers, between the input and
# output, and excluding the Embedding layer. The number of outputs (units,
# nodes, or neurons) is the dimension of the representational space for the
# layer. In other words, the amount of freedom the network is allowed when
# learning an internal representation. If a model has more hidden units
# (a higher-dimensional representation space), and/or more layers, then the
# network can learn more complex representations. However, it makes the network
# more computationally expensive and may lead to learning unwanted
# patterns—patterns that improve performance on training data but not on the
# test data. This is called overfitting.


# This function will be used to generate the embeddings for samples and their
# corresponding neighbors, which will then be used for graph regularization.
def embedding_fn(features, mode):
  """Returns the embedding corresponding to the given features.

  Args:
    features: A dictionary containing batch features returned from the
      `input_fn`, that include sample features, corresponding neighbor features,
      and neighbor weights.
    mode: Specifies if this is training, evaluation, or prediction. See
      tf.estimator.ModeKeys.

  Returns:
    The embedding that will be used for graph regularization.
  """
  is_training = (mode == tf.estimator.ModeKeys.TRAIN)
  _, embedding = feed_forward_model(features, is_training)
  return embedding


def feed_forward_model_fn(features, labels, mode, params, config):
  """Implementation of the model_fn for the base feed-forward model.

  Args:
    features: This is the first item returned from the `input_fn` passed to
      `train`, `evaluate`, and `predict`. This should be a single `Tensor` or
      `dict` of same.
    labels: This is the second item returned from the `input_fn` passed to
      `train`, `evaluate`, and `predict`. This should be a single `Tensor` or
      `dict` of same (for multi-head models). If mode is `ModeKeys.PREDICT`,
      `labels=None` will be passed. If the `model_fn`'s signature does not
      accept `mode`, the `model_fn` must still be able to handle `labels=None`.
    mode: Optional. Specifies if this training, evaluation or prediction. See
      `ModeKeys`.
    params: An HParams instance as returned by get_hyper_parameters().
    config: Optional configuration object. Will receive what is passed to
      Estimator in `config` parameter, or the default `config`. Allows updating
      things in your model_fn based on configuration such as `num_ps_replicas`,
      or `model_dir`. Unused currently.

  Returns:
     A `tf.estimator.EstimatorSpec` for the base feed-forward model. This does
     not include graph-based regularization.
  """

  is_training = mode == tf.estimator.ModeKeys.TRAIN

  # Build the computation graph.
  probabilities, _ = feed_forward_model(features, is_training)
  predictions = tf.round(probabilities)

  if mode == tf.estimator.ModeKeys.PREDICT:
    # labels will be None, and no loss to compute.
    cross_entropy_loss = None
    eval_metric_ops = None
  else:
    # Loss is required in train and eval modes.
    # Flatten 'probabilities' to 1-D.
    probabilities = tf.reshape(probabilities, shape=[-1])
    cross_entropy_loss = tf.compat.v1.keras.losses.binary_crossentropy(
        labels, probabilities)
    eval_metric_ops = {
        'accuracy': tf.compat.v1.metrics.accuracy(labels, predictions)
    }

  if is_training:
    global_step = tf.compat.v1.train.get_or_create_global_step()
    train_op = build_train_op(cross_entropy_loss, global_step)
  else:
    train_op = None

  return tf.estimator.EstimatorSpec(
      mode=mode,
      predictions={
          'probabilities': probabilities,
          'predictions': predictions
      },
      loss=cross_entropy_loss,
      train_op=train_op,
      eval_metric_ops=eval_metric_ops)


# Tf.Transform considers these features as "raw"
def _get_raw_feature_spec(schema):
  return schema_utils.schema_as_feature_spec(schema).feature_spec


def _gzip_reader_fn(filenames):
  """Small utility returning a record reader that can read gzip'ed files."""
  return tf.data.TFRecordDataset(
      filenames,
      compression_type='GZIP')


def _example_serving_receiver_fn(tf_transform_output, schema):
  """Build the serving in inputs.

  Args:
    tf_transform_output: A TFTransformOutput.
    schema: the schema of the input data.

  Returns:
    Tensorflow graph which parses examples, applying tf-transform to them.
  """
  raw_feature_spec = _get_raw_feature_spec(schema)
  raw_feature_spec.pop(LABEL_KEY)

  # We don't need the ID feature for serving.
  raw_feature_spec.pop(ID_FEATURE_KEY)

  raw_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(
      raw_feature_spec, default_batch_size=None)
  serving_input_receiver = raw_input_fn()

  transformed_features = tf_transform_output.transform_raw_features(
      serving_input_receiver.features)

  # Even though, LABEL_KEY was removed from 'raw_feature_spec', the transform
  # operation would have injected the transformed LABEL_KEY feature with a
  # default value.
  transformed_features.pop(_transformed_name(LABEL_KEY))
  return tf.estimator.export.ServingInputReceiver(
      transformed_features, serving_input_receiver.receiver_tensors)


def _eval_input_receiver_fn(tf_transform_output, schema):
  """Build everything needed for the tf-model-analysis to run the model.

  Args:
    tf_transform_output: A TFTransformOutput.
    schema: the schema of the input data.

  Returns:
    EvalInputReceiver function, which contains:

      - Tensorflow graph which parses raw untransformed features, applies the
        tf-transform preprocessing operators.
      - Set of raw, untransformed features.
      - Label against which predictions will be compared.
  """
  # Notice that the inputs are raw features, not transformed features here.
  raw_feature_spec = _get_raw_feature_spec(schema)

  # We don't need the ID feature for TFMA.
  raw_feature_spec.pop(ID_FEATURE_KEY)

  raw_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(
      raw_feature_spec, default_batch_size=None)
  serving_input_receiver = raw_input_fn()

  transformed_features = tf_transform_output.transform_raw_features(
      serving_input_receiver.features)

  labels = transformed_features.pop(_transformed_name(LABEL_KEY))
  return tfma.export.EvalInputReceiver(
      features=transformed_features,
      receiver_tensors=serving_input_receiver.receiver_tensors,
      labels=labels)


def _augment_feature_spec(feature_spec, num_neighbors):
  """Augments `feature_spec` to include neighbor features.
    Args:
      feature_spec: Dictionary of feature keys mapping to TF feature types.
      num_neighbors: Number of neighbors to use for feature key augmentation.
    Returns:
      An augmented `feature_spec` that includes neighbor feature keys.
  """
  for i in range(num_neighbors):
    feature_spec['{}{}_{}'.format(NBR_FEATURE_PREFIX, i, 'id')] = \
        tf.io.VarLenFeature(dtype=tf.string)
    # We don't care about the neighbor features corresponding to
    # _transformed_name(LABEL_KEY) because the LABEL_KEY feature will be
    # removed from the feature spec during training/evaluation.
    feature_spec['{}{}_{}'.format(NBR_FEATURE_PREFIX, i, 'text_xf')] = \
        tf.io.FixedLenFeature(shape=[HPARAMS.max_seq_length], dtype=tf.int64,
                              default_value=tf.constant(0, dtype=tf.int64,
                                                        shape=[HPARAMS.max_seq_length]))
    # The 'NL_num_nbrs' features is currently not used.

  # Set the neighbor weight feature keys.
  for i in range(num_neighbors):
    feature_spec['{}{}{}'.format(NBR_FEATURE_PREFIX, i, NBR_WEIGHT_SUFFIX)] = \
        tf.io.FixedLenFeature(shape=[1], dtype=tf.float32, default_value=[0.0])

  return feature_spec


def _input_fn(filenames, tf_transform_output, is_training, batch_size=200):
  """Generates features and labels for training or evaluation.

  Args:
    filenames: [str] list of CSV files to read data from.
    tf_transform_output: A TFTransformOutput.
    is_training: Boolean indicating if we are in training mode.
    batch_size: int First dimension size of the Tensors returned by input_fn

  Returns:
    A (features, indices) tuple where features is a dictionary of
      Tensors, and indices is a single Tensor of label indices.
  """
  transformed_feature_spec = (
      tf_transform_output.transformed_feature_spec().copy())

  # During training, NSL uses augmented training data (which includes features
  # from graph neighbors). So, update the feature spec accordingly. This needs
  # to be done because we are using different schemas for NSL training and eval,
  # but the Trainer Component only accepts a single schema.
  if is_training:
    transformed_feature_spec =_augment_feature_spec(transformed_feature_spec,
                                                    HPARAMS.num_neighbors)

  dataset = tf.data.experimental.make_batched_features_dataset(
      filenames, batch_size, transformed_feature_spec, reader=_gzip_reader_fn)

  transformed_features = tf.compat.v1.data.make_one_shot_iterator(
      dataset).get_next()
  # We pop the label because we do not want to use it as a feature while we're
  # training.
  return transformed_features, transformed_features.pop(
      _transformed_name(LABEL_KEY))


# TFX will call this function
def trainer_fn(hparams, schema):
  """Build the estimator using the high level API.
  Args:
    hparams: Holds hyperparameters used to train the model as name/value pairs.
    schema: Holds the schema of the training examples.
  Returns:
    A dict of the following:

      - estimator: The estimator that will be used for training and eval.
      - train_spec: Spec for training.
      - eval_spec: Spec for eval.
      - eval_input_receiver_fn: Input function for eval.
  """
  train_batch_size = 40
  eval_batch_size = 40

  tf_transform_output = tft.TFTransformOutput(hparams.transform_output)

  train_input_fn = lambda: _input_fn(
      hparams.train_files,
      tf_transform_output,
      is_training=True,
      batch_size=train_batch_size)

  eval_input_fn = lambda: _input_fn(
      hparams.eval_files,
      tf_transform_output,
      is_training=False,
      batch_size=eval_batch_size)

  train_spec = tf.estimator.TrainSpec(
      train_input_fn,
      max_steps=hparams.train_steps)

  serving_receiver_fn = lambda: _example_serving_receiver_fn(
      tf_transform_output, schema)

  exporter = tf.estimator.FinalExporter('imdb', serving_receiver_fn)
  eval_spec = tf.estimator.EvalSpec(
      eval_input_fn,
      steps=hparams.eval_steps,
      exporters=[exporter],
      name='imdb-eval')

  run_config = tf.estimator.RunConfig(
      save_checkpoints_steps=999, keep_checkpoint_max=1)

  run_config = run_config.replace(model_dir=hparams.serving_model_dir)

  estimator = tf.estimator.Estimator(
      model_fn=feed_forward_model_fn, config=run_config, params=HPARAMS)

  # Create a graph regularization config.
  graph_reg_config = nsl.configs.make_graph_reg_config(
      max_neighbors=HPARAMS.num_neighbors,
      multiplier=HPARAMS.graph_regularization_multiplier,
      distance_type=HPARAMS.distance_type,
      sum_over_axis=-1)

  # Invoke the Graph Regularization Estimator wrapper to incorporate
  # graph-based regularization for training.
  graph_nsl_estimator = nsl.estimator.add_graph_regularization(
      estimator,
      embedding_fn,
      optimizer_fn=optimizer_fn,
      graph_reg_config=graph_reg_config)

  # Create an input receiver for TFMA processing
  receiver_fn = lambda: _eval_input_receiver_fn(
      tf_transform_output, schema)

  return {
      'estimator': graph_nsl_estimator,
      'train_spec': train_spec,
      'eval_spec': eval_spec,
      'eval_input_receiver_fn': receiver_fn
  }
Writing imdb_trainer.py

Erstellen und die laufen Trainer Komponente, die Datei vorbei , dass wir oben erstellt.

# Uses user-provided Python function that implements a model using TensorFlow's
# Estimators API.
trainer = Trainer(
    module_file=_trainer_module_file,
    custom_executor_spec=executor_spec.ExecutorClassSpec(
        trainer_executor.Executor),
    transformed_examples=graph_augmentation.outputs['augmented_examples'],
    schema=schema_gen.outputs['schema'],
    transform_graph=transform.outputs['transform_graph'],
    train_args=trainer_pb2.TrainArgs(num_steps=10000),
    eval_args=trainer_pb2.EvalArgs(num_steps=5000))
context.run(trainer)
WARNING:absl:`custom_executor_spec` is deprecated. Please customize component directly.
WARNING:absl:`transformed_examples` is deprecated. Please use `examples` instead.
2021-07-27 09:14:50.215390: W ml_metadata/metadata_store/rdbms_metadata_access_object.cc:623] No property is defined for the Type
2021-07-27 09:14:50.219967: W ml_metadata/metadata_store/rdbms_metadata_access_object.cc:623] No property is defined for the Type
WARNING:absl:Examples artifact does not have payload_format custom property. Falling back to FORMAT_TF_EXAMPLE
WARNING:absl:Examples artifact does not have payload_format custom property. Falling back to FORMAT_TF_EXAMPLE
WARNING:absl:Examples artifact does not have payload_format custom property. Falling back to FORMAT_TF_EXAMPLE
running bdist_wheel
running build
running build_py
creating build
creating build/lib
copying imdb_transform.py -> build/lib
copying imdb_trainer.py -> build/lib
installing to /tmp/tmpi_e26_65
running install
running install_lib
copying build/lib/imdb_trainer.py -> /tmp/tmpi_e26_65
copying build/lib/imdb_transform.py -> /tmp/tmpi_e26_65
running install_egg_info
running egg_info
creating tfx_user_code_Trainer.egg-info
writing tfx_user_code_Trainer.egg-info/PKG-INFO
writing dependency_links to tfx_user_code_Trainer.egg-info/dependency_links.txt
writing top-level names to tfx_user_code_Trainer.egg-info/top_level.txt
writing manifest file 'tfx_user_code_Trainer.egg-info/SOURCES.txt'
reading manifest file 'tfx_user_code_Trainer.egg-info/SOURCES.txt'
writing manifest file 'tfx_user_code_Trainer.egg-info/SOURCES.txt'
Copying tfx_user_code_Trainer.egg-info to /tmp/tmpi_e26_65/tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d-py3.7.egg-info
running install_scripts
creating /tmp/tmpi_e26_65/tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d.dist-info/WHEEL
creating '/tmp/tmpeptp4r8g/tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d-py3-none-any.whl' and adding '/tmp/tmpi_e26_65' to it
adding 'imdb_trainer.py'
adding 'imdb_transform.py'
adding 'tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d.dist-info/METADATA'
adding 'tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d.dist-info/WHEEL'
adding 'tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d.dist-info/top_level.txt'
adding 'tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d.dist-info/RECORD'
removing /tmp/tmpi_e26_65
Processing /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/_wheels/tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d-py3-none-any.whl
Installing collected packages: tfx-user-code-Trainer
Successfully installed tfx-user-code-Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 999, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 1, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
WARNING: You are using pip version 21.1.3; however, version 21.2.1 is available.
You should consider upgrading via the '/tmpfs/src/tf_docs_env/bin/python -m pip install --upgrade pip' command.
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 999, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 1, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Not using Distribute Coordinator.
INFO:tensorflow:Not using Distribute Coordinator.
INFO:tensorflow:Running training and evaluation locally (non-distributed).
INFO:tensorflow:Running training and evaluation locally (non-distributed).
INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps 999 or save_checkpoints_secs None.
INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps 999 or save_checkpoints_secs None.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/rmsprop.py:123: calling Ones.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/rmsprop.py:123: calling Ones.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py:5049: calling gather (from tensorflow.python.ops.array_ops) with validate_indices is deprecated and will be removed in a future version.
Instructions for updating:
The `validate_indices` argument has no effect. Indices are always validated on CPU and never validated on GPU.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py:5049: calling gather (from tensorflow.python.ops.array_ops) with validate_indices is deprecated and will be removed in a future version.
Instructions for updating:
The `validate_indices` argument has no effect. Indices are always validated on CPU and never validated on GPU.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
2021-07-27 09:14:52.531561: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:14:52.531986: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
pciBusID: 0000:00:05.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2021-07-27 09:14:52.532161: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:14:52.532492: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:14:52.532785: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-07-27 09:14:52.532840: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-07-27 09:14:52.532849: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264]      0 
2021-07-27 09:14:52.532856: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0:   N 
2021-07-27 09:14:52.532977: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:14:52.533323: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:14:52.533620: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14646 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0)
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
2021-07-27 09:14:53.258544: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
2021-07-27 09:14:55.355026: I tensorflow/stream_executor/cuda/cuda_dnn.cc:359] Loaded cuDNN version 8100
2021-07-27 09:14:55.359182: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.11
2021-07-27 09:14:55.748941: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.11
INFO:tensorflow:loss = 0.69322926, step = 0
INFO:tensorflow:loss = 0.69322926, step = 0
INFO:tensorflow:global_step/sec: 211.694
INFO:tensorflow:global_step/sec: 211.694
INFO:tensorflow:loss = 0.69236696, step = 100 (0.474 sec)
INFO:tensorflow:loss = 0.69236696, step = 100 (0.474 sec)
INFO:tensorflow:global_step/sec: 275.739
INFO:tensorflow:global_step/sec: 275.739
INFO:tensorflow:loss = 0.6926228, step = 200 (0.363 sec)
INFO:tensorflow:loss = 0.6926228, step = 200 (0.363 sec)
INFO:tensorflow:global_step/sec: 274.571
INFO:tensorflow:global_step/sec: 274.571
INFO:tensorflow:loss = 0.691752, step = 300 (0.365 sec)
INFO:tensorflow:loss = 0.691752, step = 300 (0.365 sec)
INFO:tensorflow:global_step/sec: 274.726
INFO:tensorflow:global_step/sec: 274.726
INFO:tensorflow:loss = 0.69001484, step = 400 (0.364 sec)
INFO:tensorflow:loss = 0.69001484, step = 400 (0.364 sec)
INFO:tensorflow:global_step/sec: 270.769
INFO:tensorflow:global_step/sec: 270.769
INFO:tensorflow:loss = 0.6899564, step = 500 (0.369 sec)
INFO:tensorflow:loss = 0.6899564, step = 500 (0.369 sec)
INFO:tensorflow:global_step/sec: 277.988
INFO:tensorflow:global_step/sec: 277.988
INFO:tensorflow:loss = 0.6896905, step = 600 (0.359 sec)
INFO:tensorflow:loss = 0.6896905, step = 600 (0.359 sec)
INFO:tensorflow:global_step/sec: 280.015
INFO:tensorflow:global_step/sec: 280.015
INFO:tensorflow:loss = 0.6895627, step = 700 (0.357 sec)
INFO:tensorflow:loss = 0.6895627, step = 700 (0.357 sec)
INFO:tensorflow:global_step/sec: 278.32
INFO:tensorflow:global_step/sec: 278.32
INFO:tensorflow:loss = 0.68509215, step = 800 (0.359 sec)
INFO:tensorflow:loss = 0.68509215, step = 800 (0.359 sec)
INFO:tensorflow:global_step/sec: 285.247
INFO:tensorflow:global_step/sec: 285.247
INFO:tensorflow:loss = 0.688862, step = 900 (0.351 sec)
INFO:tensorflow:loss = 0.688862, step = 900 (0.351 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 999...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 999...
INFO:tensorflow:Saving checkpoints for 999 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Saving checkpoints for 999 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/saver.py:971: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to delete files with this prefix.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/saver.py:971: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to delete files with this prefix.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 999...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 999...
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-07-27T09:15:00
INFO:tensorflow:Starting evaluation at 2021-07-27T09:15:00
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt-999
2021-07-27 09:15:00.071872: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:00.072285: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
pciBusID: 0000:00:05.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2021-07-27 09:15:00.072492: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:00.072873: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:00.073192: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-07-27 09:15:00.073239: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-07-27 09:15:00.073247: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264]      0 
2021-07-27 09:15:00.073256: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0:   N 
2021-07-27 09:15:00.073432: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:00.073877: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:00.074208: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14646 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0)
INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt-999
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Evaluation [500/5000]
INFO:tensorflow:Evaluation [500/5000]
INFO:tensorflow:Evaluation [1000/5000]
INFO:tensorflow:Evaluation [1000/5000]
INFO:tensorflow:Evaluation [1500/5000]
INFO:tensorflow:Evaluation [1500/5000]
INFO:tensorflow:Evaluation [2000/5000]
INFO:tensorflow:Evaluation [2000/5000]
INFO:tensorflow:Evaluation [2500/5000]
INFO:tensorflow:Evaluation [2500/5000]
INFO:tensorflow:Evaluation [3000/5000]
INFO:tensorflow:Evaluation [3000/5000]
INFO:tensorflow:Evaluation [3500/5000]
INFO:tensorflow:Evaluation [3500/5000]
INFO:tensorflow:Evaluation [4000/5000]
INFO:tensorflow:Evaluation [4000/5000]
INFO:tensorflow:Evaluation [4500/5000]
INFO:tensorflow:Evaluation [4500/5000]
INFO:tensorflow:Evaluation [5000/5000]
INFO:tensorflow:Evaluation [5000/5000]
INFO:tensorflow:Inference Time : 5.49521s
INFO:tensorflow:Inference Time : 5.49521s
INFO:tensorflow:Finished evaluation at 2021-07-27-09:15:05
INFO:tensorflow:Finished evaluation at 2021-07-27-09:15:05
INFO:tensorflow:Saving dict for global step 999: accuracy = 0.6677, global_step = 999, loss = 0.68569756
INFO:tensorflow:Saving dict for global step 999: accuracy = 0.6677, global_step = 999, loss = 0.68569756
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 999: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt-999
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 999: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt-999
INFO:tensorflow:global_step/sec: 16.4135
INFO:tensorflow:global_step/sec: 16.4135
INFO:tensorflow:loss = 0.68576753, step = 1000 (6.092 sec)
INFO:tensorflow:loss = 0.68576753, step = 1000 (6.092 sec)
INFO:tensorflow:global_step/sec: 276.683
INFO:tensorflow:global_step/sec: 276.683
INFO:tensorflow:loss = 0.6788543, step = 1100 (0.362 sec)
INFO:tensorflow:loss = 0.6788543, step = 1100 (0.362 sec)
INFO:tensorflow:global_step/sec: 274.503
INFO:tensorflow:global_step/sec: 274.503
INFO:tensorflow:loss = 0.6834281, step = 1200 (0.365 sec)
INFO:tensorflow:loss = 0.6834281, step = 1200 (0.365 sec)
INFO:tensorflow:global_step/sec: 274.221
INFO:tensorflow:global_step/sec: 274.221
INFO:tensorflow:loss = 0.68370265, step = 1300 (0.365 sec)
INFO:tensorflow:loss = 0.68370265, step = 1300 (0.365 sec)
INFO:tensorflow:global_step/sec: 272.284
INFO:tensorflow:global_step/sec: 272.284
INFO:tensorflow:loss = 0.6844999, step = 1400 (0.367 sec)
INFO:tensorflow:loss = 0.6844999, step = 1400 (0.367 sec)
INFO:tensorflow:global_step/sec: 272.698
INFO:tensorflow:global_step/sec: 272.698
INFO:tensorflow:loss = 0.68151146, step = 1500 (0.367 sec)
INFO:tensorflow:loss = 0.68151146, step = 1500 (0.367 sec)
INFO:tensorflow:global_step/sec: 277.091
INFO:tensorflow:global_step/sec: 277.091
INFO:tensorflow:loss = 0.6727173, step = 1600 (0.361 sec)
INFO:tensorflow:loss = 0.6727173, step = 1600 (0.361 sec)
INFO:tensorflow:global_step/sec: 274.492
INFO:tensorflow:global_step/sec: 274.492
INFO:tensorflow:loss = 0.67641914, step = 1700 (0.364 sec)
INFO:tensorflow:loss = 0.67641914, step = 1700 (0.364 sec)
INFO:tensorflow:global_step/sec: 277.889
INFO:tensorflow:global_step/sec: 277.889
INFO:tensorflow:loss = 0.66065145, step = 1800 (0.360 sec)
INFO:tensorflow:loss = 0.66065145, step = 1800 (0.360 sec)
INFO:tensorflow:global_step/sec: 278.998
INFO:tensorflow:global_step/sec: 278.998
INFO:tensorflow:loss = 0.6694759, step = 1900 (0.359 sec)
INFO:tensorflow:loss = 0.6694759, step = 1900 (0.359 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1998...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1998...
INFO:tensorflow:Saving checkpoints for 1998 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Saving checkpoints for 1998 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1998...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1998...
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:global_step/sec: 229.839
INFO:tensorflow:global_step/sec: 229.839
INFO:tensorflow:loss = 0.6578656, step = 2000 (0.434 sec)
INFO:tensorflow:loss = 0.6578656, step = 2000 (0.434 sec)
INFO:tensorflow:global_step/sec: 272.315
INFO:tensorflow:global_step/sec: 272.315
INFO:tensorflow:loss = 0.67634356, step = 2100 (0.367 sec)
INFO:tensorflow:loss = 0.67634356, step = 2100 (0.367 sec)
INFO:tensorflow:global_step/sec: 282.632
INFO:tensorflow:global_step/sec: 282.632
INFO:tensorflow:loss = 0.64694864, step = 2200 (0.354 sec)
INFO:tensorflow:loss = 0.64694864, step = 2200 (0.354 sec)
INFO:tensorflow:global_step/sec: 276.824
INFO:tensorflow:global_step/sec: 276.824
INFO:tensorflow:loss = 0.66624814, step = 2300 (0.361 sec)
INFO:tensorflow:loss = 0.66624814, step = 2300 (0.361 sec)
INFO:tensorflow:global_step/sec: 269.425
INFO:tensorflow:global_step/sec: 269.425
INFO:tensorflow:loss = 0.6768476, step = 2400 (0.371 sec)
INFO:tensorflow:loss = 0.6768476, step = 2400 (0.371 sec)
INFO:tensorflow:global_step/sec: 265.784
INFO:tensorflow:global_step/sec: 265.784
INFO:tensorflow:loss = 0.6642151, step = 2500 (0.376 sec)
INFO:tensorflow:loss = 0.6642151, step = 2500 (0.376 sec)
INFO:tensorflow:global_step/sec: 277.728
INFO:tensorflow:global_step/sec: 277.728
INFO:tensorflow:loss = 0.6666919, step = 2600 (0.361 sec)
INFO:tensorflow:loss = 0.6666919, step = 2600 (0.361 sec)
INFO:tensorflow:global_step/sec: 276.673
INFO:tensorflow:global_step/sec: 276.673
INFO:tensorflow:loss = 0.6315029, step = 2700 (0.361 sec)
INFO:tensorflow:loss = 0.6315029, step = 2700 (0.361 sec)
INFO:tensorflow:global_step/sec: 266.139
INFO:tensorflow:global_step/sec: 266.139
INFO:tensorflow:loss = 0.61445504, step = 2800 (0.376 sec)
INFO:tensorflow:loss = 0.61445504, step = 2800 (0.376 sec)
INFO:tensorflow:global_step/sec: 268.77
INFO:tensorflow:global_step/sec: 268.77
INFO:tensorflow:loss = 0.6582615, step = 2900 (0.372 sec)
INFO:tensorflow:loss = 0.6582615, step = 2900 (0.372 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 2997...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 2997...
INFO:tensorflow:Saving checkpoints for 2997 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Saving checkpoints for 2997 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 2997...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 2997...
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:global_step/sec: 227.589
INFO:tensorflow:global_step/sec: 227.589
INFO:tensorflow:loss = 0.6330004, step = 3000 (0.440 sec)
INFO:tensorflow:loss = 0.6330004, step = 3000 (0.440 sec)
INFO:tensorflow:global_step/sec: 257.036
INFO:tensorflow:global_step/sec: 257.036
INFO:tensorflow:loss = 0.6225431, step = 3100 (0.389 sec)
INFO:tensorflow:loss = 0.6225431, step = 3100 (0.389 sec)
INFO:tensorflow:global_step/sec: 267.359
INFO:tensorflow:global_step/sec: 267.359
INFO:tensorflow:loss = 0.59520555, step = 3200 (0.374 sec)
INFO:tensorflow:loss = 0.59520555, step = 3200 (0.374 sec)
INFO:tensorflow:global_step/sec: 246.66
INFO:tensorflow:global_step/sec: 246.66
INFO:tensorflow:loss = 0.6436679, step = 3300 (0.406 sec)
INFO:tensorflow:loss = 0.6436679, step = 3300 (0.406 sec)
INFO:tensorflow:global_step/sec: 235.989
INFO:tensorflow:global_step/sec: 235.989
INFO:tensorflow:loss = 0.6019606, step = 3400 (0.424 sec)
INFO:tensorflow:loss = 0.6019606, step = 3400 (0.424 sec)
INFO:tensorflow:global_step/sec: 256.192
INFO:tensorflow:global_step/sec: 256.192
INFO:tensorflow:loss = 0.6313018, step = 3500 (0.390 sec)
INFO:tensorflow:loss = 0.6313018, step = 3500 (0.390 sec)
INFO:tensorflow:global_step/sec: 252.023
INFO:tensorflow:global_step/sec: 252.023
INFO:tensorflow:loss = 0.63757026, step = 3600 (0.397 sec)
INFO:tensorflow:loss = 0.63757026, step = 3600 (0.397 sec)
INFO:tensorflow:global_step/sec: 243.633
INFO:tensorflow:global_step/sec: 243.633
INFO:tensorflow:loss = 0.5859189, step = 3700 (0.410 sec)
INFO:tensorflow:loss = 0.5859189, step = 3700 (0.410 sec)
INFO:tensorflow:global_step/sec: 242.902
INFO:tensorflow:global_step/sec: 242.902
INFO:tensorflow:loss = 0.54519206, step = 3800 (0.412 sec)
INFO:tensorflow:loss = 0.54519206, step = 3800 (0.412 sec)
INFO:tensorflow:global_step/sec: 258.566
INFO:tensorflow:global_step/sec: 258.566
INFO:tensorflow:loss = 0.5704173, step = 3900 (0.387 sec)
INFO:tensorflow:loss = 0.5704173, step = 3900 (0.387 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 3996...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 3996...
INFO:tensorflow:Saving checkpoints for 3996 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Saving checkpoints for 3996 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 3996...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 3996...
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:global_step/sec: 215.072
INFO:tensorflow:global_step/sec: 215.072
INFO:tensorflow:loss = 0.59502316, step = 4000 (0.465 sec)
INFO:tensorflow:loss = 0.59502316, step = 4000 (0.465 sec)
INFO:tensorflow:global_step/sec: 252.848
INFO:tensorflow:global_step/sec: 252.848
INFO:tensorflow:loss = 0.5610915, step = 4100 (0.396 sec)
INFO:tensorflow:loss = 0.5610915, step = 4100 (0.396 sec)
INFO:tensorflow:global_step/sec: 253.373
INFO:tensorflow:global_step/sec: 253.373
INFO:tensorflow:loss = 0.5591887, step = 4200 (0.394 sec)
INFO:tensorflow:loss = 0.5591887, step = 4200 (0.394 sec)
INFO:tensorflow:global_step/sec: 251.582
INFO:tensorflow:global_step/sec: 251.582
INFO:tensorflow:loss = 0.49893284, step = 4300 (0.398 sec)
INFO:tensorflow:loss = 0.49893284, step = 4300 (0.398 sec)
INFO:tensorflow:global_step/sec: 248.965
INFO:tensorflow:global_step/sec: 248.965
INFO:tensorflow:loss = 0.5927149, step = 4400 (0.402 sec)
INFO:tensorflow:loss = 0.5927149, step = 4400 (0.402 sec)
INFO:tensorflow:global_step/sec: 257.041
INFO:tensorflow:global_step/sec: 257.041
INFO:tensorflow:loss = 0.5431159, step = 4500 (0.389 sec)
INFO:tensorflow:loss = 0.5431159, step = 4500 (0.389 sec)
INFO:tensorflow:global_step/sec: 252.152
INFO:tensorflow:global_step/sec: 252.152
INFO:tensorflow:loss = 0.5833171, step = 4600 (0.397 sec)
INFO:tensorflow:loss = 0.5833171, step = 4600 (0.397 sec)
INFO:tensorflow:global_step/sec: 250.862
INFO:tensorflow:global_step/sec: 250.862
INFO:tensorflow:loss = 0.5355993, step = 4700 (0.399 sec)
INFO:tensorflow:loss = 0.5355993, step = 4700 (0.399 sec)
INFO:tensorflow:global_step/sec: 253.628
INFO:tensorflow:global_step/sec: 253.628
INFO:tensorflow:loss = 0.619254, step = 4800 (0.394 sec)
INFO:tensorflow:loss = 0.619254, step = 4800 (0.394 sec)
INFO:tensorflow:global_step/sec: 250.356
INFO:tensorflow:global_step/sec: 250.356
INFO:tensorflow:loss = 0.54462916, step = 4900 (0.399 sec)
INFO:tensorflow:loss = 0.54462916, step = 4900 (0.399 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 4995...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 4995...
INFO:tensorflow:Saving checkpoints for 4995 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Saving checkpoints for 4995 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 4995...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 4995...
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:global_step/sec: 215.532
INFO:tensorflow:global_step/sec: 215.532
INFO:tensorflow:loss = 0.5465646, step = 5000 (0.464 sec)
INFO:tensorflow:loss = 0.5465646, step = 5000 (0.464 sec)
INFO:tensorflow:global_step/sec: 246.76
INFO:tensorflow:global_step/sec: 246.76
INFO:tensorflow:loss = 0.51135373, step = 5100 (0.405 sec)
INFO:tensorflow:loss = 0.51135373, step = 5100 (0.405 sec)
INFO:tensorflow:global_step/sec: 250.25
INFO:tensorflow:global_step/sec: 250.25
INFO:tensorflow:loss = 0.591072, step = 5200 (0.400 sec)
INFO:tensorflow:loss = 0.591072, step = 5200 (0.400 sec)
INFO:tensorflow:global_step/sec: 253.138
INFO:tensorflow:global_step/sec: 253.138
INFO:tensorflow:loss = 0.48519018, step = 5300 (0.395 sec)
INFO:tensorflow:loss = 0.48519018, step = 5300 (0.395 sec)
INFO:tensorflow:global_step/sec: 250.641
INFO:tensorflow:global_step/sec: 250.641
INFO:tensorflow:loss = 0.47934756, step = 5400 (0.399 sec)
INFO:tensorflow:loss = 0.47934756, step = 5400 (0.399 sec)
INFO:tensorflow:global_step/sec: 251.855
INFO:tensorflow:global_step/sec: 251.855
INFO:tensorflow:loss = 0.44151902, step = 5500 (0.397 sec)
INFO:tensorflow:loss = 0.44151902, step = 5500 (0.397 sec)
INFO:tensorflow:global_step/sec: 250.974
INFO:tensorflow:global_step/sec: 250.974
INFO:tensorflow:loss = 0.4974475, step = 5600 (0.398 sec)
INFO:tensorflow:loss = 0.4974475, step = 5600 (0.398 sec)
INFO:tensorflow:global_step/sec: 252.506
INFO:tensorflow:global_step/sec: 252.506
INFO:tensorflow:loss = 0.57592094, step = 5700 (0.396 sec)
INFO:tensorflow:loss = 0.57592094, step = 5700 (0.396 sec)
INFO:tensorflow:global_step/sec: 250.984
INFO:tensorflow:global_step/sec: 250.984
INFO:tensorflow:loss = 0.47122863, step = 5800 (0.398 sec)
INFO:tensorflow:loss = 0.47122863, step = 5800 (0.398 sec)
INFO:tensorflow:global_step/sec: 260.372
INFO:tensorflow:global_step/sec: 260.372
INFO:tensorflow:loss = 0.43418798, step = 5900 (0.384 sec)
INFO:tensorflow:loss = 0.43418798, step = 5900 (0.384 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 5994...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 5994...
INFO:tensorflow:Saving checkpoints for 5994 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Saving checkpoints for 5994 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 5994...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 5994...
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:global_step/sec: 221.857
INFO:tensorflow:global_step/sec: 221.857
INFO:tensorflow:loss = 0.5193545, step = 6000 (0.451 sec)
INFO:tensorflow:loss = 0.5193545, step = 6000 (0.451 sec)
INFO:tensorflow:global_step/sec: 258.13
INFO:tensorflow:global_step/sec: 258.13
INFO:tensorflow:loss = 0.5118042, step = 6100 (0.387 sec)
INFO:tensorflow:loss = 0.5118042, step = 6100 (0.387 sec)
INFO:tensorflow:global_step/sec: 256.472
INFO:tensorflow:global_step/sec: 256.472
INFO:tensorflow:loss = 0.520128, step = 6200 (0.390 sec)
INFO:tensorflow:loss = 0.520128, step = 6200 (0.390 sec)
INFO:tensorflow:global_step/sec: 271.654
INFO:tensorflow:global_step/sec: 271.654
INFO:tensorflow:loss = 0.49134195, step = 6300 (0.368 sec)
INFO:tensorflow:loss = 0.49134195, step = 6300 (0.368 sec)
INFO:tensorflow:global_step/sec: 264.465
INFO:tensorflow:global_step/sec: 264.465
INFO:tensorflow:loss = 0.48210445, step = 6400 (0.378 sec)
INFO:tensorflow:loss = 0.48210445, step = 6400 (0.378 sec)
INFO:tensorflow:global_step/sec: 256.224
INFO:tensorflow:global_step/sec: 256.224
INFO:tensorflow:loss = 0.47818863, step = 6500 (0.390 sec)
INFO:tensorflow:loss = 0.47818863, step = 6500 (0.390 sec)
INFO:tensorflow:global_step/sec: 255.261
INFO:tensorflow:global_step/sec: 255.261
INFO:tensorflow:loss = 0.5382578, step = 6600 (0.392 sec)
INFO:tensorflow:loss = 0.5382578, step = 6600 (0.392 sec)
INFO:tensorflow:global_step/sec: 272.234
INFO:tensorflow:global_step/sec: 272.234
INFO:tensorflow:loss = 0.47908917, step = 6700 (0.367 sec)
INFO:tensorflow:loss = 0.47908917, step = 6700 (0.367 sec)
INFO:tensorflow:global_step/sec: 271.08
INFO:tensorflow:global_step/sec: 271.08
INFO:tensorflow:loss = 0.47381815, step = 6800 (0.369 sec)
INFO:tensorflow:loss = 0.47381815, step = 6800 (0.369 sec)
INFO:tensorflow:global_step/sec: 259.457
INFO:tensorflow:global_step/sec: 259.457
INFO:tensorflow:loss = 0.39517635, step = 6900 (0.386 sec)
INFO:tensorflow:loss = 0.39517635, step = 6900 (0.386 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 6993...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 6993...
INFO:tensorflow:Saving checkpoints for 6993 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Saving checkpoints for 6993 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 6993...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 6993...
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:global_step/sec: 223.964
INFO:tensorflow:global_step/sec: 223.964
INFO:tensorflow:loss = 0.42984772, step = 7000 (0.446 sec)
INFO:tensorflow:loss = 0.42984772, step = 7000 (0.446 sec)
INFO:tensorflow:global_step/sec: 278.579
INFO:tensorflow:global_step/sec: 278.579
INFO:tensorflow:loss = 0.40576378, step = 7100 (0.359 sec)
INFO:tensorflow:loss = 0.40576378, step = 7100 (0.359 sec)
INFO:tensorflow:global_step/sec: 262.381
INFO:tensorflow:global_step/sec: 262.381
INFO:tensorflow:loss = 0.502881, step = 7200 (0.381 sec)
INFO:tensorflow:loss = 0.502881, step = 7200 (0.381 sec)
INFO:tensorflow:global_step/sec: 254.754
INFO:tensorflow:global_step/sec: 254.754
INFO:tensorflow:loss = 0.4110231, step = 7300 (0.393 sec)
INFO:tensorflow:loss = 0.4110231, step = 7300 (0.393 sec)
INFO:tensorflow:global_step/sec: 249.425
INFO:tensorflow:global_step/sec: 249.425
INFO:tensorflow:loss = 0.37873524, step = 7400 (0.401 sec)
INFO:tensorflow:loss = 0.37873524, step = 7400 (0.401 sec)
INFO:tensorflow:global_step/sec: 250.202
INFO:tensorflow:global_step/sec: 250.202
INFO:tensorflow:loss = 0.51536375, step = 7500 (0.400 sec)
INFO:tensorflow:loss = 0.51536375, step = 7500 (0.400 sec)
INFO:tensorflow:global_step/sec: 253.281
INFO:tensorflow:global_step/sec: 253.281
INFO:tensorflow:loss = 0.60245466, step = 7600 (0.394 sec)
INFO:tensorflow:loss = 0.60245466, step = 7600 (0.394 sec)
INFO:tensorflow:global_step/sec: 253.165
INFO:tensorflow:global_step/sec: 253.165
INFO:tensorflow:loss = 0.44345674, step = 7700 (0.395 sec)
INFO:tensorflow:loss = 0.44345674, step = 7700 (0.395 sec)
INFO:tensorflow:global_step/sec: 255.968
INFO:tensorflow:global_step/sec: 255.968
INFO:tensorflow:loss = 0.49975634, step = 7800 (0.391 sec)
INFO:tensorflow:loss = 0.49975634, step = 7800 (0.391 sec)
INFO:tensorflow:global_step/sec: 253.47
INFO:tensorflow:global_step/sec: 253.47
INFO:tensorflow:loss = 0.5979585, step = 7900 (0.394 sec)
INFO:tensorflow:loss = 0.5979585, step = 7900 (0.394 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 7992...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 7992...
INFO:tensorflow:Saving checkpoints for 7992 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Saving checkpoints for 7992 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 7992...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 7992...
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:global_step/sec: 217.451
INFO:tensorflow:global_step/sec: 217.451
INFO:tensorflow:loss = 0.4377802, step = 8000 (0.460 sec)
INFO:tensorflow:loss = 0.4377802, step = 8000 (0.460 sec)
INFO:tensorflow:global_step/sec: 256.419
INFO:tensorflow:global_step/sec: 256.419
INFO:tensorflow:loss = 0.505089, step = 8100 (0.390 sec)
INFO:tensorflow:loss = 0.505089, step = 8100 (0.390 sec)
INFO:tensorflow:global_step/sec: 252.919
INFO:tensorflow:global_step/sec: 252.919
INFO:tensorflow:loss = 0.40602866, step = 8200 (0.395 sec)
INFO:tensorflow:loss = 0.40602866, step = 8200 (0.395 sec)
INFO:tensorflow:global_step/sec: 256.346
INFO:tensorflow:global_step/sec: 256.346
INFO:tensorflow:loss = 0.43653098, step = 8300 (0.390 sec)
INFO:tensorflow:loss = 0.43653098, step = 8300 (0.390 sec)
INFO:tensorflow:global_step/sec: 260.355
INFO:tensorflow:global_step/sec: 260.355
INFO:tensorflow:loss = 0.3623276, step = 8400 (0.384 sec)
INFO:tensorflow:loss = 0.3623276, step = 8400 (0.384 sec)
INFO:tensorflow:global_step/sec: 260.462
INFO:tensorflow:global_step/sec: 260.462
INFO:tensorflow:loss = 0.39351046, step = 8500 (0.384 sec)
INFO:tensorflow:loss = 0.39351046, step = 8500 (0.384 sec)
INFO:tensorflow:global_step/sec: 252.832
INFO:tensorflow:global_step/sec: 252.832
INFO:tensorflow:loss = 0.36336032, step = 8600 (0.395 sec)
INFO:tensorflow:loss = 0.36336032, step = 8600 (0.395 sec)
INFO:tensorflow:global_step/sec: 257.437
INFO:tensorflow:global_step/sec: 257.437
INFO:tensorflow:loss = 0.45432475, step = 8700 (0.388 sec)
INFO:tensorflow:loss = 0.45432475, step = 8700 (0.388 sec)
INFO:tensorflow:global_step/sec: 255.461
INFO:tensorflow:global_step/sec: 255.461
INFO:tensorflow:loss = 0.5483323, step = 8800 (0.392 sec)
INFO:tensorflow:loss = 0.5483323, step = 8800 (0.392 sec)
INFO:tensorflow:global_step/sec: 258.549
INFO:tensorflow:global_step/sec: 258.549
INFO:tensorflow:loss = 0.39868888, step = 8900 (0.387 sec)
INFO:tensorflow:loss = 0.39868888, step = 8900 (0.387 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 8991...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 8991...
INFO:tensorflow:Saving checkpoints for 8991 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Saving checkpoints for 8991 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 8991...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 8991...
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:global_step/sec: 219.514
INFO:tensorflow:global_step/sec: 219.514
INFO:tensorflow:loss = 0.38975185, step = 9000 (0.455 sec)
INFO:tensorflow:loss = 0.38975185, step = 9000 (0.455 sec)
INFO:tensorflow:global_step/sec: 251.244
INFO:tensorflow:global_step/sec: 251.244
INFO:tensorflow:loss = 0.4212112, step = 9100 (0.398 sec)
INFO:tensorflow:loss = 0.4212112, step = 9100 (0.398 sec)
INFO:tensorflow:global_step/sec: 272.072
INFO:tensorflow:global_step/sec: 272.072
INFO:tensorflow:loss = 0.4536277, step = 9200 (0.368 sec)
INFO:tensorflow:loss = 0.4536277, step = 9200 (0.368 sec)
INFO:tensorflow:global_step/sec: 270.537
INFO:tensorflow:global_step/sec: 270.537
INFO:tensorflow:loss = 0.39709157, step = 9300 (0.370 sec)
INFO:tensorflow:loss = 0.39709157, step = 9300 (0.370 sec)
INFO:tensorflow:global_step/sec: 255.055
INFO:tensorflow:global_step/sec: 255.055
INFO:tensorflow:loss = 0.4788674, step = 9400 (0.392 sec)
INFO:tensorflow:loss = 0.4788674, step = 9400 (0.392 sec)
INFO:tensorflow:global_step/sec: 259.597
INFO:tensorflow:global_step/sec: 259.597
INFO:tensorflow:loss = 0.3967464, step = 9500 (0.385 sec)
INFO:tensorflow:loss = 0.3967464, step = 9500 (0.385 sec)
INFO:tensorflow:global_step/sec: 258.95
INFO:tensorflow:global_step/sec: 258.95
INFO:tensorflow:loss = 0.3898618, step = 9600 (0.386 sec)
INFO:tensorflow:loss = 0.3898618, step = 9600 (0.386 sec)
INFO:tensorflow:global_step/sec: 258.698
INFO:tensorflow:global_step/sec: 258.698
INFO:tensorflow:loss = 0.3142761, step = 9700 (0.387 sec)
INFO:tensorflow:loss = 0.3142761, step = 9700 (0.387 sec)
INFO:tensorflow:global_step/sec: 265.135
INFO:tensorflow:global_step/sec: 265.135
INFO:tensorflow:loss = 0.328832, step = 9800 (0.377 sec)
INFO:tensorflow:loss = 0.328832, step = 9800 (0.377 sec)
INFO:tensorflow:global_step/sec: 263.242
INFO:tensorflow:global_step/sec: 263.242
INFO:tensorflow:loss = 0.4077425, step = 9900 (0.380 sec)
INFO:tensorflow:loss = 0.4077425, step = 9900 (0.380 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 9990...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 9990...
INFO:tensorflow:Saving checkpoints for 9990 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Saving checkpoints for 9990 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 9990...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 9990...
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 10000...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 10000...
INFO:tensorflow:Saving checkpoints for 10000 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Saving checkpoints for 10000 into /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 10000...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 10000...
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-07-27T09:15:41
INFO:tensorflow:Starting evaluation at 2021-07-27T09:15:41
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt-10000
2021-07-27 09:15:41.058084: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:41.058476: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
pciBusID: 0000:00:05.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2021-07-27 09:15:41.058629: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:41.058970: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:41.059237: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-07-27 09:15:41.059282: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-07-27 09:15:41.059289: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264]      0 
2021-07-27 09:15:41.059296: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0:   N 
2021-07-27 09:15:41.059413: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:41.059723: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:41.059991: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14646 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0)
INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt-10000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Evaluation [500/5000]
INFO:tensorflow:Evaluation [500/5000]
INFO:tensorflow:Evaluation [1000/5000]
INFO:tensorflow:Evaluation [1000/5000]
INFO:tensorflow:Evaluation [1500/5000]
INFO:tensorflow:Evaluation [1500/5000]
INFO:tensorflow:Evaluation [2000/5000]
INFO:tensorflow:Evaluation [2000/5000]
INFO:tensorflow:Evaluation [2500/5000]
INFO:tensorflow:Evaluation [2500/5000]
INFO:tensorflow:Evaluation [3000/5000]
INFO:tensorflow:Evaluation [3000/5000]
INFO:tensorflow:Evaluation [3500/5000]
INFO:tensorflow:Evaluation [3500/5000]
INFO:tensorflow:Evaluation [4000/5000]
INFO:tensorflow:Evaluation [4000/5000]
INFO:tensorflow:Evaluation [4500/5000]
INFO:tensorflow:Evaluation [4500/5000]
INFO:tensorflow:Evaluation [5000/5000]
INFO:tensorflow:Evaluation [5000/5000]
INFO:tensorflow:Inference Time : 5.51757s
INFO:tensorflow:Inference Time : 5.51757s
INFO:tensorflow:Finished evaluation at 2021-07-27-09:15:46
INFO:tensorflow:Finished evaluation at 2021-07-27-09:15:46
INFO:tensorflow:Saving dict for global step 10000: accuracy = 0.8013, global_step = 10000, loss = 0.44217905
INFO:tensorflow:Saving dict for global step 10000: accuracy = 0.8013, global_step = 10000, loss = 0.44217905
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 10000: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt-10000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 10000: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt-10000
INFO:tensorflow:Performing the final export in the end of training.
INFO:tensorflow:Performing the final export in the end of training.
WARNING:tensorflow:Loading a TF2 SavedModel but eager mode seems disabled.
WARNING:tensorflow:Loading a TF2 SavedModel but eager mode seems disabled.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/saved_model/signature_def_utils_impl.py:201: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/saved_model/signature_def_utils_impl.py:201: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
INFO:tensorflow:Signatures INCLUDED in export for Classify: None
INFO:tensorflow:Signatures INCLUDED in export for Classify: None
INFO:tensorflow:Signatures INCLUDED in export for Regress: None
INFO:tensorflow:Signatures INCLUDED in export for Regress: None
INFO:tensorflow:Signatures INCLUDED in export for Predict: ['serving_default']
INFO:tensorflow:Signatures INCLUDED in export for Predict: ['serving_default']
INFO:tensorflow:Signatures INCLUDED in export for Train: None
INFO:tensorflow:Signatures INCLUDED in export for Train: None
INFO:tensorflow:Signatures INCLUDED in export for Eval: None
INFO:tensorflow:Signatures INCLUDED in export for Eval: None
INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt-10000
2021-07-27 09:15:48.407821: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:48.408315: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
pciBusID: 0000:00:05.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2021-07-27 09:15:48.408563: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:48.409077: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:48.409448: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-07-27 09:15:48.409503: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-07-27 09:15:48.409517: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264]      0 
2021-07-27 09:15:48.409527: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0:   N 
2021-07-27 09:15:48.409698: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:48.410244: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:48.410662: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14646 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0)
INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt-10000
INFO:tensorflow:Assets added to graph.
INFO:tensorflow:Assets added to graph.
INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/export/imdb/temp-1627377346/assets
INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/export/imdb/temp-1627377346/assets
INFO:tensorflow:SavedModel written to: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/export/imdb/temp-1627377346/saved_model.pb
INFO:tensorflow:SavedModel written to: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/export/imdb/temp-1627377346/saved_model.pb
INFO:tensorflow:Loss for final step: 0.47436312.
INFO:tensorflow:Loss for final step: 0.47436312.
WARNING:tensorflow:Loading a TF2 SavedModel but eager mode seems disabled.
WARNING:tensorflow:Loading a TF2 SavedModel but eager mode seems disabled.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Signatures INCLUDED in export for Classify: None
INFO:tensorflow:Signatures INCLUDED in export for Classify: None
INFO:tensorflow:Signatures INCLUDED in export for Regress: None
INFO:tensorflow:Signatures INCLUDED in export for Regress: None
INFO:tensorflow:Signatures INCLUDED in export for Predict: None
INFO:tensorflow:Signatures INCLUDED in export for Predict: None
INFO:tensorflow:Signatures INCLUDED in export for Train: None
INFO:tensorflow:Signatures INCLUDED in export for Train: None
INFO:tensorflow:Signatures INCLUDED in export for Eval: ['eval']
INFO:tensorflow:Signatures INCLUDED in export for Eval: ['eval']
WARNING:tensorflow:Export includes no default signature!
WARNING:tensorflow:Export includes no default signature!
INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt-10000
2021-07-27 09:15:48.855551: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:48.856008: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
pciBusID: 0000:00:05.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2021-07-27 09:15:48.856222: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:48.856639: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:48.856952: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-07-27 09:15:48.857013: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-07-27 09:15:48.857021: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264]      0 
2021-07-27 09:15:48.857027: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0:   N 
2021-07-27 09:15:48.857170: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:48.857568: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-07-27 09:15:48.857913: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14646 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0)
INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-Serving/model.ckpt-10000
INFO:tensorflow:Assets added to graph.
INFO:tensorflow:Assets added to graph.
INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-TFMA/temp-1627377348/assets
INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-TFMA/temp-1627377348/assets
INFO:tensorflow:SavedModel written to: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-TFMA/temp-1627377348/saved_model.pb
INFO:tensorflow:SavedModel written to: /tmp/tfx-interactive-2021-07-27T09_11_59.954153-vciyoyjw/Trainer/model_run/9/Format-TFMA/temp-1627377348/saved_model.pb
WARNING:absl:Support for estimator-based executor and model export will be deprecated soon. Please use export structure <ModelExportPath>/serving_model_dir/saved_model.pb"
WARNING:absl:Support for estimator-based executor and model export will be deprecated soon. Please use export structure <ModelExportPath>/eval_model_dir/saved_model.pb"

Werfen Sie einen Blick auf das trainierte Modell , das aus exportiert wurde Trainer .

train_uri = trainer.outputs['model'].get()[0].uri
serving_model_path = os.path.join(train_uri, 'Format-Serving')
exported_model = tf.saved_model.load(serving_model_path)
exported_model.graph.get_operations()[:10] + ["..."]
[<tf.Operation 'global_step/Initializer/zeros' type=Const>,
 <tf.Operation 'global_step' type=VarHandleOp>,
 <tf.Operation 'global_step/IsInitialized/VarIsInitializedOp' type=VarIsInitializedOp>,
 <tf.Operation 'global_step/Assign' type=AssignVariableOp>,
 <tf.Operation 'global_step/Read/ReadVariableOp' type=ReadVariableOp>,
 <tf.Operation 'input_example_tensor' type=Placeholder>,
 <tf.Operation 'ParseExample/ParseExampleV2/names' type=Const>,
 <tf.Operation 'ParseExample/ParseExampleV2/sparse_keys' type=Const>,
 <tf.Operation 'ParseExample/ParseExampleV2/dense_keys' type=Const>,
 <tf.Operation 'ParseExample/ParseExampleV2/ragged_keys' type=Const>,
 '...']

Lassen Sie uns die Metriken des Modells mit Tensorboard visualisieren.


# Get the URI of the output artifact representing the training logs,
# which is a directory
model_run_dir = trainer.outputs['model_run'].get()[0].uri

%load_ext tensorboard
%tensorboard --logdir {model_run_dir}

Model-Serving

Die Graphregularisierung wirkt sich nur auf den Trainingsworkflow aus, indem der Verlustfunktion ein Regularisierungsterm hinzugefügt wird. Dadurch bleiben die Modellbewertungs- und Serving-Workflows unverändert. Es ist aus dem gleichen Grund , dass wir auch nachgeschaltete TFX - Komponenten , die normalerweise kommen , nachdem der Trainer Komponente wie der Evaluator, Pusher, usw. weggelassen haben

Abschluss

Wir haben die Verwendung der Graph-Regularisierung unter Verwendung des Neural Structured Learning (NSL)-Frameworks in einer TFX-Pipeline demonstriert, selbst wenn die Eingabe keinen expliziten Graphen enthält. Wir haben uns die Aufgabe der Sentimentklassifizierung von IMDB-Filmrezensionen angesehen, für die wir ein Ähnlichkeitsdiagramm basierend auf Rezensionseinbettungen synthetisiert haben. Wir ermutigen Benutzer, weiter zu experimentieren, indem sie verschiedene Einbettungen für die Graphkonstruktion verwenden, Hyperparameter variieren, den Umfang der Überwachung ändern und verschiedene Modellarchitekturen definieren.