이 튜토리얼에서 그래프 정규화를 설명 신경 구조화 학습 프레임 워크와 TFX 파이프 라인에서 감정 분류를위한 엔드 - 투 - 엔드 워크 플로우를 보여줍니다.
개요
리뷰의 텍스트를 사용하여 양 또는 음 등이 노트북, 분류의 영화 리뷰. 이 이진 분류, 기계 학습 문제의 중요하고 널리 적용 가지 예입니다.
주어진 입력에서 그래프를 작성하여 이 노트북에서 그래프 정규화 사용을 시연할 것입니다. 입력에 명시적 그래프가 포함되지 않은 경우 NSL(신경 구조적 학습) 프레임워크를 사용하여 그래프 정규화 모델을 구축하는 일반적인 방법은 다음과 같습니다.
- 입력의 각 텍스트 샘플에 대한 임베딩을 생성합니다. 이는 같은 사전 훈련 모델을 사용하여 수행 할 수 있습니다 word2vec , 회전 , BERT 등
- 'L2' 거리, '코사인' 거리 등과 같은 유사성 메트릭을 사용하여 이러한 임베딩을 기반으로 그래프를 작성합니다. 그래프의 노드는 샘플에 해당하고 그래프의 모서리는 샘플 쌍 간의 유사성에 해당합니다.
- 위의 합성 그래프와 샘플 기능에서 훈련 데이터를 생성합니다. 결과 훈련 데이터에는 원래 노드 기능 외에 이웃 기능이 포함됩니다.
- Estimators를 사용하여 기본 모델로 신경망을 만듭니다.
- 와 기본 모델 감싸
add_graph_regularization
새로운 견적 그래프 모델을 만들기 위해 상기 NSL 프레임 워크에 의해 제공되는 래퍼 함수. 이 새 모델은 훈련 목표의 정규화 항으로 그래프 정규화 손실을 포함합니다. - 그래프 Estimator 모델을 훈련하고 평가합니다.
이 자습서에서는 여러 사용자 지정 TFX 구성 요소와 사용자 지정 그래프 정규화 트레이너 구성 요소를 사용하여 위의 워크플로를 TFX 파이프라인에 통합합니다.
아래는 TFX 파이프라인의 개략도입니다. 주황색 상자는 기성품 TFX 구성 요소를 나타내고 분홍색 상자는 사용자 지정 TFX 구성 요소를 나타냅니다.
핍 업그레이드
로컬에서 실행할 때 시스템에서 Pip를 업그레이드하지 않으려면 Colab에서 실행 중인지 확인하세요. 물론 로컬 시스템은 별도로 업그레이드할 수 있습니다.
try:
import colab
!pip install --upgrade pip
except:
pass
필수 패키지 설치
!pip install -q -U \
tfx==1.2.0 \
neural-structured-learning \
tensorflow-hub \
tensorflow-datasets
런타임을 다시 시작하셨습니까?
Google Colab을 사용하는 경우 위의 셀을 처음 실행할 때 런타임을 다시 시작해야 합니다(런타임 > 런타임 다시 시작...). Colab이 패키지를 로드하는 방식 때문입니다.
종속성 및 가져오기
import apache_beam as beam
import gzip as gzip_lib
import numpy as np
import os
import pprint
import shutil
import tempfile
import urllib
import uuid
pp = pprint.PrettyPrinter()
import tensorflow as tf
import neural_structured_learning as nsl
import tfx
from tfx.components.evaluator.component import Evaluator
from tfx.components.example_gen.import_example_gen.component import ImportExampleGen
from tfx.components.example_validator.component import ExampleValidator
from tfx.components.model_validator.component import ModelValidator
from tfx.components.pusher.component import Pusher
from tfx.components.schema_gen.component import SchemaGen
from tfx.components.statistics_gen.component import StatisticsGen
from tfx.components.trainer import executor as trainer_executor
from tfx.components.trainer.component import Trainer
from tfx.components.transform.component import Transform
from tfx.dsl.components.base import executor_spec
from tfx.orchestration.experimental.interactive.interactive_context import InteractiveContext
from tfx.proto import evaluator_pb2
from tfx.proto import example_gen_pb2
from tfx.proto import pusher_pb2
from tfx.proto import trainer_pb2
from tfx.types import artifact
from tfx.types import artifact_utils
from tfx.types import channel
from tfx.types import standard_artifacts
from tfx.types.standard_artifacts import Examples
from tfx.dsl.component.experimental.annotations import InputArtifact
from tfx.dsl.component.experimental.annotations import OutputArtifact
from tfx.dsl.component.experimental.annotations import Parameter
from tfx.dsl.component.experimental.decorators import component
from tensorflow_metadata.proto.v0 import anomalies_pb2
from tensorflow_metadata.proto.v0 import schema_pb2
from tensorflow_metadata.proto.v0 import statistics_pb2
import tensorflow_data_validation as tfdv
import tensorflow_transform as tft
import tensorflow_model_analysis as tfma
import tensorflow_hub as hub
import tensorflow_datasets as tfds
print("TF Version: ", tf.__version__)
print("Eager mode: ", tf.executing_eagerly())
print(
"GPU is",
"available" if tf.config.list_physical_devices("GPU") else "NOT AVAILABLE")
print("NSL Version: ", nsl.__version__)
print("TFX Version: ", tfx.__version__)
print("TFDV version: ", tfdv.__version__)
print("TFT version: ", tft.__version__)
print("TFMA version: ", tfma.__version__)
print("Hub version: ", hub.__version__)
print("Beam version: ", beam.__version__)
TF Version: 2.5.2 Eager mode: True GPU is available NSL Version: 1.3.1 TFX Version: 1.2.0 TFDV version: 1.2.0 TFT version: 1.2.0 TFMA version: 0.33.0 Hub version: 0.12.0 Beam version: 2.34.0
IMDB 데이터세트
IMDB의 데이터 세트 로부터 5 만 개 영화 리뷰의 텍스트가 들어 인터넷 영화 데이터베이스 . 교육용 리뷰 25,000개와 테스트용 리뷰 25,000개로 나뉩니다. 훈련 및 테스트 세트가 긍정적이고 부정적인 리뷰의 동일한 수를 포함하는 의미 균형된다. 또한 레이블이 지정되지 않은 추가 영화 리뷰가 50,000개 있습니다.
전처리된 IMDB 데이터세트 다운로드
다음 코드는 TFDS를 사용하여 IMDB 데이터 세트를 다운로드합니다(또는 이미 다운로드된 경우 캐시된 복사본 사용). 이 노트북의 속도를 높이기 위해 레이블이 지정된 리뷰 10,000개와 레이블이 없는 리뷰 10,000개만 교육에 사용하고 테스트 리뷰 10,000개를 평가에 사용합니다.
train_set, eval_set = tfds.load(
"imdb_reviews:1.0.0",
split=["train[:10000]+unsupervised[:10000]", "test[:10000]"],
shuffle_files=False)
훈련 세트의 몇 가지 리뷰를 살펴보겠습니다.
for tfrecord in train_set.take(4):
print("Review: {}".format(tfrecord["text"].numpy().decode("utf-8")[:300]))
print("Label: {}\n".format(tfrecord["label"].numpy()))
Review: This was an absolutely terrible movie. Don't be lured in by Christopher Walken or Michael Ironside. Both are great actors, but this must simply be their worst role in history. Even their great acting could not redeem this movie's ridiculous storyline. This movie is an early nineties US propaganda pi Label: 0 Review: I have been known to fall asleep during films, but this is usually due to a combination of things including, really tired, being warm and comfortable on the sette and having just eaten a lot. However on this occasion I fell asleep because the film was rubbish. The plot development was constant. Cons Label: 0 Review: Mann photographs the Alberta Rocky Mountains in a superb fashion, and Jimmy Stewart and Walter Brennan give enjoyable performances as they always seem to do. <br /><br />But come on Hollywood - a Mountie telling the people of Dawson City, Yukon to elect themselves a marshal (yes a marshal!) and to e Label: 0 Review: This is the kind of film for a snowy Sunday afternoon when the rest of the world can go ahead with its own business as you descend into a big arm-chair and mellow for a couple of hours. Wonderful performances from Cher and Nicolas Cage (as always) gently row the plot along. There are no rapids to cr Label: 1
def _dict_to_example(instance):
"""Decoded CSV to tf example."""
feature = {}
for key, value in instance.items():
if value is None:
feature[key] = tf.train.Feature()
elif value.dtype == np.integer:
feature[key] = tf.train.Feature(
int64_list=tf.train.Int64List(value=value.tolist()))
elif value.dtype == np.float32:
feature[key] = tf.train.Feature(
float_list=tf.train.FloatList(value=value.tolist()))
else:
feature[key] = tf.train.Feature(
bytes_list=tf.train.BytesList(value=value.tolist()))
return tf.train.Example(features=tf.train.Features(feature=feature))
examples_path = tempfile.mkdtemp(prefix="tfx-data")
train_path = os.path.join(examples_path, "train.tfrecord")
eval_path = os.path.join(examples_path, "eval.tfrecord")
for path, dataset in [(train_path, train_set), (eval_path, eval_set)]:
with tf.io.TFRecordWriter(path) as writer:
for example in dataset:
writer.write(
_dict_to_example({
"label": np.array([example["label"].numpy()]),
"text": np.array([example["text"].numpy()]),
}).SerializeToString())
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/ipykernel_launcher.py:7: DeprecationWarning: Converting `np.integer` or `np.signedinteger` to a dtype is deprecated. The current result is `np.dtype(np.int_)` which is not strictly correct. Note that the result depends on the system. To ensure stable results use may want to use `np.int64` or `np.int32`. import sys
대화식으로 TFX 구성 요소 실행
당신이 TFX 구성 요소를 구성 및 취득 할 InteractiveContext 내에서 대화 형으로 각각을 실행에 따라 세포에서 ExecutionResult
객체. 이는 각 구성 요소에 대한 종속성이 충족되는 시기에 따라 TFX DAG에서 구성 요소를 실행하는 오케스트레이터의 프로세스를 미러링합니다.
context = InteractiveContext()
WARNING:absl:InteractiveContext pipeline_root argument not provided: using temporary directory /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9 as root for pipeline outputs. WARNING:absl:InteractiveContext metadata_connection_config not provided: using SQLite ML Metadata database at /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/metadata.sqlite.
ExampleGen 구성 요소
모든 ML 개발 프로세스에서 코드 개발을 시작할 때의 첫 번째 단계는 교육 및 테스트 데이터 세트를 수집하는 것입니다. ExampleGen
구성 요소는 TFX 파이프 라인에 데이터를 제공합니다.
ExampleGen 구성 요소를 만들고 실행합니다.
input_config = example_gen_pb2.Input(splits=[
example_gen_pb2.Input.Split(name='train', pattern='train.tfrecord'),
example_gen_pb2.Input.Split(name='eval', pattern='eval.tfrecord')
])
example_gen = ImportExampleGen(input_base=examples_path, input_config=input_config)
context.run(example_gen, enable_cache=True)
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features. WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter. WARNING:apache_beam.io.tfrecordio:Couldn't find python-snappy so the implementation of _TFRecordUtil._masked_crc32c is not as fast as it could be.
for artifact in example_gen.outputs['examples'].get():
print(artifact)
print('\nexample_gen.outputs is a {}'.format(type(example_gen.outputs)))
print(example_gen.outputs)
print(example_gen.outputs['examples'].get()[0].split_names)
Artifact(artifact: id: 1 type_id: 14 uri: "/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/ImportExampleGen/examples/1" properties { key: "split_names" value { string_value: "[\"train\", \"eval\"]" } } custom_properties { key: "file_format" value { string_value: "tfrecords_gzip" } } custom_properties { key: "input_fingerprint" value { string_value: "split:train,num_files:1,total_bytes:27706811,xor_checksum:1638618106,sum_checksum:1638618106\nsplit:eval,num_files:1,total_bytes:13374744,xor_checksum:1638618111,sum_checksum:1638618111" } } custom_properties { key: "payload_format" value { string_value: "FORMAT_TF_EXAMPLE" } } custom_properties { key: "span" value { int_value: 0 } } custom_properties { key: "state" value { string_value: "published" } } custom_properties { key: "tfx_version" value { string_value: "1.2.0" } } state: LIVE , artifact_type: id: 14 name: "Examples" properties { key: "span" value: INT } properties { key: "split_names" value: STRING } properties { key: "version" value: INT } ) example_gen.outputs is a <class 'dict'> {'examples': Channel( type_name: Examples artifacts: [Artifact(artifact: id: 1 type_id: 14 uri: "/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/ImportExampleGen/examples/1" properties { key: "split_names" value { string_value: "[\"train\", \"eval\"]" } } custom_properties { key: "file_format" value { string_value: "tfrecords_gzip" } } custom_properties { key: "input_fingerprint" value { string_value: "split:train,num_files:1,total_bytes:27706811,xor_checksum:1638618106,sum_checksum:1638618106\nsplit:eval,num_files:1,total_bytes:13374744,xor_checksum:1638618111,sum_checksum:1638618111" } } custom_properties { key: "payload_format" value { string_value: "FORMAT_TF_EXAMPLE" } } custom_properties { key: "span" value { int_value: 0 } } custom_properties { key: "state" value { string_value: "published" } } custom_properties { key: "tfx_version" value { string_value: "1.2.0" } } state: LIVE , artifact_type: id: 14 name: "Examples" properties { key: "span" value: INT } properties { key: "split_names" value: STRING } properties { key: "version" value: INT } )] additional_properties: {} additional_custom_properties: {} )} ["train", "eval"]
구성 요소의 출력에는 2개의 아티팩트가 포함됩니다.
- 교육 예제(10,000개의 레이블이 지정된 리뷰 + 10,000개의 레이블이 없는 리뷰)
- 평가 예(10,000개의 레이블이 지정된 리뷰)
IdentExamples 사용자 지정 구성 요소
NSL을 사용하려면 각 인스턴스에 고유 ID가 있어야 합니다. 모든 분할의 모든 인스턴스에 이러한 고유 ID를 추가하는 사용자 지정 구성 요소를 만듭니다. 우리는 활용 아파치 빔을 필요한 경우 쉽게 대규모 데이터 세트로 확장 할 수 있도록.
def make_example_with_unique_id(example, id_feature_name):
"""Adds a unique ID to the given `tf.train.Example` proto.
This function uses Python's 'uuid' module to generate a universally unique
identifier for each example.
Args:
example: An instance of a `tf.train.Example` proto.
id_feature_name: The name of the feature in the resulting `tf.train.Example`
that will contain the unique identifier.
Returns:
A new `tf.train.Example` proto that includes a unique identifier as an
additional feature.
"""
result = tf.train.Example()
result.CopyFrom(example)
unique_id = uuid.uuid4()
result.features.feature.get_or_create(
id_feature_name).bytes_list.MergeFrom(
tf.train.BytesList(value=[str(unique_id).encode('utf-8')]))
return result
@component
def IdentifyExamples(orig_examples: InputArtifact[Examples],
identified_examples: OutputArtifact[Examples],
id_feature_name: Parameter[str],
component_name: Parameter[str]) -> None:
# Get a list of the splits in input_data
splits_list = artifact_utils.decode_split_names(
split_names=orig_examples.split_names)
# For completeness, encode the splits names and payload_format.
# We could also just use input_data.split_names.
identified_examples.split_names = artifact_utils.encode_split_names(
splits=splits_list)
# TODO(b/168616829): Remove populating payload_format after tfx 0.25.0.
identified_examples.set_string_custom_property(
"payload_format",
orig_examples.get_string_custom_property("payload_format"))
for split in splits_list:
input_dir = artifact_utils.get_split_uri([orig_examples], split)
output_dir = artifact_utils.get_split_uri([identified_examples], split)
os.mkdir(output_dir)
with beam.Pipeline() as pipeline:
(pipeline
| 'ReadExamples' >> beam.io.ReadFromTFRecord(
os.path.join(input_dir, '*'),
coder=beam.coders.coders.ProtoCoder(tf.train.Example))
| 'AddUniqueId' >> beam.Map(make_example_with_unique_id, id_feature_name)
| 'WriteIdentifiedExamples' >> beam.io.WriteToTFRecord(
file_path_prefix=os.path.join(output_dir, 'data_tfrecord'),
coder=beam.coders.coders.ProtoCoder(tf.train.Example),
file_name_suffix='.gz'))
return
identify_examples = IdentifyExamples(
orig_examples=example_gen.outputs['examples'],
component_name=u'IdentifyExamples',
id_feature_name=u'id')
context.run(identify_examples, enable_cache=False)
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter. WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
StatisticsGen 구성 요소
StatisticsGen
구성 요소는 데이터 세트에 대한 기술 통계를 계산합니다. 생성하는 통계는 검토를 위해 시각화할 수 있으며, 예를 들어 유효성 검사 및 스키마 추론에 사용됩니다.
StatisticsGen 구성 요소를 생성하고 실행합니다.
# Computes statistics over data for visualization and example validation.
statistics_gen = StatisticsGen(
examples=identify_examples.outputs["identified_examples"])
context.run(statistics_gen, enable_cache=True)
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
SchemaGen 구성 요소
SchemaGen
구성 요소는 StatisticsGen에서 통계를 바탕으로 데이터에 대한 스키마를 생성합니다. 각 기능의 데이터 유형과 범주형 기능의 법적 값 범위를 추론하려고 합니다.
SchemaGen 구성 요소를 만들고 실행합니다.
# Generates schema based on statistics files.
schema_gen = SchemaGen(
statistics=statistics_gen.outputs['statistics'], infer_feature_shape=False)
context.run(schema_gen, enable_cache=True)
WARNING: Logging before InitGoogleLogging() is written to STDERR I1204 11:42:13.777263 6839 rdbms_metadata_access_object.cc:686] No property is defined for the Type
생성 된 아티팩트 단지입니다 schema.pbtxt
의 텍스트 표현을 포함한 schema_pb2.Schema
protobuf를 :
train_uri = schema_gen.outputs['schema'].get()[0].uri
schema_filename = os.path.join(train_uri, 'schema.pbtxt')
schema = tfx.utils.io_utils.parse_pbtxt_file(
file_name=schema_filename, message=schema_pb2.Schema())
그것은 사용하여 시각 될 수 tfdv.display_schema()
(우리는 후속 실험실에서 더 자세하게 살펴 보겠습니다을) :
tfdv.display_schema(schema)
ExampleValidator 구성 요소
ExampleValidator
StatisticsGen에서 통계에서는 schemagen의 스키마에 기초하여, 이상 검출을 행한다. 누락된 값, 잘못된 유형의 값 또는 허용 가능한 값의 도메인 외부에 있는 범주형 값과 같은 문제를 찾습니다.
ExampleValidator 구성 요소를 만들고 실행합니다.
# Performs anomaly detection based on statistics and data schema.
validate_stats = ExampleValidator(
statistics=statistics_gen.outputs['statistics'],
schema=schema_gen.outputs['schema'])
context.run(validate_stats, enable_cache=False)
SynthesizeGraph 구성 요소
그래프 구성에는 텍스트 샘플에 대한 임베딩을 만든 다음 유사성 함수를 사용하여 임베딩을 비교하는 작업이 포함됩니다.
우리는에 묻어을 만들 pretrained 스위블 묻어을 사용합니다 tf.train.Example
입력의 각 샘플에 대한 형식입니다. 우리는의 결과 묻어 저장합니다 TFRecord
샘플의 ID와 함께 형식을. 이것은 중요하며 나중에 그래프의 해당 노드와 샘플 임베딩을 일치시킬 수 있습니다.
샘플 임베딩이 있으면 이를 사용하여 유사성 그래프를 만들 것입니다. 즉, 이 그래프의 노드는 샘플에 해당하고 이 그래프의 가장자리는 노드 쌍 간의 유사성에 해당합니다.
Neural Structured Learning은 샘플 임베딩을 기반으로 그래프를 작성하기 위한 그래프 작성 라이브러리를 제공합니다. 그것은 그들 사이에 묻어 및 빌드 가장자리를 비교하기 위해 유사성 척도로 코사인 유사성을 사용합니다. 또한 최종 그래프에서 유사하지 않은 가장자리를 버리는 데 사용할 수 있는 유사성 임계값을 지정할 수 있습니다. 다음 예에서 유사성 임계값으로 0.99를 사용하면 111,066개의 양방향 에지가 있는 그래프가 생성됩니다.
swivel_url = 'https://tfhub.dev/google/tf2-preview/gnews-swivel-20dim/1'
hub_layer = hub.KerasLayer(swivel_url, input_shape=[], dtype=tf.string)
def _bytes_feature(value):
"""Returns a bytes_list from a string / byte."""
return tf.train.Feature(bytes_list=tf.train.BytesList(value=value))
def _float_feature(value):
"""Returns a float_list from a float / double."""
return tf.train.Feature(float_list=tf.train.FloatList(value=value))
def create_embedding_example(example):
"""Create tf.Example containing the sample's embedding and its ID."""
sentence_embedding = hub_layer(tf.sparse.to_dense(example['text']))
# Flatten the sentence embedding back to 1-D.
sentence_embedding = tf.reshape(sentence_embedding, shape=[-1])
feature_dict = {
'id': _bytes_feature(tf.sparse.to_dense(example['id']).numpy()),
'embedding': _float_feature(sentence_embedding.numpy().tolist())
}
return tf.train.Example(features=tf.train.Features(feature=feature_dict))
def create_dataset(uri):
tfrecord_filenames = [os.path.join(uri, name) for name in os.listdir(uri)]
return tf.data.TFRecordDataset(tfrecord_filenames, compression_type='GZIP')
def create_embeddings(train_path, output_path):
dataset = create_dataset(train_path)
embeddings_path = os.path.join(output_path, 'embeddings.tfr')
feature_map = {
'label': tf.io.FixedLenFeature([], tf.int64),
'id': tf.io.VarLenFeature(tf.string),
'text': tf.io.VarLenFeature(tf.string)
}
with tf.io.TFRecordWriter(embeddings_path) as writer:
for tfrecord in dataset:
tensor_dict = tf.io.parse_single_example(tfrecord, feature_map)
embedding_example = create_embedding_example(tensor_dict)
writer.write(embedding_example.SerializeToString())
def build_graph(output_path, similarity_threshold):
embeddings_path = os.path.join(output_path, 'embeddings.tfr')
graph_path = os.path.join(output_path, 'graph.tsv')
graph_builder_config = nsl.configs.GraphBuilderConfig(
similarity_threshold=similarity_threshold,
lsh_splits=32,
lsh_rounds=15,
random_seed=12345)
nsl.tools.build_graph_from_config([embeddings_path], graph_path,
graph_builder_config)
"""Custom Artifact type"""
class SynthesizedGraph(tfx.types.artifact.Artifact):
"""Output artifact of the SynthesizeGraph component"""
TYPE_NAME = 'SynthesizedGraphPath'
PROPERTIES = {
'span': standard_artifacts.SPAN_PROPERTY,
'split_names': standard_artifacts.SPLIT_NAMES_PROPERTY,
}
@component
def SynthesizeGraph(identified_examples: InputArtifact[Examples],
synthesized_graph: OutputArtifact[SynthesizedGraph],
similarity_threshold: Parameter[float],
component_name: Parameter[str]) -> None:
# Get a list of the splits in input_data
splits_list = artifact_utils.decode_split_names(
split_names=identified_examples.split_names)
# We build a graph only based on the 'Split-train' split which includes both
# labeled and unlabeled examples.
train_input_examples_uri = os.path.join(identified_examples.uri,
'Split-train')
output_graph_uri = os.path.join(synthesized_graph.uri, 'Split-train')
os.mkdir(output_graph_uri)
print('Creating embeddings...')
create_embeddings(train_input_examples_uri, output_graph_uri)
print('Synthesizing graph...')
build_graph(output_graph_uri, similarity_threshold)
synthesized_graph.split_names = artifact_utils.encode_split_names(
splits=['Split-train'])
return
synthesize_graph = SynthesizeGraph(
identified_examples=identify_examples.outputs['identified_examples'],
component_name=u'SynthesizeGraph',
similarity_threshold=0.99)
context.run(synthesize_graph, enable_cache=False)
Creating embeddings... Synthesizing graph...
train_uri = synthesize_graph.outputs["synthesized_graph"].get()[0].uri
os.listdir(train_uri)
['Split-train']
graph_path = os.path.join(train_uri, "Split-train", "graph.tsv")
print("node 1\t\t\t\t\tnode 2\t\t\t\t\tsimilarity")
!head {graph_path}
print("...")
!tail {graph_path}
node 1 node 2 similarity 8c4f4c09-3dfa-4b8f-b3eb-e1596f7509ed 638cfa94-ebb5-4182-bb18-a8f4cc332131 0.990838 638cfa94-ebb5-4182-bb18-a8f4cc332131 8c4f4c09-3dfa-4b8f-b3eb-e1596f7509ed 0.990838 8c4f4c09-3dfa-4b8f-b3eb-e1596f7509ed 1f9023b1-d312-4fc5-b87f-52636c7b0ea8 0.990184 1f9023b1-d312-4fc5-b87f-52636c7b0ea8 8c4f4c09-3dfa-4b8f-b3eb-e1596f7509ed 0.990184 292e3cc8-7c6b-4463-98d8-5dbfa88a75f9 1ec31309-2b4a-4a4c-9f72-083f201d54a7 0.992471 1ec31309-2b4a-4a4c-9f72-083f201d54a7 292e3cc8-7c6b-4463-98d8-5dbfa88a75f9 0.992471 d5560e01-40d9-4cc0-9cd0-23355c7378f2 b78d8ee6-e404-44bf-a5bc-977b883d1913 0.992505 b78d8ee6-e404-44bf-a5bc-977b883d1913 d5560e01-40d9-4cc0-9cd0-23355c7378f2 0.992505 e138ef2e-4fe4-44b0-a4dc-8b01266b7ae6 b78d8ee6-e404-44bf-a5bc-977b883d1913 0.992823 b78d8ee6-e404-44bf-a5bc-977b883d1913 e138ef2e-4fe4-44b0-a4dc-8b01266b7ae6 0.992823 ... 11f44e7c-8393-4d17-8810-ca1f5e60e692 029e39a4-cd35-4e33-bdb1-8547f56a1ca7 0.991879 029e39a4-cd35-4e33-bdb1-8547f56a1ca7 11f44e7c-8393-4d17-8810-ca1f5e60e692 0.991879 4bdebeac-2f54-47a2-889c-3c2cf190e2dd 5eb7cfca-1f3d-4a32-9746-ebcca805b1d0 0.991046 5eb7cfca-1f3d-4a32-9746-ebcca805b1d0 4bdebeac-2f54-47a2-889c-3c2cf190e2dd 0.991046 e75e90af-8093-484a-883f-9f545a126208 3b2258ce-d8d7-40d5-ba1f-d771e7ddc56f 0.991198 3b2258ce-d8d7-40d5-ba1f-d771e7ddc56f e75e90af-8093-484a-883f-9f545a126208 0.991198 ce73d577-0f4d-4919-aaee-bbf8aadb12ec ba933752-a08b-4615-9b90-0731c8bfc23d 0.990260 ba933752-a08b-4615-9b90-0731c8bfc23d ce73d577-0f4d-4919-aaee-bbf8aadb12ec 0.990260 d20d75c6-eb13-41f5-865c-e6e54725fe13 648ff28d-6860-4e8f-a411-d1577a1d78ca 0.991317 648ff28d-6860-4e8f-a411-d1577a1d78ca d20d75c6-eb13-41f5-865c-e6e54725fe13 0.991317
wc -l {graph_path}
222132 /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/SynthesizeGraph/synthesized_graph/6/Split-train/graph.tsv
변환 구성 요소
는 Transform
요소를 행한다 데이터 변환 기능 및 엔지니어링. 결과에는 훈련 또는 추론 전에 데이터를 사전 처리하기 위해 훈련 및 제공 중에 사용되는 입력 TensorFlow 그래프가 포함됩니다. 이 그래프는 모델 훈련의 결과인 SavedModel의 일부가 됩니다. 훈련과 서빙 모두에 동일한 입력 그래프가 사용되기 때문에 전처리는 항상 동일하며 한 번만 작성하면 됩니다.
Transform 구성 요소에는 작업 중인 데이터 및/또는 모델에 필요할 수 있는 기능 엔지니어링의 임의적인 복잡성 때문에 다른 많은 구성 요소보다 더 많은 코드가 필요합니다. 필요한 처리를 정의하는 코드 파일을 사용할 수 있어야 합니다.
각 샘플에는 다음 세 가지 기능이 포함됩니다.
- ID : 시료의 노드 ID.
- text_xf : 단어 ID가 포함 된의 INT64 목록.
- label_xf : 검토 대상 클래스를 식별 INT64 싱글 0 = 네거티브, 포지티브 = 1.
하자가 포함 된 모듈 정의 preprocessing_fn()
우리는에 전달할 것이라고 기능을 Transform
구성 요소 :
_transform_module_file = 'imdb_transform.py'
%%writefile {_transform_module_file}
import tensorflow as tf
import tensorflow_transform as tft
SEQUENCE_LENGTH = 100
VOCAB_SIZE = 10000
OOV_SIZE = 100
def tokenize_reviews(reviews, sequence_length=SEQUENCE_LENGTH):
reviews = tf.strings.lower(reviews)
reviews = tf.strings.regex_replace(reviews, r" '| '|^'|'$", " ")
reviews = tf.strings.regex_replace(reviews, "[^a-z' ]", " ")
tokens = tf.strings.split(reviews)[:, :sequence_length]
start_tokens = tf.fill([tf.shape(reviews)[0], 1], "<START>")
end_tokens = tf.fill([tf.shape(reviews)[0], 1], "<END>")
tokens = tf.concat([start_tokens, tokens, end_tokens], axis=1)
tokens = tokens[:, :sequence_length]
tokens = tokens.to_tensor(default_value="<PAD>")
pad = sequence_length - tf.shape(tokens)[1]
tokens = tf.pad(tokens, [[0, 0], [0, pad]], constant_values="<PAD>")
return tf.reshape(tokens, [-1, sequence_length])
def preprocessing_fn(inputs):
"""tf.transform's callback function for preprocessing inputs.
Args:
inputs: map from feature keys to raw not-yet-transformed features.
Returns:
Map from string feature key to transformed feature operations.
"""
outputs = {}
outputs["id"] = inputs["id"]
tokens = tokenize_reviews(_fill_in_missing(inputs["text"], ''))
outputs["text_xf"] = tft.compute_and_apply_vocabulary(
tokens,
top_k=VOCAB_SIZE,
num_oov_buckets=OOV_SIZE)
outputs["label_xf"] = _fill_in_missing(inputs["label"], -1)
return outputs
def _fill_in_missing(x, default_value):
"""Replace missing values in a SparseTensor.
Fills in missing values of `x` with the default_value.
Args:
x: A `SparseTensor` of rank 2. Its dense shape should have size at most 1
in the second dimension.
default_value: the value with which to replace the missing values.
Returns:
A rank 1 tensor where missing values of `x` have been filled in.
"""
if not isinstance(x, tf.sparse.SparseTensor):
return x
return tf.squeeze(
tf.sparse.to_dense(
tf.SparseTensor(x.indices, x.values, [x.dense_shape[0], 1]),
default_value),
axis=1)
Writing imdb_transform.py
작성하고 실행 Transform
위에 생성 된 파일을 참조, 구성 요소입니다.
# Performs transformations and feature engineering in training and serving.
transform = Transform(
examples=identify_examples.outputs['identified_examples'],
schema=schema_gen.outputs['schema'],
module_file=_transform_module_file)
context.run(transform, enable_cache=True)
running bdist_wheel running build running build_py creating build creating build/lib copying imdb_transform.py -> build/lib installing to /tmp/tmpiem_52h3 running install running install_lib copying build/lib/imdb_transform.py -> /tmp/tmpiem_52h3 running install_egg_info running egg_info creating tfx_user_code_Transform.egg-info writing tfx_user_code_Transform.egg-info/PKG-INFO writing dependency_links to tfx_user_code_Transform.egg-info/dependency_links.txt writing top-level names to tfx_user_code_Transform.egg-info/top_level.txt writing manifest file 'tfx_user_code_Transform.egg-info/SOURCES.txt' reading manifest file 'tfx_user_code_Transform.egg-info/SOURCES.txt' writing manifest file 'tfx_user_code_Transform.egg-info/SOURCES.txt' Copying tfx_user_code_Transform.egg-info to /tmp/tmpiem_52h3/tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50-py3.7.egg-info running install_scripts creating /tmp/tmpiem_52h3/tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50.dist-info/WHEEL creating '/tmp/tmps6mh09_9/tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50-py3-none-any.whl' and adding '/tmp/tmpiem_52h3' to it adding 'imdb_transform.py' adding 'tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50.dist-info/METADATA' adding 'tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50.dist-info/WHEEL' adding 'tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50.dist-info/top_level.txt' adding 'tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50.dist-info/RECORD' removing /tmp/tmpiem_52h3 /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. setuptools.SetuptoolsDeprecationWarning, listing git files failed - pretending there aren't any I1204 11:43:54.715353 6839 rdbms_metadata_access_object.cc:686] No property is defined for the Type I1204 11:43:54.719055 6839 rdbms_metadata_access_object.cc:686] No property is defined for the Type Processing /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/_wheels/tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50-py3-none-any.whl Installing collected packages: tfx-user-code-Transform Successfully installed tfx-user-code-Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50 Processing /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/_wheels/tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50-py3-none-any.whl Installing collected packages: tfx-user-code-Transform Successfully installed tfx-user-code-Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50 WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_transform/tf_utils.py:261: Tensor.experimental_ref (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Use ref() instead. WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_transform/tf_utils.py:261: Tensor.experimental_ref (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Use ref() instead. Processing /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/_wheels/tfx_user_code_Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50-py3-none-any.whl WARNING:root:This output type hint will be ignored and not used for type-checking purposes. Typically, output type hints for a PTransform are single (or nested) types wrapped by a PCollection, PDone, or None. Got: Tuple[Dict[str, Union[NoneType, _Dataset]], Union[Dict[str, Dict[str, PCollection]], NoneType]] instead. Installing collected packages: tfx-user-code-Transform Successfully installed tfx-user-code-Transform-0.0+074f608d1f54105225e2fee77ebe4b6159a009eca01b5a0791099840a2185d50 WARNING:absl:Tables initialized inside a tf.function will be re-initialized on every invocation of the function. This re-initialization can have significant impact on performance. Consider lifting them out of the graph context using `tf.init_scope`.: compute_and_apply_vocabulary/apply_vocab/text_file_init/InitializeTableFromTextFileV2 WARNING:absl:Tables initialized inside a tf.function will be re-initialized on every invocation of the function. This re-initialization can have significant impact on performance. Consider lifting them out of the graph context using `tf.init_scope`.: compute_and_apply_vocabulary/apply_vocab/text_file_init/InitializeTableFromTextFileV2 WARNING:root:This output type hint will be ignored and not used for type-checking purposes. Typically, output type hints for a PTransform are single (or nested) types wrapped by a PCollection, PDone, or None. Got: Tuple[Dict[str, Union[NoneType, _Dataset]], Union[Dict[str, Dict[str, PCollection]], NoneType]] instead. WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter. INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/transform_graph/7/.temp_path/tftransform_tmp/41946dd1d2594124b929c5ec8c7f82cd/assets 2021-12-04 11:44:05.216878: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them. INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/transform_graph/7/.temp_path/tftransform_tmp/41946dd1d2594124b929c5ec8c7f82cd/assets INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/transform_graph/7/.temp_path/tftransform_tmp/50d5168031d643728b9fd8d8ede0362b/assets INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/transform_graph/7/.temp_path/tftransform_tmp/50d5168031d643728b9fd8d8ede0362b/assets
Transform
성분 출력의 2 개 종류가 :
-
transform_graph
(이 그래프가 게재 및 평가 모델을 포함한다) 전처리 조작을 수행 할 수있는 그래프이다. -
transformed_examples
전처리 된 교육 및 평가 데이터를 나타냅니다.
transform.outputs
{'transform_graph': Channel( type_name: TransformGraph artifacts: [Artifact(artifact: id: 7 type_id: 25 uri: "/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/transform_graph/7" custom_properties { key: "name" value { string_value: "transform_graph" } } custom_properties { key: "producer_component" value { string_value: "Transform" } } custom_properties { key: "state" value { string_value: "published" } } custom_properties { key: "tfx_version" value { string_value: "1.2.0" } } state: LIVE , artifact_type: id: 25 name: "TransformGraph" )] additional_properties: {} additional_custom_properties: {} ), 'transformed_examples': Channel( type_name: Examples artifacts: [Artifact(artifact: id: 8 type_id: 14 uri: "/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/transformed_examples/7" properties { key: "split_names" value { string_value: "[\"train\", \"eval\"]" } } custom_properties { key: "name" value { string_value: "transformed_examples" } } custom_properties { key: "producer_component" value { string_value: "Transform" } } custom_properties { key: "state" value { string_value: "published" } } custom_properties { key: "tfx_version" value { string_value: "1.2.0" } } state: LIVE , artifact_type: id: 14 name: "Examples" properties { key: "span" value: INT } properties { key: "split_names" value: STRING } properties { key: "version" value: INT } )] additional_properties: {} additional_custom_properties: {} ), 'updated_analyzer_cache': Channel( type_name: TransformCache artifacts: [Artifact(artifact: id: 9 type_id: 26 uri: "/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/updated_analyzer_cache/7" custom_properties { key: "name" value { string_value: "updated_analyzer_cache" } } custom_properties { key: "producer_component" value { string_value: "Transform" } } custom_properties { key: "state" value { string_value: "published" } } custom_properties { key: "tfx_version" value { string_value: "1.2.0" } } state: LIVE , artifact_type: id: 26 name: "TransformCache" )] additional_properties: {} additional_custom_properties: {} ), 'pre_transform_schema': Channel( type_name: Schema artifacts: [Artifact(artifact: id: 10 type_id: 19 uri: "/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/pre_transform_schema/7" custom_properties { key: "name" value { string_value: "pre_transform_schema" } } custom_properties { key: "producer_component" value { string_value: "Transform" } } custom_properties { key: "state" value { string_value: "published" } } custom_properties { key: "tfx_version" value { string_value: "1.2.0" } } state: LIVE , artifact_type: id: 19 name: "Schema" )] additional_properties: {} additional_custom_properties: {} ), 'pre_transform_stats': Channel( type_name: ExampleStatistics artifacts: [Artifact(artifact: id: 11 type_id: 17 uri: "/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/pre_transform_stats/7" custom_properties { key: "name" value { string_value: "pre_transform_stats" } } custom_properties { key: "producer_component" value { string_value: "Transform" } } custom_properties { key: "state" value { string_value: "published" } } custom_properties { key: "tfx_version" value { string_value: "1.2.0" } } state: LIVE , artifact_type: id: 17 name: "ExampleStatistics" properties { key: "span" value: INT } properties { key: "split_names" value: STRING } )] additional_properties: {} additional_custom_properties: {} ), 'post_transform_schema': Channel( type_name: Schema artifacts: [Artifact(artifact: id: 12 type_id: 19 uri: "/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/post_transform_schema/7" custom_properties { key: "name" value { string_value: "post_transform_schema" } } custom_properties { key: "producer_component" value { string_value: "Transform" } } custom_properties { key: "state" value { string_value: "published" } } custom_properties { key: "tfx_version" value { string_value: "1.2.0" } } state: LIVE , artifact_type: id: 19 name: "Schema" )] additional_properties: {} additional_custom_properties: {} ), 'post_transform_stats': Channel( type_name: ExampleStatistics artifacts: [Artifact(artifact: id: 13 type_id: 17 uri: "/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/post_transform_stats/7" custom_properties { key: "name" value { string_value: "post_transform_stats" } } custom_properties { key: "producer_component" value { string_value: "Transform" } } custom_properties { key: "state" value { string_value: "published" } } custom_properties { key: "tfx_version" value { string_value: "1.2.0" } } state: LIVE , artifact_type: id: 17 name: "ExampleStatistics" properties { key: "span" value: INT } properties { key: "split_names" value: STRING } )] additional_properties: {} additional_custom_properties: {} ), 'post_transform_anomalies': Channel( type_name: ExampleAnomalies artifacts: [Artifact(artifact: id: 14 type_id: 21 uri: "/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/post_transform_anomalies/7" custom_properties { key: "name" value { string_value: "post_transform_anomalies" } } custom_properties { key: "producer_component" value { string_value: "Transform" } } custom_properties { key: "state" value { string_value: "published" } } custom_properties { key: "tfx_version" value { string_value: "1.2.0" } } state: LIVE , artifact_type: id: 21 name: "ExampleAnomalies" properties { key: "span" value: INT } properties { key: "split_names" value: STRING } )] additional_properties: {} additional_custom_properties: {} )}
상기 들여다보세요 transform_graph
유물을 : 그것은 3 개 하위 디렉토리를 포함하는 디렉토리를 가리키는 :
train_uri = transform.outputs['transform_graph'].get()[0].uri
os.listdir(train_uri)
['transform_fn', 'transformed_metadata', 'metadata']
transform_fn
서브 디렉토리에는 실제 전처리 그래프를 포함한다. metadata
하위 디렉토리는 원본 데이터의 스키마를 포함합니다. transformed_metadata
하위 디렉토리는 전처리 된 데이터의 스키마를 포함합니다.
변환된 예제 중 일부를 살펴보고 실제로 의도한 대로 처리되었는지 확인하십시오.
def pprint_examples(artifact, n_examples=3):
print("artifact:", artifact)
uri = os.path.join(artifact.uri, "Split-train")
print("uri:", uri)
tfrecord_filenames = [os.path.join(uri, name) for name in os.listdir(uri)]
print("tfrecord_filenames:", tfrecord_filenames)
dataset = tf.data.TFRecordDataset(tfrecord_filenames, compression_type="GZIP")
for tfrecord in dataset.take(n_examples):
serialized_example = tfrecord.numpy()
example = tf.train.Example.FromString(serialized_example)
pp.pprint(example)
pprint_examples(transform.outputs['transformed_examples'].get()[0])
artifact: Artifact(artifact: id: 8 type_id: 14 uri: "/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/transformed_examples/7" properties { key: "split_names" value { string_value: "[\"train\", \"eval\"]" } } custom_properties { key: "name" value { string_value: "transformed_examples" } } custom_properties { key: "producer_component" value { string_value: "Transform" } } custom_properties { key: "state" value { string_value: "published" } } custom_properties { key: "tfx_version" value { string_value: "1.2.0" } } state: LIVE , artifact_type: id: 14 name: "Examples" properties { key: "span" value: INT } properties { key: "split_names" value: STRING } properties { key: "version" value: INT } ) uri: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/transformed_examples/7/Split-train tfrecord_filenames: ['/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Transform/transformed_examples/7/Split-train/transformed_examples-00000-of-00001.gz'] features { feature { key: "id" value { bytes_list { value: "d62bf114-6f90-4a7d-ad60-559924f2582b" } } } feature { key: "label_xf" value { int64_list { value: 0 } } } feature { key: "text_xf" value { int64_list { value: 13 value: 8 value: 14 value: 32 value: 338 value: 310 value: 15 value: 95 value: 27 value: 10001 value: 9 value: 31 value: 1173 value: 3153 value: 43 value: 495 value: 10060 value: 214 value: 26 value: 71 value: 142 value: 19 value: 8 value: 204 value: 339 value: 27 value: 74 value: 181 value: 238 value: 9 value: 440 value: 67 value: 74 value: 71 value: 94 value: 100 value: 22 value: 5442 value: 8 value: 1573 value: 607 value: 530 value: 8 value: 15 value: 6 value: 32 value: 378 value: 6292 value: 207 value: 2276 value: 388 value: 0 value: 84 value: 1023 value: 154 value: 65 value: 155 value: 52 value: 0 value: 10080 value: 7871 value: 65 value: 250 value: 74 value: 3202 value: 20 value: 10000 value: 3720 value: 10020 value: 10008 value: 1282 value: 3862 value: 3 value: 53 value: 3952 value: 110 value: 1879 value: 17 value: 3153 value: 14 value: 166 value: 19 value: 2 value: 1023 value: 1007 value: 9405 value: 9 value: 2 value: 15 value: 12 value: 14 value: 4504 value: 4 value: 109 value: 158 value: 1202 value: 7 value: 174 value: 505 value: 12 } } } } features { feature { key: "id" value { bytes_list { value: "bc94a341-63f1-417c-8a1b-c723a29e67e4" } } } feature { key: "label_xf" value { int64_list { value: 0 } } } feature { key: "text_xf" value { int64_list { value: 13 value: 7 value: 23 value: 75 value: 494 value: 5 value: 748 value: 2155 value: 307 value: 91 value: 19 value: 8 value: 6 value: 499 value: 763 value: 5 value: 2 value: 1690 value: 4 value: 200 value: 593 value: 57 value: 1244 value: 120 value: 2364 value: 3 value: 4407 value: 21 value: 0 value: 10081 value: 3 value: 263 value: 42 value: 6947 value: 2 value: 169 value: 185 value: 21 value: 8 value: 5143 value: 7 value: 1339 value: 2155 value: 81 value: 0 value: 18 value: 14 value: 1468 value: 0 value: 86 value: 986 value: 14 value: 2259 value: 1790 value: 562 value: 3 value: 284 value: 200 value: 401 value: 5 value: 668 value: 19 value: 17 value: 58 value: 1934 value: 4 value: 45 value: 14 value: 4212 value: 113 value: 43 value: 135 value: 7 value: 753 value: 7 value: 224 value: 23 value: 1155 value: 179 value: 4 value: 0 value: 18 value: 19 value: 7 value: 191 value: 0 value: 2047 value: 4 value: 10 value: 3 value: 283 value: 42 value: 401 value: 5 value: 668 value: 4 value: 90 value: 234 value: 10023 value: 227 } } } } features { feature { key: "id" value { bytes_list { value: "bc8a0f68-45eb-4993-b757-52d92db1cd5a" } } } feature { key: "label_xf" value { int64_list { value: 0 } } } feature { key: "text_xf" value { int64_list { value: 13 value: 4577 value: 7158 value: 0 value: 10047 value: 3778 value: 3346 value: 9 value: 2 value: 758 value: 1915 value: 3 value: 2280 value: 1511 value: 3 value: 2003 value: 10020 value: 225 value: 786 value: 382 value: 16 value: 39 value: 203 value: 361 value: 5 value: 93 value: 11 value: 11 value: 19 value: 220 value: 21 value: 341 value: 2 value: 10000 value: 966 value: 0 value: 77 value: 4 value: 6677 value: 464 value: 10071 value: 5 value: 10042 value: 630 value: 2 value: 10044 value: 404 value: 2 value: 10044 value: 3 value: 5 value: 10008 value: 0 value: 1259 value: 630 value: 106 value: 10042 value: 6721 value: 10 value: 49 value: 21 value: 0 value: 2071 value: 20 value: 1292 value: 4 value: 0 value: 431 value: 11 value: 11 value: 166 value: 67 value: 2342 value: 5815 value: 12 value: 575 value: 21 value: 0 value: 1691 value: 537 value: 4 value: 0 value: 3605 value: 307 value: 0 value: 10054 value: 1563 value: 3115 value: 467 value: 4577 value: 3 value: 1069 value: 1158 value: 5 value: 23 value: 4279 value: 6677 value: 464 value: 20 value: 10004 } } } }
GraphAugmentation 구성 요소
샘플 기능과 합성 그래프가 있으므로 신경 구조화 학습을 위한 증강 훈련 데이터를 생성할 수 있습니다. NSL 프레임워크는 그래프와 샘플 기능을 결합하여 그래프 정규화를 위한 최종 훈련 데이터를 생성하는 라이브러리를 제공합니다. 결과 훈련 데이터에는 원본 샘플 기능과 해당 이웃의 기능이 포함됩니다.
이 튜토리얼에서는 무방향 모서리를 고려하고 샘플당 최대 3개의 이웃을 사용하여 그래프 이웃으로 훈련 데이터를 보강합니다.
def split_train_and_unsup(input_uri):
'Separate the labeled and unlabeled instances.'
tmp_dir = tempfile.mkdtemp(prefix='tfx-data')
tfrecord_filenames = [
os.path.join(input_uri, filename) for filename in os.listdir(input_uri)
]
train_path = os.path.join(tmp_dir, 'train.tfrecord')
unsup_path = os.path.join(tmp_dir, 'unsup.tfrecord')
with tf.io.TFRecordWriter(train_path) as train_writer, \
tf.io.TFRecordWriter(unsup_path) as unsup_writer:
for tfrecord in tf.data.TFRecordDataset(
tfrecord_filenames, compression_type='GZIP'):
example = tf.train.Example()
example.ParseFromString(tfrecord.numpy())
if ('label_xf' not in example.features.feature or
example.features.feature['label_xf'].int64_list.value[0] == -1):
writer = unsup_writer
else:
writer = train_writer
writer.write(tfrecord.numpy())
return train_path, unsup_path
def gzip(filepath):
with open(filepath, 'rb') as f_in:
with gzip_lib.open(filepath + '.gz', 'wb') as f_out:
shutil.copyfileobj(f_in, f_out)
os.remove(filepath)
def copy_tfrecords(input_uri, output_uri):
for filename in os.listdir(input_uri):
input_filename = os.path.join(input_uri, filename)
output_filename = os.path.join(output_uri, filename)
shutil.copyfile(input_filename, output_filename)
@component
def GraphAugmentation(identified_examples: InputArtifact[Examples],
synthesized_graph: InputArtifact[SynthesizedGraph],
augmented_examples: OutputArtifact[Examples],
num_neighbors: Parameter[int],
component_name: Parameter[str]) -> None:
# Get a list of the splits in input_data
splits_list = artifact_utils.decode_split_names(
split_names=identified_examples.split_names)
train_input_uri = os.path.join(identified_examples.uri, 'Split-train')
eval_input_uri = os.path.join(identified_examples.uri, 'Split-eval')
train_graph_uri = os.path.join(synthesized_graph.uri, 'Split-train')
train_output_uri = os.path.join(augmented_examples.uri, 'Split-train')
eval_output_uri = os.path.join(augmented_examples.uri, 'Split-eval')
os.mkdir(train_output_uri)
os.mkdir(eval_output_uri)
# Separate the labeled and unlabeled examples from the 'Split-train' split.
train_path, unsup_path = split_train_and_unsup(train_input_uri)
output_path = os.path.join(train_output_uri, 'nsl_train_data.tfr')
pack_nbrs_args = dict(
labeled_examples_path=train_path,
unlabeled_examples_path=unsup_path,
graph_path=os.path.join(train_graph_uri, 'graph.tsv'),
output_training_data_path=output_path,
add_undirected_edges=True,
max_nbrs=num_neighbors)
print('nsl.tools.pack_nbrs arguments:', pack_nbrs_args)
nsl.tools.pack_nbrs(**pack_nbrs_args)
# Downstream components expect gzip'ed TFRecords.
gzip(output_path)
# The test examples are left untouched and are simply copied over.
copy_tfrecords(eval_input_uri, eval_output_uri)
augmented_examples.split_names = identified_examples.split_names
return
# Augments training data with graph neighbors.
graph_augmentation = GraphAugmentation(
identified_examples=transform.outputs['transformed_examples'],
synthesized_graph=synthesize_graph.outputs['synthesized_graph'],
component_name=u'GraphAugmentation',
num_neighbors=3)
context.run(graph_augmentation, enable_cache=False)
nsl.tools.pack_nbrs arguments: {'labeled_examples_path': '/tmp/tfx-datajju3fxrq/train.tfrecord', 'unlabeled_examples_path': '/tmp/tfx-datajju3fxrq/unsup.tfrecord', 'graph_path': '/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/SynthesizeGraph/synthesized_graph/6/Split-train/graph.tsv', 'output_training_data_path': '/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/GraphAugmentation/augmented_examples/8/Split-train/nsl_train_data.tfr', 'add_undirected_edges': True, 'max_nbrs': 3}
pprint_examples(graph_augmentation.outputs['augmented_examples'].get()[0], 6)
artifact: Artifact(artifact: id: 15 type_id: 14 uri: "/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/GraphAugmentation/augmented_examples/8" properties { key: "split_names" value { string_value: "[\"train\", \"eval\"]" } } custom_properties { key: "name" value { string_value: "augmented_examples" } } custom_properties { key: "producer_component" value { string_value: "GraphAugmentation" } } custom_properties { key: "state" value { string_value: "published" } } custom_properties { key: "tfx_version" value { string_value: "1.2.0" } } state: LIVE , artifact_type: id: 14 name: "Examples" properties { key: "span" value: INT } properties { key: "split_names" value: STRING } properties { key: "version" value: INT } ) uri: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/GraphAugmentation/augmented_examples/8/Split-train tfrecord_filenames: ['/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/GraphAugmentation/augmented_examples/8/Split-train/nsl_train_data.tfr.gz'] features { feature { key: "NL_num_nbrs" value { int64_list { value: 0 } } } feature { key: "id" value { bytes_list { value: "d62bf114-6f90-4a7d-ad60-559924f2582b" } } } feature { key: "label_xf" value { int64_list { value: 0 } } } feature { key: "text_xf" value { int64_list { value: 13 value: 8 value: 14 value: 32 value: 338 value: 310 value: 15 value: 95 value: 27 value: 10001 value: 9 value: 31 value: 1173 value: 3153 value: 43 value: 495 value: 10060 value: 214 value: 26 value: 71 value: 142 value: 19 value: 8 value: 204 value: 339 value: 27 value: 74 value: 181 value: 238 value: 9 value: 440 value: 67 value: 74 value: 71 value: 94 value: 100 value: 22 value: 5442 value: 8 value: 1573 value: 607 value: 530 value: 8 value: 15 value: 6 value: 32 value: 378 value: 6292 value: 207 value: 2276 value: 388 value: 0 value: 84 value: 1023 value: 154 value: 65 value: 155 value: 52 value: 0 value: 10080 value: 7871 value: 65 value: 250 value: 74 value: 3202 value: 20 value: 10000 value: 3720 value: 10020 value: 10008 value: 1282 value: 3862 value: 3 value: 53 value: 3952 value: 110 value: 1879 value: 17 value: 3153 value: 14 value: 166 value: 19 value: 2 value: 1023 value: 1007 value: 9405 value: 9 value: 2 value: 15 value: 12 value: 14 value: 4504 value: 4 value: 109 value: 158 value: 1202 value: 7 value: 174 value: 505 value: 12 } } } } features { feature { key: "NL_num_nbrs" value { int64_list { value: 0 } } } feature { key: "id" value { bytes_list { value: "bc94a341-63f1-417c-8a1b-c723a29e67e4" } } } feature { key: "label_xf" value { int64_list { value: 0 } } } feature { key: "text_xf" value { int64_list { value: 13 value: 7 value: 23 value: 75 value: 494 value: 5 value: 748 value: 2155 value: 307 value: 91 value: 19 value: 8 value: 6 value: 499 value: 763 value: 5 value: 2 value: 1690 value: 4 value: 200 value: 593 value: 57 value: 1244 value: 120 value: 2364 value: 3 value: 4407 value: 21 value: 0 value: 10081 value: 3 value: 263 value: 42 value: 6947 value: 2 value: 169 value: 185 value: 21 value: 8 value: 5143 value: 7 value: 1339 value: 2155 value: 81 value: 0 value: 18 value: 14 value: 1468 value: 0 value: 86 value: 986 value: 14 value: 2259 value: 1790 value: 562 value: 3 value: 284 value: 200 value: 401 value: 5 value: 668 value: 19 value: 17 value: 58 value: 1934 value: 4 value: 45 value: 14 value: 4212 value: 113 value: 43 value: 135 value: 7 value: 753 value: 7 value: 224 value: 23 value: 1155 value: 179 value: 4 value: 0 value: 18 value: 19 value: 7 value: 191 value: 0 value: 2047 value: 4 value: 10 value: 3 value: 283 value: 42 value: 401 value: 5 value: 668 value: 4 value: 90 value: 234 value: 10023 value: 227 } } } } features { feature { key: "NL_num_nbrs" value { int64_list { value: 0 } } } feature { key: "id" value { bytes_list { value: "bc8a0f68-45eb-4993-b757-52d92db1cd5a" } } } feature { key: "label_xf" value { int64_list { value: 0 } } } feature { key: "text_xf" value { int64_list { value: 13 value: 4577 value: 7158 value: 0 value: 10047 value: 3778 value: 3346 value: 9 value: 2 value: 758 value: 1915 value: 3 value: 2280 value: 1511 value: 3 value: 2003 value: 10020 value: 225 value: 786 value: 382 value: 16 value: 39 value: 203 value: 361 value: 5 value: 93 value: 11 value: 11 value: 19 value: 220 value: 21 value: 341 value: 2 value: 10000 value: 966 value: 0 value: 77 value: 4 value: 6677 value: 464 value: 10071 value: 5 value: 10042 value: 630 value: 2 value: 10044 value: 404 value: 2 value: 10044 value: 3 value: 5 value: 10008 value: 0 value: 1259 value: 630 value: 106 value: 10042 value: 6721 value: 10 value: 49 value: 21 value: 0 value: 2071 value: 20 value: 1292 value: 4 value: 0 value: 431 value: 11 value: 11 value: 166 value: 67 value: 2342 value: 5815 value: 12 value: 575 value: 21 value: 0 value: 1691 value: 537 value: 4 value: 0 value: 3605 value: 307 value: 0 value: 10054 value: 1563 value: 3115 value: 467 value: 4577 value: 3 value: 1069 value: 1158 value: 5 value: 23 value: 4279 value: 6677 value: 464 value: 20 value: 10004 } } } } features { feature { key: "NL_num_nbrs" value { int64_list { value: 0 } } } feature { key: "id" value { bytes_list { value: "06e86044-477f-47b9-babf-825fbb5af70c" } } } feature { key: "label_xf" value { int64_list { value: 1 } } } feature { key: "text_xf" value { int64_list { value: 13 value: 8 value: 6 value: 0 value: 251 value: 4 value: 18 value: 20 value: 2 value: 6783 value: 2295 value: 2338 value: 52 value: 0 value: 468 value: 4 value: 0 value: 189 value: 73 value: 153 value: 1294 value: 17 value: 90 value: 234 value: 935 value: 16 value: 25 value: 10024 value: 92 value: 2 value: 192 value: 4218 value: 3317 value: 3 value: 10098 value: 20 value: 2 value: 356 value: 4 value: 565 value: 334 value: 382 value: 36 value: 6989 value: 3 value: 6065 value: 2510 value: 16 value: 203 value: 7264 value: 2849 value: 0 value: 86 value: 346 value: 50 value: 26 value: 58 value: 10020 value: 5 value: 1464 value: 58 value: 2081 value: 2969 value: 42 value: 2 value: 2364 value: 3 value: 1402 value: 10062 value: 138 value: 147 value: 614 value: 115 value: 29 value: 90 value: 105 value: 2 value: 223 value: 18 value: 9 value: 160 value: 324 value: 3 value: 24 value: 12 value: 1252 value: 0 value: 2142 value: 10 value: 1832 value: 111 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 } } } } features { feature { key: "NL_num_nbrs" value { int64_list { value: 0 } } } feature { key: "id" value { bytes_list { value: "072dc782-850b-4286-8f4f-2f6f527db6cf" } } } feature { key: "label_xf" value { int64_list { value: 1 } } } feature { key: "text_xf" value { int64_list { value: 13 value: 16 value: 423 value: 23 value: 1367 value: 30 value: 0 value: 363 value: 12 value: 153 value: 3174 value: 9 value: 8 value: 18 value: 26 value: 667 value: 338 value: 1372 value: 0 value: 86 value: 46 value: 9200 value: 282 value: 0 value: 10091 value: 4 value: 0 value: 694 value: 10028 value: 52 value: 362 value: 26 value: 202 value: 39 value: 216 value: 5 value: 27 value: 5822 value: 19 value: 52 value: 58 value: 362 value: 26 value: 202 value: 39 value: 474 value: 0 value: 10029 value: 4 value: 2 value: 243 value: 143 value: 386 value: 3 value: 0 value: 386 value: 579 value: 2 value: 132 value: 57 value: 725 value: 88 value: 140 value: 30 value: 27 value: 33 value: 1359 value: 29 value: 8 value: 567 value: 35 value: 106 value: 230 value: 60 value: 0 value: 3041 value: 5 value: 7879 value: 28 value: 281 value: 110 value: 111 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 value: 1 } } } } features { feature { key: "NL_num_nbrs" value { int64_list { value: 0 } } } feature { key: "id" value { bytes_list { value: "27da61c0-3dff-46e9-8588-d12176b3798f" } } } feature { key: "label_xf" value { int64_list { value: 1 } } } feature { key: "text_xf" value { int64_list { value: 13 value: 8 value: 6 value: 2 value: 18 value: 69 value: 140 value: 27 value: 83 value: 31 value: 1877 value: 905 value: 9 value: 10057 value: 31 value: 43 value: 2115 value: 36 value: 32 value: 2057 value: 6133 value: 10 value: 6 value: 32 value: 2474 value: 1614 value: 3 value: 2707 value: 990 value: 4 value: 10067 value: 9 value: 2 value: 1532 value: 242 value: 90 value: 3757 value: 3 value: 90 value: 10026 value: 0 value: 242 value: 6 value: 260 value: 31 value: 24 value: 4 value: 0 value: 84 value: 497 value: 177 value: 1151 value: 777 value: 9 value: 397 value: 552 value: 7726 value: 10051 value: 34 value: 14 value: 379 value: 33 value: 1829 value: 9 value: 123 value: 0 value: 916 value: 10028 value: 7 value: 64 value: 571 value: 12 value: 8 value: 18 value: 27 value: 687 value: 9 value: 30 value: 5609 value: 16 value: 25 value: 99 value: 117 value: 66 value: 2 value: 130 value: 21 value: 8 value: 842 value: 7726 value: 10051 value: 6 value: 338 value: 1107 value: 3 value: 24 value: 10020 value: 29 value: 53 value: 1476 } } } }
트레이너 구성 요소
Trainer
구성 요소 기차 모델은 TensorFlow를 사용하여.
포함하는 파이썬 모듈 만들기 trainer_fn
추정량을 반환해야합니다 기능을. 당신이 Keras 모델을 만드는 선호하는 경우에, 이렇게 한 후 사용 추정량로 변환 할 수 있습니다 keras.model_to_estimator()
.
# Setup paths.
_trainer_module_file = 'imdb_trainer.py'
%%writefile {_trainer_module_file}
import neural_structured_learning as nsl
import tensorflow as tf
import tensorflow_model_analysis as tfma
import tensorflow_transform as tft
from tensorflow_transform.tf_metadata import schema_utils
NBR_FEATURE_PREFIX = 'NL_nbr_'
NBR_WEIGHT_SUFFIX = '_weight'
LABEL_KEY = 'label'
ID_FEATURE_KEY = 'id'
def _transformed_name(key):
return key + '_xf'
def _transformed_names(keys):
return [_transformed_name(key) for key in keys]
# Hyperparameters:
#
# We will use an instance of `HParams` to inclue various hyperparameters and
# constants used for training and evaluation. We briefly describe each of them
# below:
#
# - max_seq_length: This is the maximum number of words considered from each
# movie review in this example.
# - vocab_size: This is the size of the vocabulary considered for this
# example.
# - oov_size: This is the out-of-vocabulary size considered for this example.
# - distance_type: This is the distance metric used to regularize the sample
# with its neighbors.
# - graph_regularization_multiplier: This controls the relative weight of the
# graph regularization term in the overall
# loss function.
# - num_neighbors: The number of neighbors used for graph regularization. This
# value has to be less than or equal to the `num_neighbors`
# argument used above in the GraphAugmentation component when
# invoking `nsl.tools.pack_nbrs`.
# - num_fc_units: The number of units in the fully connected layer of the
# neural network.
class HParams(object):
"""Hyperparameters used for training."""
def __init__(self):
### dataset parameters
# The following 3 values should match those defined in the Transform
# Component.
self.max_seq_length = 100
self.vocab_size = 10000
self.oov_size = 100
### Neural Graph Learning parameters
self.distance_type = nsl.configs.DistanceType.L2
self.graph_regularization_multiplier = 0.1
# The following value has to be at most the value of 'num_neighbors' used
# in the GraphAugmentation component.
self.num_neighbors = 1
### Model Architecture
self.num_embedding_dims = 16
self.num_fc_units = 64
HPARAMS = HParams()
def optimizer_fn():
"""Returns an instance of `tf.Optimizer`."""
return tf.compat.v1.train.RMSPropOptimizer(
learning_rate=0.0001, decay=1e-6)
def build_train_op(loss, global_step):
"""Builds a train op to optimize the given loss using gradient descent."""
with tf.name_scope('train'):
optimizer = optimizer_fn()
train_op = optimizer.minimize(loss=loss, global_step=global_step)
return train_op
# Building the model:
#
# A neural network is created by stacking layers—this requires two main
# architectural decisions:
# * How many layers to use in the model?
# * How many *hidden units* to use for each layer?
#
# In this example, the input data consists of an array of word-indices. The
# labels to predict are either 0 or 1. We will use a feed-forward neural network
# as our base model in this tutorial.
def feed_forward_model(features, is_training, reuse=tf.compat.v1.AUTO_REUSE):
"""Builds a simple 2 layer feed forward neural network.
The layers are effectively stacked sequentially to build the classifier. The
first layer is an Embedding layer, which takes the integer-encoded vocabulary
and looks up the embedding vector for each word-index. These vectors are
learned as the model trains. The vectors add a dimension to the output array.
The resulting dimensions are: (batch, sequence, embedding). Next is a global
average pooling 1D layer, which reduces the dimensionality of its inputs from
3D to 2D. This fixed-length output vector is piped through a fully-connected
(Dense) layer with 16 hidden units. The last layer is densely connected with a
single output node. Using the sigmoid activation function, this value is a
float between 0 and 1, representing a probability, or confidence level.
Args:
features: A dictionary containing batch features returned from the
`input_fn`, that include sample features, corresponding neighbor features,
and neighbor weights.
is_training: a Python Boolean value or a Boolean scalar Tensor, indicating
whether to apply dropout.
reuse: a Python Boolean value for reusing variable scope.
Returns:
logits: Tensor of shape [batch_size, 1].
representations: Tensor of shape [batch_size, _] for graph regularization.
This is the representation of each example at the graph regularization
layer.
"""
with tf.compat.v1.variable_scope('ff', reuse=reuse):
inputs = features[_transformed_name('text')]
embeddings = tf.compat.v1.get_variable(
'embeddings',
shape=[
HPARAMS.vocab_size + HPARAMS.oov_size, HPARAMS.num_embedding_dims
])
embedding_layer = tf.nn.embedding_lookup(embeddings, inputs)
pooling_layer = tf.compat.v1.layers.AveragePooling1D(
pool_size=HPARAMS.max_seq_length, strides=HPARAMS.max_seq_length)(
embedding_layer)
# Shape of pooling_layer is now [batch_size, 1, HPARAMS.num_embedding_dims]
pooling_layer = tf.reshape(pooling_layer, [-1, HPARAMS.num_embedding_dims])
dense_layer = tf.compat.v1.layers.Dense(
16, activation='relu')(
pooling_layer)
output_layer = tf.compat.v1.layers.Dense(
1, activation='sigmoid')(
dense_layer)
# Graph regularization will be done on the penultimate (dense) layer
# because the output layer is a single floating point number.
return output_layer, dense_layer
# A note on hidden units:
#
# The above model has two intermediate or "hidden" layers, between the input and
# output, and excluding the Embedding layer. The number of outputs (units,
# nodes, or neurons) is the dimension of the representational space for the
# layer. In other words, the amount of freedom the network is allowed when
# learning an internal representation. If a model has more hidden units
# (a higher-dimensional representation space), and/or more layers, then the
# network can learn more complex representations. However, it makes the network
# more computationally expensive and may lead to learning unwanted
# patterns—patterns that improve performance on training data but not on the
# test data. This is called overfitting.
# This function will be used to generate the embeddings for samples and their
# corresponding neighbors, which will then be used for graph regularization.
def embedding_fn(features, mode):
"""Returns the embedding corresponding to the given features.
Args:
features: A dictionary containing batch features returned from the
`input_fn`, that include sample features, corresponding neighbor features,
and neighbor weights.
mode: Specifies if this is training, evaluation, or prediction. See
tf.estimator.ModeKeys.
Returns:
The embedding that will be used for graph regularization.
"""
is_training = (mode == tf.estimator.ModeKeys.TRAIN)
_, embedding = feed_forward_model(features, is_training)
return embedding
def feed_forward_model_fn(features, labels, mode, params, config):
"""Implementation of the model_fn for the base feed-forward model.
Args:
features: This is the first item returned from the `input_fn` passed to
`train`, `evaluate`, and `predict`. This should be a single `Tensor` or
`dict` of same.
labels: This is the second item returned from the `input_fn` passed to
`train`, `evaluate`, and `predict`. This should be a single `Tensor` or
`dict` of same (for multi-head models). If mode is `ModeKeys.PREDICT`,
`labels=None` will be passed. If the `model_fn`'s signature does not
accept `mode`, the `model_fn` must still be able to handle `labels=None`.
mode: Optional. Specifies if this training, evaluation or prediction. See
`ModeKeys`.
params: An HParams instance as returned by get_hyper_parameters().
config: Optional configuration object. Will receive what is passed to
Estimator in `config` parameter, or the default `config`. Allows updating
things in your model_fn based on configuration such as `num_ps_replicas`,
or `model_dir`. Unused currently.
Returns:
A `tf.estimator.EstimatorSpec` for the base feed-forward model. This does
not include graph-based regularization.
"""
is_training = mode == tf.estimator.ModeKeys.TRAIN
# Build the computation graph.
probabilities, _ = feed_forward_model(features, is_training)
predictions = tf.round(probabilities)
if mode == tf.estimator.ModeKeys.PREDICT:
# labels will be None, and no loss to compute.
cross_entropy_loss = None
eval_metric_ops = None
else:
# Loss is required in train and eval modes.
# Flatten 'probabilities' to 1-D.
probabilities = tf.reshape(probabilities, shape=[-1])
cross_entropy_loss = tf.compat.v1.keras.losses.binary_crossentropy(
labels, probabilities)
eval_metric_ops = {
'accuracy': tf.compat.v1.metrics.accuracy(labels, predictions)
}
if is_training:
global_step = tf.compat.v1.train.get_or_create_global_step()
train_op = build_train_op(cross_entropy_loss, global_step)
else:
train_op = None
return tf.estimator.EstimatorSpec(
mode=mode,
predictions={
'probabilities': probabilities,
'predictions': predictions
},
loss=cross_entropy_loss,
train_op=train_op,
eval_metric_ops=eval_metric_ops)
# Tf.Transform considers these features as "raw"
def _get_raw_feature_spec(schema):
return schema_utils.schema_as_feature_spec(schema).feature_spec
def _gzip_reader_fn(filenames):
"""Small utility returning a record reader that can read gzip'ed files."""
return tf.data.TFRecordDataset(
filenames,
compression_type='GZIP')
def _example_serving_receiver_fn(tf_transform_output, schema):
"""Build the serving in inputs.
Args:
tf_transform_output: A TFTransformOutput.
schema: the schema of the input data.
Returns:
Tensorflow graph which parses examples, applying tf-transform to them.
"""
raw_feature_spec = _get_raw_feature_spec(schema)
raw_feature_spec.pop(LABEL_KEY)
# We don't need the ID feature for serving.
raw_feature_spec.pop(ID_FEATURE_KEY)
raw_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(
raw_feature_spec, default_batch_size=None)
serving_input_receiver = raw_input_fn()
transformed_features = tf_transform_output.transform_raw_features(
serving_input_receiver.features)
# Even though, LABEL_KEY was removed from 'raw_feature_spec', the transform
# operation would have injected the transformed LABEL_KEY feature with a
# default value.
transformed_features.pop(_transformed_name(LABEL_KEY))
return tf.estimator.export.ServingInputReceiver(
transformed_features, serving_input_receiver.receiver_tensors)
def _eval_input_receiver_fn(tf_transform_output, schema):
"""Build everything needed for the tf-model-analysis to run the model.
Args:
tf_transform_output: A TFTransformOutput.
schema: the schema of the input data.
Returns:
EvalInputReceiver function, which contains:
- Tensorflow graph which parses raw untransformed features, applies the
tf-transform preprocessing operators.
- Set of raw, untransformed features.
- Label against which predictions will be compared.
"""
# Notice that the inputs are raw features, not transformed features here.
raw_feature_spec = _get_raw_feature_spec(schema)
# We don't need the ID feature for TFMA.
raw_feature_spec.pop(ID_FEATURE_KEY)
raw_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(
raw_feature_spec, default_batch_size=None)
serving_input_receiver = raw_input_fn()
transformed_features = tf_transform_output.transform_raw_features(
serving_input_receiver.features)
labels = transformed_features.pop(_transformed_name(LABEL_KEY))
return tfma.export.EvalInputReceiver(
features=transformed_features,
receiver_tensors=serving_input_receiver.receiver_tensors,
labels=labels)
def _augment_feature_spec(feature_spec, num_neighbors):
"""Augments `feature_spec` to include neighbor features.
Args:
feature_spec: Dictionary of feature keys mapping to TF feature types.
num_neighbors: Number of neighbors to use for feature key augmentation.
Returns:
An augmented `feature_spec` that includes neighbor feature keys.
"""
for i in range(num_neighbors):
feature_spec['{}{}_{}'.format(NBR_FEATURE_PREFIX, i, 'id')] = \
tf.io.VarLenFeature(dtype=tf.string)
# We don't care about the neighbor features corresponding to
# _transformed_name(LABEL_KEY) because the LABEL_KEY feature will be
# removed from the feature spec during training/evaluation.
feature_spec['{}{}_{}'.format(NBR_FEATURE_PREFIX, i, 'text_xf')] = \
tf.io.FixedLenFeature(shape=[HPARAMS.max_seq_length], dtype=tf.int64,
default_value=tf.constant(0, dtype=tf.int64,
shape=[HPARAMS.max_seq_length]))
# The 'NL_num_nbrs' features is currently not used.
# Set the neighbor weight feature keys.
for i in range(num_neighbors):
feature_spec['{}{}{}'.format(NBR_FEATURE_PREFIX, i, NBR_WEIGHT_SUFFIX)] = \
tf.io.FixedLenFeature(shape=[1], dtype=tf.float32, default_value=[0.0])
return feature_spec
def _input_fn(filenames, tf_transform_output, is_training, batch_size=200):
"""Generates features and labels for training or evaluation.
Args:
filenames: [str] list of CSV files to read data from.
tf_transform_output: A TFTransformOutput.
is_training: Boolean indicating if we are in training mode.
batch_size: int First dimension size of the Tensors returned by input_fn
Returns:
A (features, indices) tuple where features is a dictionary of
Tensors, and indices is a single Tensor of label indices.
"""
transformed_feature_spec = (
tf_transform_output.transformed_feature_spec().copy())
# During training, NSL uses augmented training data (which includes features
# from graph neighbors). So, update the feature spec accordingly. This needs
# to be done because we are using different schemas for NSL training and eval,
# but the Trainer Component only accepts a single schema.
if is_training:
transformed_feature_spec =_augment_feature_spec(transformed_feature_spec,
HPARAMS.num_neighbors)
dataset = tf.data.experimental.make_batched_features_dataset(
filenames, batch_size, transformed_feature_spec, reader=_gzip_reader_fn)
transformed_features = tf.compat.v1.data.make_one_shot_iterator(
dataset).get_next()
# We pop the label because we do not want to use it as a feature while we're
# training.
return transformed_features, transformed_features.pop(
_transformed_name(LABEL_KEY))
# TFX will call this function
def trainer_fn(hparams, schema):
"""Build the estimator using the high level API.
Args:
hparams: Holds hyperparameters used to train the model as name/value pairs.
schema: Holds the schema of the training examples.
Returns:
A dict of the following:
- estimator: The estimator that will be used for training and eval.
- train_spec: Spec for training.
- eval_spec: Spec for eval.
- eval_input_receiver_fn: Input function for eval.
"""
train_batch_size = 40
eval_batch_size = 40
tf_transform_output = tft.TFTransformOutput(hparams.transform_output)
train_input_fn = lambda: _input_fn(
hparams.train_files,
tf_transform_output,
is_training=True,
batch_size=train_batch_size)
eval_input_fn = lambda: _input_fn(
hparams.eval_files,
tf_transform_output,
is_training=False,
batch_size=eval_batch_size)
train_spec = tf.estimator.TrainSpec(
train_input_fn,
max_steps=hparams.train_steps)
serving_receiver_fn = lambda: _example_serving_receiver_fn(
tf_transform_output, schema)
exporter = tf.estimator.FinalExporter('imdb', serving_receiver_fn)
eval_spec = tf.estimator.EvalSpec(
eval_input_fn,
steps=hparams.eval_steps,
exporters=[exporter],
name='imdb-eval')
run_config = tf.estimator.RunConfig(
save_checkpoints_steps=999, keep_checkpoint_max=1)
run_config = run_config.replace(model_dir=hparams.serving_model_dir)
estimator = tf.estimator.Estimator(
model_fn=feed_forward_model_fn, config=run_config, params=HPARAMS)
# Create a graph regularization config.
graph_reg_config = nsl.configs.make_graph_reg_config(
max_neighbors=HPARAMS.num_neighbors,
multiplier=HPARAMS.graph_regularization_multiplier,
distance_type=HPARAMS.distance_type,
sum_over_axis=-1)
# Invoke the Graph Regularization Estimator wrapper to incorporate
# graph-based regularization for training.
graph_nsl_estimator = nsl.estimator.add_graph_regularization(
estimator,
embedding_fn,
optimizer_fn=optimizer_fn,
graph_reg_config=graph_reg_config)
# Create an input receiver for TFMA processing
receiver_fn = lambda: _eval_input_receiver_fn(
tf_transform_output, schema)
return {
'estimator': graph_nsl_estimator,
'train_spec': train_spec,
'eval_spec': eval_spec,
'eval_input_receiver_fn': receiver_fn
}
Writing imdb_trainer.py
작성하고 실행 Trainer
그것을 우리가 위에서 만든 파일을 전달 구성 요소를.
# Uses user-provided Python function that implements a model using TensorFlow's
# Estimators API.
trainer = Trainer(
module_file=_trainer_module_file,
custom_executor_spec=executor_spec.ExecutorClassSpec(
trainer_executor.Executor),
transformed_examples=graph_augmentation.outputs['augmented_examples'],
schema=schema_gen.outputs['schema'],
transform_graph=transform.outputs['transform_graph'],
train_args=trainer_pb2.TrainArgs(num_steps=10000),
eval_args=trainer_pb2.EvalArgs(num_steps=5000))
context.run(trainer)
WARNING:absl:`custom_executor_spec` is deprecated. Please customize component directly. WARNING:absl:`transformed_examples` is deprecated. Please use `examples` instead. /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. setuptools.SetuptoolsDeprecationWarning, listing git files failed - pretending there aren't any I1204 11:44:36.000404 6839 rdbms_metadata_access_object.cc:686] No property is defined for the Type I1204 11:44:36.003713 6839 rdbms_metadata_access_object.cc:686] No property is defined for the Type WARNING:absl:Examples artifact does not have payload_format custom property. Falling back to FORMAT_TF_EXAMPLE WARNING:absl:Examples artifact does not have payload_format custom property. Falling back to FORMAT_TF_EXAMPLE WARNING:absl:Examples artifact does not have payload_format custom property. Falling back to FORMAT_TF_EXAMPLE running bdist_wheel running build running build_py creating build creating build/lib copying imdb_trainer.py -> build/lib copying imdb_transform.py -> build/lib installing to /tmp/tmpyr89v7kz running install running install_lib copying build/lib/imdb_trainer.py -> /tmp/tmpyr89v7kz copying build/lib/imdb_transform.py -> /tmp/tmpyr89v7kz running install_egg_info running egg_info creating tfx_user_code_Trainer.egg-info writing tfx_user_code_Trainer.egg-info/PKG-INFO writing dependency_links to tfx_user_code_Trainer.egg-info/dependency_links.txt writing top-level names to tfx_user_code_Trainer.egg-info/top_level.txt writing manifest file 'tfx_user_code_Trainer.egg-info/SOURCES.txt' reading manifest file 'tfx_user_code_Trainer.egg-info/SOURCES.txt' writing manifest file 'tfx_user_code_Trainer.egg-info/SOURCES.txt' Copying tfx_user_code_Trainer.egg-info to /tmp/tmpyr89v7kz/tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d-py3.7.egg-info running install_scripts creating /tmp/tmpyr89v7kz/tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d.dist-info/WHEEL creating '/tmp/tmpl71r0gnq/tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d-py3-none-any.whl' and adding '/tmp/tmpyr89v7kz' to it adding 'imdb_trainer.py' adding 'imdb_transform.py' adding 'tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d.dist-info/METADATA' adding 'tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d.dist-info/WHEEL' adding 'tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d.dist-info/top_level.txt' adding 'tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d.dist-info/RECORD' removing /tmp/tmpyr89v7kz Processing /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/_wheels/tfx_user_code_Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d-py3-none-any.whl Installing collected packages: tfx-user-code-Trainer Successfully installed tfx-user-code-Trainer-0.0+b990a2c6a4f23081880867efa3bd3c38db9d7bd0a87a0c9b277ae63714defc8d INFO:tensorflow:Using config: {'_model_dir': '/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 999, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true graph_options { rewrite_options { meta_optimizer_iterations: ONE } } , '_keep_checkpoint_max': 1, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1} INFO:tensorflow:Using config: {'_model_dir': '/tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 999, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true graph_options { rewrite_options { meta_optimizer_iterations: ONE } } , '_keep_checkpoint_max': 1, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1} INFO:tensorflow:Not using Distribute Coordinator. INFO:tensorflow:Not using Distribute Coordinator. INFO:tensorflow:Running training and evaluation locally (non-distributed). INFO:tensorflow:Running training and evaluation locally (non-distributed). INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps 999 or save_checkpoints_secs None. INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps 999 or save_checkpoints_secs None. WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version. Instructions for updating: Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts. WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version. Instructions for updating: Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts. INFO:tensorflow:Calling model_fn. INFO:tensorflow:Calling model_fn. WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/rmsprop.py:123: calling Ones.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/rmsprop.py:123: calling Ones.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py:5049: calling gather (from tensorflow.python.ops.array_ops) with validate_indices is deprecated and will be removed in a future version. Instructions for updating: The `validate_indices` argument has no effect. Indices are always validated on CPU and never validated on GPU. WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py:5049: calling gather (from tensorflow.python.ops.array_ops) with validate_indices is deprecated and will be removed in a future version. Instructions for updating: The `validate_indices` argument has no effect. Indices are always validated on CPU and never validated on GPU. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Create CheckpointSaverHook. INFO:tensorflow:Create CheckpointSaverHook. INFO:tensorflow:Graph was finalized. INFO:tensorflow:Graph was finalized. INFO:tensorflow:Running local_init_op. INFO:tensorflow:Running local_init_op. INFO:tensorflow:Done running local_init_op. INFO:tensorflow:Done running local_init_op. INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0... INFO:tensorflow:Saving checkpoints for 0 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Saving checkpoints for 0 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0... INFO:tensorflow:loss = 0.6933001, step = 0 INFO:tensorflow:loss = 0.6933001, step = 0 INFO:tensorflow:global_step/sec: 220.68 INFO:tensorflow:global_step/sec: 220.68 INFO:tensorflow:loss = 0.69297814, step = 100 (0.454 sec) INFO:tensorflow:loss = 0.69297814, step = 100 (0.454 sec) INFO:tensorflow:global_step/sec: 288.754 INFO:tensorflow:global_step/sec: 288.754 INFO:tensorflow:loss = 0.6923192, step = 200 (0.347 sec) INFO:tensorflow:loss = 0.6923192, step = 200 (0.347 sec) INFO:tensorflow:global_step/sec: 286.927 INFO:tensorflow:global_step/sec: 286.927 INFO:tensorflow:loss = 0.6908457, step = 300 (0.348 sec) INFO:tensorflow:loss = 0.6908457, step = 300 (0.348 sec) INFO:tensorflow:global_step/sec: 286.211 INFO:tensorflow:global_step/sec: 286.211 INFO:tensorflow:loss = 0.6921471, step = 400 (0.350 sec) INFO:tensorflow:loss = 0.6921471, step = 400 (0.350 sec) INFO:tensorflow:global_step/sec: 282.252 INFO:tensorflow:global_step/sec: 282.252 INFO:tensorflow:loss = 0.69014025, step = 500 (0.354 sec) INFO:tensorflow:loss = 0.69014025, step = 500 (0.354 sec) INFO:tensorflow:global_step/sec: 288.814 INFO:tensorflow:global_step/sec: 288.814 INFO:tensorflow:loss = 0.6904064, step = 600 (0.346 sec) INFO:tensorflow:loss = 0.6904064, step = 600 (0.346 sec) INFO:tensorflow:global_step/sec: 275.969 INFO:tensorflow:global_step/sec: 275.969 INFO:tensorflow:loss = 0.6891232, step = 700 (0.363 sec) INFO:tensorflow:loss = 0.6891232, step = 700 (0.363 sec) INFO:tensorflow:global_step/sec: 280.819 INFO:tensorflow:global_step/sec: 280.819 INFO:tensorflow:loss = 0.69049495, step = 800 (0.356 sec) INFO:tensorflow:loss = 0.69049495, step = 800 (0.356 sec) INFO:tensorflow:global_step/sec: 278.558 INFO:tensorflow:global_step/sec: 278.558 INFO:tensorflow:loss = 0.68652004, step = 900 (0.359 sec) INFO:tensorflow:loss = 0.68652004, step = 900 (0.359 sec) INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 999... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 999... INFO:tensorflow:Saving checkpoints for 999 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Saving checkpoints for 999 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/saver.py:971: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version. Instructions for updating: Use standard file APIs to delete files with this prefix. WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/saver.py:971: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version. Instructions for updating: Use standard file APIs to delete files with this prefix. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 999... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 999... INFO:tensorflow:Calling model_fn. INFO:tensorflow:Calling model_fn. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Starting evaluation at 2021-12-04T11:44:45 INFO:tensorflow:Starting evaluation at 2021-12-04T11:44:45 INFO:tensorflow:Graph was finalized. INFO:tensorflow:Graph was finalized. INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt-999 INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt-999 INFO:tensorflow:Running local_init_op. INFO:tensorflow:Running local_init_op. INFO:tensorflow:Done running local_init_op. INFO:tensorflow:Done running local_init_op. INFO:tensorflow:Evaluation [500/5000] INFO:tensorflow:Evaluation [500/5000] INFO:tensorflow:Evaluation [1000/5000] INFO:tensorflow:Evaluation [1000/5000] INFO:tensorflow:Evaluation [1500/5000] INFO:tensorflow:Evaluation [1500/5000] INFO:tensorflow:Evaluation [2000/5000] INFO:tensorflow:Evaluation [2000/5000] INFO:tensorflow:Evaluation [2500/5000] INFO:tensorflow:Evaluation [2500/5000] INFO:tensorflow:Evaluation [3000/5000] INFO:tensorflow:Evaluation [3000/5000] INFO:tensorflow:Evaluation [3500/5000] INFO:tensorflow:Evaluation [3500/5000] INFO:tensorflow:Evaluation [4000/5000] INFO:tensorflow:Evaluation [4000/5000] INFO:tensorflow:Evaluation [4500/5000] INFO:tensorflow:Evaluation [4500/5000] INFO:tensorflow:Evaluation [5000/5000] INFO:tensorflow:Evaluation [5000/5000] INFO:tensorflow:Inference Time : 5.56428s INFO:tensorflow:Inference Time : 5.56428s INFO:tensorflow:Finished evaluation at 2021-12-04-11:44:51 INFO:tensorflow:Finished evaluation at 2021-12-04-11:44:51 INFO:tensorflow:Saving dict for global step 999: accuracy = 0.7047, global_step = 999, loss = 0.68605316 INFO:tensorflow:Saving dict for global step 999: accuracy = 0.7047, global_step = 999, loss = 0.68605316 INFO:tensorflow:Saving 'checkpoint_path' summary for global step 999: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt-999 INFO:tensorflow:Saving 'checkpoint_path' summary for global step 999: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt-999 INFO:tensorflow:global_step/sec: 16.1827 INFO:tensorflow:global_step/sec: 16.1827 INFO:tensorflow:loss = 0.68512, step = 1000 (6.179 sec) INFO:tensorflow:loss = 0.68512, step = 1000 (6.179 sec) INFO:tensorflow:global_step/sec: 278.496 INFO:tensorflow:global_step/sec: 278.496 INFO:tensorflow:loss = 0.6872438, step = 1100 (0.360 sec) INFO:tensorflow:loss = 0.6872438, step = 1100 (0.360 sec) INFO:tensorflow:global_step/sec: 276.552 INFO:tensorflow:global_step/sec: 276.552 INFO:tensorflow:loss = 0.6817854, step = 1200 (0.361 sec) INFO:tensorflow:loss = 0.6817854, step = 1200 (0.361 sec) INFO:tensorflow:global_step/sec: 271.064 INFO:tensorflow:global_step/sec: 271.064 INFO:tensorflow:loss = 0.6696973, step = 1300 (0.369 sec) INFO:tensorflow:loss = 0.6696973, step = 1300 (0.369 sec) INFO:tensorflow:global_step/sec: 275.856 INFO:tensorflow:global_step/sec: 275.856 INFO:tensorflow:loss = 0.6826827, step = 1400 (0.362 sec) INFO:tensorflow:loss = 0.6826827, step = 1400 (0.362 sec) INFO:tensorflow:global_step/sec: 270.879 INFO:tensorflow:global_step/sec: 270.879 INFO:tensorflow:loss = 0.6712682, step = 1500 (0.369 sec) INFO:tensorflow:loss = 0.6712682, step = 1500 (0.369 sec) INFO:tensorflow:global_step/sec: 277.073 INFO:tensorflow:global_step/sec: 277.073 INFO:tensorflow:loss = 0.67981917, step = 1600 (0.361 sec) INFO:tensorflow:loss = 0.67981917, step = 1600 (0.361 sec) INFO:tensorflow:global_step/sec: 270.234 INFO:tensorflow:global_step/sec: 270.234 INFO:tensorflow:loss = 0.67373323, step = 1700 (0.370 sec) INFO:tensorflow:loss = 0.67373323, step = 1700 (0.370 sec) INFO:tensorflow:global_step/sec: 279.658 INFO:tensorflow:global_step/sec: 279.658 INFO:tensorflow:loss = 0.66337496, step = 1800 (0.358 sec) INFO:tensorflow:loss = 0.66337496, step = 1800 (0.358 sec) INFO:tensorflow:global_step/sec: 279.271 INFO:tensorflow:global_step/sec: 279.271 INFO:tensorflow:loss = 0.6738259, step = 1900 (0.358 sec) INFO:tensorflow:loss = 0.6738259, step = 1900 (0.358 sec) INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1998... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1998... INFO:tensorflow:Saving checkpoints for 1998 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Saving checkpoints for 1998 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1998... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1998... INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:global_step/sec: 237.544 INFO:tensorflow:global_step/sec: 237.544 INFO:tensorflow:loss = 0.66583055, step = 2000 (0.421 sec) INFO:tensorflow:loss = 0.66583055, step = 2000 (0.421 sec) INFO:tensorflow:global_step/sec: 277.133 INFO:tensorflow:global_step/sec: 277.133 INFO:tensorflow:loss = 0.6637004, step = 2100 (0.361 sec) INFO:tensorflow:loss = 0.6637004, step = 2100 (0.361 sec) INFO:tensorflow:global_step/sec: 272.248 INFO:tensorflow:global_step/sec: 272.248 INFO:tensorflow:loss = 0.6696273, step = 2200 (0.367 sec) INFO:tensorflow:loss = 0.6696273, step = 2200 (0.367 sec) INFO:tensorflow:global_step/sec: 277.247 INFO:tensorflow:global_step/sec: 277.247 INFO:tensorflow:loss = 0.6513475, step = 2300 (0.361 sec) INFO:tensorflow:loss = 0.6513475, step = 2300 (0.361 sec) INFO:tensorflow:global_step/sec: 276.598 INFO:tensorflow:global_step/sec: 276.598 INFO:tensorflow:loss = 0.6662655, step = 2400 (0.362 sec) INFO:tensorflow:loss = 0.6662655, step = 2400 (0.362 sec) INFO:tensorflow:global_step/sec: 272.004 INFO:tensorflow:global_step/sec: 272.004 INFO:tensorflow:loss = 0.6493275, step = 2500 (0.368 sec) INFO:tensorflow:loss = 0.6493275, step = 2500 (0.368 sec) INFO:tensorflow:global_step/sec: 279.613 INFO:tensorflow:global_step/sec: 279.613 INFO:tensorflow:loss = 0.64058864, step = 2600 (0.358 sec) INFO:tensorflow:loss = 0.64058864, step = 2600 (0.358 sec) INFO:tensorflow:global_step/sec: 279.725 INFO:tensorflow:global_step/sec: 279.725 INFO:tensorflow:loss = 0.6401115, step = 2700 (0.357 sec) INFO:tensorflow:loss = 0.6401115, step = 2700 (0.357 sec) INFO:tensorflow:global_step/sec: 275.868 INFO:tensorflow:global_step/sec: 275.868 INFO:tensorflow:loss = 0.66073626, step = 2800 (0.363 sec) INFO:tensorflow:loss = 0.66073626, step = 2800 (0.363 sec) INFO:tensorflow:global_step/sec: 279.9 INFO:tensorflow:global_step/sec: 279.9 INFO:tensorflow:loss = 0.61275744, step = 2900 (0.357 sec) INFO:tensorflow:loss = 0.61275744, step = 2900 (0.357 sec) INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 2997... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 2997... INFO:tensorflow:Saving checkpoints for 2997 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Saving checkpoints for 2997 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 2997... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 2997... INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:global_step/sec: 239.223 INFO:tensorflow:global_step/sec: 239.223 INFO:tensorflow:loss = 0.6508343, step = 3000 (0.418 sec) INFO:tensorflow:loss = 0.6508343, step = 3000 (0.418 sec) INFO:tensorflow:global_step/sec: 278.547 INFO:tensorflow:global_step/sec: 278.547 INFO:tensorflow:loss = 0.65112776, step = 3100 (0.359 sec) INFO:tensorflow:loss = 0.65112776, step = 3100 (0.359 sec) INFO:tensorflow:global_step/sec: 279.487 INFO:tensorflow:global_step/sec: 279.487 INFO:tensorflow:loss = 0.63657844, step = 3200 (0.358 sec) INFO:tensorflow:loss = 0.63657844, step = 3200 (0.358 sec) INFO:tensorflow:global_step/sec: 277.617 INFO:tensorflow:global_step/sec: 277.617 INFO:tensorflow:loss = 0.6216135, step = 3300 (0.360 sec) INFO:tensorflow:loss = 0.6216135, step = 3300 (0.360 sec) INFO:tensorflow:global_step/sec: 279.256 INFO:tensorflow:global_step/sec: 279.256 INFO:tensorflow:loss = 0.64972967, step = 3400 (0.358 sec) INFO:tensorflow:loss = 0.64972967, step = 3400 (0.358 sec) INFO:tensorflow:global_step/sec: 281.028 INFO:tensorflow:global_step/sec: 281.028 INFO:tensorflow:loss = 0.6309604, step = 3500 (0.356 sec) INFO:tensorflow:loss = 0.6309604, step = 3500 (0.356 sec) INFO:tensorflow:global_step/sec: 282.144 INFO:tensorflow:global_step/sec: 282.144 INFO:tensorflow:loss = 0.59252113, step = 3600 (0.355 sec) INFO:tensorflow:loss = 0.59252113, step = 3600 (0.355 sec) INFO:tensorflow:global_step/sec: 275.802 INFO:tensorflow:global_step/sec: 275.802 INFO:tensorflow:loss = 0.5944205, step = 3700 (0.363 sec) INFO:tensorflow:loss = 0.5944205, step = 3700 (0.363 sec) INFO:tensorflow:global_step/sec: 273.658 INFO:tensorflow:global_step/sec: 273.658 INFO:tensorflow:loss = 0.63925326, step = 3800 (0.365 sec) INFO:tensorflow:loss = 0.63925326, step = 3800 (0.365 sec) INFO:tensorflow:global_step/sec: 274.902 INFO:tensorflow:global_step/sec: 274.902 INFO:tensorflow:loss = 0.6255677, step = 3900 (0.365 sec) INFO:tensorflow:loss = 0.6255677, step = 3900 (0.365 sec) INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 3996... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 3996... INFO:tensorflow:Saving checkpoints for 3996 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Saving checkpoints for 3996 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 3996... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 3996... INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:global_step/sec: 235.472 INFO:tensorflow:global_step/sec: 235.472 INFO:tensorflow:loss = 0.5732498, step = 4000 (0.424 sec) INFO:tensorflow:loss = 0.5732498, step = 4000 (0.424 sec) INFO:tensorflow:global_step/sec: 277.885 INFO:tensorflow:global_step/sec: 277.885 INFO:tensorflow:loss = 0.59263897, step = 4100 (0.360 sec) INFO:tensorflow:loss = 0.59263897, step = 4100 (0.360 sec) INFO:tensorflow:global_step/sec: 272.498 INFO:tensorflow:global_step/sec: 272.498 INFO:tensorflow:loss = 0.6244205, step = 4200 (0.367 sec) INFO:tensorflow:loss = 0.6244205, step = 4200 (0.367 sec) INFO:tensorflow:global_step/sec: 273.911 INFO:tensorflow:global_step/sec: 273.911 INFO:tensorflow:loss = 0.5709779, step = 4300 (0.365 sec) INFO:tensorflow:loss = 0.5709779, step = 4300 (0.365 sec) INFO:tensorflow:global_step/sec: 272.385 INFO:tensorflow:global_step/sec: 272.385 INFO:tensorflow:loss = 0.57497543, step = 4400 (0.367 sec) INFO:tensorflow:loss = 0.57497543, step = 4400 (0.367 sec) INFO:tensorflow:global_step/sec: 277.073 INFO:tensorflow:global_step/sec: 277.073 INFO:tensorflow:loss = 0.62753403, step = 4500 (0.361 sec) INFO:tensorflow:loss = 0.62753403, step = 4500 (0.361 sec) INFO:tensorflow:global_step/sec: 279.972 INFO:tensorflow:global_step/sec: 279.972 INFO:tensorflow:loss = 0.5253285, step = 4600 (0.357 sec) INFO:tensorflow:loss = 0.5253285, step = 4600 (0.357 sec) INFO:tensorflow:global_step/sec: 283.916 INFO:tensorflow:global_step/sec: 283.916 INFO:tensorflow:loss = 0.5570012, step = 4700 (0.353 sec) INFO:tensorflow:loss = 0.5570012, step = 4700 (0.353 sec) INFO:tensorflow:global_step/sec: 286.699 INFO:tensorflow:global_step/sec: 286.699 INFO:tensorflow:loss = 0.54549825, step = 4800 (0.348 sec) INFO:tensorflow:loss = 0.54549825, step = 4800 (0.348 sec) INFO:tensorflow:global_step/sec: 287.171 INFO:tensorflow:global_step/sec: 287.171 INFO:tensorflow:loss = 0.58005756, step = 4900 (0.348 sec) INFO:tensorflow:loss = 0.58005756, step = 4900 (0.348 sec) INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 4995... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 4995... INFO:tensorflow:Saving checkpoints for 4995 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Saving checkpoints for 4995 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 4995... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 4995... INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:global_step/sec: 246.035 INFO:tensorflow:global_step/sec: 246.035 INFO:tensorflow:loss = 0.55126476, step = 5000 (0.406 sec) INFO:tensorflow:loss = 0.55126476, step = 5000 (0.406 sec) INFO:tensorflow:global_step/sec: 286.048 INFO:tensorflow:global_step/sec: 286.048 INFO:tensorflow:loss = 0.5440348, step = 5100 (0.350 sec) INFO:tensorflow:loss = 0.5440348, step = 5100 (0.350 sec) INFO:tensorflow:global_step/sec: 288.158 INFO:tensorflow:global_step/sec: 288.158 INFO:tensorflow:loss = 0.530152, step = 5200 (0.347 sec) INFO:tensorflow:loss = 0.530152, step = 5200 (0.347 sec) INFO:tensorflow:global_step/sec: 282.667 INFO:tensorflow:global_step/sec: 282.667 INFO:tensorflow:loss = 0.61745214, step = 5300 (0.354 sec) INFO:tensorflow:loss = 0.61745214, step = 5300 (0.354 sec) INFO:tensorflow:global_step/sec: 283.025 INFO:tensorflow:global_step/sec: 283.025 INFO:tensorflow:loss = 0.5531441, step = 5400 (0.354 sec) INFO:tensorflow:loss = 0.5531441, step = 5400 (0.354 sec) INFO:tensorflow:global_step/sec: 284.596 INFO:tensorflow:global_step/sec: 284.596 INFO:tensorflow:loss = 0.55586976, step = 5500 (0.351 sec) INFO:tensorflow:loss = 0.55586976, step = 5500 (0.351 sec) INFO:tensorflow:global_step/sec: 283.212 INFO:tensorflow:global_step/sec: 283.212 INFO:tensorflow:loss = 0.5627943, step = 5600 (0.353 sec) INFO:tensorflow:loss = 0.5627943, step = 5600 (0.353 sec) INFO:tensorflow:global_step/sec: 281.121 INFO:tensorflow:global_step/sec: 281.121 INFO:tensorflow:loss = 0.45171082, step = 5700 (0.356 sec) INFO:tensorflow:loss = 0.45171082, step = 5700 (0.356 sec) INFO:tensorflow:global_step/sec: 281.568 INFO:tensorflow:global_step/sec: 281.568 INFO:tensorflow:loss = 0.51796657, step = 5800 (0.355 sec) INFO:tensorflow:loss = 0.51796657, step = 5800 (0.355 sec) INFO:tensorflow:global_step/sec: 272.14 INFO:tensorflow:global_step/sec: 272.14 INFO:tensorflow:loss = 0.570162, step = 5900 (0.368 sec) INFO:tensorflow:loss = 0.570162, step = 5900 (0.368 sec) INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 5994... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 5994... INFO:tensorflow:Saving checkpoints for 5994 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Saving checkpoints for 5994 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 5994... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 5994... INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:global_step/sec: 234.957 INFO:tensorflow:global_step/sec: 234.957 INFO:tensorflow:loss = 0.5400977, step = 6000 (0.425 sec) INFO:tensorflow:loss = 0.5400977, step = 6000 (0.425 sec) INFO:tensorflow:global_step/sec: 259.23 INFO:tensorflow:global_step/sec: 259.23 INFO:tensorflow:loss = 0.4981569, step = 6100 (0.386 sec) INFO:tensorflow:loss = 0.4981569, step = 6100 (0.386 sec) INFO:tensorflow:global_step/sec: 267.697 INFO:tensorflow:global_step/sec: 267.697 INFO:tensorflow:loss = 0.5613683, step = 6200 (0.373 sec) INFO:tensorflow:loss = 0.5613683, step = 6200 (0.373 sec) INFO:tensorflow:global_step/sec: 266.623 INFO:tensorflow:global_step/sec: 266.623 INFO:tensorflow:loss = 0.48216385, step = 6300 (0.375 sec) INFO:tensorflow:loss = 0.48216385, step = 6300 (0.375 sec) INFO:tensorflow:global_step/sec: 266.123 INFO:tensorflow:global_step/sec: 266.123 INFO:tensorflow:loss = 0.4599746, step = 6400 (0.376 sec) INFO:tensorflow:loss = 0.4599746, step = 6400 (0.376 sec) INFO:tensorflow:global_step/sec: 269.688 INFO:tensorflow:global_step/sec: 269.688 INFO:tensorflow:loss = 0.4796008, step = 6500 (0.371 sec) INFO:tensorflow:loss = 0.4796008, step = 6500 (0.371 sec) INFO:tensorflow:global_step/sec: 258.906 INFO:tensorflow:global_step/sec: 258.906 INFO:tensorflow:loss = 0.5626136, step = 6600 (0.386 sec) INFO:tensorflow:loss = 0.5626136, step = 6600 (0.386 sec) INFO:tensorflow:global_step/sec: 261.596 INFO:tensorflow:global_step/sec: 261.596 INFO:tensorflow:loss = 0.5001174, step = 6700 (0.382 sec) INFO:tensorflow:loss = 0.5001174, step = 6700 (0.382 sec) INFO:tensorflow:global_step/sec: 266.467 INFO:tensorflow:global_step/sec: 266.467 INFO:tensorflow:loss = 0.44604325, step = 6800 (0.376 sec) INFO:tensorflow:loss = 0.44604325, step = 6800 (0.376 sec) INFO:tensorflow:global_step/sec: 267.785 INFO:tensorflow:global_step/sec: 267.785 INFO:tensorflow:loss = 0.4936733, step = 6900 (0.373 sec) INFO:tensorflow:loss = 0.4936733, step = 6900 (0.373 sec) INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 6993... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 6993... INFO:tensorflow:Saving checkpoints for 6993 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Saving checkpoints for 6993 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 6993... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 6993... INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:global_step/sec: 231.159 INFO:tensorflow:global_step/sec: 231.159 INFO:tensorflow:loss = 0.44407076, step = 7000 (0.433 sec) INFO:tensorflow:loss = 0.44407076, step = 7000 (0.433 sec) INFO:tensorflow:global_step/sec: 259.935 INFO:tensorflow:global_step/sec: 259.935 INFO:tensorflow:loss = 0.4649738, step = 7100 (0.385 sec) INFO:tensorflow:loss = 0.4649738, step = 7100 (0.385 sec) INFO:tensorflow:global_step/sec: 261.497 INFO:tensorflow:global_step/sec: 261.497 INFO:tensorflow:loss = 0.48575532, step = 7200 (0.382 sec) INFO:tensorflow:loss = 0.48575532, step = 7200 (0.382 sec) INFO:tensorflow:global_step/sec: 264.401 INFO:tensorflow:global_step/sec: 264.401 INFO:tensorflow:loss = 0.5566124, step = 7300 (0.378 sec) INFO:tensorflow:loss = 0.5566124, step = 7300 (0.378 sec) INFO:tensorflow:global_step/sec: 263.189 INFO:tensorflow:global_step/sec: 263.189 INFO:tensorflow:loss = 0.485472, step = 7400 (0.380 sec) INFO:tensorflow:loss = 0.485472, step = 7400 (0.380 sec) INFO:tensorflow:global_step/sec: 262.158 INFO:tensorflow:global_step/sec: 262.158 INFO:tensorflow:loss = 0.39120063, step = 7500 (0.381 sec) INFO:tensorflow:loss = 0.39120063, step = 7500 (0.381 sec) INFO:tensorflow:global_step/sec: 266.983 INFO:tensorflow:global_step/sec: 266.983 INFO:tensorflow:loss = 0.35777277, step = 7600 (0.374 sec) INFO:tensorflow:loss = 0.35777277, step = 7600 (0.374 sec) INFO:tensorflow:global_step/sec: 267.642 INFO:tensorflow:global_step/sec: 267.642 INFO:tensorflow:loss = 0.5350034, step = 7700 (0.374 sec) INFO:tensorflow:loss = 0.5350034, step = 7700 (0.374 sec) INFO:tensorflow:global_step/sec: 269.459 INFO:tensorflow:global_step/sec: 269.459 INFO:tensorflow:loss = 0.42015103, step = 7800 (0.371 sec) INFO:tensorflow:loss = 0.42015103, step = 7800 (0.371 sec) INFO:tensorflow:global_step/sec: 267.026 INFO:tensorflow:global_step/sec: 267.026 INFO:tensorflow:loss = 0.54285204, step = 7900 (0.375 sec) INFO:tensorflow:loss = 0.54285204, step = 7900 (0.375 sec) INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 7992... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 7992... INFO:tensorflow:Saving checkpoints for 7992 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Saving checkpoints for 7992 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 7992... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 7992... INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:global_step/sec: 226.498 INFO:tensorflow:global_step/sec: 226.498 INFO:tensorflow:loss = 0.36296645, step = 8000 (0.441 sec) INFO:tensorflow:loss = 0.36296645, step = 8000 (0.441 sec) INFO:tensorflow:global_step/sec: 262.545 INFO:tensorflow:global_step/sec: 262.545 INFO:tensorflow:loss = 0.5328135, step = 8100 (0.381 sec) INFO:tensorflow:loss = 0.5328135, step = 8100 (0.381 sec) INFO:tensorflow:global_step/sec: 264.823 INFO:tensorflow:global_step/sec: 264.823 INFO:tensorflow:loss = 0.42400876, step = 8200 (0.377 sec) INFO:tensorflow:loss = 0.42400876, step = 8200 (0.377 sec) INFO:tensorflow:global_step/sec: 270.946 INFO:tensorflow:global_step/sec: 270.946 INFO:tensorflow:loss = 0.4334933, step = 8300 (0.369 sec) INFO:tensorflow:loss = 0.4334933, step = 8300 (0.369 sec) INFO:tensorflow:global_step/sec: 271.252 INFO:tensorflow:global_step/sec: 271.252 INFO:tensorflow:loss = 0.44592458, step = 8400 (0.369 sec) INFO:tensorflow:loss = 0.44592458, step = 8400 (0.369 sec) INFO:tensorflow:global_step/sec: 272.492 INFO:tensorflow:global_step/sec: 272.492 INFO:tensorflow:loss = 0.44213057, step = 8500 (0.367 sec) INFO:tensorflow:loss = 0.44213057, step = 8500 (0.367 sec) INFO:tensorflow:global_step/sec: 273.226 INFO:tensorflow:global_step/sec: 273.226 INFO:tensorflow:loss = 0.46779203, step = 8600 (0.366 sec) INFO:tensorflow:loss = 0.46779203, step = 8600 (0.366 sec) INFO:tensorflow:global_step/sec: 261.518 INFO:tensorflow:global_step/sec: 261.518 INFO:tensorflow:loss = 0.5460389, step = 8700 (0.382 sec) INFO:tensorflow:loss = 0.5460389, step = 8700 (0.382 sec) INFO:tensorflow:global_step/sec: 277.202 INFO:tensorflow:global_step/sec: 277.202 INFO:tensorflow:loss = 0.5019726, step = 8800 (0.361 sec) INFO:tensorflow:loss = 0.5019726, step = 8800 (0.361 sec) INFO:tensorflow:global_step/sec: 276.724 INFO:tensorflow:global_step/sec: 276.724 INFO:tensorflow:loss = 0.45209432, step = 8900 (0.361 sec) INFO:tensorflow:loss = 0.45209432, step = 8900 (0.361 sec) INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 8991... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 8991... INFO:tensorflow:Saving checkpoints for 8991 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Saving checkpoints for 8991 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 8991... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 8991... INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:global_step/sec: 226.305 INFO:tensorflow:global_step/sec: 226.305 INFO:tensorflow:loss = 0.34912163, step = 9000 (0.442 sec) INFO:tensorflow:loss = 0.34912163, step = 9000 (0.442 sec) INFO:tensorflow:global_step/sec: 271.186 INFO:tensorflow:global_step/sec: 271.186 INFO:tensorflow:loss = 0.5445255, step = 9100 (0.369 sec) INFO:tensorflow:loss = 0.5445255, step = 9100 (0.369 sec) INFO:tensorflow:global_step/sec: 267.761 INFO:tensorflow:global_step/sec: 267.761 INFO:tensorflow:loss = 0.35654712, step = 9200 (0.373 sec) INFO:tensorflow:loss = 0.35654712, step = 9200 (0.373 sec) INFO:tensorflow:global_step/sec: 262.439 INFO:tensorflow:global_step/sec: 262.439 INFO:tensorflow:loss = 0.42294815, step = 9300 (0.381 sec) INFO:tensorflow:loss = 0.42294815, step = 9300 (0.381 sec) INFO:tensorflow:global_step/sec: 262.881 INFO:tensorflow:global_step/sec: 262.881 INFO:tensorflow:loss = 0.45307142, step = 9400 (0.380 sec) INFO:tensorflow:loss = 0.45307142, step = 9400 (0.380 sec) INFO:tensorflow:global_step/sec: 264.643 INFO:tensorflow:global_step/sec: 264.643 INFO:tensorflow:loss = 0.43050554, step = 9500 (0.378 sec) INFO:tensorflow:loss = 0.43050554, step = 9500 (0.378 sec) INFO:tensorflow:global_step/sec: 270.757 INFO:tensorflow:global_step/sec: 270.757 INFO:tensorflow:loss = 0.40443382, step = 9600 (0.369 sec) INFO:tensorflow:loss = 0.40443382, step = 9600 (0.369 sec) INFO:tensorflow:global_step/sec: 268.755 INFO:tensorflow:global_step/sec: 268.755 INFO:tensorflow:loss = 0.37255523, step = 9700 (0.372 sec) INFO:tensorflow:loss = 0.37255523, step = 9700 (0.372 sec) INFO:tensorflow:global_step/sec: 264.603 INFO:tensorflow:global_step/sec: 264.603 INFO:tensorflow:loss = 0.4721123, step = 9800 (0.378 sec) INFO:tensorflow:loss = 0.4721123, step = 9800 (0.378 sec) INFO:tensorflow:global_step/sec: 273.682 INFO:tensorflow:global_step/sec: 273.682 INFO:tensorflow:loss = 0.52799636, step = 9900 (0.365 sec) INFO:tensorflow:loss = 0.52799636, step = 9900 (0.365 sec) INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 9990... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 9990... INFO:tensorflow:Saving checkpoints for 9990 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Saving checkpoints for 9990 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 9990... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 9990... INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 10000... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 10000... INFO:tensorflow:Saving checkpoints for 10000 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Saving checkpoints for 10000 into /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 10000... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 10000... INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs). INFO:tensorflow:Calling model_fn. INFO:tensorflow:Calling model_fn. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Starting evaluation at 2021-12-04T11:45:25 INFO:tensorflow:Starting evaluation at 2021-12-04T11:45:25 INFO:tensorflow:Graph was finalized. INFO:tensorflow:Graph was finalized. INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt-10000 INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt-10000 INFO:tensorflow:Running local_init_op. INFO:tensorflow:Running local_init_op. INFO:tensorflow:Done running local_init_op. INFO:tensorflow:Done running local_init_op. INFO:tensorflow:Evaluation [500/5000] INFO:tensorflow:Evaluation [500/5000] INFO:tensorflow:Evaluation [1000/5000] INFO:tensorflow:Evaluation [1000/5000] INFO:tensorflow:Evaluation [1500/5000] INFO:tensorflow:Evaluation [1500/5000] INFO:tensorflow:Evaluation [2000/5000] INFO:tensorflow:Evaluation [2000/5000] INFO:tensorflow:Evaluation [2500/5000] INFO:tensorflow:Evaluation [2500/5000] INFO:tensorflow:Evaluation [3000/5000] INFO:tensorflow:Evaluation [3000/5000] INFO:tensorflow:Evaluation [3500/5000] INFO:tensorflow:Evaluation [3500/5000] INFO:tensorflow:Evaluation [4000/5000] INFO:tensorflow:Evaluation [4000/5000] INFO:tensorflow:Evaluation [4500/5000] INFO:tensorflow:Evaluation [4500/5000] INFO:tensorflow:Evaluation [5000/5000] INFO:tensorflow:Evaluation [5000/5000] INFO:tensorflow:Inference Time : 5.60779s INFO:tensorflow:Inference Time : 5.60779s INFO:tensorflow:Finished evaluation at 2021-12-04-11:45:30 INFO:tensorflow:Finished evaluation at 2021-12-04-11:45:30 INFO:tensorflow:Saving dict for global step 10000: accuracy = 0.8008, global_step = 10000, loss = 0.4497029 INFO:tensorflow:Saving dict for global step 10000: accuracy = 0.8008, global_step = 10000, loss = 0.4497029 INFO:tensorflow:Saving 'checkpoint_path' summary for global step 10000: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt-10000 INFO:tensorflow:Saving 'checkpoint_path' summary for global step 10000: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt-10000 INFO:tensorflow:Performing the final export in the end of training. INFO:tensorflow:Performing the final export in the end of training. WARNING:tensorflow:Loading a TF2 SavedModel but eager mode seems disabled. WARNING:tensorflow:Loading a TF2 SavedModel but eager mode seems disabled. INFO:tensorflow:Calling model_fn. INFO:tensorflow:Calling model_fn. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Done calling model_fn. WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/saved_model/signature_def_utils_impl.py:201: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version. Instructions for updating: This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info. WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/saved_model/signature_def_utils_impl.py:201: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version. Instructions for updating: This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info. INFO:tensorflow:Signatures INCLUDED in export for Classify: None INFO:tensorflow:Signatures INCLUDED in export for Classify: None INFO:tensorflow:Signatures INCLUDED in export for Regress: None INFO:tensorflow:Signatures INCLUDED in export for Regress: None INFO:tensorflow:Signatures INCLUDED in export for Predict: ['serving_default'] INFO:tensorflow:Signatures INCLUDED in export for Predict: ['serving_default'] INFO:tensorflow:Signatures INCLUDED in export for Train: None INFO:tensorflow:Signatures INCLUDED in export for Train: None INFO:tensorflow:Signatures INCLUDED in export for Eval: None INFO:tensorflow:Signatures INCLUDED in export for Eval: None INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt-10000 INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt-10000 INFO:tensorflow:Assets added to graph. INFO:tensorflow:Assets added to graph. INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/export/imdb/temp-1638618330/assets INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/export/imdb/temp-1638618330/assets INFO:tensorflow:SavedModel written to: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/export/imdb/temp-1638618330/saved_model.pb INFO:tensorflow:SavedModel written to: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/export/imdb/temp-1638618330/saved_model.pb INFO:tensorflow:Loss for final step: 0.43356365. INFO:tensorflow:Loss for final step: 0.43356365. WARNING:tensorflow:Loading a TF2 SavedModel but eager mode seems disabled. WARNING:tensorflow:Loading a TF2 SavedModel but eager mode seems disabled. INFO:tensorflow:Calling model_fn. INFO:tensorflow:Calling model_fn. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Signatures INCLUDED in export for Classify: None INFO:tensorflow:Signatures INCLUDED in export for Classify: None INFO:tensorflow:Signatures INCLUDED in export for Regress: None INFO:tensorflow:Signatures INCLUDED in export for Regress: None INFO:tensorflow:Signatures INCLUDED in export for Predict: None INFO:tensorflow:Signatures INCLUDED in export for Predict: None INFO:tensorflow:Signatures INCLUDED in export for Train: None INFO:tensorflow:Signatures INCLUDED in export for Train: None INFO:tensorflow:Signatures INCLUDED in export for Eval: ['eval'] INFO:tensorflow:Signatures INCLUDED in export for Eval: ['eval'] WARNING:tensorflow:Export includes no default signature! WARNING:tensorflow:Export includes no default signature! INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt-10000 INFO:tensorflow:Restoring parameters from /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-Serving/model.ckpt-10000 INFO:tensorflow:Assets added to graph. INFO:tensorflow:Assets added to graph. INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-TFMA/temp-1638618332/assets INFO:tensorflow:Assets written to: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-TFMA/temp-1638618332/assets INFO:tensorflow:SavedModel written to: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-TFMA/temp-1638618332/saved_model.pb INFO:tensorflow:SavedModel written to: /tmp/tfx-interactive-2021-12-04T11_41_51.482724-py59cet9/Trainer/model_run/9/Format-TFMA/temp-1638618332/saved_model.pb WARNING:absl:Support for estimator-based executor and model export will be deprecated soon. Please use export structure <ModelExportPath>/serving_model_dir/saved_model.pb" WARNING:absl:Support for estimator-based executor and model export will be deprecated soon. Please use export structure <ModelExportPath>/eval_model_dir/saved_model.pb"
에서 내 보낸 된 훈련 모델에서 슬쩍 가져 Trainer
.
train_uri = trainer.outputs['model'].get()[0].uri
serving_model_path = os.path.join(train_uri, 'Format-Serving')
exported_model = tf.saved_model.load(serving_model_path)
exported_model.graph.get_operations()[:10] + ["..."]
[<tf.Operation 'global_step/Initializer/zeros' type=Const>, <tf.Operation 'global_step' type=VarHandleOp>, <tf.Operation 'global_step/IsInitialized/VarIsInitializedOp' type=VarIsInitializedOp>, <tf.Operation 'global_step/Assign' type=AssignVariableOp>, <tf.Operation 'global_step/Read/ReadVariableOp' type=ReadVariableOp>, <tf.Operation 'input_example_tensor' type=Placeholder>, <tf.Operation 'ParseExample/ParseExampleV2/names' type=Const>, <tf.Operation 'ParseExample/ParseExampleV2/sparse_keys' type=Const>, <tf.Operation 'ParseExample/ParseExampleV2/dense_keys' type=Const>, <tf.Operation 'ParseExample/ParseExampleV2/ragged_keys' type=Const>, '...']
Tensorboard를 사용하여 모델의 메트릭을 시각화해 보겠습니다.
#docs_infra: no_execute
# Get the URI of the output artifact representing the training logs,
# which is a directory
model_run_dir = trainer.outputs['model_run'].get()[0].uri
%load_ext tensorboard
%tensorboard --logdir {model_run_dir}
모델 서빙
그래프 정규화는 손실 함수에 정규화 항을 추가하여 훈련 워크플로에만 영향을 줍니다. 결과적으로 모델 평가 및 제공 워크플로는 변경되지 않은 상태로 유지됩니다. 우리가 일반적 등 평가자, 푸셔, 같은 트레이너 구성 요소 쫓을 다운 스트림 TFX 구성 요소를 생략 한 것과 같은 이유입니다
결론
입력에 명시적 그래프가 포함되어 있지 않은 경우에도 TFX 파이프라인에서 NSL(신경 구조적 학습) 프레임워크를 사용하여 그래프 정규화를 사용하는 방법을 시연했습니다. 리뷰 임베딩을 기반으로 유사도 그래프를 합성한 IMDB 영화 리뷰의 감정 분류 작업을 고려했습니다. 그래프 구성에 다양한 임베딩을 사용하고, 하이퍼파라미터를 변경하고, 감독의 양을 변경하고, 다양한 모델 아키텍처를 정의하여 사용자가 더 많은 실험을 하도록 권장합니다.