SavedModel 포맷 사용하기

TensorFlow.org에서 보기

구글 코랩(Colab)에서 실행하기

깃허브(GitHub) 소스 보기

Download notebook

SavedModel에는 가중치 및 연산을 포함한 완전한 텐서플로 프로그램이 포함됩니다. 기존에 설계했던 모델 코드를 실행할 필요가 없어 공유하거나 (TFLite, TensorFlow.js, TensorFlow Serving, TFHub와 같은 환경으로) 배포하는 데 유용합니다.

파이썬 모델 코드를 가지고 있고 파이썬 내에서 가중치를 불러오고 싶다면, 체크포인트 훈련 가이드를 참조하세요.

빠른 소개를 위해 이 섹션에서는 미리 훈련된 케라스 모델을 내보내고 그 모델로 이미지 분류 요청을 처리합니다. 나머지 가이드에서는 세부 정보와 SavedModel을 만드는 다른 방법에 대해 설명합니다.

import tensorflow as tf
from matplotlib import pyplot as plt
import numpy as np

file = tf.keras.utils.get_file(
    "grace_hopper.jpg",
    "https://storage.googleapis.com/download.tensorflow.org/example_images/grace_hopper.jpg")
img = tf.keras.preprocessing.image.load_img(file, target_size=[224, 224])
plt.imshow(img)
plt.axis('off')
x = tf.keras.preprocessing.image.img_to_array(img)
x = tf.keras.applications.mobilenet.preprocess_input(
    x[tf.newaxis,...])

Downloading data from https://storage.googleapis.com/download.tensorflow.org/example_images/grace_hopper.jpg
65536/61306 [================================] - 0s 0us/step

실행 예제로 그레이스 호퍼(Grace Hopper)의 이미지와 사용이 쉬운 케라스 사전 훈련 이미지 분류 모델을 사용할 것입니다. 사용자 정의 모델도 사용할 수 있는데, 자세한 것은 나중에 설명합니다.

#tf.keras.applications.vgg19.decode_predictions
labels_path = tf.keras.utils.get_file('ImageNetLabels.txt','https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt')
imagenet_labels = np.array(open(labels_path).read().splitlines())

Downloading data from https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt
16384/10484 [==============================================] - 0s 0us/step

pretrained_model = tf.keras.applications.MobileNet()
result_before_save = pretrained_model(x)
print()

decoded = imagenet_labels[np.argsort(result_before_save)[0,::-1][:5]+1]

print("저장 전 결과:\n", decoded)

Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/mobilenet/mobilenet_1_0_224_tf.h5
17227776/17225924 [==============================] - 1s 0us/step

저장 전 결과:
 ['military uniform' 'bow tie' 'suit' 'bearskin' 'pickelhaube']

이 이미지의 가장 가능성 있는 예측은 "군복"입니다.

tf.saved_model.save(pretrained_model, "/tmp/mobilenet/1/")

WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/training/tracking/tracking.py:111: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/training/tracking/tracking.py:111: Layer.updates (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
INFO:tensorflow:Assets written to: /tmp/mobilenet/1/assets

저장 경로의 마지막 경로 요소(여기서는 1/)는 모델의 버전 번호인 텐서플로 서빙(TensorFlow Serving) 컨벤션을 따릅니다 - 텐서플로 서빙과 같은 도구가 최신 모델을 구분할 수 있게 합니다.

SavedModel은 시그니처(signatures)라 불리는 이름있는 함수를 가집니다. 케라스 모델은 serving_default 시그니처 키를 사용하여 정방향 패스(forward pass)를 내보냅니다. SavedModel 커맨드 라인 인터페이스는 디스크에 저장된 SavedModel을 검사할 때 유용합니다.

saved_model_cli show --dir /tmp/mobilenet/1 --tag_set serve --signature_def serving_default

2020-09-23 04:08:44.664120: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
The given SavedModel SignatureDef contains the following input(s):
  inputs['input_1'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 224, 224, 3)
      name: serving_default_input_1:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['predictions'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 1000)
      name: StatefulPartitionedCall:0
Method name is: tensorflow/serving/predict

파이썬에서 tf.saved_model.load로 SavedModel을 다시 불러오고 해군대장 호퍼(Admiral Hopper)의 이미지가 어떻게 분류되는지 볼 수 있습니다.

loaded = tf.saved_model.load("/tmp/mobilenet/1/")
print(list(loaded.signatures.keys()))  # ["serving_default"]

['serving_default']

가져온 시그니처는 항상 딕셔너리를 반환합니다.

infer = loaded.signatures["serving_default"]
print(infer.structured_outputs)

{'predictions': TensorSpec(shape=(None, 1000), dtype=tf.float32, name='predictions')}

SavedModel로부터 추론을 실행하면 처음 모델과 같은 결과를 제공합니다.

labeling = infer(tf.constant(x))[pretrained_model.output_names[0]]

decoded = imagenet_labels[np.argsort(labeling)[0,::-1][:5]+1]

print("저장과 불러오기 이후의 결과:\n", decoded)

저장과 불러오기 이후의 결과:
 ['military uniform' 'bow tie' 'suit' 'bearskin' 'pickelhaube']

텐서플로 서빙으로 모델 배포하기

SavedModel은 파이썬에서 사용하기에 적합하지만, 일반적으로 프로덕션 환경에서는 추론을 위한 전용 서비스를 사용합니다. 이는 텐서플로 서빙을 사용한 SavedModel로 쉽게 구성할 수 있습니다.

tensorflow_model_server를 노트북이나 로컬 머신에 설치하는 방법을 포함한 텐서플로 서빙에 대한 자세한 내용은 TensorFlow Serving REST 튜토리얼을 참조하십시오. 간단한 예를 들면 앞서 내보낸 mobilenet 모델을 배포하기 위해 모델 경로를 SavedModel 디렉토리로 설정합니다:

nohup tensorflow_model_server \
  --rest_api_port=8501 \
  --model_name=mobilenet \
  --model_base_path="/tmp/mobilenet" >server.log 2>&1

이제 요청을 보냅니다.

!pip install -q requests
import json
import numpy
import requests
data = json.dumps({"signature_name": "serving_default",
                   "instances": x.tolist()})
headers = {"content-type": "application/json"}
json_response = requests.post('http://localhost:8501/v1/models/mobilenet:predict',
                              data=data, headers=headers)
predictions = numpy.array(json.loads(json_response.text)["predictions"])

predictions의 결과는 파이썬에서와 같습니다.

SavedModel 포맷

SavedModel은 변수값과 상수를 포함하고 직렬화된 시그니처와 이를 실행하는 데 필요한 상태를 담은 디렉토리입니다.

ls /tmp/mobilenet/1  # assets    saved_model.pb  variables

assets  saved_model.pb  variables

saved_model.pb 파일은 각각 하나의 함수로 된 이름있는 시그니처 세트를 포함합니다.

SavedModel에는 다중 시그니처 세트(saved_model_cli의 tag_set 매개변수 값으로 확인된 다중 MetaGraph)를 포함할 수 있지만 이런 경우는 드뭅니다. 다중 시그니처 세트를 작성하는 API에는 tf.Estimator.experimental_export_all_saved_models 및 TensorFlow 1.x의 tf.saved_model.Builder가 포함됩니다.

saved_model_cli show --dir /tmp/mobilenet/1 --tag_set serve

2020-09-23 04:08:50.883963: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
The given SavedModel MetaGraphDef contains SignatureDefs with the following keys:
SignatureDef key: "__saved_model_init_op"
SignatureDef key: "serving_default"

variables 디렉토리에는 일반적인 훈련 체크포인트 파일이 있습니다(훈련 체크포인트 가이드 참조).

ls /tmp/mobilenet/1/variables

variables.data-00000-of-00001  variables.index

assets 디렉토리에는 텐서플로 그래프(TensorFlow graph)에서 사용되는 파일들, 예를 들어 상수 테이블을 초기화하는 데 사용되는 텍스트 파일들이 있습니다. 이번 예제에서는 사용되지 않습니다.

SavedModel은 텐서플로 그래프에서 사용되지 않는 파일을 위해 assets.extra 디렉토리를 가질 수 있는데, 예를 들면 사용자가 SavedModel과 함께 사용할 파일입니다. 텐서플로 자체는 이 디렉토리를 사용하지 않습니다.

사용자 정의 모델 내보내기

첫 번째 섹션에서는, tf.saved_model.save가 tf.keras.Model 객체에 대한 시그니처를 자동으로 결정했습니다. 이는 케라스의 Model 객체가 내보내기 위한 명시적 메서드와 입력 크기를 가지기 때문에 작동했습니다. tf.saved_model.save는 저수준(low-level) 모델 설계 API와도 잘 작동하지만, 모델을 텐서플로 서빙에 배포할 계획이라면 시그니처로 사용할 함수를 지정해야 합니다.

class CustomModule(tf.Module):

  def __init__(self):
    super(CustomModule, self).__init__()
    self.v = tf.Variable(1.)

  @tf.function
  def __call__(self, x):
    return x * self.v

  @tf.function(input_signature=[tf.TensorSpec([], tf.float32)])
  def mutate(self, new_v):
    self.v.assign(new_v)

module = CustomModule()

이 모듈은 tf.function 데코레이터가 적용된 두 메서드를 가지고 있습니다. 이 함수들은 SavedModel에 포함되어 있으므로 tf.saved_model.load 함수를 사용하여 파이썬 프로그램에 함께 로드됩니다. 하지만 명시적 선언 없이는 텐서플로 서빙과 같은 시그니처 배포 도구와 saved_model_cli가 접근할 수 없습니다.

module.mutate는 input_signature를 가지고 있어서 계산 그래프를 SavedModel에 저장하기 위한 정보가 이미 충분히 있습니다. __call__은 시그니처가 없기에 저장하기 전 이 메서드를 호출해야 합니다.

module(tf.constant(0.))
tf.saved_model.save(module, "/tmp/module_no_signatures")

INFO:tensorflow:Assets written to: /tmp/module_no_signatures/assets

input_signature가 없는 함수의 경우, 저장 전에 사용된 입력의 크기는 함수가 불려진 이후에 사용될 것입니다. 스칼라값으로 __call__을 호출했으므로 스칼라값만 받아들일 것입니다

imported = tf.saved_model.load("/tmp/module_no_signatures")
assert 3. == imported(tf.constant(3.)).numpy()
imported.mutate(tf.constant(2.))
assert 6. == imported(tf.constant(3.)).numpy()

함수는 벡터와 같은 새로운 형식을 수용하지 않습니다.

imported(tf.constant([3.]))

ValueError: Could not find matching function to call for canonicalized inputs ((,), {}). Only existing signatures are [((TensorSpec(shape=(), dtype=tf.float32, name=u'x'),), {})].

get_concrete_function을 사용해 입력 크기를 함수 호출 없이 추가할 수 있습니다. 이 함수는 매개변수 값으로 Tensor 대신 입력 크기와 데이터 타입을 나타내는 tf.TensorSpec 객체를 받습니다. 크기가 None이면 모든 크기가 수용 가능합니다. 또는 각 축의 크기(axis size)를 담은 리스트일 수도 있습니다. 축 크기가 'None'이면 그 축에 대해 임의의 크기를 사용할 수 있습니다. 또한 tf.TensorSpecs는 이름을 가질 수 있는데, 기본값은 함수의 매개변수 키워드(여기서는 "x")입니다.

module.__call__.get_concrete_function(x=tf.TensorSpec([None], tf.float32))
tf.saved_model.save(module, "/tmp/module_no_signatures")
imported = tf.saved_model.load("/tmp/module_no_signatures")
assert [3.] == imported(tf.constant([3.])).numpy()

INFO:tensorflow:Assets written to: /tmp/module_no_signatures/assets

tf.keras.Model과 tf.Module과 같은 객체에 포함된 함수와 변수는 가져올 때 사용할 수 있지만 많은 파이썬의 타입과 속성은 잃어버립니다. 파이썬 프로그램 자체는 SavedModel에 저장되지 않습니다.

내보낼 함수를 시그니처로 지정하지 못했기에 시그니처는 없습니다.

saved_model_cli show --dir /tmp/module_no_signatures --tag_set serve

2020-09-23 04:08:53.608259: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
The given SavedModel MetaGraphDef contains SignatureDefs with the following keys:
SignatureDef key: "__saved_model_init_op"

내보낼 시그니처 지정하기

어떤 함수가 시그니처라는 것을 나타내려면 저장할 때 signatures 매개변수를 지정합니다.

call = module.__call__.get_concrete_function(tf.TensorSpec(None, tf.float32))
tf.saved_model.save(module, "/tmp/module_with_signature", signatures=call)

INFO:tensorflow:Assets written to: /tmp/module_with_signature/assets

먼저 tf.function 객체를 get_concrete_function 메서드를 사용해 ConcreteFunction 객체로 바꾸었습니다. 이것은 함수가 고정된 input_signature 없이 만들어지고 함수와 연관된 명시적인 Tensor 입력이 없었으므로 필수적입니다.

saved_model_cli show --dir /tmp/module_with_signature --tag_set serve --signature_def serving_default

2020-09-23 04:08:55.921542: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
The given SavedModel SignatureDef contains the following input(s):
  inputs['x'] tensor_info:
      dtype: DT_FLOAT
      shape: unknown_rank
      name: serving_default_x:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['output_0'] tensor_info:
      dtype: DT_FLOAT
      shape: unknown_rank
      name: StatefulPartitionedCall:0
Method name is: tensorflow/serving/predict

imported = tf.saved_model.load("/tmp/module_with_signature")
signature = imported.signatures["serving_default"]
assert [3.] == signature(x=tf.constant([3.]))["output_0"].numpy()
imported.mutate(tf.constant(2.))
assert [6.] == signature(x=tf.constant([3.]))["output_0"].numpy()
assert 2. == imported.v.numpy()

하나의 시그니처를 내보냈고 키는 기본값인 "serving_default"가 됩니다. 여러 시그니처를 내보내려면 딕셔너리로 전달합니다.

@tf.function(input_signature=[tf.TensorSpec([], tf.string)])
def parse_string(string_input):
  return imported(tf.strings.to_number(string_input))

signatures = {"serving_default": parse_string,
              "from_float": imported.signatures["serving_default"]}

tf.saved_model.save(imported, "/tmp/module_with_multiple_signatures", signatures)

INFO:tensorflow:Assets written to: /tmp/module_with_multiple_signatures/assets

saved_model_cli show --dir /tmp/module_with_multiple_signatures --tag_set serve

2020-09-23 04:08:58.328632: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
The given SavedModel MetaGraphDef contains SignatureDefs with the following keys:
SignatureDef key: "__saved_model_init_op"
SignatureDef key: "from_float"
SignatureDef key: "serving_default"

saved_model_cli는 커맨드 라인에서 SavedModel을 직접 실행할 수도 있습니다.

saved_model_cli run --dir /tmp/module_with_multiple_signatures --tag_set serve --signature_def serving_default --input_exprs="string_input='3.'"
saved_model_cli run --dir /tmp/module_with_multiple_signatures --tag_set serve --signature_def from_float --input_exprs="x=3."

2020-09-23 04:09:00.572292: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-09-23 04:09:01.943289: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1
2020-09-23 04:09:01.953084: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:01.953505: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
pciBusID: 0000:00:05.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2020-09-23 04:09:01.953539: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-09-23 04:09:01.955512: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2020-09-23 04:09:01.957417: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-09-23 04:09:01.957767: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2020-09-23 04:09:01.959779: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2020-09-23 04:09:01.960992: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2020-09-23 04:09:01.965284: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2020-09-23 04:09:01.965430: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:01.965876: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:01.966225: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-09-23 04:09:01.966576: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-09-23 04:09:01.973951: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 2000129999 Hz
2020-09-23 04:09:01.974421: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5717c70 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-09-23 04:09:01.974453: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-09-23 04:09:02.068987: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:02.069582: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x57abbd0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-09-23 04:09:02.069624: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Tesla V100-SXM2-16GB, Compute Capability 7.0
2020-09-23 04:09:02.069895: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:02.070348: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
pciBusID: 0000:00:05.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2020-09-23 04:09:02.070403: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-09-23 04:09:02.070464: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2020-09-23 04:09:02.070492: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-09-23 04:09:02.070518: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2020-09-23 04:09:02.070542: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2020-09-23 04:09:02.070566: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2020-09-23 04:09:02.070591: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2020-09-23 04:09:02.070691: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:02.071179: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:02.071549: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-09-23 04:09:02.071617: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-09-23 04:09:02.644733: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-09-23 04:09:02.644788: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0 
2020-09-23 04:09:02.644801: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N 
2020-09-23 04:09:02.645036: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:02.645518: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:02.645906: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 53 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0)
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/tools/saved_model_cli.py:444: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.loader.load or tf.compat.v1.saved_model.load. There will be a new function for importing SavedModels in Tensorflow 2.0.
INFO:tensorflow:Restoring parameters from /tmp/module_with_multiple_signatures/variables/variables
2020-09-23 04:09:02.680933: I tensorflow/stream_executor/cuda/cuda_driver.cc:775] failed to allocate 53.75M (56360960 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2020-09-23 04:09:02.682002: I tensorflow/stream_executor/cuda/cuda_driver.cc:775] failed to allocate 48.38M (50724864 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
Result for output key output_0:
6.0
2020-09-23 04:09:03.619578: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-09-23 04:09:04.984406: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1
2020-09-23 04:09:04.993964: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:04.994371: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
pciBusID: 0000:00:05.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2020-09-23 04:09:04.994405: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-09-23 04:09:04.996426: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2020-09-23 04:09:04.998344: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-09-23 04:09:04.998713: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2020-09-23 04:09:05.000767: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2020-09-23 04:09:05.001962: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2020-09-23 04:09:05.006433: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2020-09-23 04:09:05.006581: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:05.007035: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:05.007400: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-09-23 04:09:05.007749: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-09-23 04:09:05.015144: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 2000129999 Hz
2020-09-23 04:09:05.015571: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5fcd0c0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-09-23 04:09:05.015600: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-09-23 04:09:05.106599: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:05.107183: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x60611d0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-09-23 04:09:05.107222: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Tesla V100-SXM2-16GB, Compute Capability 7.0
2020-09-23 04:09:05.107481: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:05.107896: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
pciBusID: 0000:00:05.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2020-09-23 04:09:05.107949: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-09-23 04:09:05.108001: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2020-09-23 04:09:05.108026: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-09-23 04:09:05.108047: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2020-09-23 04:09:05.108070: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2020-09-23 04:09:05.108092: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2020-09-23 04:09:05.108115: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2020-09-23 04:09:05.108227: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:05.108662: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:05.109021: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-09-23 04:09:05.109079: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-09-23 04:09:05.673193: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-09-23 04:09:05.673250: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0 
2020-09-23 04:09:05.673259: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N 
2020-09-23 04:09:05.673489: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:05.673925: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-09-23 04:09:05.674297: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 53 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0)
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/tools/saved_model_cli.py:444: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.loader.load or tf.compat.v1.saved_model.load. There will be a new function for importing SavedModels in Tensorflow 2.0.
INFO:tensorflow:Restoring parameters from /tmp/module_with_multiple_signatures/variables/variables
2020-09-23 04:09:05.707789: I tensorflow/stream_executor/cuda/cuda_driver.cc:775] failed to allocate 53.75M (56360960 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
2020-09-23 04:09:05.708856: I tensorflow/stream_executor/cuda/cuda_driver.cc:775] failed to allocate 48.38M (50724864 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory
Result for output key output_0:
6.0

가져온 모델 미세 튜닝하기

변수 객체가 사용 가능하므로 imported 함수를 통해 역전파할 수 있습니다.

optimizer = tf.optimizers.SGD(0.05)

def train_step():
  with tf.GradientTape() as tape:
    loss = (10. - imported(tf.constant(2.))) ** 2
  variables = tape.watched_variables()
  grads = tape.gradient(loss, variables)
  optimizer.apply_gradients(zip(grads, variables))
  return loss

for _ in range(10):
  # "v"는 5로 수렴, "loss"는 0으로 수렴
  print("loss={:.2f} v={:.2f}".format(train_step(), imported.v.numpy()))

loss=36.00 v=3.20
loss=12.96 v=3.92
loss=4.67 v=4.35
loss=1.68 v=4.61
loss=0.60 v=4.77
loss=0.22 v=4.86
loss=0.08 v=4.92
loss=0.03 v=4.95
loss=0.01 v=4.97
loss=0.00 v=4.98

SavedModel의 제어 흐름

tf.function에 들어갈 수 있는 것은 모두 SavedModel에 들어갈 수 있습니다. AutoGraph를 사용하면 Tensor에 의존하는 조건부 논리를 파이썬 제어 흐름으로 표현할 수 있습니다.

@tf.function(input_signature=[tf.TensorSpec([], tf.int32)])
def control_flow(x):
  if x < 0:
    tf.print("유효하지 않음!")
  else:
    tf.print(x % 3)

to_export = tf.Module()
to_export.control_flow = control_flow
tf.saved_model.save(to_export, "/tmp/control_flow")

INFO:tensorflow:Assets written to: /tmp/control_flow/assets

imported = tf.saved_model.load("/tmp/control_flow")
imported.control_flow(tf.constant(-1))  # 유효하지 않음!
imported.control_flow(tf.constant(2))   # 2
imported.control_flow(tf.constant(3))   # 0

유효하지 않음!
2
0

추정기(Estimator)의 SavedModel

추정기는 tf.Estimator.export_saved_model을 통해 SavedModel을 내보냅니다. 자세한 내용은 Estimator 가이드를 참조하십시오.

input_column = tf.feature_column.numeric_column("x")
estimator = tf.estimator.LinearClassifier(feature_columns=[input_column])

def input_fn():
  return tf.data.Dataset.from_tensor_slices(
    ({"x": [1., 2., 3., 4.]}, [1, 1, 0, 0])).repeat(200).shuffle(64).batch(16)
estimator.train(input_fn)

serving_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(
  tf.feature_column.make_parse_example_spec([input_column]))
export_path = estimator.export_saved_model(
  "/tmp/from_estimator/", serving_input_fn)

INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpdx24k4d1
INFO:tensorflow:Using config: {'_model_dir': '/tmp/tmpdx24k4d1', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
INFO:tensorflow:Calling model_fn.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/canned/linear.py:1481: Layer.add_variable (from tensorflow.python.keras.engine.base_layer_v1) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.add_weight` method instead.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/keras/optimizer_v2/ftrl.py:112: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpdx24k4d1/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 0.6931472, step = 0
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 50...
INFO:tensorflow:Saving checkpoints for 50 into /tmp/tmpdx24k4d1/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 50...
INFO:tensorflow:Loss for final step: 0.4231811.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/saved_model/signature_def_utils_impl.py:145: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
INFO:tensorflow:Signatures INCLUDED in export for Classify: ['serving_default', 'classification']
INFO:tensorflow:Signatures INCLUDED in export for Regress: ['regression']
INFO:tensorflow:Signatures INCLUDED in export for Predict: ['predict']
INFO:tensorflow:Signatures INCLUDED in export for Train: None
INFO:tensorflow:Signatures INCLUDED in export for Eval: None
INFO:tensorflow:Restoring parameters from /tmp/tmpdx24k4d1/model.ckpt-50
INFO:tensorflow:Assets added to graph.
INFO:tensorflow:No assets to write.
INFO:tensorflow:SavedModel written to: /tmp/from_estimator/temp-1600834147/saved_model.pb

이 SavedModel은 텐서플로 서빙에 배포하는 데 유용한 직렬화된 tf.Example 프로토콜 버퍼를 사용합니다. 그러나 tf.saved_model.load로 불러오고 파이썬에서 실행할 수도 있습니다.

imported = tf.saved_model.load(export_path)

def predict(x):
  example = tf.train.Example()
  example.features.feature["x"].float_list.value.extend([x])
  return imported.signatures["predict"](
    examples=tf.constant([example.SerializeToString()]))

print(predict(1.5))
print(predict(3.5))

{'all_class_ids': <tf.Tensor: shape=(1, 2), dtype=int32, numpy=array([[0, 1]], dtype=int32)>, 'logistic': <tf.Tensor: shape=(1, 1), dtype=float32, numpy=array([[0.5608568]], dtype=float32)>, 'classes': <tf.Tensor: shape=(1, 1), dtype=string, numpy=array([[b'1']], dtype=object)>, 'class_ids': <tf.Tensor: shape=(1, 1), dtype=int64, numpy=array([[1]])>, 'all_classes': <tf.Tensor: shape=(1, 2), dtype=string, numpy=array([[b'0', b'1']], dtype=object)>, 'probabilities': <tf.Tensor: shape=(1, 2), dtype=float32, numpy=array([[0.43914315, 0.5608568 ]], dtype=float32)>, 'logits': <tf.Tensor: shape=(1, 1), dtype=float32, numpy=array([[0.24464023]], dtype=float32)>}
{'all_class_ids': <tf.Tensor: shape=(1, 2), dtype=int32, numpy=array([[0, 1]], dtype=int32)>, 'logistic': <tf.Tensor: shape=(1, 1), dtype=float32, numpy=array([[0.20554812]], dtype=float32)>, 'classes': <tf.Tensor: shape=(1, 1), dtype=string, numpy=array([[b'0']], dtype=object)>, 'class_ids': <tf.Tensor: shape=(1, 1), dtype=int64, numpy=array([[0]])>, 'all_classes': <tf.Tensor: shape=(1, 2), dtype=string, numpy=array([[b'0', b'1']], dtype=object)>, 'probabilities': <tf.Tensor: shape=(1, 2), dtype=float32, numpy=array([[0.7944519 , 0.20554814]], dtype=float32)>, 'logits': <tf.Tensor: shape=(1, 1), dtype=float32, numpy=array([[-1.3519722]], dtype=float32)>}

tf.estimator.export.build_server_input_receiver_fn를 사용해 tf.train.Example이 아닌 원시 텐서를 가지는 입력 함수를 만들 수 있습니다.

C++에서 SavedModel 불러오기

SavedModel의 C++ 버전 loader는 SessionOptions 및 RunOptions을 허용하며 경로에서 SavedModel을 불러오는 API를 제공합니다. 불러 올 그래프와 연관된 태그를 지정해야합니다. 불러온 SavedModel의 버전은 SavedModelBundle이라고 하며 MetaGraphDef와 불러온 세션을 포함합니다.

const string export_dir = ...
SavedModelBundle bundle;
...
LoadSavedModel(session_options, run_options, export_dir, {kSavedModelTagTrain},
               &bundle);

$ bazel build tensorflow/python/tools:saved_model_cli

usage: saved_model_cli show [-h] --dir DIR [--all]
[--tag_set TAG_SET] [--signature_def SIGNATURE_DEF_KEY]

$ saved_model_cli show --dir /tmp/saved_model_dir
The given SavedModel contains the following tag-sets:
serve
serve, gpu

$ saved_model_cli show --dir /tmp/saved_model_dir --tag_set serve
The given SavedModel `MetaGraphDef` contains `SignatureDefs` with the
following keys:
SignatureDef key: "classify_x2_to_y3"
SignatureDef key: "classify_x_to_y"
SignatureDef key: "regress_x2_to_y3"
SignatureDef key: "regress_x_to_y"
SignatureDef key: "regress_x_to_y2"
SignatureDef key: "serving_default"

$ saved_model_cli show --dir /tmp/saved_model_dir --tag_set serve,gpu

$ saved_model_cli show --dir \
/tmp/saved_model_dir --tag_set serve --signature_def serving_default
The given SavedModel SignatureDef contains the following input(s):
  inputs['x'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 1)
      name: x:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['y'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 1)
      name: y:0
Method name is: tensorflow/serving/predict

$ saved_model_cli show --dir /tmp/saved_model_dir --all
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['classify_x2_to_y3']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['inputs'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: x2:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['scores'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: y3:0
  Method name is: tensorflow/serving/classify

...

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['x'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: x:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['y'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: y:0
  Method name is: tensorflow/serving/predict

usage: saved_model_cli run [-h] --dir DIR --tag_set TAG_SET --signature_def
                           SIGNATURE_DEF_KEY [--inputs INPUTS]
                           [--input_exprs INPUT_EXPRS]
                           [--input_examples INPUT_EXAMPLES] [--outdir OUTDIR]
                           [--overwrite] [--tf_debug]

--inputs <INPUTS>

`<input_key>=[[1],[2],[3]]`

`<input_key>=np.ones((32,32,3))`

`<input_key>=[{"age":[22,24],"education":["BS","MS"]}]`