tensorflow::serving::AspiredVersionsManager

#include <aspired_versions_manager.h>

A manager that implements the Target<Loader> API which uses aspired-versions callbacks to dictate which servable versions to load.

Summary

This manager also uses that API to infer which ones to unload: If a given servable version is currently loaded, and is omitted from an aspired-versions callback invocation pertaining to its servable stream, this manager interprets that omission as an implicit instruction to unload the version. See below for details.

(The implicit-unload semantics facilitates stateless Source implementations, whereby a given iteration of the Source's logic simply decides which versions of a servable ought to be loaded, without needing to know what it has decided in the past.)

This manager makes transitions between versions of a servable stream using a configured AspiredVersionPolicy. The manager prefers unloading before loading to free up resources in the server when deciding among transitions suggested by the policy.

Inheritance

Inherits from: tensorflow::serving::Manager, Target< std::unique_ptr< Loader > >

Constructors and Destructors

~AspiredVersionsManager()

Public types

PreLoadHook using
BasicManager::PreLoadHook

Friend classes

internal::AspiredVersionsManagerTargetImpl
friend class
test_util::AspiredVersionsManagerTestAccess
friend class

Public static functions

Create(Options options, std::unique_ptr< AspiredVersionsManager > *manager)
Status

Public functions

GetAspiredVersionsCallback() override
Source< std::unique_ptr< Loader > >::AspiredVersionsCallback
Returns a callback to set the list of aspired versions for a particular servable stream, using Loaders.
ListAvailableServableIds() const override
virtual std::vector< ServableId >
Gets a list of all available servable ids, i.e.

Structs

tensorflow::serving::AspiredVersionsManager::Options

Config options and pluggable objects that will be used by the AspiredVersionsManager.

Public types

PreLoadHook

BasicManager::PreLoadHook PreLoadHook

Friend classes

internal::AspiredVersionsManagerTargetImpl

friend class internal::AspiredVersionsManagerTargetImpl

test_util::AspiredVersionsManagerTestAccess

friend class test_util::AspiredVersionsManagerTestAccess

Public static functions

Create

Status Create(
  Options options,
  std::unique_ptr< AspiredVersionsManager > *manager
)

Public functions

GetAspiredVersionsCallback

Source< std::unique_ptr< Loader > >::AspiredVersionsCallback GetAspiredVersionsCallback() override

Returns a callback to set the list of aspired versions for a particular servable stream, using Loaders.

ListAvailableServableIds

virtual std::vector< ServableId > ListAvailableServableIds() const override

Gets a list of all available servable ids, i.e.

each of these can be retrieved using GetServableHandle.

~AspiredVersionsManager

 ~AspiredVersionsManager() override