tensorflow::serving::AspiredVersionsManager

#include <aspired_versions_manager.h>

A manager that implements the Target<Loader> API which uses aspired-versions callbacks to dictate which servable versions to load.

Summary

This manager also uses that API to infer which ones to unload: If a given servable version is currently loaded, and is omitted from an aspired-versions callback invocation pertaining to its servable stream, this manager interprets that omission as an implicit instruction to unload the version. See below for details.

(The implicit-unload semantics facilitates stateless Source implementations, whereby a given iteration of the Source's logic simply decides which versions of a servable ought to be loaded, without needing to know what it has decided in the past.)

This manager makes transitions between versions of a servable stream using a configured AspiredVersionPolicy. The manager prefers unloading before loading to free up resources in the server when deciding among transitions suggested by the policy.

Inheritance

Public types

BasicManager::PreLoadHook PreLoadHook

Friend classes

friend class internal::AspiredVersionsManagerTargetImpl

friend class test_util::AspiredVersionsManagerTestAccess

Public static functions

Status Create(
  Options options,
  std::unique_ptr< AspiredVersionsManager > *manager
)

Public functions

Source< std::unique_ptr< Loader > >::AspiredVersionsCallback GetAspiredVersionsCallback() override

virtual std::vector< ServableId > ListAvailableServableIds() const override

 ~AspiredVersionsManager() override