tensorflow::serving::LoaderHarness

#include <loader_harness.h>

LoaderHarness is a widget the Manager uses to hold on to and talk to a Loader while it owns it.

Summary

It tracks the overall state of a Servable such that Manager can determine which state transitions to make at what times.

A manager implementation can also add some additional state with each harness. For example, a manager could put ACL or lifecycle metadata here. The ownership is maintained by the harness.

This class is thread-safe.

Constructors and Destructors
`LoaderHarness(const ServableId & id, std::unique_ptr< Loader > loader, const Options & options)`
`LoaderHarness(const ServableId & id, std::unique_ptr< Loader > loader, std::unique_ptr< T > additional_state, const Options & options)` Constructor to create a harness with additional state, which a manager needs.
`~LoaderHarness()` Legal to destruct iff current state is one of kNew, kDisabled or kError.

Public types
`State{ kNew, kLoadRequested, kLoadApproved, kLoading, kReady, kUnloadRequested, kQuiescing, kQuiesced, kUnloading, kDisabled, kError }`	enum State of the underlying servable, from the perspective of the LoaderHarness and for the purpose of communication between it and a Manager.

Public functions
`DoneQuiescing()`	`Status` Transitions the state to kQuiesced, which means that there are no more live handles to this servable available in the frontend.
`Error(const Status & status)`	`void` Transitions the state to kError and invokes 'options_.error_callback'.
`Load()`	`Status` Transitions to kLoading, delegates to Servable::Load(), then transitions either to kReady if Load() succeeds, or to kError (and invokes 'options_.
`LoadApproved()`	`Status` Transitions to kLoadApproved.
`LoadRequested()`	`Status` Transitions the state of the harness to kLoadRequested iff its current state is kNew.
`StartQuiescing()`	`Status` Transitions the state to kQuiescing, which means that we would like to not give out any more handles to this servable.
`Unload()`	`Status` Transitions to kUnloading, delegates to Servable::Unload(), then transitions to kDisabled when Unload() is done.
`UnloadRequested()`	`Status` Transitions the state of the harness to kUnloadRequested iff its current state is kReady.
`additional_state()`	`T *` Gets the additional state.
`cancel_load_retry()`	`bool`
`id() const`	`ServableId` Returns the identifier of underlying Servable.
`loader() const`	`Loader *` Returns a pointer to the wrapped loader.
`loader_state_snapshot()`	`ServableStateSnapshot< T >` Returns the current overall state snapshot of the underlying Servable.
`set_cancel_load_retry(bool value)`	`void` Cancels retrying the load of the servable.
`state()`	`State` Returns the current state of underlying Servable.
`status()`	`Status` Whether anything has gone wrong with this servable.

Public static functions
`StateDebugString(State state)`	`string`

Structs
tensorflow::serving::LoaderHarness::Options	Options to configure a LoaderHarness.

Public types

State

 State

State of the underlying servable, from the perspective of the LoaderHarness and for the purpose of communication between it and a Manager.

Not equivalent to the semantic servable states in servable_state.h.

Valid transitions: kNew>kLoading>kReady>kQuiescing>kQuiesced>kUnloading>kDisabled as well as: any_state>kError.

Properties
`kDisabled`	'loader_->Unload()' has finished.
`kError`	An error has occurred, either during 'loader_->Load()' or outside of the harness (and was reported to the harness via a call to Error()).
`kLoadApproved`	The servable has been approved for loading, e.g. resources have been set aside for it.
`kLoadRequested`	The manager has been requested to load this servable.
`kLoading`	'loader_->Load()' has been called.
`kNew`	Initial state.
`kQuiesced`	The servable has been made unavailable for serving.
`kQuiescing`	The servable is going to be made unavailable for serving.
`kReady`	'loader_->Load()' has succeeded.
`kUnloadRequested`	The manager has been requested to unload this servable.
`kUnloading`	'loader_->Unload()' has been called.

Public functions

DoneQuiescing

Status DoneQuiescing()

Transitions the state to kQuiesced, which means that there are no more live handles to this servable available in the frontend.

At this point, we can unload this object.

REQUIRES: State is kQuiescing when called. Otherwise DCHECK-fails, transitions to state kError and invokes 'options_.error_callback'.

Error

void Error(
  const Status & status
)

Transitions the state to kError and invokes 'options_.error_callback'.

Load

Status Load()

Transitions to kLoading, delegates to Servable::Load(), then transitions either to kReady if Load() succeeds, or to kError (and invokes 'options_.

error_callback') if Load() fails. This call may take a long time.

We retry the Servable::Load() according to the options set during construction of this class. We stop retrying and give up if 1. we have reached max_num_load_retries or, 2. if cancel_load() is set to true.

REQUIRES: State is kLoadApproved when called. Otherwise DCHECK-fails, transitions to state kError and invokes 'options_.error_callback'.

LoadApproved

Status LoadApproved()

Transitions to kLoadApproved.

REQUIRES: State is kLoadRequested when called. Otherwise DCHECK-fails, transitions to state kError and invokes 'options_.error_callback'.

LoadRequested

Status LoadRequested()

Transitions the state of the harness to kLoadRequested iff its current state is kNew.

The test-and-change is done transactionally, so this method can be used to ensure that at most one Load() request can proceed.

LoaderHarness

 LoaderHarness(
  const ServableId & id,
  std::unique_ptr< Loader > loader,
  const Options & options
)

LoaderHarness

 LoaderHarness(
  const ServableId & id,
  std::unique_ptr< Loader > loader,
  std::unique_ptr< T > additional_state,
  const Options & options
)

Constructor to create a harness with additional state, which a manager needs.

StartQuiescing

Status StartQuiescing()

Transitions the state to kQuiescing, which means that we would like to not give out any more handles to this servable.

REQUIRES: State is kUnloadRequested when called. Otherwise DCHECK-fails, transitions to state kError and invokes 'options_.error_callback'.

Unload

Status Unload()

Transitions to kUnloading, delegates to Servable::Unload(), then transitions to kDisabled when Unload() is done.

REQUIRES: State is kQuiesced when called. Otherwise DCHECK-fails, transitions to state kError and invokes 'options_.error_callback'.

UnloadRequested

Status UnloadRequested()

Transitions the state of the harness to kUnloadRequested iff its current state is kReady.

The test-and-change is done transactionally, so this method can be used to ensure that at most one Load() request can proceed.

additional_state

T * additional_state()

Gets the additional state.

Returns nullptr if the type mismatches or if it wasn't set.

cancel_load_retry

bool cancel_load_retry()

id

ServableId id() const

Returns the identifier of underlying Servable.

loader

Loader * loader() const

Returns a pointer to the wrapped loader.

Ownership remains with this class.

loader_state_snapshot

ServableStateSnapshot< T > loader_state_snapshot()

Returns the current overall state snapshot of the underlying Servable.

set_cancel_load_retry

void set_cancel_load_retry(
  bool value
)

Cancels retrying the load of the servable.

This is best-effort, and does not preempt a Load() which is already happening, only subsequent calls.

If the retries are cancelled, the servable goes into a state dependent on the last Load() called on it. If the last Load() was successful, it will be in state kReady, else in kError.

state

State state()

Returns the current state of underlying Servable.

status

Status status()

Whether anything has gone wrong with this servable.

If state is kError, this will be non-OK. If not OK, the error could be something that occurred in a Source or SourceAdapter, in the Loader, in the Manager, or elsewhere. All errors pertaining to the servable are reported here, regardless of origin.

~LoaderHarness

 ~LoaderHarness()

Legal to destruct iff current state is one of kNew, kDisabled or kError.

Check-fails if violated.

Public static functions

StateDebugString

string StateDebugString(
  State state
)

tensorflow::serving::LoaderHarness

Summary

Constructors and Destructors

Public types

Public functions

Public static functions

Structs

Public types

State

Public functions

DoneQuiescing

Error

Load

LoadApproved

LoadRequested

LoaderHarness

LoaderHarness

StartQuiescing

Unload

UnloadRequested

additional_state

cancel_load_retry

id

loader

loader_state_snapshot

set_cancel_load_retry

state

status

~LoaderHarness

Public static functions

StateDebugString