tensorflow::Env

This is an abstract class.

#include <env.h>

An interface used by the tensorflow implementation to access operating system functionality like the filesystem etc.

Summary

Callers may wish to provide a custom Env object to get fine grain control.

All Env implementations are safe for concurrent access from multiple threads without any external synchronization.

Inheritance

Direct Known Subclasses:tensorflow::EnvWrapper

Constructors and Destructors

Env()
~Env()

Public functions

CreateDir(const string & dirname)
Creates the specified directory.
DeleteDir(const string & dirname)
Deletes the specified directory.
DeleteFile(const string & fname)
Deletes the named file.
DeleteRecursively(const string & dirname, int64 *undeleted_files, int64 *undeleted_dirs)
Deletes the specified directory and all subdirectories and files underneath it.
FileExists(const string & fname)
bool
Returns true iff the named file exists.
GetChildren(const string & dir, std::vector< string > *result)
Stores in *result the names of the children of the specified directory.
GetFileSize(const string & fname, uint64 *file_size)
Stores the size of fname in *file_size.
GetFileSystemForFile(const string & fname, FileSystem **result)
virtual Status
Returns the FileSystem object to handle operations on the file specified by 'fname'.
GetMatchingPaths(const string & pattern, std::vector< string > *results)
virtual Status
Given a pattern, stores in *results the set of paths that matches that pattern.
GetRegisteredFileSystemSchemes(std::vector< string > *schemes)
virtual Status
Returns the file system schemes registered for this Env.
GetSymbolFromLibrary(void *handle, const char *symbol_name, void **symbol)=0
virtual Status
IsDirectory(const string & fname)
Returns whether the given path is a directory or not.
LoadLibrary(const char *library_filename, void **handle)=0
virtual Status
MatchPath(const string & path, const string & pattern)=0
virtual bool
Returns true if the path matches the given pattern.
NewAppendableFile(const string & fname, std::unique_ptr< WritableFile > *result)
Creates an object that either appends to an existing file, or writes to a new file (if the file does not exist to begin with).
NewRandomAccessFile(const string & fname, std::unique_ptr< RandomAccessFile > *result)
Creates a brand new random access read-only file with the specified name.
NewReadOnlyMemoryRegionFromFile(const string & fname, std::unique_ptr< ReadOnlyMemoryRegion > *result)
Creates a readonly region of memory with the file context.
NewWritableFile(const string & fname, std::unique_ptr< WritableFile > *result)
Creates an object that writes to a new file with the specified name.
NowMicros()=0
virtual uint64
Returns the number of micro-seconds since some fixed point in time.
NowSeconds()
virtual uint64
Returns the number of seconds since some fixed point in time.
RecursivelyCreateDir(const string & dirname)
Creates the specified directory and all the necessary subdirectories.
RegisterFileSystem(const string & scheme, FileSystemRegistry::Factory factory)
virtual Status
RenameFile(const string & src, const string & target)
Renames file src to target.
SchedClosure(std::function< void()> closure)=0
virtual void
SchedClosureAfter(int64 micros, std::function< void()> closure)=0
virtual void
SleepForMicroseconds(int64 micros)=0
virtual void
Sleeps/delays the thread for the prescribed number of micro-seconds.
StartThread(const ThreadOptions & thread_options, const string & name, std::function< void()> fn) TF_MUST_USE_RESULT=0
virtual Thread *
Returns a new thread that is running fn() and is identified (for debugging/performance-analysis) by "name".
Stat(const string & fname, FileStatistics *stat)
Obtains statistics for the given path.

Public static functions

Default()
Env *
Returns a default environment suitable for the current operating system.

Public functions

CreateDir

Status CreateDir(
  const string & dirname
)

Creates the specified directory.

Typical return codes

  • OK - successfully created the directory.
  • ALREADY_EXISTS - directory already exists.
  • PERMISSION_DENIED - dirname is not writable.

DeleteDir

Status DeleteDir(
  const string & dirname
)

Deletes the specified directory.

DeleteFile

Status DeleteFile(
  const string & fname
)

Deletes the named file.

DeleteRecursively

Status DeleteRecursively(
  const string & dirname,
  int64 *undeleted_files,
  int64 *undeleted_dirs
)

Deletes the specified directory and all subdirectories and files underneath it.

undeleted_files and undeleted_dirs stores the number of files and directories that weren't deleted (unspecified if the return status is not OK). REQUIRES: undeleted_files, undeleted_dirs to be not null. Typical return codes

  • OK - dirname exists and we were able to delete everything underneath.
  • NOT_FOUND - dirname doesn't exist
  • PERMISSION_DENIED - dirname or some descendant is not writable
  • UNIMPLEMENTED - Some underlying functions (like Delete) are not implemented

Env

 Env()

FileExists

bool FileExists(
  const string & fname
)

Returns true iff the named file exists.

GetChildren

Status GetChildren(
  const string & dir,
  std::vector< string > *result
)

Stores in *result the names of the children of the specified directory.

The names are relative to "dir".

Original contents of *results are dropped.

GetFileSize

Status GetFileSize(
  const string & fname,
  uint64 *file_size
)

Stores the size of fname in *file_size.

GetFileSystemForFile

virtual Status GetFileSystemForFile(
  const string & fname,
  FileSystem **result
)

Returns the FileSystem object to handle operations on the file specified by 'fname'.

The FileSystem object is used as the implementation for the file system related (non-virtual) functions that follow. Returned FileSystem object is still owned by the Env object and will

GetMatchingPaths

virtual Status GetMatchingPaths(
  const string & pattern,
  std::vector< string > *results
)

Given a pattern, stores in *results the set of paths that matches that pattern.

*results is cleared.

pattern must match all of a name, not just a substring. pattern: { term } term: '*': matches any sequence of non-'/' characters '?': matches a single non-'/' character '[' [ '^' ] { match-list } ']': matches any single character (not) on the list c: matches character c (c != '*', '?', '\', '[') '\' c: matches character c character-range: c: matches character c (c != '\', '-', ']') '\' c: matches character c lo '-' hi: matches character c for lo <= c <= hi

Typical return codes

  • OK - no errors
  • UNIMPLEMENTED - Some underlying functions (like GetChildren) are not implemented The default implementation uses a combination of GetChildren, MatchPath and IsDirectory.

GetRegisteredFileSystemSchemes

virtual Status GetRegisteredFileSystemSchemes(
  std::vector< string > *schemes
)

Returns the file system schemes registered for this Env.

GetSymbolFromLibrary

virtual Status GetSymbolFromLibrary(
  void *handle,
  const char *symbol_name,
  void **symbol
)=0

IsDirectory

Status IsDirectory(
  const string & fname
)

Returns whether the given path is a directory or not.

Typical return codes (not guaranteed exhaustive):

  • OK - The path exists and is a directory.
  • FAILED_PRECONDITION - The path exists and is not a directory.
  • NOT_FOUND - The path entry does not exist.
  • PERMISSION_DENIED - Insufficient permissions.
  • UNIMPLEMENTED - The file factory doesn't support directories.

LoadLibrary

virtual Status LoadLibrary(
  const char *library_filename,
  void **handle
)=0

MatchPath

virtual bool MatchPath(
  const string & path,
  const string & pattern
)=0

Returns true if the path matches the given pattern.

The wildcards allowed in pattern are described below (GetMatchingPaths).

NewAppendableFile

Status NewAppendableFile(
  const string & fname,
  std::unique_ptr< WritableFile > *result
)

Creates an object that either appends to an existing file, or writes to a new file (if the file does not exist to begin with).

On success, stores a pointer to the new file in *result and returns OK. On failure stores NULL in *result and returns non-OK.

The returned file will only be accessed by one thread at a time.

The ownership of the returned WritableFile is passed to the caller and the object should be deleted when is not used. The file object shouldn't live longer than the Env object.

NewRandomAccessFile

Status NewRandomAccessFile(
  const string & fname,
  std::unique_ptr< RandomAccessFile > *result
)

Creates a brand new random access read-only file with the specified name.

On success, stores a pointer to the new file in *result and returns OK. On failure stores NULL in *result and returns non-OK. If the file does not exist, returns a non-OK status.

The returned file may be concurrently accessed by multiple threads.

The ownership of the returned RandomAccessFile is passed to the caller and the object should be deleted when is not used. The file object shouldn't live longer than the Env object.

NewReadOnlyMemoryRegionFromFile

Status NewReadOnlyMemoryRegionFromFile(
  const string & fname,
  std::unique_ptr< ReadOnlyMemoryRegion > *result
)

Creates a readonly region of memory with the file context.

On success, it returns a pointer to read-only memory region from the content of file fname. The ownership of the region is passed to the caller. On failure stores nullptr in *result and returns non-OK.

The returned memory region can be accessed from many threads in parallel.

The ownership of the returned ReadOnlyMemoryRegion is passed to the caller and the object should be deleted when is not used. The memory region object shouldn't live longer than the Env object.

NewWritableFile

Status NewWritableFile(
  const string & fname,
  std::unique_ptr< WritableFile > *result
)

Creates an object that writes to a new file with the specified name.

Deletes any existing file with the same name and creates a new file. On success, stores a pointer to the new file in *result and returns OK. On failure stores NULL in *result and returns non-OK.

The returned file will only be accessed by one thread at a time.

The ownership of the returned WritableFile is passed to the caller and the object should be deleted when is not used. The file object shouldn't live longer than the Env object.

NowMicros

virtual uint64 NowMicros()=0

Returns the number of micro-seconds since some fixed point in time.

Only useful for computing deltas of time.

NowSeconds

virtual uint64 NowSeconds()

Returns the number of seconds since some fixed point in time.

Only useful for computing deltas of time.

RecursivelyCreateDir

Status RecursivelyCreateDir(
  const string & dirname
)

Creates the specified directory and all the necessary subdirectories.

Typical return codes.

  • OK - successfully created the directory and sub directories, even if they were already created.
  • PERMISSION_DENIED - dirname or some subdirectory is not writable.

RegisterFileSystem

virtual Status RegisterFileSystem(
  const string & scheme,
  FileSystemRegistry::Factory factory
)

RenameFile

Status RenameFile(
  const string & src,
  const string & target
)

Renames file src to target.

If target already exists, it will be replaced.

SchedClosure

virtual void SchedClosure(
  std::function< void()> closure
)=0

SchedClosureAfter

virtual void SchedClosureAfter(
  int64 micros,
  std::function< void()> closure
)=0

SleepForMicroseconds

virtual void SleepForMicroseconds(
  int64 micros
)=0

Sleeps/delays the thread for the prescribed number of micro-seconds.

StartThread

virtual Thread * StartThread(
  const ThreadOptions & thread_options,
  const string & name,
  std::function< void()> fn
) TF_MUST_USE_RESULT=0

Returns a new thread that is running fn() and is identified (for debugging/performance-analysis) by "name".

Caller takes ownership of the result and must delete it eventually (the deletion will block until fn() stops running).

Stat

Status Stat(
  const string & fname,
  FileStatistics *stat
)

Obtains statistics for the given path.

~Env

virtual  ~Env()=default

Public static functions

Default

Env * Default()

Returns a default environment suitable for the current operating system.

Sophisticated users may wish to provide their own Env implementation instead of relying on this default environment.

The result of Default() belongs to this library and must never be deleted.