ML Community Day is November 9! Join us for updates from TensorFlow, JAX, and more Learn more

File I/O wrappers without thread locking.

Used in the notebooks

Used in the tutorials

The main roles of the module are:

  1. To provide an API that is close to Python's file I/O objects, and
  2. To provide an implementation based on TensorFlow's C++ FileSystem API.

The C++ FileSystem API supports multiple file system implementations, including local files, Google Cloud Storage (using a gs:// prefix, and HDFS (using an hdfs:// prefix). TensorFlow exports these as, so that you can use these implementations for saving and loading checkpoints, writing to TensorBoard logs, and accessing training data (among other uses). However, if all your files are local, you can use the regular Python file API without any problem.

Once you obtain a GFile object, you can use it in most ways as you would any Python's file object:

with open("/tmp/x", "w") as f:
with"/tmp/x") as f:

The difference is that you can specify URI schemes to use other filesystems (e.g., gs:// for GCS, s3:// for S3, etc.), if they are supported. Using file:// as an example, we have:

with"file:///tmp/x", "w") as f:

You can also read all lines of a file directly:

with"file:///tmp/x", "w") as f:
['asdf\n', 'qwer\n']

You can iterate over the lines:

with"file:///tmp/x", "w") as f: