TensorFlow Extended (TFX) is an end-to-end platform for deploying production ML pipelines
When you’re ready to move your models from research to production, use TFX to create and manage a production pipeline.
How it works
When you’re ready to go beyond training a single model, or ready to put your amazing model to work and move it to production, TFX is there to help you build a complete ML pipeline.
A TFX pipeline is a sequence of components that implement an ML pipeline which is specifically designed for scalable, high-performance machine learning tasks. That includes modeling, training, serving inference, and managing deployments to online, native mobile, and JavaScript targets. To learn more, read our TFX User Guide.
The pipeline components are built using TFX libraries which can also be used individually. Below is an overview of those underlying libraries.
TensorFlow Data Validation
TensorFlow Data Validation (TFDV) helps developers understand, validate, and monitor their ML data at scale. TFDV is used to analyze and validate petabytes of data at Google every day, and has a proven track record in helping TFX users maintain the health of their ML pipelines.
TensorFlow Transform
When applying machine learning to real world datasets, a lot of effort is required to preprocess data into a suitable format. This includes converting between formats, tokenizing and stemming text and forming vocabularies, and performing a variety of numerical operations such as normalization. You can do it all with tf.Transform.
TensorFlow Model Analysis
TensorFlow Model Analysis (TFMA) enables developers to compute and visualize evaluation metrics for their models. Before deploying any machine learning (ML) model, ML developers need to evaluate model performance to ensure that it meets specific quality thresholds and behaves as expected for all relevant slices of data. For example, a model may have an acceptable AUC over the entire eval dataset, but underperform on specific slices. TFMA gives developers the tools to create a deep understanding of their model performance.
TensorFlow Serving
Machine Learning (ML) serving systems need to support model versioning (for model updates with a rollback option) and multiple models (for experimentation via A/B testing), while ensuring that concurrent models achieve high throughput on hardware accelerators (GPUs and TPUs) with low latency. TensorFlow Serving has proven performance handling tens of millions of inferences per second at Google.
Solutions to common problems
Explore step-by-step tutorials to help you with your projects.
In this example we'll be processing a widely used dataset containing census data, and training a model to do classification. Along the way we'll be transforming the data using tf.Transform.
This guide trains a neural network model to classify images of clothing, like sneakers and shirts, saves the trained model, and then serves it with TensorFlow Serving. The focus is on TensorFlow Serving, rather than the modeling and training in TensorFlow.
This example colab notebook illustrates how TensorFlow Data Validation (TFDV) can be used to investigate and visualize your dataset. That includes looking at descriptive statistics, inferring a schema, checking for and fixing anomalies, and checking for drift and skew in our dataset.
News & announcements
Check out our blog and YouTube playlist for additional TFX content,
and subscribe to our monthly TensorFlow newsletter to get the
latest announcements sent directly to your inbox.
A new (experimental) Colab notebook environment for iterative development of TFX components & pipelines, using InteractiveContext.
Watch this keynote presentation from TensorFlow World 2019 and learn about how TFX was designed to meet the needs of production ML pipelines.
Learn about the orchestration and components which make up TFX ML Pipelines, and watch a complete end-to-end demo!
Learn how you can create a semi-custom TFX component simply by replacing the Executor for an existing component.