When you’re ready to move your models from research to production, use TFX to create and manage a production pipeline.
When you’re ready to go beyond training a single model, or ready to put your amazing model to work and move it to production, TFX is there to help you build a complete ML pipeline.
A TFX pipeline is a sequence of components that implement an ML pipeline which is specifically designed for scalable, high-performance machine learning tasks. That includes modeling, training, serving inference, and managing deployments to online, native mobile, and JavaScript targets. To learn more, read our TFX User Guide.
The pipeline components are built using TFX libraries which can also be used individually. Below is an overview of those underlying libraries.
TensorFlow Data Validation (TFDV) helps developers understand, validate, and monitor their ML data at scale. TFDV is used to analyze and validate petabytes of data at Google every day, and has a proven track record in helping TFX users maintain the health of their ML pipelines.
When applying machine learning to real world datasets, a lot of effort is required to preprocess data into a suitable format. This includes converting between formats, tokenizing and stemming text and forming vocabularies, and performing a variety of numerical operations such as normalization. You can do it all with tf.Transform.
TensorFlow Model Analysis (TFMA) enables developers to compute and visualize evaluation metrics for their models. Before deploying any machine learning (ML) model, ML developers need to evaluate model performance to ensure that it meets specific quality thresholds and behaves as expected for all relevant slices of data. For example, a model may have an acceptable AUC over the entire eval dataset, but underperform on specific slices. TFMA gives developers the tools to create a deep understanding of their model performance.
Machine Learning (ML) serving systems need to support model versioning (for model updates with a rollback option) and multiple models (for experimentation via A/B testing), while ensuring that concurrent models achieve high throughput on hardware accelerators (GPUs and TPUs) with low latency. TensorFlow Serving has proven performance handling tens of millions of inferences per second at Google.
Explore step-by-step tutorials to help you with your projects.
In this example we'll be processing a widely used dataset containing census data, and training a model to do classification. Along the way we'll be transforming the data using tf.Transform.
This guide trains a neural network model to classify images of clothing, like sneakers and shirts, saves the trained model, and then serves it with TensorFlow Serving. The focus is on TensorFlow Serving, rather than the modeling and training in TensorFlow.
This example colab notebook illustrates how TensorFlow Data Validation (TFDV) can be used to investigate and visualize your dataset. That includes looking at descriptive statistics, inferring a schema, checking for and fixing anomalies, and checking for drift and skew in our dataset.
What is TensorFlow Extended and how can it help with your ML pipeline? Learn about the main concepts of TFX, how it all fits together, and how Google leverages it for their business.
Learn from Google’s experience in applying Beam for ML pipelines, including how TFX uses Beam and why Beam was chosen.
Learn how to create a complete TFX ML pipeline using several ready-made components, configure them for many typical ML use cases with a high level Python API, and execute them with an orchestration system of your choice such as Apache Airflow or Kubeflow.
