RSVP for your your local TensorFlow Everywhere event today!


Airflow-specific TFX Component.

This class wrap a component run into its own PythonOperator in Airflow.

parent_dag An AirflowPipeline instance as the pipeline DAG.
component An instance of base_node.BaseNode that holds all properties of a logical component.
component_launcher_class The class of the launcher to launch the component.
pipeline_info An instance of data_types.PipelineInfo that holds pipeline properties.
enable_cache Whether or not cache is enabled for this component run.
metadata_connection_config A config proto for metadata connection.
beam_pipeline_args Pipeline arguments for Beam powered Components.
additional_pipeline_args Additional pipeline args.
component_config Component config to launch the component.

dag Returns the Operator's DAG if set, otherwise raises an error
dag_id Returns dag id if it has one or an adhoc + owner
deps Returns the list of dependencies for the operator. These differ from execution context dependencies in that they are specific to tasks and can be extended/overridden by subclasses.
downstream_list @property: list of tasks directly downstream
downstream_task_ids @property: list of ids of tasks directly downstream


priority_weight_total Total priority weight for the task. It might include all upstream or downstream tasks. depending on the weight rule.

  • WeightRule.ABSOLUTE - only own weight
  • WeightRule.DOWNSTREAM - adds priority weight of all downstream tasks
  • WeightRule.UPSTREAM - adds priority weight of all upstream tasks
schedule_interval The schedule interval of the DAG always wins over individual tasks so that tasks within a DAG always line up. The task still needs a schedule_interval as it may not be attached to a DAG.
task_type @property: type of the task
upstream_list @property: list of tasks directly upstream
upstream_task_ids @property: list of ids of tasks directly upstream



Adds only new items to item set


Clears the state of task instances associated with the task, following the parameters specified.


Performs dry run for the operator - just render template fields.


This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.



Get the direct relative ids to the current task, upstream or downstream.


Get the direct relatives to the current task, upstream or downstream.

For an operator, gets the URL that the external links specified in extra_links should point to.

:raise ValueError: The error message of a ValueError will be passed on through to the fronted to show up as a tooltip on the disabled link :param dttm: The datetime parsed execution date for the URL being searched for :param link_name: The name of the link we're looking for the URL for. Should be one of the options specified in extra_links :return: A URL


Get a flat list of relatives' ids, either upstream or downstream.


Get a flat list of relatives, either upstream or downstream.


Stringified DAGs and operators contain exactly these fields.


Get a set of task instance related to this task for a specific date range.


Fetch a Jinja template environment from the DAG or instantiate empty environment if no DAG.


Returns True if the Operator has been assigned to a DAG.


Override this method to cleanup subprocesses when a task instance gets killed. Any use of the threading, subprocess or multiprocessing module within an operator needs to be cleaned up or it will leave ghost processes behind.


This hook is triggered right after self.execute() is called. It is passed the execution context and any results returned by the operator.


This hook is triggered right before self.execute() is called.


Hook that is triggered after the templated fields get replaced by their content. If you need your operator to alter the content of the file before the template is rendered, it should override this method to do so.


Render a templated string. The content can be a collection holding multiple templated strings and will be templated recursively.

:param content: Content to template. Only strings can be templated (may be inside collection). :type content: Any :param context: Dict with values to apply on templated content :type context: dict :param jinja_env: Jinja environment. Can be provided to avoid re-creating Jinja environments during recursion. :type jinja_env: jinja2.Environment :param seen_oids: template fields already rendered (to avoid RecursionError on circular dependencies) :type seen_oids: set :return: Templated content


Template all attributes listed in template_fields. Note this operation is irreversible.

:param context: Dict with values to apply on content :type context: dict :param jinja_env: Jinja environment :type jinja_env: jinja2.Environment



Run a set of task instances for a date range.


Set a task or a task list to be directly downstream from the current task.


Set a task or a task list to be directly upstream from the current task.


See TaskInstance.xcom_pull()


See TaskInstance.xcom_push()


Return self==value.


Return a >= b. Computed by @total_ordering from (not a < b).


Return a > b. Computed by @total_ordering from (not a < b) and (a != b).


Return a <= b. Computed by @total_ordering from (a < b) or (a == b).


Return self<value.


Return self!=value.




operator_extra_links ()
pool ''



ui_color '#ffefeb'
ui_fgcolor '#000'