Kubernetes

Kubernetes Executor

The Kubernetes Executor allows you to run tasks on Kubernetes as Pods.

Kubernetes Operator

The KubernetesPodOperator allows you to create Pods on Kubernetes.

Pod Mutation Hook

Your local Airflow settings file can define a pod_mutation_hook function that has the ability to mutate pod objects before sending them to the Kubernetes client for scheduling. It receives a single argument as a reference to pod objects, and is expected to alter its attributes.

This could be used, for instance, to add sidecar or init containers to every worker pod launched by KubernetesExecutor or KubernetesPodOperator.

def pod_mutation_hook(pod: Pod):
  pod.annotations['airflow.apache.org/launched-by'] = 'Tests'