EmrAddStepsOperator(job_flow_id=None, job_flow_name=None, cluster_states=None, aws_conn_id='aws_default', steps=None, *args, **kwargs)[source]¶
An operator that adds steps to an existing EMR job_flow.
job_flow_id (str) – id of the JobFlow to add steps to. (templated)
job_flow_name (str) – name of the JobFlow to add steps to. Use as an alternative to passing job_flow_id. will search for id of JobFlow with matching name in one of the states in param cluster_states. Exactly one cluster like this should exist or will fail. (templated)
cluster_states (list) – Acceptable cluster states when searching for JobFlow id by job_flow_name. (templated)
aws_conn_id (str) – aws connection to uses
steps (list) – boto3 style steps to be added to the jobflow. (templated)
do_xcom_push (bool) – if True, job_flow_id is pushed to XCom with key job_flow_id.