Reputation: 4323
In this example, I see that the EmrCreateJobFlowOperator receives the aws/emr connections that were setup in Airflow UI:
cluster_creator = EmrCreateJobFlowOperator(
task_id='create_job_flow',
job_flow_overrides=JOB_FLOW_OVERRIDES,
aws_conn_id='aws_default',
emr_conn_id='emr_conn_id',
dag=dag)
In Airflow UI, in the tabs of connections, how can I add my AWS credentials so the DAG can pick them up? I don't see any type of connections for AWS. Any idea?
Upvotes: 2
Views: 4209
Reputation: 1111
The aws_default
picks up credentials from environment variables or ~/.aws/credentials
. You can choose your deployment mode as decide where you want to put the secret.
However if you want to add a connection string via UI, you can go to Admin -> Connections and edit the keys there.
Upvotes: 3