CODEWITHSUNDEEP
pythonairflow
David Masip
David Masip

Reputation: 2491

Airflow templating issue

I have a simple Airflow dag:

from datetime import timedelta


from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.utils.dates import days_ago

default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': days_ago(2),
    'email': ['[email protected]'],
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=5),
}
dag = DAG(
    'tutorial',
    default_args=default_args,
    description='A simple tutorial DAG',
    schedule_interval=timedelta(days=1),
)

templated_command = """
echo "{{ ds }}"
echo "{{ macros.ds_add(ds, 7)}}"
echo "Param: {{ params.my_param }}"
"""

t3 = BashOperator(
    task_id='templated',
    depends_on_past=False,
    bash_command=templated_command,
    params={'my_param': templated_command},
    dag=dag,
)

I would like to be able to pass a template to params. However, this doesn't seem to work the way I am doing it, since the rendered template outputs:

echo "2020-09-17"
echo "2020-09-24"
echo "Param: 
echo "{{ ds }}"
echo "{{ macros.ds_add(ds, 7)}}"
echo "Param: {{ params.my_param }}"
"

It is effectively rendering the bash_command but not the params. Is there a way to solve this and render the params properly?

Upvotes: 1

Views: 414

Answers (1)

A.B
A.B

Reputation: 20445

You are passing whole template as value to my_param in Params, which is a string, for your actual template it does renders every (ds, macros.ds_add(ds, 7) and even param.my_param) right

whats happening is, you get echo "{{ ds }}" echo "{{ macros.ds_add(ds, 7)}}" echo "Param: {{ params.my_param }}" " displayed again as string because it is the value of my_param

I dont know why you want to pass whole temple as string to my_param?

Upvotes: 1

Related Questions