CODEWITHSUNDEEP
pythonairflowjinja2keyword-argumentf-string
Sara
Sara

Reputation: 43

F-string with jinja templating in airflow to pass dynamic values to op_kwargs

I am trying to pull values using xcom_pull in airflow dynamically

The below mentioned formatting doesn't work for me when I piece together jinja templating with f-strings in op_kwargs. Appreciate if anyone can help me here.

op_kwargs={'names':"{{ ti.xcom_pull(key = '" + f'name{i+1}' + ", task_ids='places' ) }}"}

Upvotes: 2

Views: 3341

Answers (1)

Elad Kalif
Elad Kalif

Reputation: 15931

Using fstring require to set proper number of brackets for Jinja. You can do:

op_kwargs={'names': f"{{{{ ti.xcom_pull(key='name{i+1}', task_ids='places') }}}}"}

Example (This is just a minimal example for your parameters to clarify how this works):

from datetime import datetime
from airflow import DAG
from airflow.operators.python import PythonOperator

default_args = {
    'owner': 'airflow',
    'start_date': datetime(2017, 2, 1)
}


def func1(ti):
    ti.xcom_push(key="name2", value="helloworld")


def func2(names):
    print(names)


with DAG('fstring_dag', default_args=default_args, catchup=False, schedule=None):

    a = PythonOperator(
        task_id='places',
        python_callable=func1,
    )

    i = 1

    b = PythonOperator(
        task_id='places2',
        python_callable=func2,
        op_kwargs={'names': f"{{{{ ti.xcom_pull(key='name{i+1}', task_ids='places') }}}}"}
    )
    a >> b

Render tab:

enter image description here

Log:

enter image description here

Upvotes: 3

Related Questions