CODEWITHSUNDEEP
pythongoogle-bigqueryjinja2airflowgoogle-cloud-composer
Alex Lipkin
Alex Lipkin

Reputation: 21

Apache Airflow: Pass variable in jinja include

I am currently using following statement in Apache airflow

s10_test_load = bigquery.BigQueryInsertJobOperator(
    task_id="10_test_load",
    configuration={
            "query": {
                        "query": "{% include './scripts/10_test_load.sql' %}",                           
                        "useLegacySql": False,
            },                  
    },
)

This statement is working fine and is executing sql from the script file.

I would like dynamically use the file name from a variable like following

file_name='./scripts/10_test_load.sql'

s10_test_load = bigquery.BigQueryInsertJobOperator(
    task_id="10_test_load",
    configuration={
            "query": {
                        "query": "{% include file_name %}",                          
                        "useLegacySql": False,
            },                  
    },
)

How I can pass a python variable inside jinja include?

Upvotes: 2

Views: 2753

Answers (1)

itroulli
itroulli

Reputation: 2094

You could use f-string formating and add an extra pair of curly braces like this:

file_name='bq/queries/10_test_load.sql'

t2 = BigQueryInsertJobOperator(
   task_id="10_test_load",
    configuration={
        "query": {
            "query": f"{{% include '{file_name}' %}}",                          
            "useLegacySql": False,
        },                  
    },             
)

This gets rendered the desired way: enter image description here

Upvotes: 2

Related Questions