CODEWITHSUNDEEP
python-3.xgoogle-bigqueryairflowairflow-2.x
Kulasangar
Kulasangar

Reputation: 9454

ERROR - Exception rendering Jinja template for task from BigQueryExecuteQueryOperator in Airflow

I've been getting this following issue, for the last couple of weeks out of no where.

ERROR - Exception rendering Jinja template for task 'sometask', field 'sql'

jinja2.exceptions.TemplateNotFound: sql/test.sql

I have a task to execute a piece of sql through BigQueryExecuteQueryOperator. This is how my task looks like

tt = BigQueryExecuteQueryOperator(
        task_id="test1",
        sql=const.sqlfolder + "test.sql",
        destination_dataset_table="target_table",
    )

I also do have a user_defined_macros which i'm referring to in the SQL file.

user_defined_macros={
        "test_function": const.TEST_FUNCTION
}

and this is how i'm referring to it in the SQL file:

SELECT
    {{ test_function }} as test,
    *
FROM
    sometable

Airflow version that I'm using is 2.4.3. I did go through some of the similar questions, but those seem not related to this at all. One was for BashOperator where we have to leave a space at the end of the path of the file or try using the variable through Params rather than macros.

This issue happens once in a while during daily runs.

Any help is really appreciated, if anyone has gone through the same.

Upvotes: 1

Views: 1745

Answers (1)

Scott C
Scott C

Reputation: 1

I got this error and it was because I didn't define the template_searchpath parameter in my dag definition. That parameter tells the dag where to locate your sql file(s).

Upvotes: 0

Related Questions