CODEWITHSUNDEEP
airflowdbt
DreamingFighter
DreamingFighter

Reputation: 41

How to reduce parser process when running dbt on Airflow

I am running dbt version 1.0.4 on Airflow. My ETL pipeline is running fine.

But I notice that dbt takes a long time to parse files every time it run on Airflow. Some lines in the log: [2022-06-14 05:06:54,523] {subprocess.py:78} INFO - 05:06:54.506639 [debug] [MainThread]: Parsing macros/common/helpers/dropif.sql

[2022-06-14 05:06:55,826] {subprocess.py:78} INFO - 05:06:55.809703 [debug] [MainThread]: 1605: jinja rendering because of STATIC_PARSER flag. file: mart/domain_1/model_1.sql

Since my project is quite big, it takes a looooong time to actual run the query.

So, is there any way for me to bypass the parsing?

Upvotes: 0

Views: 799

Answers (1)

Bảo Nguyễn Cao
Bảo Nguyễn Cao

Reputation: 1

I add --no-static-parser. But I want to reduce more parsing time.

Upvotes: 0

Related Questions