CODEWITHSUNDEEP
airflowdbt
Serda Shehu
Serda Shehu

Reputation: 85

No such file or directory - Airflow

I have my airflow project with the structure as below

airflow
    |
    |----dags
    |      |----dag.py
    |
    |----dbt-redshift
             |----models
                   |----model.sql

I have included the dbt-redshift directory in the volumes section as

volumes:
  -./dbt-redshift:/opt/airflow/dbt-redshift

And I'm trying to run the dbt inside the DAG using a bash operator

 dbt_task = BashOperator(task_id='dbt', bash_command="cd ~/dbt-redshift && dbt run", dag=dag)

But when i execute the DAG i get the error

cd: /home/***/dbt-redshift no such file or directory

I'm not sure I understand how these directories are located inside the airflow project.

Upvotes: 0

Views: 2164

Answers (1)

Daniel T
Daniel T

Reputation: 621

You are mounting the volume inside the container to /opt/airflow/dbt-redshift, but the BashOperator references ~/dbt-redshit with ~ resolving to /home/airflow.

(Assuming you are using the apache/airflow image)

Either change the command used by the BashOperator to reference /opt/airflow/dbt-redshift or change the volume to mount to the home directory.

Upvotes: 1

Related Questions