CODEWITHSUNDEEP
pytestairflow
adan11
adan11

Reputation: 787

Airflow - Pytest - fixture 'dag' not found

I am trying to write some integrity tests for my DAG's in airflow. I am currently testing for cyclic DAGs using the test_cycle function found in airflow.utils.dag_cycle_tester. My code is below:

import glob
import importlib.util
import os
import pytest
from airflow.models import DAG
from airflow.utils.dag_cycle_tester import test_cycle

DAG_PATH = os.path.join(
 os.path.dirname(__file__), "..", "..", "dags/**/*.py"
)
DAG_FILES = glob.glob(DAG_PATH, recursive=True)

print(DAG_FILES)

@pytest.mark.parametrize("dag_file", DAG_FILES)
def test_dag_integrity(dag_file):
    module_name, _ = os.path.splitext(dag_file)
    module_path = os.path.join(DAG_PATH, dag_file)
    mod_spec = importlib.util.spec_from_file_location(module_name, 
    module_path)
    module = importlib.util.module_from_spec(mod_spec)
    mod_spec.loader.exec_module(module)
    dag_objects = [var for var in vars(module).values() if isinstance(var, 
    DAG)]
    assert dag_objects
    for dag in dag_objects:
        test_cycle(dag)

For some reason when i run this using pytest tests/ , i get the below error:

___________ERROR at setup of test_cycle ___________

def test_cycle(dag):

E fixture 'dag' not found

available fixtures: anyio_backend, anyio_backend_name, anyio_backend_options, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, celery_config, celery_enable_logging, cov, doctest_namespace, monkeypatch, no_cover, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory use 'pytest --fixtures [testpath]' for help on them.

Any ideas on what causes this would be much appreciated. Thanks!

Upvotes: 2

Views: 970

Answers (1)

Jorrick Sleijster
Jorrick Sleijster

Reputation: 1156

Although the failure is actually in another function that what you are showing, I can explain what is happening.

Problem:
Pytest picks every function up starting with test_. Thereby as your function test_cycle starts with test_ it is also going to run that as an individual test. In the individual test it tries to find the fixture dag, which isn't present.

Therefore the solution is:
Rename the function to cycle_dag and that should fix it :)

Upvotes: 3

Related Questions