CODEWITHSUNDEEP
airflow
Denis Kokorin
Denis Kokorin

Reputation: 895

Airflow var.json masking

I configured Airflow (v 2.2.2) DAG to read variables from secrets backend. Variables are used by BashOperator via jinja template.

The problem is that I can't make variable masking to work using var.json syntax.

The following example works for me and password is masked in Airflow logs:

BashOperator(
    task_id='my_id',
    bash_command="some-command --user={{ var.value.MY_USER }} -- password={{ var.json.MY_PASSWORD }}",
    ...
)

Now I want to store username and password in the same variable. Credentials are retrieved, but unfortunately password is not masked in logs.

BashOperator(
    task_id='my_id',
    bash_command="some-command --user={{ var.json.VARIABLE_NAME_SECRET.user }} -- password={{ var.json.VARIABLE_NAME_SECRET.password }}",
    ...
)

Is it possible to mask value in dictionary when variable is retrieved already json-deserialized?

Upvotes: 1

Views: 395

Answers (1)

Hussein Awala
Hussein Awala

Reputation: 5100

Maybe there is a bug in the masking method, instead, you can store your username and password in an airflow connection, then access them with the same method:

BashOperator(
    task_id='my_id',
    bash_command="some-command --user={{ conn.connection_id.login }} -- password={{ conn.connection_id.password }}",
    ...
)

Upvotes: 1

Related Questions