CODEWITHSUNDEEP
pythonairflowairflow-scheduler
Jose Pato
Jose Pato

Reputation: 111

Airflow Hash "#" in day-of-week field not running appropriately

Hi we are having trouble scheduling task with a # config as the standard says, to configure for the first, second, third, fourth or fifth day of the week for a given month.

Hash (#) '#' is allowed for the day-of-week field, and must be followed by a number between one and five. It allows specifying constructs such as "the second Friday" of a given month.[19] For example, entering "5#3" in the day-of-week field corresponds to the third Friday of every month.

As is mention here https://en.wikipedia.org/wiki/Cron Here is my dag config... as you can se, this should run today, the second monday of every month, and it did't run. As this, I have for every day of the month and only on few occasion the task has run successfully. What I'm doing wrong?

with DAG(dag_id='dag_126_73618_3012xx1the2', default_args=args , concurrency=3, catchup=True, tags=['account_126', 'form'], schedule_interval='30 12 * * 1#2', start_date=datetime(2021 ,8, 1, 0, 0)) as dag:

Here is my complete config

from airflow import DAG from airflow.utils.dates import days_ago, datetime, timedelta from lkf_operator import LKFLogin, CreateAndAssignTask

args = { 'owner' : 'airflow', 'email' : ['[email protected]', '[email protected]'], 'email_on_failure' : True, 'retries' : 3, 'retry_delay' : timedelta(seconds=30), }

params = { 'username' : '[email protected]', 'api_key' : 'xxxxxx', }

with DAG(dag_id='dag_126_73618_3012xx1the2', default_args=args , concurrency=3, catchup=True, tags=['account_126', 'form'], schedule_interval='30 12 * * 1#2', start_date=datetime(2021 ,8, 1, 0, 0)) as dag:

do_run_lkf_login_73618_1 = LKFLogin(
    name ='LKF Login',
    task_id ='run_lkf_login_73618_1', 
    params = params 
)

do_run_segundo_lunes_del_mes_73618_2 = CreateAndAssignTask(
    name ='Segundo Lunes del Mes',
    task_id ='run_segundo_lunes_del_mes_73618_2', 
    params = {'form_id': 73618, 'assinge_user_id': 126, 'answers': {'61148df86f960aaaa3e4e445': 71925, '611492cc5a6316b13ee4e3f2':

'Segundo Lunes del Mes', '61148df86f960aaaa3e4e446': '{% $today %}', '61148df86f960aaaa3e4e447': 'pendiente'}}

)

do_run_lkf_login_73618_1.set_downstream([do_run_segundo_lunes_del_mes_73618_2])

What is stranger is that for some days it works and from other it dosen't and is the same code, except one or two strings

dags

Upvotes: 0

Views: 655

Answers (1)

Jarek Potiuk
Jarek Potiuk

Reputation: 20047

I am afraid Airflow does not implement the complete crontab with all the flavours. You can check https://crontab.guru/ - this nicely summarizes the crontab specification that is implemented by Airflow.

However, very soon in Airflow 2.2 this problem will be largely solved. See https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-39+Richer+scheduler_interval - in ~ month when Airflow 2.2 is out, you will be able to use much more flexible and easier to define schedule.

Upvotes: 0

Related Questions