CODEWITHSUNDEEP
airflowairflow-scheduler
Felipe FB
Felipe FB

Reputation: 1342

Airflow doesn't recognize DAG scheduling

I'm trying to make weekly, monthly airflow schedules but it's not working. Could anyone report what may be happening? If I make weekly, monthly scheduling, it stays still, as if it were turned off. No error message, simply does not execute. I sent a code example to demonstrate how I'm scheduling ... is there any other way to do this scheduling?

import airflow
import os
import six
import time
from datetime import datetime, timedelta
from airflow import DAG
from airflow import AirflowException
from airflow.models import BaseOperator
from airflow.utils.decorators import apply_defaults
from airflow.operators.slack_operator import SlackAPIPostOperator

default_args = {
    'owner': 'bexs-data',
    'start_date': airflow.utils.dates.days_ago(0),
    'depends_on_past': False,
    'email': ['[email protected]'],
    'email_on_failure': False,
    'email_on_retry': False,
    'depends_on_past': False,
    # If a task fails, retry it once after waiting
    # at least 5 minutes
    'retries': 1,
    'retry_delay': timedelta(minutes=5),
    'on_failure_callback': slack_msg
}

dag = DAG(
    dag_id=nm_dag,
    default_args=default_args,
    schedule_interval='51 18 * * 4', 
    dagrun_timeout=timedelta(minutes=60)
)

Upvotes: 2

Views: 9196

Answers (1)

dlamblin
dlamblin

Reputation: 45321

There is documentation around not doing the following: 'start_date': airflow.utils.dates.days_ago(0), because in this way there will never be a 1 week interval that has passed since your start date, which means the first interval doesn't close, and the first run doesn't get scheduled.

Suggestion: pick a fixed day on a day-of-the-week 4 (Thursday?) from last week as your start_date.

Airflow will accept a datetime or a string. Use airflow.utils.timezone's datetime for a non-UTC schedule. For example, either:

default_args = {
  'owner': 'your-unix-user-id-or-ldap-etc',
  'start_date': '2018-1-1',
...
}

or

from airflow.utils.timezone import datetime

default_args = {
  'owner': 'your-unix-user-id-or-ldap-etc',
  'start_date': datetime(2018, 1, 1),
...
}

Upvotes: 5

Related Questions