CODEWITHSUNDEEP
ownerairflow
Max.H
Max.H

Reputation: 161

How should I run task with "owner" not shell owner in AIRFLOW

My task code is the following.

from airflow.models import DAG
from airflow.operators import BashOperator
from datetime import datetime, timedelta
rootdir = "/tmp/airflow"
default_args = {
    'owner': 'max',
    'depends_on_past': False,
    'start_date': datetime.now(),
    'email': ['[email protected]'],
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=5),
}
dag = DAG('test3', default_args=default_args,
                    schedule_interval='*/2 * * * *')
t1 = BashOperator(
    task_id='test3-task1',
    bash_command='date  >> {rootdir}/test3-task1.out'.format(rootdir=rootdir),
    owner='max',
    dag=dag)
t2 = BashOperator(
    task_id='test3-task2',
    bash_command='whoami',
    retries=3,
    owner='max',
    dag=dag)

Then I run the command "airflow test test3 test3-task2 2016-07-25" with 'airflow' user of linux. The result of output "whoami" is "airflow". But I hope that the output result is "owner" of task.

What is my wrong ?

Thanks

the following is the output result.

[2016-07-25 11:22:37,716] {bash_operator.py:64} INFO - Temporary script location :/tmp/airflowtmpoYNJE8//tmp/airflowtmpoYNJE8/test3-task2U1lpom

[2016-07-25 11:22:37,716] {bash_operator.py:65} INFO - Running command: whoami

[2016-07-25 11:22:37,722] {bash_operator.py:73} INFO - Output:

[2016-07-25 11:22:37,725] {bash_operator.py:77} INFO - airflow

[2016-07-25 11:22:37,725] {bash_operator.py:80} INFO - Command exited with return code 0

Upvotes: 0

Views: 1720

Answers (2)

Oleg
Oleg

Reputation: 21

You can use the "run_as_user" parameter under default_args

default_args = {
    'owner': 'max',
    'depends_on_past': False,
    'start_date': datetime.now(),
    'email': ['[email protected]'],
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=5),
    'run_as_user': 'max'
}

Upvotes: 1

highlycaffeinated
highlycaffeinated

Reputation: 19867

It doesn't look like what you're trying to do is supported. Looking at the source code for both the bash_operator and the BaseOperator, neither makes any attempt to change users before executing the task, unfortunately.

Upvotes: 0

Related Questions