CODEWITHSUNDEEP
pythondockerairflow
sometimesiwritecode
sometimesiwritecode

Reputation: 3213

How to pass an argument to a docker container using airflow's DockerOperator

I have a docker image that runs a python script.

The Dockerfile looks like:

FROM python:3.6
ADD . /


RUN pip install -r requirements.txt
ENTRYPOINT ["python3.6", "./main.py"]

As I understand it, using ENTRYPOINT instead of CMD to run the python script allows parameters to be passed to main.py that can be parsed with argparse within main.py

I have succesfully passed in arguments and retrieved and used them in main.py when running the docker container (named param-test) with the following command:

docker run -it param-test "cats"

I now wish to do the equivalent of the above 'docker run' command via Apache Airflow.

I have the following code that creates a DockerOperator within airflow succesfully but i am unclear how to pass the argument in this code configuration like i do in the above docker run command:

DockerOperator(dag=dag,
            task_id='my_task_name',
            image='my_docker_container_ecr_url'
)

How do i pass the param "cats" to the DockerOperator in an equivalent way to running:

docker run -it param-test "cats"

so that my python script cat use the cats param?

EDIT: python code in main.py that uses the cats param passed in from docker:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('city')
args = parser.parse_args()
print(args.cat_var)

Upvotes: 1

Views: 2812

Answers (1)

Tarun Lalwani
Tarun Lalwani

Reputation: 146520

Below should work

DockerOperator(dag=dag,
            task_id='my_task_name',
            image='my_docker_container_ecr_url',
            command='cats'
)

Upvotes: 1

Related Questions