CODEWITHSUNDEEP
pythoncelery
Miguel_LZ
Miguel_LZ

Reputation: 11

Celery task not detecting error from subprocess

I have the following task with runs a python scripts using Popen (I already tried with check_output, call with the same result)

@app.task
def TaskName(filename):
    try:
       proc = subprocess.Popen(['python', filename],stdout=subprocess.PIPE, 
       stderr=subprocess.PIPE): 
       taskoutp, taskerror = proc.communicate()
       print('Output: ', taskoutp)
       print('Error: ', taskerror)
    Except:
       raise Exception('Task error: ', taskerror)

If I generate an error in the subprocess it is displayed in the worker cmd window as if it were a normal output/print, the task remains indefinetly with the status 'STARTED' even when I manually close the window opened by the Popen subprocess.

What can I do so the tasks not only prints the error but actually stops and changes its status to FAILURE?

Upvotes: 1

Views: 502

Answers (1)

Sowjanya R Bhat
Sowjanya R Bhat

Reputation: 1168

If you want to store results of your task , you could use this parameter result_backend or CELERY_RESULT_BACKEND depending on the version of celery you're using.

This parameter can be used for acknowledging a task only after it is processed :

task_acks_late
Default: Disabled.

Late ack means the task messages will be acknowledged after the task has been executed, not just before (the default behavior).

Complete list of Configuration options can be found here => https://docs.celeryproject.org/en/stable/userguide/configuration.html

Upvotes: 1

Related Questions