CODEWITHSUNDEEP
djangopython-3.4python-multithreading
Filip Kilibarda
Filip Kilibarda

Reputation: 2668

ThreadPoolExecutor fails when run with manage.py

# test.py
# python 3.4.5

import time
from concurrent.futures import ThreadPoolExecutor

def a():
    time.sleep(1)
    print("success")

executor = ThreadPoolExecutor(1)
executor.submit(a).result()

The above snippet works when run like

$ python test.py 
success

But fails when run like

$ python manage.py shell < test.py 
Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/var/www/cgi-bin/tracking/lib64/python3.4/site-packages/django/core/management/__init__.py", line 363, in execute_from_command_line
    utility.execute()
  File "/var/www/cgi-bin/tracking/lib64/python3.4/site-packages/django/core/management/__init__.py", line 355, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/var/www/cgi-bin/tracking/lib64/python3.4/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/var/www/cgi-bin/tracking/lib64/python3.4/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/var/www/cgi-bin/tracking/lib64/python3.4/site-packages/django/core/management/commands/shell.py", line 101, in handle
    exec(sys.stdin.read())
  File "<string>", line 11, in <module>
  File "/usr/lib64/python3.4/concurrent/futures/_base.py", line 395, in result
    return self.__get_result()
  File "/usr/lib64/python3.4/concurrent/futures/_base.py", line 354, in __get_result
    raise self._exception
  File "/usr/lib64/python3.4/concurrent/futures/thread.py", line 54, in run
    result = self.fn(*self.args, **self.kwargs)
  File "<string>", line 7, in a
NameError: name 'time' is not defined

Which is really strange to me. What is it about running the script with the manage.py shell command that results in the time module being undefined in the function a?

Upvotes: 1

Views: 575

Answers (2)

soloidx
soloidx

Reputation: 749

Checking in the Django implementation (django/core/management/commands/shell.py line 83):

# Execute stdin if it has anything to read and exit.
# Not supported on Windows due to select.select() limitations.
if sys.platform != 'win32' and select.select([sys.stdin], [], [], 0)[0]:
    exec(sys.stdin.read())
    return

The developers did not add a globals() scope in the exec() method, that means you are importing time and ThreadPoolExecutor in the 'locals()' dictionary of the handle() scope (in shell.py) but after, when you try to use inside a() it tries to search in the locals() dictionary of the "a" scope and in the globals() dictionary so it throws an import error, you can see an example in this snippet:

command = """
import time
def b():
    time.sleep(1)
b()
"""
def a():
    exec(command)
a()

and try to change exec(command) by exec(command, globals())

Upvotes: 1

Lemayzeur
Lemayzeur

Reputation: 8525

I think it's not working because you did not set the environment variable DJANGO_SETTING_MODULE to your settings, and call django.setup() or set the path to sys.path.append('path/')
(NOT SURE)

But these 2 options can work like a charm:

Either you import the module time inside the function:

from concurrent.futures import ThreadPoolExecutor

def a():
    import time
    time.sleep(1)
    print("success")

executor = ThreadPoolExecutor(1)
executor.submit(a).result()

or just import time at the beginning like you did, and use the module as a global one:

from concurrent.futures import ThreadPoolExecutor
import time

def a():
    global time
    time.sleep(1)
    print("success")

executor = ThreadPoolExecutor(1)
executor.submit(a).result()

Upvotes: 1

Related Questions