CODEWITHSUNDEEP
pythonpython-2.7timeraspberry-piscrapy
Praind
Praind

Reputation: 1581

Importing function from module takes very long

When i import my self written function of a module in a python script it takes about 6 seconds to load. The function contains only about 50 lines of code but that shouldn't even matter since it has not been executed yet right?

This is the script that loads the function:

#/usr/bin/env python

import time
print(time.clock())
from os import chdir
print(time.clock())
from os.path import abspath, dirname
print(time.clock())
from Project.controllers.SpiderController import RunSpider
print(time.clock())

And the output is as follows:

0.193569
0.194114
0.194458
6.315348

I also tried to import the whole module but the result was the same.

What could be the cause of that?

Some side notes:

Upvotes: 7

Views: 4394

Answers (1)

willeM_ Van Onsem
willeM_ Van Onsem

Reputation: 477666

but that shouldn't even matter since it has not been executed yet right?

The code of the function itself is not executed, but the code in the file is executed. This is logical, since that file might contain decorators, library calls, inner constants, etc. It is even possible that the function is build (so that an algorithm constructs the function).

With from <module> import <item> you do a almost normal import, but you create only one reference to an item in that package.

So it can take a long time if there is a program written in the module (that is not scoped in an if __name__ == '__main__':) or when you import a large amount of additional libraries.

It is for instance possible to construct a function like:

def foo(x):
    return x + 5

def bar(y):
    return y * 2

def qux(x):
    return foo(bar(x))

If you then run from module import qux, then it will first have to define foo and bar, since qux depends on these.

Furthermore although the code itself is not executed, the interpreter will analyze the function: it will transform the source code into a syntax tree and do some analysis (which variables are local, etc.).

Finally note that a package typically has an __init__.py file, that initializes the package. That file is also executed and can take considerable time as well. For instance some packages that have a database connection, will already set up the connection to that database, and it can take some time before the database responds to the connection.

Upvotes: 4

Related Questions