CODEWITHSUNDEEP
pythonperformancemodule
orlp
orlp

Reputation: 117701

Is there any way to speed up an import?

I have a CLI application that requires sympy. The speed of the CLI application matters - it's used a lot in a user feedback loop.

However, simply doing import sympy takes a full second. This gets incredibly annoying in a tight feedback loop. Is there anyway to 'preload' or optimize a module when a script is run again without a change to the module?

Upvotes: 8

Views: 6966

Answers (3)

jrwren
jrwren

Reputation: 17908

I took a look at what happens when you run import sympy, and it imports all of sympy.

https://github.com/sympy/sympy/blob/master/sympy/__init__.py

If you are only using certain parts of sympy, then only import those parts that you need.

It would be nice if you could do this:

import sympy.sets

But (as you point out) that imports sympy and then sets.

One solution is to write your own importer. You can do this with the help of the imp module.

import imp
sets = imp.load_module("sets", open("sympy/sets/__init__.py"), "sympy/sets/__init__.py", ('.py', 'U', 1))

But, even that may not optimize enough. Taking a look at sympy/sets/__init__.py I see that it does this:

from .sets import (Set, Interval, Union, EmptySet, FiniteSet, ProductSet,
    Intersection, imageset, Complement, SymmetricDifference)
from .fancysets import TransformationSet, ImageSet, Range, ComplexRegion
from .contains import Contains
from .conditionset import ConditionSet

Maybe you can import only the sets module from simpy sets namespace?

import imp
sets = imp.load_module("sets", open("sympy/sets/set.py") "sympy/sets/set.py", ('.py', 'U', 1))

Upvotes: 2

dot.Py
dot.Py

Reputation: 5157

You should test if importing only the modules that you are using in the code improves the loading time.

IE:

from sympy import mod1, mod2, mod3

vs

import sympy

You should read these previous questions:

Python import X or from X import Y? (performance)

improving speed of Python module import

'import module' vs. 'from module import function'

Upvotes: 0

Alfe
Alfe

Reputation: 59436

Obviously sympy does a lot when being imported. It could be initialization of internal data structures or similar. You could call this a flaw in the design of the sympy library.

Your only choice in this case would be to avoid redoing this initialization.

I assume that you find this behavior annoying because you intend to do it often. I propose to avoid doing it often. A way to achieve this could be to create a server which is started just once, imports sympy upon its startup, and then offers a service (via interprocess communication) which allows you to do whatever you want to do with sympy.

If this could be an option for you, I could elaborate on how to do this.

Upvotes: 2

Related Questions