CODEWITHSUNDEEP
pythonoptimizationcython
VictorNorman
VictorNorman

Reputation: 165

How to figure out why cython-izing code slows it down?

We have some code written in python that uses a few classes that are really just "structs" -- instances of these classes just have a bunch of fields in them and no methods. Example:

class ResProperties:
    def __init__(self):
        self.endDayUtilities = 0
        self.marginalUtilities = []
        self.held = 0
        self.idleResource = True
        self.experience = 0.0
        self.resSetAside = 0
        self.unitsGatheredToday = 0

Our main code uses a bunch of instances of this class.

To speed up the code, I thought I'd cython-ized this class:

cdef class ResProperties:

    cdef public float endDayUtilities
    cdef public list marginalUtilities
    cdef public int held
    cdef public int idleResource
    cdef public float experience
    cdef public int resSetAside
    cdef public int unitsGatheredToday

    def __init__(self):
        self.endDayUtilities = 0
        # etc: code just like above.

However, the result is that the code runs something like 25% slower now!

How do I figure out what is causing the code to run slower now?

Thanks.

Upvotes: 2

Views: 252

Answers (1)

kindall
kindall

Reputation: 184071

You converted these classes to Cython but are still using them from Python code?

Conversion of data from C to Python and back is going to incur overhead. For example, your endDayUtilities member is a C-style float. When you access it from Python, a float() object must be constructed before your Python code can do anything with it. The same thing has to happen in reverse when you assign to that attribute from Python.

Off the top of my head, I'd estimate the performance overhead of these data conversions at... oh, about 25%. :-)

You're not going to see a performance boost until you move some of your code that uses that data to Cython. Basically, the more you can stay in C-land, the better you'll do. Going back and forth will kill you.

As another, simpler approach, you might want to try Psyco or PyPy instead of Cython.

Upvotes: 5

Related Questions