CODEWITHSUNDEEP
pythoncnumpy
philE
philE

Reputation: 1703

Speed costs of calling Python function from C extension

I am writing a program that involves simulating evolution paths of a few different variables. The bulk of the program is in Python, but I am writing code for the simulation loops (~15k) as a C-extension to improve speed. I would, however, still like to take advantage of Numpy's random number generators (here). I know that I can call Python functions from my extension, but will that slow down the C loops, thus negating the purpose of writing the extension in the first place?

Upvotes: 3

Views: 1203

Answers (1)

sth
sth

Reputation: 229934

When you call a Python function from a C extension you get several parts of overhead:

  1. The C variables you want to pass to the python function need to be wrapped in Python objects. This costs about as much as it would if you would create the objects directly in Python.
  2. The Python function needs to be called. This will cost as much as a normal Python function call.
  3. The return value needs to be unwrapped for C and it's python value will be garbage collected. This also costs just about as much as a normal Python return value that goes out of scope.

To judge if it's worth it really depends on the amount of code that is done in C and how much python object manipulation you can avoid there. If you only have a single for loop in C that calls a python function it's probably not worth it. If you do a lot of data manipulation that can be done in C instead, you will save all that even if you call a Python function somewhere.

But the fastest way would probably to use the numpy C API and call the numpy functions directly from C. This would allow you to avoid most overhead by not having to wrap all the parameters into Python objects, whil still using numpy's functionality.

Upvotes: 2

Related Questions