CODEWITHSUNDEEP
pythonnumpycython
Silmathoron
Silmathoron

Reputation: 2001

Cython: dynamic "cdef extern from xxx"

I am trying to get direct access to numpy functions random/randomkit.h to use random generators in a multithreaded application with cython (i.e. without the gil).

To that end I am trying to access the header file from numpy's folder using the following code:

import os
import numpy

str_rkdir = os.path.dirname(os.path.abspath(numpy.__file__))
str_randomkit_h = str_rkdir + "/random/randomkit.h"

cdef extern from str_randomkit_h:
   ctypedef struct rk_state
   cdef unsigned long rk_random(rk_state * state) nogil
   cdef signed long rk_gauss(rk_state * state) nogil
   cdef void rk_seed(unsigned long seed, rk_state * state) nogil

However, this does not work with pyximport because cython complains about the str_randomkit_h string (I guess it doesn't interpret the python code before compiling)... is there a simple way around this?

As a dirty workaround, I'm dynamically generating the .pxd before the cython compilation but this does not really feel nice so I was wondering whether there was a better way to do it.

Upvotes: 1

Views: 1172

Answers (1)

Dimitris Fasarakis Hilliard
Dimitris Fasarakis Hilliard

Reputation: 160687

Dynamically you can't get it work that way because, as you also stated, there is no interpreting going on, only compiling. DEFs won't help either because the functions that can be used in the preprocessing stage are limited. I don't believe there is some other sneaky way to do this; you need to use pyximport or a setup.py script.

With pyximport you can supply appropriate arguments to the install function. The directory/ies containing the include file(s) can be given as a value in the setup_args dictionary with the key 'include_dirs':

>>> from numpy import random as rnd    
>>> from pyximport import install
>>> install(setup_args={'include_dirs':rnd.__path__[0] + '/'})

Of course, doing this implies you also change your .pxd file to supply the header name as a string literal:

cdef extern from "randomkit.h": 
    ...

Alternatively, using a setup script is pretty straightforward. Again, you just supply the include_dirs after computing them in some fashion.

Upvotes: 1

Related Questions