CODEWITHSUNDEEP
pythonpython-3.ximportmodulepython-c-api
Peter Varo
Peter Varo

Reputation: 12150

How to create submodules with the Python C API?

I want to create the following module and object structure:

main_module
|
`--sub_module
   |
   +--ObjectOne
   |
   `--sub_sub_module
      |
      `--ObjectTwo

When I only had main_module.sub_module everything worked as it should, as I created an empty __init__.py inside the main_module folder (since main_module does not have any objects at the moment), and I placed sub_module.so next to it.

However, when I tried to create two separate c modules to make main_module.sub_module.sub_sub_module work (that is sub_module.so and sub_sub_module.so) and then I added two __init__.pys which both of them imported their extension modules relative to them, then I got into ImportErrors (as wrappers around the undefined symbol error "raised" from the shared libraries), because sub_sub_module needs a few of the C-level definitions from sub_module. Which makes me think, it would be way more easier, to create a single shared library which somehow creates "virtual" modules inside it, instead of the unnecessary library linking..

So my question is: Is it possible? If so, how? Or, is there a better way to achieve what I'm looking for?

Upvotes: 0

Views: 1002

Answers (1)

tynn
tynn

Reputation: 39843

I think I got your problem now. For a long and detailed answer you should read Providing a C API for an Extension Module.

Portability therefore requires not to make any assumptions about symbol visibility. This means that all symbols in extension modules should be declared static, except for the module’s initialization function, in order to avoid name clashes with other extension modules [...]. And it means that symbols that should be accessible from other extension modules must be exported in a different way.

To achieve this you should use Capsules. For these every function you want to use in another module should be stored inside a void* array. Then you create a capsule with PyCapsule_New(void*, const char*, PyCapsule_Destructor) using the array as first argument. This object must be added to your sub_module. Then you can call PyCapsule_Import(const char*, int) in your sub_sub_module to import the array and access the functions you need.

Upvotes: 2

Related Questions