CODEWITHSUNDEEP
pythoncythonpython-c-api
Lei Wang
Lei Wang

Reputation: 271

Is Python C module extension version incompatible?

I compile a python c module extension in Python 3.6. It works well in Python 3.6 environment but does not work in Python 3.7 and 3.8 environments, it gets error cannot import name 'cygrpc'.

I wondering is this the expected behavior? If yes, what is the reason it.

Upvotes: 4

Views: 1096

Answers (1)

sytech
sytech

Reputation: 40921

It depends on how the extension is compiled and what Python C API interfaces it uses. But yes, C extensions generally have to be compiled to target a specific version of Python. In absence of knowledge of how the extension module is created, I would assume it needs to be recompiled.

Usually, the Python C API changes every release. If that change breaks something in your extension, naturally, you'll have to update your extension for the new API interfaces. However, even if your changes are source-compatible (no source code changes are needed to support the new Python version) it does not mean that the compiled binaries are necessarily compatible. This is where the Binary interface (ABI) comes into play.

See also: Stable Application Binary Interface which describes the reason behind this:

The reason is primarily the evolution of struct definitions, where addition of a new field, or changing the type of a field, might not break the API, but can break the ABI. As a consequence, extension modules need to be recompiled for every Python release [...]

There is also a mention of a limited case in which binaries are compatible across versions:

Since Python 3.2, a subset of the API has been declared to guarantee a stable ABI. Extension modules wishing to use this API (called “limited API”) need to define Py_LIMITED_API. A number of interpreter details then become hidden from the extension module; in return, a module is built that works on any 3.x version (x>=2) without recompilation.

So, if you only use parts of the stable API and define Py_LIMITED_API, then the compiled module may be compatible across Python versions without recompilation.

Upvotes: 5

Related Questions