CODEWITHSUNDEEP
pythoncoptimizationcpython
Charitoo
Charitoo

Reputation: 1872

How to know what part of CPython is implemented in C?

Parts of the standard library of CPython that were written in C are faster than the parts that implemented in Python. To optimize the code, it would be good if one uses the functions which are implemented in C. My question is how can you determine or know the parts of the standard library is implemented in C?

Upvotes: 0

Views: 185

Answers (2)

MSeifert
MSeifert

Reputation: 152870

Parts of the standard library of CPython that were written in C are faster than the parts that implemented in Python. To optimise your code, it good if you use the functions and implemented in C.

While that's correct it's only half of the story. All builtins are implemented in C, a lot of standard-library modules are completely or partly implemented in C. So everything already uses C functions.

For example collections.Counter is a pure-Python class but the collections._count_elements function (Python 3) is implemented in C and used by Counter so that it can "count faster". But makes that Counter a C function?

So it's not a clear-cut thing and you shouldn't expect a Python part to be necessarily (much) slower than if it were implemented in C. Also "written in C" or "written in Python" is kind of an implementation detail. So what's written in Python now could be re-implemented as C functions in a future version (probably also vise-versa but that happens less frequently or not at all).

how can you determine or know the parts of the standard library that were implemented in C?

You have to investigate that yourself. Some modules are available with a C implementation and a Python implementation (for example StringIO vs. cStringIO (python 2)) and other modules are completely implemented in C (for example itertools), others are partly implemented in C (for example collections).

Fortunately the CPython source code is available at GitHub but it still requires to look at the Lib folder to check if there's a Python implementation. If there is no Python implementation it's almost certainly completely written in C but if there's a .py file (or in a subfolder) you still need to check what's imported there. For example collections imports (and overrides) a lot of things from _collections which is implemented in C.

Upvotes: 4

Khoyo
Khoyo

Reputation: 1247

My question is how can you determine or know the parts of the standard library that were implemented in C?

You can read the sources of the standard library.

Python Modules

C modules

Or, what you should probably do, is measure the performance of your code, and then act based on that.

Upvotes: 0

Related Questions