CODEWITHSUNDEEP
pythondecimalcython
MathTolliob
MathTolliob

Reputation: 61

How to boost Decimal module in Python

I have written a code using the Python module Decimal, but it takes some time to be executed.

So, I would like to boost it. After profiling it, I’ve seen that a function could be improved.

Consequently, I have two questions:

  1. Is the cdecimal module the same as the decimal module ?
  2. If not, how could I install the cdecimal module on my mac? « pip3 install pypi-cdecimal » gives the following error:
    Collecting pypi-cdecimal
      Using cached             https://files.pythonhosted.org/packages/c6/3e/0e8408545ef9bca6e11956c1e78215b820f0193669afe8750f9cbaa054d1/pypi-cdecimal-2.3.tar.gz

    Complete output from command python setup.py egg_info:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/lf/v8d843ms57dd27cslfdcbrlc0000gn/T/pip-install-wvun3np6/pypi-cdecimal/setup.py", line 426, in <module>
        ext_modules = [cdecimal_ext(MACHINE)]
      File "/private/var/folders/lf/v8d843ms57dd27cslfdcbrlc0000gn/T/pip-install-wvun3np6/pypi-cdecimal/setup.py", line 315, in cdecimal_ext
        config_vars = configure(machine, cc, py_size_t)
      File "/private/var/folders/lf/v8d843ms57dd27cslfdcbrlc0000gn/T/pip-install-wvun3np6/pypi-cdecimal/setup.py", line 230, in configure
        os.chmod("./configure", 0x1ed) # pip removes execute permissions.
    FileNotFoundError: [Errno 2] No such file or directory: './configure'

  1. Is it possible to cythonize my code? Could me cythonize using Decimal variables?

  2. If any of these could be helpful, should I have to switch to numpy?

Upvotes: 1

Views: 1150

Answers (1)

Draconis
Draconis

Reputation: 3461

As a general rule, cmodule is the version of module written in C (instead of pure Python). Python 3 will choose to use the c version automatically if it's available, and ever since CPython 3.3, cdecimal has been built-in.

So if you're using CPython (and if you don't know what that is then you definitely are), you're already using cdecimal every time you import decimal.

The problem is, working with 300 digits of precision is expensive. This is just a fundamental limitation. Numpy isn't optimized for that sort of thing and won't help (since none of the built-in Numpy types are capable of it). I doubt Cython will help either, since it gets its speed from using native C data types (same as Numpy). The best way to make your code run faster is to reduce that precision.

As for how to reduce precision, that depends on your problem. For example, a Naïve Bayes model might involve multiplying tiny, tiny numbers with a hundred zeroes after the decimal point before you get to anything significant. So you might think you need a Decimal with that many digits. In practice, you can take the logarithm and add instead of multiplying, and now your precision needs are solved while still giving the right answer.

Upvotes: 2

Related Questions