CODEWITHSUNDEEP
pythonmultithreadingperformancenumpy
drjrm3
drjrm3

Reputation: 4728

Multithreaded summation of arrays in Python

I have two arrays of size n1 X n2 and I want them to be added together in parallel. I have an openMP enabled build of Python, but when I set export OMP_NUM_THREADS=4 inside of my bash shell before execution, I don't see my code being multithreaded. Is it possible to perform arr = arr + tarr in a mulithreaded way?

#!/usr/bin/env python

import numpy as np

n1 = 20000
n2 = 20000

arr = np.random.random_sample((n1,n2))

for i in range(10):
    tarr = np.random.random_sample((n1,n2))
    arr = arr+tarr

Upvotes: 1

Views: 934

Answers (1)

ali_m
ali_m

Reputation: 74242

Your BLAS linkage would only be only relevant for linear algebra operations (matrix products, solving linear systems etc.). numpy itself does not multithread basic elementwise arithmetic operations on arrays (such as addition, non-matrix multiplication, exponentiation etc.).

One of the simplest options for multithreading that calculation would be to use numexpr:

In [1]: import numpy as np

In [2]: import numexpr as ne

In [3]: n1, n2 = 5000, 5000

In [4]: x = np.random.randn(n1, n2)

In [5]: %%timeit y = np.random.randn(n1, n2)
   ...: x + y
   ...: 
1 loops, best of 3: 245 ms per loop

In [6]: %%timeit y = np.random.randn(n1, n2)
   ...: ne.evaluate('x + y')
   ...: 
10 loops, best of 3: 83.6 ms per loop

Upvotes: 1

Related Questions