CODEWITHSUNDEEP
pythonnumpypython-3.5openblas
Ram Idavalapati
Ram Idavalapati

Reputation: 714

Numpy dot operation is not using all cpu cores

This is happening when the row size of b is from 2 to 15 (i.e from (2, 10000) to (15, 10000)).

Example:

import numpy as np

a = np.random.rand(10**4, 10**4)

def dot(a, b_row_size):
    b = np.random.rand(b_row_size, 10**4)

    for i in range(10):
        # dot operation
        x = np.dot(a, b.T)

# Using all CPU cores
dot(a, 1)

# Using only one CPU core
dot(a, 2)

# Using only one CPU core
dot(a, 5)

# Using only one CPU core
dot(a, 15)

# Using all CPU cores
dot(a, 16)

# Using all CPU cores
dot(a, 50)

np.show_config()

openblas_lapack_info:
    define_macros = [('HAVE_CBLAS', None)]
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
lapack_opt_info:
    define_macros = [('HAVE_CBLAS', None)]
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
blas_mkl_info:
  NOT AVAILABLE
lapack_mkl_info:
  NOT AVAILABLE
blas_opt_info:
    define_macros = [('HAVE_CBLAS', None)]
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
blis_info:
  NOT AVAILABLE
openblas_info:
    define_macros = [('HAVE_CBLAS', None)]
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c

Upvotes: 5

Views: 4945

Answers (1)

venkat
venkat

Reputation: 161

Numpy dot operation is not using all cpu cores

numpy.show_config() is clearly showing that it is using OpenBLAS at underline level.

So OpenBLAS is the actual one that is responsible for parallel computation.

But in sgemm OpenBLAS won't parallelize the computation up to certain threshold (In your case the row size of b is 2 to 15).

As a workaround, you can change the threshold value (GEMM_MULTITHREAD_THRESHOLD) in sgemm file and compile the OpenBLAS with numpy

Change the GEMM_MULTITHREAD_THRESHOLD value from 4 to 0 to parallelize all the sgemm computations.

Upvotes: 3

Related Questions