CODEWITHSUNDEEP
macospysparkpython-3.6
vinaykp
vinaykp

Reputation: 549

what is Intel MKL FATAL ERROR: Cannot load libmkl_core.dylib. while running pyspark in MacOs?

Pyspark is installed in my Mac. This error I am getting while I try to run Pyspark from the command line.

I am installing Pyspark using homebrew and following instructions in this blog(https://medium.com/@roshinijohri/spark-with-jupyter-notebook-on-macos-2-0-0-and-higher-c61b971b5007). When I start running Pyspark in the shell I am getting the following error.

    Python 3.7.1 (default, Dec 14 2018, 13:28:58) 
    [Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    Intel MKL FATAL ERROR: Cannot load libmkl_core.dylib.

Upvotes: 14

Views: 21316

Answers (6)

Musebe Ivan
Musebe Ivan

Reputation: 182

Unconventionally try running your application that requires the mkl library outside the conda environment.

Upvotes: 0

The Ambidextrous
The Ambidextrous

Reputation: 117

If you are getting this error after running makemigrations command in Django, it is highly likely that your virtual environment is not activated.

This happens because when the venv is not activated, python starts wondering off to other directories looking for different libraries.

Running source venv/bin/activate (where venv is name of your virtual environment) fixes issue

Upvotes: 0

ishaj
ishaj

Reputation: 101

I was also getting the same error. Then I tried to create a virtual environment and then this error went off.

Upvotes: 1

msc
msc

Reputation: 135

First of all, I tried all the solutions above and neither of them solved this issue for me. For the folks that have a similar situation to me, I'm writing the solution that solved this bug below.

In my case, I have an anaconda virtual environment and doing all of the stuff in there.

Instead of removing mkl, or installing nomkl and etc. I installed it again by using the command:

conda install -c anaconda mkl

This solved the issue for me which I'm trying to solve for a day. I hope it also would be useful for you folks too.

Upvotes: 5

Uncaught Exception
Uncaught Exception

Reputation: 2179

enter image description here

Update your mkl package. That should fix this.

Upvotes: 1

sk311
sk311

Reputation: 107

MKL is Intel's "Math Kernel Library", and it basically handles calculations. The new version of Numpy uses MKL by default, and that is likely why this error is showing up.

Try to update numpy,

conda update numpy

or disable MKL by

conda install nomkl

Hope that helped! Reference: https://github.com/BVLC/caffe/issues/3884

Upvotes: 9

Related Questions