CODEWITHSUNDEEP
pythonmacosmatplotlib
gcamargo
gcamargo

Reputation: 3971

Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized

Getting the error message when using matplotlib:

Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized OMP: Hint: This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

Upvotes: 116

Views: 106775

Answers (17)

Tuấn Nguyễn
Tuấn Nguyễn

Reputation: 1

Hello it seem this error came from torch, you can tried this, note that my python version is 3.10.15

conda install pytorch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 -c pytorch

Upvotes: 0

Moncef
Moncef

Reputation: 369

I had the same error on my Mac. I tried setting KMP_DUPLICATE_LIB_OK to True using: os.environ['KMP_DUPLICATE_LIB_OK']='True'

but it didn't work for me!

I was able to solve it by running this command:

conda install nomkl

If you're using Anaconda. Open it and go to Environments, choose your environment and select open terminal then run the command.

Upvotes: 0

AbeLinkon
AbeLinkon

Reputation: 1735

I have Windows 10, and experienced this issue when trying to use easyocr. After a couple evenings I figured out the issue was that I somehow didn't have numpy installed. That led me to a different error where I needed to reinstall torchvision.

After that, it works great.

Upvotes: 0

kilojoules
kilojoules

Reputation: 10093

For me, this problem came up when I imported pytorch after numpy. Importing them in this order fixed my problem:

import torch
import numpy as np

Upvotes: 12

Erfan Khalaji
Erfan Khalaji

Reputation: 63

I am using Macbook M1 and I faced the same issue. I solved this problem after removing the mkl package from my conda environment using the following command:

conda remove mkl

This issue happens because mkl is developed for Intel users and many of my packages were comming from mkl. After you remove this package, you will be required to reinstall many packages that you use through mkl. Also, in my case, when I tried to install pandas afterwards, there was a dependency issue. I solved this issue as well after updating all conda packages using the following command:

conda update --all

A useful link that helped me figure this out:

Upvotes: 2

LauraD.
LauraD.

Reputation: 1

I had the same problem. Nothing you suggested solved the issue. I found that a possible cause is that you have multiple OpenMP libraries installed on your machine and they conflict with each other. Plus, I found that the problem was numpy and I did the upgrade (conda update numpy) and FINALLY IT WORKED!!!

Upvotes: 0

aliceincodeland
aliceincodeland

Reputation: 1

I was getting the same error as mentioned in the original question when I ran a code with Tensorflow on my macOS Monterey. I tried installing nomkl and removing mkl as suggested in many of the previous answers. However this gave me trouble on running readcsv module of pandas and many other modules from different packages. A friend told me that newer versions of macOS have trouble with the usual Tensorflow and therefore pypi has launched a special version of TF called tf-nightly.

https://pypi.org/project/tf-nightly/#description

This installation solved the problem for me.

Upvotes: 0

gcamargo
gcamargo

Reputation: 3971

Do the following to solve the issue:

import os

os.environ['KMP_DUPLICATE_LIB_OK']='True'

Answer found at: https://github.com/dmlc/xgboost/issues/1715

Be aware of potential side-effects:

but that may cause crashes or silently produce incorrect results.

Upvotes: 105

TransferOrbit
TransferOrbit

Reputation: 227

Confronted with the same error #15, none of the solutions to-date (5 Feb 2021) fully worked despite being helpful. However, I did manage to solve it while avoiding: dithering with dylib libraries, installing from source, or setting the environment variable KMP_DUPLICATE_LIB_OK=TRUE and its downsides of being an “unsafe, unsupported, undocumented workaround” and its potential “crashes or silently produce incorrect results”.

The trouble was that conda wasn’t picking up the non-mkl builds of tensorflow (v2.0.0) despite loading the nomkl package. What finally made this solution work was to:

  • ensure I was loading packages from the defaults channel (ie. from a channel with a non-mkl version of tensorflow. As of 5 Feb 2021, conda-forge does not have a tensorflow version of 2.0 or greater).
  • specify the precise build of the tensorflow version I wanted: tensorflow>=2.*=eigen_py37h153756e_0. Without this, conda kept loading the mkl_... version of the package despite the nomkl package also being loaded.

I created a conda environment using the following environment.yml file (as per the conda documentation for managing environments) :

name: tf_nomkl
channels:
  - conda-forge
  - defaults
dependencies:
  - nomkl
  - python>=3.7
  - numpy
  - scipy
  - pandas
  - jupyter
  - jupyterlab
  - nb_conda
  - nb_conda_kernels
  - ipykernel
  - pathlib
  - matplotlib
  - seaborn
  - tensorflow>=2.*=eigen_py37h153756e_0

You could try to do the same without an environment.yml file, but it’s better to load all the packages you want in an environment in one go if you can. This solution works on MacOS Big Sur v11.1.

Upvotes: 1

gingerL
gingerL

Reputation: 1

conda install --revision 0 doesn't solve UnsatisfiableError: The following specifications... for me. So I manually install nomkl and remove mkl and mil-service in Anaconda-Navigator environment, and it works great for me!

Upvotes: 0

el3ati2
el3ati2

Reputation: 495

Check if there's an update for the mkl package in your env (anaconda).

I was able to solve my case simply by updating mkl.

conda install -c intel mkl

(macOS Catalina 10.15.5)

Upvotes: 2

J.E.K
J.E.K

Reputation: 1461

I had the same issue on macOS and found the following reasons:

Problem:

I had a conda environment where Numpy, SciPy and TensorFlow were installed.

Conda is using Intel(R) MKL Optimizations, see docs:

Anaconda has packaged MKL-powered binary versions of some of the most popular numerical/scientific Python libraries into MKL Optimizations for improved performance.

The Intel MKL functions (e.g. FFT, LAPACK, BLAS) are threaded with the OpenMP technology.

But on macOS you do not need MKL, because the Accelerate Framework comes with its own optimization algorithms and already uses OpenMP. That is the reason for the error message: OMP Error #15: ...

Workaround:

You should install all packages without MKL support:

conda install nomkl

and then use

conda install numpy scipy pandas tensorflow

followed by

conda remove mkl mkl-service

For more information see conda MKL Optimizations.

Upvotes: 74

ttt
ttt

Reputation: 876

Try to change the backend of matplotlib.

For example, Tkagg backend causes this problem in my case. I changed it to Qt5Agg

matplotlib.use('Qt5Agg')

and it helps.

Upvotes: 1

user11874275
user11874275

Reputation: 121

I had the same issue in a conda environment where TensorFlow was installed. After doing

  • pip uninstall tensorflow
  • pip install tensorflow

the problem was gone.

Upvotes: 12

sjcoding
sjcoding

Reputation: 954

This is a better solution, if applicable. Else, anyway gcamargo’s solution is likely to work. However, it comes with a warning "that it may cause crashes or silently produce incorrect results"

I had the same error on my Mac with a python program using numpy, keras, and matplotlib. I solved it with

conda install nomkl

Answer found at: https://github.com/dmlc/xgboost/issues/1715

Upvotes: 81

Jayhou
Jayhou

Reputation: 51

Had same issue in OSX when updating tensoflow to 1.13 using conda.

  • Solution 1: /gcamargo worked but 3x slower per training epoch.
  • Solution 2: /sjcoding worked and removed serious warining but also 3x slower in training.
  • Solution 3: that restored performance was: Install pip in new conda env and use pip to install tensorflow. Using conda-forge also worked but version of tf is old.

Apparently the new Intel-MKL optimizations in Anaconda are broken for OSX tensorflow.

Upvotes: 5

ltjds
ltjds

Reputation: 231

So, for those of you getting this same issue with lightgbm, I found in the documentation that you can

  1. pip uninstall lightgbm
  2. pip install lightgbm
  3. Run the following in anaconda environmnet (if you're running Conda)
ln -sf `ls -d "$(brew --cellar libomp)"/*/lib`/* $CONDA_PREFIX/lib

These three things worked for me.

Upvotes: 1

Related Questions