M1 Mac returns OSError: Library not loaded

Question

I am trying to import forecasting models from the darts package using from darts.models import ExponentialSmoothing and the following error is returned:

OSError: dlopen(/Users/sp/opt/anaconda3/envs/august/lib/python3.9/site-packages/lightgbm/lib_lightgbm.so, 0x0006): Library not loaded: /usr/local/opt/libomp/lib/libomp.dylib
Referenced from: /Users/sp/opt/anaconda3/envs/august/lib/python3.9/site-packages/lightgbm/lib_lightgbm.so
Reason: tried: '/usr/local/opt/libomp/lib/libomp.dylib' (no such file), '/usr/local/lib/libomp.dylib' (no such file), '/usr/lib/libomp.dylib' (no such file)

I have seen a lot of people have the same issue when installing LightGBM and have tried all those fixes. I have tried brew install libomp, I have uninstalled and reinstalled Anaconda, I have used different environments, I have installed gcc and cmake and nothing has worked. I have a MacBook Pro with an M1 chip and am using Spyder with Anaconda. Does anyone have an answer to this?

My code, which is taken directly from the darts package site, and its output is as follows:

import pandas as pd

import darts

from darts import TimeSeries

# Read a pandas DataFrame

df = pd.read_csv('/Users/sp/Documents/Skripsie/airline-passenger-traffic.csv')

# Create a TimeSeries, specifying the time and value columns

series = TimeSeries.from_dataframe(df, '1949-01', '112')

# Set aside the last 36 months as a validation series

train, val = series[:-36], series[-36:]

from darts.models import ExponentialSmoothing

The Traceback:

Traceback (most recent call last):

  File "/var/folders/51/7qn6p5813zd3dw8p2c_t3btr0000gn/T/ipykernel_68567/3838417146.py", line 1, in <module>
    from darts.models import ExponentialSmoothing

  File "/Users/sp/opt/anaconda3/envs/august/lib/python3.9/site-packages/darts/models/__init__.py", line 46, in <module>
    from darts.models.forecasting.gradient_boosted_model import LightGBMModel

  File "/Users/sp/opt/anaconda3/envs/august/lib/python3.9/site-packages/darts/models/forecasting/gradient_boosted_model.py", line 13, in <module>
    import lightgbm as lgb

  File "/Users/sp/opt/anaconda3/envs/august/lib/python3.9/site-packages/lightgbm/__init__.py", line 8, in <module>
    from .basic import Booster, Dataset, Sequence, register_logger

  File "/Users/sp/opt/anaconda3/envs/august/lib/python3.9/site-packages/lightgbm/basic.py", line 110, in <module>
    _LIB = _load_lib()

  File "/Users/sp/opt/anaconda3/envs/august/lib/python3.9/site-packages/lightgbm/basic.py", line 101, in _load_lib
    lib = ctypes.cdll.LoadLibrary(lib_path[0])

  File "/Users/sp/opt/anaconda3/envs/august/lib/python3.9/ctypes/__init__.py", line 460, in LoadLibrary
    return self._dlltype(name)

  File "/Users/sp/opt/anaconda3/envs/august/lib/python3.9/ctypes/__init__.py", line 382, in __init__
    self._handle = _dlopen(self._name, mode)

OSError: dlopen(/Users/sp/opt/anaconda3/envs/august/lib/python3.9/site-packages/lightgbm/lib_lightgbm.so, 0x0006): Library not loaded: /usr/local/opt/libomp/lib/libomp.dylib
  Referenced from: /Users/sp/opt/anaconda3/envs/august/lib/python3.9/site-packages/lightgbm/lib_lightgbm.so
  Reason: tried: '/usr/local/opt/libomp/lib/libomp.dylib' (no such file), '/usr/local/lib/libomp.dylib' (no such file), '/usr/lib/libomp.dylib' (no such file)

Answer 1

You should install OpenMP according to documentation.

It can be done with the following command:

brew install libomp

This solved my problem, I hope it will help someone else

Answer 2

sjp00,

Your post could have been from me almost verbatim. I went through a very similar process. After pulling my hair out, this is what worked. TL;DR is that for Darts to work with LightGBM on an apple M1 computer conda-forge needs to be used. Default conda will not work (but may at a future date).

Reading this issue helped clarify the situation, https://github.com/microsoft/LightGBM/issues/4843

specifically, StrikerRUS's comment:

"If I understood correctly, default conda channels lack support of macOS-arm64 packages."(https://github.com/microsoft/LightGBM/issues/4843#issuecomment-1012313487)

and

One difference could be that I'm using Conda instead of miniconda?

Yeah! That is my main assumption! I believe that LightGBM is being compiled for arm64 architecture but conda (and Python) is x64-based and is run via Rosetta 2 translation. So architecture mismatch happens.

(https://github.com/microsoft/LightGBM/issues/4843#issuecomment-1061332231)

On my laptop which is a MacBook Pro (16-inch, 2021), with an Apple M1 Pro chip, and macOS Monterey Version 12.5.1 installed and the following conda setup:

conda info

 active environment : lightgbm_test_env
active env location : /Users/bhudson/opt/anaconda3/envs/lightgbm_test_env
        shell level : 2
   user config file : /Users/bhudson/.condarc  populated config files : /Users/bhudson/.condarc
      conda version : 4.14.0
conda-build version : 3.21.9
     python version : 3.9.7.final.0
   virtual packages : __osx=10.16=0
                      __unix=0=0
                      __archspec=1=x86_64
   base environment : /Users/bhudson/opt/anaconda3  (writable)
  conda av data dir : /Users/bhudson/opt/anaconda3/etc/conda   conda av metadata url : None
       channel URLs : https://conda.anaconda.org/conda-forge/osx-64
                      https://conda.anaconda.org/conda-forge/noarch
                      https://repo.anaconda.com/pkgs/main/osx-64
                      https://repo.anaconda.com/pkgs/main/noarch
                      https://repo.anaconda.com/pkgs/r/osx-64
                      https://repo.anaconda.com/pkgs/r/noarch
      package cache : /Users/bhudson/opt/anaconda3/pkgs
                      /Users/bhudson/.conda/pkgs
   envs directories : /Users/bhudson/opt/anaconda3/envs
                      /Users/bhudson/.conda/envs
           platform : osx-64
         user-agent : conda/4.14.0 requests/2.26.0 CPython/3.9.7 Darwin/21.6.0 OSX/10.16
            UID:GID : 501:20
         netrc file : None
       offline mode : False

This is what finally worked for me. First make and activate a clean python 3.9 environment

conda create -n lightgbm_test_env python=3.9
conda activate lightgbm_test_env

Make sure that conda forge is added as a channel (and that is prioritized)

conda config --add channels conda-forge
conda config --set channel_priority strict

Then install all of U8Darts with LightGBM at the same time.

conda install -c conda-forge -c pytorch u8darts-all lightgbm

The "-c conda-forge" flag may not be strictly necessary, but I left it in for extra redundancy.