CODEWITHSUNDEEP
pythonpandasmacosapple-m1arm64
mak
mak

Reputation: 43

How to use pandas on M1 mac? (without rosetta or changing to x86 environment in any other way)

Last I wrote a python project was less than 2 months ago and everything worked fine. I'm not sure if while working on other project I messed something up on my mac but now when trying to run python files which used to run perfectly, the following error appears:

dlopen(/opt/homebrew/lib/python3.9/site-packages/pandas/_libs/interval.cpython-39-darwin.so, 0x0002): tried: '/opt/homebrew/lib/python3.9/site-packages/pandas/_libs/interval.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/interval.cpython-39-darwin.so' (no such file), '/usr/lib/interval.cpython-39-darwin.so' (no such file)

I understand there is an issue with the architecture x86 vs arm so I tried seeing what platform the terminal is on with:

python -c 'import platform; print(platform.platform())'

which confirmed it was arm64. Doing some googling and looking at similar issues such as Trouble installing Pandas on new MacBook Air M1 it seems like it would be possible to run the python project in an x86 environment, however like already mentioned, it worked fine before, and it seems there was no update since, so what could have happened that pandas (and perhaps other libs) no longer work on arm, and how can it be reverted?

Upvotes: 4

Views: 1447

Answers (1)

Karan Mittal
Karan Mittal

Reputation: 181

You should try using miniforge.

its definition from its GitHub repository:

This repository holds a minimal installer for Conda specific to conda-forge. Miniforge allows you to install the conda package manager with the following features pre-configured:

Its main feature that will be useful for us

An emphasis on supporting various CPU architectures (x86_64, ppc64le, and aarch64 including Apple M1).

The Process I use:

  1. Create a conda environment and usually go with "python3.9".
  2. Install the packages from the conda, most of them are available but some are not.
  3. After trying and installing all the packages possible with miniforge, I use PIP for the remaining packages.

This workflow has worked pretty well for me and hope it helps you. I want to utilize the native m1 performance and I think you will be able to see the difference.

By default, miniforge only downloads arm compatible builds of python packages. till now I have not faced any major issue working with most data science libraries, except with PyTorch.

Upvotes: 1

Related Questions