R (RStudio) can't find valid tensorflow installation (Mac M1)

Question

I am trying to get tensorflow up and running in R but several attempts have failed. First my specs: Computer: MacBook Pro M1 Max OS: MacOS Ventura 13.1 R version: version 4.2.2 (2022-10-31) [arch: aarch64] with RStudio 2022.12.0 Build 353

I followed initially these guidelines: https://tensorflow.rstudio.com/install/ and https://tensorflow.rstudio.com/install/local_gpu

In R I then run the following in a new session:

library(reticulate)
virtualenv_create("r-reticulate")

library(tensorflow)
tf$constant("Hello Tensorflow!")

I receive the following output:

> library(reticulate)
> virtualenv_create("r-reticulate")
virtualenv: r-reticulate
> library(tensorflow)
> tf$constant("Hello Tensorflow!")
Error: Valid installation of TensorFlow not found.

Python environments searched for 'tensorflow' package:
 /Users/daniel/Library/r-miniconda-arm64/envs/r-reticulate/bin/python3.8

Python exception encountered:
 Traceback (most recent call last):
  File "/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/reticulate/python/rpytools/loader.py", line 39, in _import_hook
    module = _import(
  File "/Users/daniel/Library/r-miniconda-arm64/envs/r-reticulate/lib/python3.8/site-packages/tensorflow/__init__.py", line 443, in <module>
    _ll.load_library(_plugin_dir)
  File "/Users/daniel/Library/r-miniconda-arm64/envs/r-reticulate/lib/python3.8/site-packages/tensorflow/python/framework/load_library.py", line 151, in load_library
    py_tf.TF_LoadLibrary(lib)
tensorflow.python.framework.errors_impl.NotFoundError: dlopen(/Users/daniel/Library/r-miniconda-arm64/envs/r-reticulate/lib/python3.8/site-packages/tensorflow-plugins/libmetal_plugin.dylib, 0x0006): symbol not found in 

When I type 'python3 --version' in the Terminal, it returns: Python 3.10.9 and when I type 'type -a python3' it gives me the following paths:

 python3 is /opt/homebrew/bin/python3
 
 python3 is /Library/Frameworks/Python.framework/Versions/3.11/bin/python3 

 python3 is /usr/local/bin/python3 

 python3 is /usr/bin/python3 

This looked a bit odd to me to have these different paths, but I don't know because I haven't worked with Python installations before; also note that the initial Error Message returns this path (/Users/daniel/Library/r-miniconda-arm64/envs/r-reticulate/bin/python3.8), which is not listed

In additional trials, I specified the virtual environment path to any of these python paths, such as:

virtualenv_create("r-reticulate", python = "/Library/Frameworks/Python.framework/Versions/3.11/bin/python3")

The Error message is the same though.

I also tried the following solutions to reinstall and specify differently.

https://github.com/rstudio/tensorflow/issues/new?assignees=&labels=Installation&template=installation-issue.md&title=Installation+Issue

https://gist.github.com/juliasilge/035d54c59436604d6142ebebf29aa224

Installing TensorFlow on M1 Chip - Issues. - PackagesNotFoundError: The following packages are not available from current channels:

In the global menu --> python: I have selected '/usr/local/bin/python3.11'. When I try to install tensorflow for Python it returns the following:

py_install('tensorflow') Using virtual environment '~/.virtualenvs/r-reticulate' ...

• '/Users/daniel/.virtualenvs/r-reticulate/bin/python' -m pip install --upgrade --no-user 'pip' 'wheel' 'setuptools' sh: /Users/daniel/.virtualenvs/r-reticulate/bin/python: No such file or directory Error: Error installing package(s): "'pip'", "'wheel'", "'setuptools'" In addition: Warning message: In system2(command, args, ...) : error in running command

Thanks for any pointers you can provide!

---

After reinstalling miniconda and tensorflow dependencies for metal (following apple's developer instructions). I get the following: tf$constant("Hello Tensorflow!") List of 20 $ python : chr "/Users/daniel/miniconda3/bin/python3.10" $ libpython : chr "/Users/daniel/miniconda3/lib/libpython3.10.dylib" $ pythonhome : chr "/Users/daniel/miniconda3:/Users/daniel/miniconda3" $ pythonpath : chr "/Users/daniel/miniconda3/lib/python310.zip:/Users/daniel/miniconda3/lib/python3.10:/Users/daniel/miniconda3/lib"| truncated $ prefix : chr "/Users/daniel/miniconda3" $ exec_prefix : chr "/Users/daniel/miniconda3" $ base_exec_prefix : chr "/Users/daniel/miniconda3" $ virtualenv : chr "" $ virtualenv_activate : chr "" $ version_string : chr "3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:25:13) [Clang 14.0.6 ]" $ version : chr "3.10" $ architecture : chr "64bit" $ anaconda : logi FALSE $ conda : logi TRUE $ numpy :List of 2 ..$ path : chr "/Users/daniel/miniconda3/lib/python3.10/site-packages/numpy" ..$ version:Class 'numeric_version' hidden list of 1 .. ..$ : int [1:3] 1 22 3 $ required_module : chr "tensorflow" $ required_module_path: chr "/Users/daniel/miniconda3/lib/python3.10/site-packages/tensorflow" $ available : logi TRUE $ python_versions : chr "/Users/daniel/miniconda3/bin/python3.10" $ forced : chr "use_python function"

• attr(*, "class")= chr "py_config" Error: Python module tensorflow was not found.

Detected Python configuration:

Answer 1

I got it working for tensorflow 2.8.0, doing the following steps:

1. Install Miniconda: https://docs.conda.io/en/main/miniconda.html
2. Open Terminal
3. Create Conda env: conda create --name rstudio-tf-2.8
4. Activate env: conda activate rstudio-tf-2.8
5. Downgrade Python: conda install python=3.8
6. Install tensorflow-deps: conda install -c apple tensorflow-deps==2.8.0
7. Install tensorflow-macos: python -m pip install tensorflow-macos==2.8.0
8. Install tensorflow-metal: python -m pip install tensorflow-metal==0.4.0
9. Install tensorflow: conda install -c anaconda tensorflow==2.8.0
10. Edit ~/.Renviron: echo "RETICULATE_PYTHON=~/miniforge3/envs/rstudio-tf-2.8/bin/python" >> ~/.Renviron
11. Check installed packages: conda list | grep tensorflow

It should look like this:

➜  ~ conda list | grep tensorflow                 
tensorflow                2.8.0           cpu_py39h2839aeb_0    conda-forge
tensorflow-base           2.8.0           cpu_py39hcf13c84_0    conda-forge
tensorflow-deps           2.8.0                         0    apple
tensorflow-estimator      2.8.0           cpu_py39hfad2b03_0    conda-forge
tensorflow-macos          2.8.0                    pypi_0    pypi
tensorflow-metal          0.4.0                    pypi_0    pypi

12. Go to R-Studio and restart R-Session Session -> Restart R
13. Init and test tensorflow with following R script:

reticulate::use_python("~/miniforge3/envs/rstudio-tf-2.8/bin/python")
reticulate::use_condaenv("rstudio-tf-2.8", conda="~/miniforge3/bin/conda", required = TRUE)

library(tensorflow)
tf_config() 
tf_version() 
tf$config$list_logical_devices()

14. If tf_config() crashes go back to Terminal and upgrade to python 3.9 conda install python=3.9
15. Check again if all deps are listed conda list | grep tensorflow, for me tensorflow-macos and metal were missing after the python upgrade, I had to repeat step 8 and 9
16. Repeat steps 12 and 13 - it should work now

I don't know how to avoid the crash, if I install from the beginning python 3.9 the crash still happens and get fixed by downgrading to python 3.8. My guess is come dependency issue, which I couldn't figure out.