CODEWITHSUNDEEP
pythondaskazure-machine-learning-serviceopencensusazureml-python-sdk
Arnab Biswas
Arnab Biswas

Reputation: 4605

Conda/Pip Environment creation fails with azureml & adlfs/opencensus-ext-azure

I am trying to create a local development environment using conda with azureml libraries. Following environment.yml file works fine.

name: cortixml_azure_env
channels:
  - conda-forge
  - defaults
dependencies:
  - python=3.8.3
  - pandas
  - numpy
  - flake8
  - black
  - pip
  - pip:
    - pyarrow
    - pytest
    - rope
    - dask[dataframe,distributed]
    - azure-storage-blob
    - opencensus-ext-azure
    - azureml-core
    - azureml-pipeline-steps
    - azureml-pipeline-core
    - azureml-pipeline
    - azureml-mlflow
    - scikit-learn
    - lightgbm
    - xgboost

But the moment, I add adlfs under pip installable, it get stuck at "Installing pip dependencies:" for hours and finally fails. This happens for opencensus-ext-azure as well.

Any suggestions?

Upvotes: 1

Views: 1021

Answers (1)

nferreira78
nferreira78

Reputation: 1164

Without knowing anything about your error message, from my experience, this problem usually happens because you are trying to install a library version that is not compatible with the remaining environment.

In other words, you are not specifying the version of adlfs that you want to install, so it tries to install the latest available and it is very likely that adlfs has dependencies conflicting with other packages in that file. That explains why it works well before adding and then fails when you add it.

In a nutshell, you have to create a fresh install using the environment.yml you have and then try to pip install adlfs yourself. It's likely that it will throw an incompatibility warning message. Use those messages to guide you on the right version of adlfs that will be compatible with the remaining packages.

Then finally export that working environment using:

conda env export --name {environment-name} > environment.yml

so you can then re-create the final environment anywhere else you need with:

conda env create --name {environment-name} --file environment.yml

Upvotes: 1

Related Questions