CODEWITHSUNDEEP
pythonpipanaconda
chicony
chicony

Reputation: 21

Exporting top level dependencies from conda environment including pip installations

If I do conda env export --from-history -f environment.yml I get all top level dependencies I installed but I don't get packages I installed via pip. If on the other hand, I use conda env export -f environment.yml, I get the pip packages along with all of the dependencies (rather than only the top level ones). Is there a way to export so that I get top level dependencies, including pip packages?

Upvotes: 2

Views: 1564

Answers (2)

zeit
zeit

Reputation: 377

Solution from @gwerbin

I found a script work for this purpose recently, here is the link of the codes. conda_env_export.py

As the codes are short, I could paste the code below. Note that I modify some places for my personal usage.

import re
import subprocess
import sys
import yaml
import fire


CONDA_PATH = '/home/ubuntu/miniconda3/condabin/conda'


def export_env(history_only=False):
    cmd = [CONDA_PATH, 'env', 'export']
    if history_only:
        cmd.append('--from-history')

    cp = subprocess.run(cmd, stdout=subprocess.PIPE)
    try:
        cp.check_returncode()
    except:
        raise
    else:
        return yaml.safe_load(cp.stdout)


def _is_history_dep(d, history_deps):
    if not isinstance(d, str):
        return False
    d_prefix = re.sub(r'=.*', '', d)
    return d_prefix in history_deps


def _get_pip_deps(full_deps):
    for dep in full_deps:
        if isinstance(dep, dict) and 'pip' in dep:
            return dep


def _combine_env_data(env_data_full, env_data_hist):
    deps_full = env_data_full['dependencies']
    deps_hist = env_data_hist['dependencies']
    deps = [dep for dep in deps_full if _is_history_dep(dep, deps_hist)]

    pip_deps = _get_pip_deps(deps_full)

    env_data = {}
    env_data['channels'] = env_data_full['channels']
    env_data['dependencies'] = deps
    env_data['dependencies'].append(pip_deps)

    return env_data


def main(s_save=None):
    env_data_full = export_env()
    env_data_hist = export_env(history_only=True)
    env_data = _combine_env_data(env_data_full, env_data_hist)
    if s_save:
        with open(s_save, 'w') as y:
            yaml.dump(env_data, y)
    else:
        yaml.dump(env_data, sys.stdout)


if __name__ == '__main__':
    fire.Fire(main)

Tutorials

Save the codes above as conda_export_minimal.py.

Specify CONDA_PATH with your conda's path (could be found by running command whereis conda).

Then run as follow:

python conda_export_minimal.py --s_save=env.yml

A new file env.yml will appear in your current directory, which contains minimal dependencies about conda and pip.

Upvotes: 2

merv
merv

Reputation: 76750

Nothing in the current CLI can do this.

As @Peter mentioned, it's likely simplest just to export both and manually copy over the - pip: section. Another option is to do the --from-history export and then a pip freeze > requirements.txt. The requirements.txt can then be added as in this answer.

Upvotes: 1

Related Questions