CODEWITHSUNDEEP
pythonpip
user2546460
user2546460

Reputation: 1443

How to make pip "dry-run"?

For developing a script that runs pip install it would be useful to have a --dry-run function.

I came across the --no-install option. But this one is deprecated and on-call references this. There are hints to unpack a package only, but I can't find a unpack option in the pip documentation.

Upvotes: 125

Views: 71499

Answers (8)

Marge Folsom
Marge Folsom

Reputation: 1

Another answer has already suggested using --dry-run for newer pip versions.

A useful tip when one would prefer to install a version of a package that depends on a specific sub package version.

Installing a new package in an existing environment

pip install delta-spark --dry-run | grep 'Would install'
Would install delta-spark-2.4.0 py4j-0.10.9.7 pyspark-3.4.0

However, if an existing pyspark==3.2.1 version is preferred

pip install delta-spark pyspark==3.2.1 --dry-run | grep 'Would install'
Would install delta-spark-2.0.2

Upvotes: 0

funnydman
funnydman

Reputation: 11326

Eventually it was approved and released in 22.2 version

According to the documentation, --dry-run:

Don’t actually install anything, just print what would be. Can be used in combination with --ignore-installed to ‘resolve’ the requirements.

For example:

pip install flask --dry-run

prints information about what packages would be installed without installing them (but downloading):

Collecting flask
  Using cached Flask-2.2.2-py3-none-any.whl (101 kB)
Collecting Jinja2>=3.0
  Using cached Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting Werkzeug>=2.2.2
  Using cached Werkzeug-2.2.2-py3-none-any.whl (232 kB)
Collecting click>=8.0
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting itsdangerous>=2.0
  Using cached itsdangerous-2.1.2-py3-none-any.whl (15 kB)
Collecting MarkupSafe>=2.0
  Using cached MarkupSafe-2.1.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Would install Flask-2.2.2 Jinja2-3.1.2 MarkupSafe-2.1.1 Werkzeug-2.2.2 click-8.1.3 itsdangerous-2.1.2

There are a few other options that were added recently (they're still experimental, pip gives a warning):

Tell me what pip installed:

pip install flask --report /tmp/report.json

Tell me what pip would install:

pip install flask --dry-run --report /tmp/report.json

Tell me what pip would install in an empty environment (aka resolve the requirements):

pip install flask --dry-run --ignore-installed --report /tmp/report.json

Upvotes: 72

silviot
silviot

Reputation: 4768

The pip-sync command from pip-tools can report what packages would be installed, but will also output the ones that are installed but not in the requirements file. Its dry run option is -n

$ pip install pip-tools
$ pip-sync -n requirements.txt 
Would uninstall:
  pip-tools
Would install:
  requests

Here's the help from pip-sync:

pip-sync --help
Usage: pip-sync [OPTIONS] [SRC_FILES]...

  Synchronize virtual environment with requirements.txt.

Options:
  --version               Show the version and exit.
  -n, --dry-run           Only show what would happen, don't change anything
  --force                 Proceed even if conflicts are found
  -f, --find-links TEXT   Look for archives in this directory or on this HTML
                          page
  -i, --index-url TEXT    Change index URL (defaults to PyPI)
  --extra-index-url TEXT  Add additional index URL to search
  --trusted-host TEXT     Mark this host as trusted, even though it does not
                          have valid or any HTTPS.
  --no-index              Ignore package index (only looking at --find-links
                          URLs instead)
  -q, --quiet             Give less output
  --user                  Restrict attention to user directory
  --cert TEXT             Path to alternate CA bundle.
  --client-cert TEXT      Path to SSL client certificate, a single file
                          containing the private key and the certificate in
                          PEM format.
  --help                  Show this message and exit.

Upvotes: 12

MarcH
MarcH

Reputation: 19746

[Ugly hack disclaimer] on Linux you can try to install in the system location as a user who does not have permission to install in the /usr/ directory (pip3 install --system on Debian/Ubuntu/etc.) The command fails with "Permission denied" but only after logging what is missing and what is not.

Makes you wonder how hard an actual dry-run option would really be to implement?

Upvotes: 2

Magus
Magus

Reputation: 2992

This approach uses the current environment you are using and overrides paths pythons sees, it uses a similar approach to pipenv and virtualenv, but without any additional env creation, no changes to the current env, self-cleaning, and taking your current installed packages in the final dependency list solving algorythm, you know, just like a proper dry run.

  1. pip has a --target flag that lets you point to some arbitrary directory to install packages to, instead of system packages
  2. add that path to your PYTHONPATH temporarily, so you can figure out what was installed
  3. if it fails, well, then it failed, otherwise you can just generate a freeze list
  4. finally, a sweet diff for your cognitive convenience

So, the script would be something like this:

export DRY_RUN=/tmp/python-dry-run #points to the temp dir
mkdir -p $DRY_RUN #creates the dir
pip install -r requirements.txt --target $DRY_RUN #dry run happens here
PYTHONPATH=$DRY_RUN pip freeze > requirements-dry-run.txt #dry run output
diff --color requirements.txt requirements-dry-run.txt #red removed, green added
rm -rf $DRY_RUN #cleanup

caveats, warnings and opinions:

  1. try not to export PYTHONPATH directly in your shell, that might make things confusing, instead pass it as a temp variable at the beginning of the command, just like I did there

  2. pip freeze sucks, it usually leads to dependencies getting deadlocked in the future, so try your best to maintain your requirements file manually

  3. if you're asking for this, you're probably in a dependency hell. An easy way of getting rid of requirements.txt deadlock nightmare is to just to get rid of mid and minor versions completely and letting pip figure out the dependencies with a bit more leeway. So, if you are using example==1.4.5 in your requirements.txt, make it example>=1

Upvotes: 3

nealmcb
nealmcb

Reputation: 13451

Yes - pip should have a dry-run option, to indicate what would happen in a complex situation. It is dangerous when running pip install downgrades packages without asking you. We need some way to ask what would happen if we run pip install -r requirements.txt without laboriously searching thru all the requirements and comparing them to the currently installed ones.

It looks like setup.py used to have a dry-run. Folks are asking for it elsewhere.

Some progress in that direction can be found here:

Update: Activity continues on the official bug related to this: Add `pip install --dry-run` or similar, to get resolution result · Issue #53 · pypa/pip. It is blocked on the development of the dependency resolver, which is nearing completion

Upvotes: 43

kizzx2
kizzx2

Reputation: 19213

With pip version 9 there's a new option --format freeze leading to an elegant one line solution for the pip install -r use case:

pip list --format freeze | diff - requirements.txt

Upvotes: -2

Angelos
Angelos

Reputation: 1640

It appears you are right, it has been deprecated (ref).

If by trial run you mean testing it out before actually installing a package in a certain place, presumably before a system wide install, then you can simply run it sandboxed using a virtual environment and then simply discard the environment.

virtualenv /tmp/venv; /tmp/venv/bin/pip install flask; rm -rf /tmp/venv

Not as succinct as using a dry-run argument to pip, but it does the job. Also if you want to do a dry run of a series of package installations, omit the deletion at the end.

In a script you can distil it into a procedure:

#!/bin/bash

TMP_DIR='/tmp/venv'

function dry_run (){
    if [ ! -d "$TMP_DIR" ]; then
            virtualenv /tmp/venv
    fi
    /tmp/venv/bin/pip install $1
}

dry_run flask
dry_run uwsgi
rm -rf $TMP_DIR

If you want to do a dry run that tests that the new install(s) play well with system wide deployed, then use virtualenv's system-site-packages option.

virtualenv --system-site-packages /tmp/venv; /tmp/venv/bin/pip install flask; rm -rf /tmp/venv

Upvotes: 18

Related Questions