CODEWITHSUNDEEP
pythonpip
Arash Hatami
Arash Hatami

Reputation: 5541

Where is pip cache folder?

Where is the Python pip cache folder? I had an error during installation and now reinstall packages using cache files. Where is that directory? I want to take a backup of them for installation in the future. Is it possible?

For example, I have this one

Using cached cssselect-0.9.1.tar.gz

I searched google for this directory but nothing I saw, is learning how to install from a folder, I want to find the default cache directory.

And another question: Will these cache files stay in that directory, or will they be removed soon?

Upvotes: 174

Views: 239986

Answers (7)

Hugo
Hugo

Reputation: 29314

It depends on the operating system.

With pip 20.1 or later, you can find it with:

pip cache dir

For example with macOS:

$ pip cache dir
/Users/hugo/Library/Caches/pip

Docs:

Upvotes: 157

Steve Barnes
Steve Barnes

Reputation: 28370

Just to note that archiving your cache directory is not usually a very good idea as:

  • the location may change as may the structure with different versions of pip
  • You will end up with an archive with any other packages that you may have installed, (and possibly updated or uninstalled), in your history on that machine.
  • Your archive will only be suitable for use on the same platform, version of python, etc.

If you need to archive all of the files needed for a specific package, e.g. for transfer to an air-gapped system, then the better option is to do something like, (this example is a Windows one):

pip download --dest=pkgname_download pkgname
cd pkgname_download
python -m tarfile -c ../pkgname.tar.gz *
cd ..
rmdir /S /Q pkgname_download

This will download your specified package (pkgname) and all of its dependencies for you, you can specify multiple packages but only one download directory. You can also specify a specific version using the normal pip syntax if you need to. If you need to do the same for all of the packages in your system or the current virtual environment you can use:

pip freeze > requires.txt

then rather than using a pkgname to download use -r requires.txt in the above example. The best bit is that you can use this even if your target air-gapped machine is a different python/platform/abi/implementation to your internet connected one you can pass the appropriate information to pip download and it will still do the job for you. For details see:

pip download --help

Upvotes: 0

Siddhesh Shinde
Siddhesh Shinde

Reputation: 33

Linux: ~/.cache/pip

MacOS: ~/Library/Caches/pip

Windows: %LocalAppData%\pip\Cache

Reference Docs: https://pip.pypa.io/en/stable/topics/caching/#where-is-the-cache-stored

Upvotes: 1

C Dorman
C Dorman

Reputation: 549

Note that what pip caches is not necessarily human readable, and doing pip cache list does not necessarily list all the files that have been cached in some fashion. In addition to the .whl files that appear when you ask pip to list the cache, there is an http directory where network caching is done as well. On linux, it is in ~/.cache/pip/http. Files there do not appear to be listed when you do pip cache list

If you ask pip to install comm==0.1.3, it will look in the regular cache and when the network request is made, it will look in the network cache under a hashed key. The function _FileCacheMixin.encode() in the file pip/_internal/_vendor/cachecontrol/caches/file_cache.py (in pip version 23.0.1) turns the URL into a key. On my machine, the key is '8856a20c..[etc]' and there is a file ~/.cache/pip/http/8/8/5/6/a/8856a2..[etc] that is the whl file.

Upon attempting to install comm==0.1.3, pip will report:

Collecting comm==0.1.3
  Using cached comm-0.1.3-py3-none-any.whl (6.6 kB)

But pip cache list | grep comm comes up with nothing. This is a known issue. See: https://github.com/pypa/pip/issues/10460

Upvotes: 1

ptim
ptim

Reputation: 15587

The default location for the cache directory depends on the Operating System:

Unix

~/.cache/pip and it respects the XDG_CACHE_HOME directory.

macOS

~/Library/Caches/pip

Windows

<CSIDL_LOCAL_APPDATA>\pip\Cache

Wheel Cache

pip will read from the subdirectory wheels within the pip cache directory and use any packages found there. [snip]

https://pip.pypa.io/en/latest/reference/pip_install/#caching

The location of the cache directory can be changed via the command line option --cache-dir.

Upvotes: 177

Rabash
Rabash

Reputation: 4769

Pythonic and cross-platform way:

import pip
from distutils.version import LooseVersion

if LooseVersion(pip.__version__) < LooseVersion('10'):
    # older pip version
    from pip.utils.appdirs import user_cache_dir
else:
    # newer pip version
    from pip._internal.utils.appdirs import user_cache_dir

print(user_cache_dir('pip'))
print(user_cache_dir('wheel'))

Under the hood, it normalizes paths, manages different locations for exotic and ordinary operating systems and platforms, performs Windows registry lookup.

It may worth mentioning, if you have different Python versions installed, 2.x'es and 3.x'es, they all do share the same cache location.

Upvotes: 30

fredrik
fredrik

Reputation: 10281

You can backup the associated wheel rather than attempting to perform a backup of the cache folder.

Download the wheel for csselect of version 0.9.1 into /tmp/wheelhouse:

pip wheel --wheel-dir=/tmp/wheelhouse cssselect==0.9.1

Install the downloaded wheel:

pip install /tmp/wheelhouse/cssselect-0.9.1-py2-none-any.whl

Upvotes: 10

Related Questions