CODEWITHSUNDEEP
pythonpython-3.xvirtualenvtravis-cidistutils
Uri
Uri

Reputation: 3301

DeprecationWarning in Python 3.6 and 3.7 (with Pillow, distutils, imp)

Please look at this issue:

DeprecationWarning in Python 3.6 and 3.7

  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/PIL/TiffImagePlugin.py", line 57, in <module>
    import distutils.version
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/distutils/__init__.py", line 4, in <module>
    import imp
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/imp.py", line 33, in <module>
    DeprecationWarning, stacklevel=2)
DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
The command "python -W error::DeprecationWarning manage.py test" exited with 1.

The people who commented there said it's an issue with virtualenv, which is fixed in virtualenv 16.3.0 (see the comments).

I tried to upgrade to the latest virtualenv (virtualenv==16.4.3) in the tests but the tests still fail in Python 3.6 and 3.7 with the DeprecationWarning when run with deprecation warnings enabled:

What is the problem?

You can see the code in the relevant branches in Speedy Net.

For example, the branch uri_run_tests_with_deprecation_warnings_2019-04-02_a.

Update: I understand that the problem is that the virtual environment is activated before the virtualenv upgrade. So the version activated is not the same as the version after the upgrade. Is it possible to run tests on Travis with the latest virtualenv release, or do I have to wait until Travis upgrades it?

I created a new issue for setuptools: github.com/pypa/setuptools/issues/1933

Upvotes: 3

Views: 7630

Answers (1)

aaron
aaron

Reputation: 43098

The cause: virtual environments created with outdated virtualenv

virtualenv embeds its own distutils.

Before virtualenv==16.3.0, the embedded distutils imported the deprecated imp module.

Travis uses cached Python virtual environments, each with updated in-virtual-env versions of:

  • currently virtualenv==16.6.0 for Python 3.6, and
  • currently virtualenv==16.7.8 for Python 3.7.

However, the virtual environments were created by Travis cookbooks with virtualenv==15.1.0.

The fix: recreate virtual environments (with updated virtualenv)

In the case of Travis CI, in cookbooks/travis_build_environment/recipes/virtualenv.rb:

- execute 'pip install virtualenv==15.1.0'
+ execute 'pip install virtualenv==16.3.0'

I have submitted a PR for the fix at travis-ci/travis-cookbooks#1065.

The workaround: reinstall distutils (with updated virtualenv)

This may be sufficient for those who don't want to (or can't) recreate their virtual environments.

After deep-diving into the source code of Travis CI and virtualenv, I found that you can actually install distutils inside a virtual environment with the updated in-virtual-env version of virtualenv:

before_script:
  - python -c 'import os, virtualenv; virtualenv.install_distutils(os.environ["VIRTUAL_ENV"])'

MCVE of travis.yml:

language: python
python:
  - "3.5"
  - "3.6"
  - "3.7"
os: linux
dist: bionic
install: skip
before_script:
  - python -c 'import os, virtualenv; virtualenv.install_distutils(os.environ["VIRTUAL_ENV"])'
script:
  - python -W error::DeprecationWarning -c 'import distutils'

Upvotes: 3

Related Questions