Python's setuptools is failing to install at random in my Github Actions workflow

Question

For the past few weeks I've noticed my Github Actions workflows failing randomly when installing the setuptools library. Usually running the workflow once or twice will clear the issue, but I am unsure on how to avoid it happening in the first place.

• Github Actions runs on: ubuntu-latest
• Python version: 3.10.13 (tested with 3.11.6 and 3.11.8)
• setuptools version: set to 69.1.1 in my pyproject.toml, but poetry decides to update it to 69.2.0

The installation is made through Poetry without a virtualenv (poetry config virtualenvs.create false && poetry install --no-root).

Error log:

...
- Updating setuptools (65.5.0 -> 69.1.1)

  ChefInstallError

  Failed to install setuptools >= 40.8.0.
  
  Output:
  Updating dependencies
  Resolving dependencies...
  
  Package operations: 1 install, 0 updates, 0 removals
  
    - Installing setuptools (69.2.0)
  
    CalledProcessError
  
    Command '['/tmp/tmpd2m4qzu7/.venv/bin/python', '-I', '-W', 'ignore', '-c', '\nimport importlib.util\nimport json\nimport sys\n\nfrom pathlib import Path\n\nspec = importlib.util.spec_from_file_location(\n    "packaging", Path(r"/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/packaging/__init__.py")\n)\npackaging = importlib.util.module_from_spec(spec)\nsys.modules[spec.name] = packaging\n\nspec = importlib.util.spec_from_file_location(\n    "packaging.tags", Path(r"/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/packaging/tags.py")\n)\npackaging_tags = importlib.util.module_from_spec(spec)\nspec.loader.exec_module(packaging_tags)\n\nprint(\n    json.dumps([(t.interpreter, t.abi, t.platform) for t in packaging_tags.sys_tags()])\n)\n']' returned non-zero exit status 1.
  
    at /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/subprocess.py:526 in run
         522│             # We don't call process.wait() as .__exit__ does that for us.
         523│             raise
         524│         retcode = process.poll()
         525│         if check and retcode:
      →  526│             raise CalledProcessError(retcode, process.args,
         527│                                      output=stdout, stderr=stderr)
         528│     return CompletedProcess(process.args, retcode, stdout, stderr)
         529│ 
         530│ 
  
  The following error occurred when trying to handle this error:
  
  
    EnvCommandError
  
    Command ['/tmp/tmpd2m4qzu7/.venv/bin/python', '-I', '-W', 'ignore', '-c', '\nimport importlib.util\nimport json\nimport sys\n\nfrom pathlib import Path\n\nspec = importlib.util.spec_from_file_location(\n    "packaging", Path(r"/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/packaging/__init__.py")\n)\npackaging = importlib.util.module_from_spec(spec)\nsys.modules[spec.name] = packaging\n\nspec = importlib.util.spec_from_file_location(\n    "packaging.tags", Path(r"/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/packaging/tags.py")\n)\npackaging_tags = importlib.util.module_from_spec(spec)\nspec.loader.exec_module(packaging_tags)\n\nprint(\n    json.dumps([(t.interpreter, t.abi, t.platform) for t in packaging_tags.sys_tags()])\n)\n'] errored with the following return code 1
    
    Error output:
    Traceback (most recent call last):
      File "<string>", line 18, in <module>
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/packaging/tags.py", line 26, in <module>
      File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/packaging/_manylinux.py", line 10, in <module>
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 879, in exec_module
      File "<frozen importlib._bootstrap_external>", line 1016, in get_code
      File "<frozen importlib._bootstrap_external>", line 1073, in get_data
    FileNotFoundError: [Errno 2] No such file or directory: '/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/packaging/_elffile.py'
    
  
    at /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/poetry/utils/env/base_env.py:342 in _run
        338│                 output = subprocess.check_output(
        339│                     cmd, stderr=stderr, env=env, text=True, **kwargs
        340│                 )
        341│         except CalledProcessError as e:
      → 342│             raise EnvCommandError(e)
        343│ 
        344│         return output
        345│ 
        346│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:
  
  Cannot install setuptools.
  
  
  
  Error:
  

  at /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/poetry/installation/chef.py:102 in install
       98│             InstalledRepository.load(self._env),
       99│         )
      100│         installer.update(True)
      101│         if installer.run() != 0:
    → 102│             raise ChefInstallError(requirements, io.fetch_output(), io.fetch_error())
      103│ 
      104│ 
      105│ class Chef:
      106│     def __init__(

Cannot install build-system.requires for odfpy.

I've tried changing the python version in case it is a bug with a specific version of Python, and I've also tried installing setuptools before Poetry with pip in case it is a poetry bug, but it made no difference.

It has been working perfectly well for months, and now 1 in every 3 runs will fail with this error. Any ideas?

Answer 1

Well I don't know why I didn't test this before (tunnel vision, I guess), but I managed to fix the issue by limiting the max version of the setuptools library to 69.1.* in my pyproject.toml like this:

setuptools = "~69.1"

In case someone else gets this problem, the bug seems to be caused by setuptool version 69.2.0, but version 69.1.1 works just fine.