CODEWITHSUNDEEP
pythoncircleciflake8
Ibolit
Ibolit

Reputation: 9730

Flake8 times out on circle-ci

We have a flake8 build stage in our circle-ci workflow, and more often than not this step fails due to timeout:

Too long with no output (exceeded 10m0s): context deadline exceeded

At the same time, this same stage runs quite ok locally on our macbooks:

% time make lint
poetry run black  .
All done! ✨ 🍰 ✨
226 files left unchanged.
isort -y
Skipped 2 files
PYTHONPATH=/path/to/project poetry run flake8 --show-source
0
make lint  44.00s user 4.90s system 102% cpu 47.810 total

We tried to debug the issue by adding the -vv flag to flake8 thinking we would get some plugin name that takes too long, but we don't even have the timestamps in the log:

flake8.processor          ForkPoolWorker-31   1004 WARNING  Plugin requested optional parameter "visitor" but this is not an available parameter.
flake8.processor          ForkPoolWorker-8   1080 WARNING  Plugin requested optional parameter "visitor" but this is not an available parameter.
flake8.bugbear            ForkPoolWorker-26   1082 INFO     Optional warning B950 not present in selected warnings: ['E', 'F', 'W', 'C90']. Not firing it at all.

Are there any known reasons why flake8 would freeze on CircleCI? How can one debug the issue?

Upvotes: 1

Views: 628

Answers (3)

Hans Bambel
Hans Bambel

Reputation: 956

When using a virtual-environment such as venv you should ignore the folder in the [flake8]-config (that's what happened to me). Assuming you are creating a virtualenv with virtualenv .venv it would look like this:

[flake8]
exclude = .venv

The same was for my coverage which was fixed by adding an omit to that config (solution found here):

# pyproject.toml file content
[tool.coverage.run]
omit = [
    "tests/*",
    ".venv/*",
]

Upvotes: 2

Matthieu Dsprz
Matthieu Dsprz

Reputation: 477

I've also experienced a timeout in circle-ci only, but it was due to the specific way dependencies are installed on the pipeline, creating a .venv folder which was not excluded in flake8 configuration.

The -v option helped me to notice the huge amount of files flkae8 was analyzing.

Upvotes: 0

Ibolit
Ibolit

Reputation: 9730

For now, the solution we seem to have found was to limit the number of cores running flake8:

.flake8

[flake8]
...
jobs = 6

Not sure it is the correct solution, but there you go. I will accept a better solution if there is one.

Upvotes: 0

Related Questions