CODEWITHSUNDEEP
pythonpython-poetry
Granitosaurus
Granitosaurus

Reputation: 21436

Python poetry - how to install optional dependencies?

Python's poetry dependency manager allows specifying optional dependencies via command:

$ poetry add --optional redis

Which results in this configuration:

[tool.poetry.dependencies]
python = "^3.8"
redis = {version="^3.4.1", optional=true}

However how do you actually install them? Docs seem to hint to:

$ poetry install -E redis

but that just throws and error:

Installing dependencies from lock file

[ValueError]
Extra [redis] is not specified.

Upvotes: 59

Views: 83206

Answers (6)

SaTa
SaTa

Reputation: 2682

I had a similar problem and it seems that the at least from version 1.4.2, we can use optional groups. I first add the optional group to pyproject.toml file:

[tool.poetry.group.optionalgroup]
optional = true

[tool.poetry.group.optionalgroup.dependencies]

Then, add the desried package to this group:

poetry add optionalpackage --group optionalgroup

And then install it with this command

poetry install --with optionalgroup

Check optional groups in poetry docs for more information.

Upvotes: 3

LC117
LC117

Reputation: 786

There are different way to handle optional arguments. E.g. if you have in your .toml :

[...]
vedo = { version = "^2024.5.1", optional = true }
open3d = { version = "^0.18", optional = true }
trimesh = { version = "^3.23", optional = true }

[tool.poetry.extras]
geometry = ["vedo", "open3d", "trimesh"]

You could install them using:

poetry install --extras geometry

Upvotes: 1

9769953
9769953

Reputation: 12201

This is now possible (with Poetry version 1.2; perhaps even an earlier version), using the "extras" group:

poetry add redis --group=extras

It will appear in the section

[tool.poetry.group.extras.dependencies]

which is also newer style (compared to [tool.poetry.extras] or [tool.poetry.extras.dependencies]

See the documentation. Interestingely, this still follows the older style, [tool.poetry.extras], and doesn't show the use of poetry add, but the above result is what I get.

Upvotes: 13

一年又一年
一年又一年

Reputation: 91

Up-voted Drachenfels's answer.

Dev dependency could not be optional, otherwise, no matter how you tweak it with extras or retry with poetry install -E, it will just never get installed.

This sounds like a bug but somehow by design,

...this is not something I want to add. Extras will be referenced in the distributions metadata when packaging the project but development dependencies do not which will lead to a broken extras.

— concluded in Poetry PR#606 comment by one maintainer. See here for detailed context: https://github.com/python-poetry/poetry/pull/606#issuecomment-437943927

I would say that I can accept the fact that optional dev-dependency cannot be implemented. However, at least Poetry should warn me when I have such a config. If so, I wouldn't have been confused for a long time, reading each corner of the help manual and found nothing helpful.

I found some people did get trap in this problem (Is poetry ignoring extras or pyproject.toml is misconfigured?) but their questions are closed, marked duplicated and re-linked to this question. Thus I decided to answer here and give more details about this problem.

Upvotes: 7

Arne
Arne

Reputation: 20147

You need to add a tool.poetry.extras group to your pyproject.toml if you want to use the -E flag during install, as described in this section of the docs:

[tool.poetry.extras]
caching = ["redis"]

The key refers to the word that you use with poetry install -E, and the value is a list of packages that were marked as --optional when they were added. There currently is no support for making optional packages part of a specific group during their addition, so you have to maintain this section in your pyproject.toml file by hand.

The reason behind this additional layer of abstraction is that extra-installs usually refer to some optional functionality (in this case caching) that is enabled through the installation of one or more dependencies (in this case just redis). poetry simply mimics setuptools' definition of extra-installs here, which might explain why it's so sparingly documented.

Upvotes: 61

Drachenfels
Drachenfels

Reputation: 3286

I will add that not only you have to have this extras section added by hand, as well your optional dependencies cannot be in dev section.

Example of code that won't work:

[tool.poetry]
name = "yolo"
version = "1.0.0"
description = ""
authors = []

[tool.poetry.dependencies]
python = "2.7"
Django = "*"

[tool.poetry.dev-dependencies]
pytest = "*"
ipdb = {version = "*", optional = true}

[tool.poetry.extras]
dev_tools = ["ipdb"]

But this WILL work:

[tool.poetry]
name = "yolo"
version = "1.0.0"
description = ""
authors = []

[tool.poetry.dependencies]
python = "2.7"
Django = "*"
ipdb = {version = "*", optional = true}

[tool.poetry.dev-dependencies]
pytest = "*"

[tool.poetry.extras]
dev_tools = ["ipdb"]

Upvotes: 41

Related Questions