CODEWITHSUNDEEP
pythonkedro
WestFlame
WestFlame

Reputation: 435

Access configuration in the pipelines definition (not only nodes)

I want to dynamically alter pipelines based on the configuration provided. Is there possibility to pass configuration based on the environment to the register_pipelines() or to the create_pipeline() functions?

I've read the documentation about configuration (https://kedro.readthedocs.io/en/stable/kedro_project_setup/configuration.html) but there is not specified how can one get the necessary information to use ConfigLoader it (mainly environment) that was used in kedro run --env= command.

E.g. Similarly when creating nodes, is it possible to create pipelines with create_pipeline("params:model_params")?

Upvotes: 0

Views: 374

Answers (2)

WestFlame
WestFlame

Reputation: 435

I've finally found that there is (imho not mentioned in the documentation) hook after_context_created (https://github.com/kedro-org/kedro/blob/main/kedro/framework/hooks/specs.py#L289), which I can use to fit my needs.

Upvotes: 2

datajoely
datajoely

Reputation: 1516

To take advantage of configuration environments simply create a new folder under conf/ e.g. conf/prod and then running kedro run --env prod will run things with the configuration under that folder.

The environment name should also be available in many of the hooks you can extend kedro with, although the node itself shouldn't be aware of which environment is currently configured.

Upvotes: 0

Related Questions