CODEWITHSUNDEEP
pythonprefect
chaos
chaos

Reputation: 400

How to organize prefect flow with the using of cmd arguments?

Hi I am trying to apply prefect to my project which is using the library click in dealing with command-line paras. Below is a demo code snippet:

@click.command()
@click.option(
    "-p",
    "--pages",
    type=int,
    default=0,
    help="...",
)
def main(pages):
    print("Running...")
    if pages > 0:
      a()
    else:
      b()
    print("Finished without errors.")

if __name__ == "__main__":
    main()
    another_method()

The doc of prefect mentioned about the example :

flow = Flow("hello-flow", tasks=[hello_task])
flow.register(project_name="tester")

But what if I need to run the program by let say poetry run main.py -p 10, where I need to give a fixed command-line para and also run with poetry. In that case how should I organize or refactor my code to fit with Prefect?

Upvotes: 0

Views: 1354

Answers (1)

Anna Geller
Anna Geller

Reputation: 1758

In general, with Prefect you don't have to use any of this (neither click nor poetry) in order to be able to run your flow from CLI because Prefect ships with its own CLI. In order to start your flow run from a CLI, you can use:

prefect run -p /path/to/your/flow_file.py

Let's assume you have a flow that looks as follows:

from prefect import task, Flow, Parameter

@task(log_stdout=True)
def hello_world(name):
    print(f"Hello {name}!")

with Flow("mini-example") as flow:
    name = Parameter("name", default="world")
    hw = hello_world(name)

If you want to run it locally with a different parameter value than "world", you can use --param option:

prefect run -p /path/to/your/flow_file.py --param name=Marvin

And then, once you are ready to deploy your project to your Prefect backend, you can register it:

prefect register --project yourprojectname -p /path/to/your/flow_file.py

And then you can even trigger a remote flow run that will be stored in a backend (i.e. Prefect Cloud or Server) using:

prefect run --name mini-example --project yourprojectname

Upvotes: 3

Related Questions