CODEWITHSUNDEEP
mlflow
Corey Zumar
Corey Zumar

Reputation: 81

How do I set a custom gunicorn worker timeout when serving an MLflow model with the "mlflow models serve" CLI?

When serving an MLflow Python model with the "pyfunc" backend (https://github.com/mlflow/mlflow/blob/master/mlflow/pyfunc/backend.py), how can I set a custom gunicorn worker timeout? The default timeout of 60 seconds may be insufficient when serving large models that take a long time to load.

Upvotes: 3

Views: 4044

Answers (2)

Mark Loyman
Mark Loyman

Reputation: 2180

mlflow allows setting these options from the cli:

Example: mlflow models serve ... --timeout 180

Official documentation (mlflow models serve --help):

-t, --timeout TEXT Timeout in seconds to serve a request (default: 60).

Upvotes: -1

Corey Zumar
Corey Zumar

Reputation: 81

As of MLflow 1.2, you can set a custom gunicorn timeout by specifying the GUNICORN_CMD_ARGS environment variable. The following example serves a model with a worker timeout of 120 seconds

GUNICORN_CMD_ARGS="--timeout 120" mlflow models serve --model-uri /path/to/model

Upvotes: 5

Related Questions