CODEWITHSUNDEEP
pythonamazon-s3botominiomlflow
mirekphd
mirekphd

Reputation: 6743

S3UploadFailedError due to MissingContentLength when calling PutObject in MLflow using MinIO

When trying to save / upload a file using mlflow.log_artifact() to MinIO, our MLflow users are suddenly getting this error in previously working code, raised in boto3 package (used by the mlflow package, and with S3-compatible local MinIO server acting as a data lake for our local installation of MLflow):

S3UploadFailedError: Failed to upload ./dict-20231204.2.json to mlflow/24/<run_id>/artifacts/dict-20231204.2.json: An error occurred (MissingContentLength) when calling the PutObject operation: You must provide the Content-Length HTTP header.

It is raised here: [..]/python3.11/site-packages/boto3/s3/transfer.py:378, in S3Transfer.upload_file()

Any solutions or at least workarounds to restore file uploads to MinIO buckets?

Upvotes: 0

Views: 203

Answers (1)

mirekphd
mirekphd

Reputation: 6743

A quick workaround is to downgrade the S3 client package that raises the error, thus:

$ pip install boto3"<1.36.0"

I have established that the issue (possibly an incompatibility with our legacy MinIO server) has been introduced in version 1.36.0 of boto3 (used by mlflow) and persists in version 1.36.2.

Upvotes: 1

Related Questions