boto3 config error with MLflow but not with direct Python

Question

I am struggling with the following issue.

I setup a MLflow server on an EC2 instance, with permission to read and write on S3. I check that everything works fine doing the following in Python:

import boto3
s3 = boto3.client('s3')
s3.list_buckets()
file_content = "This is a test file."
file_name = 'test-file.txt'
bucket_name = 'my-bucket'
s3.put_object(
         Bucket=bucket_name, 
         Key=file_name, 
         Body=file_content)

The code does write the file to S3.

But when trying to use the MLflow server from a remote client, by setting:

train_server = f"http://{public_ip}:{mlflow_port}"
mlflow.set_tracking_uri(train_server)

the mlflow_logartifact in the following code does not work:

with mlflow.start_run() as run:
    mlflow.log_param("param1", 5)
    mlflow.log_artifact("local_artifact.txt")

I get error:

    mlflow.log_artifact("local_artifact.txt")
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/mlflow/tracking/fluent.py", line 1109, in log_artifact
    MlflowClient().log_artifact(run_id, local_path, artifact_path)
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/mlflow/tracking/client.py", line 1914, in log_artifact
    self._tracking_client.log_artifact(run_id, local_path, artifact_path)
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/mlflow/tracking/_tracking_service/client.py", line 812, in log_artifact
    artifact_repo.log_artifact(local_path, artifact_path)
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/mlflow/store/artifact/s3_artifact_repo.py", line 177, in log_artifact
    s3_client=self._get_s3_client(), local_file=local_file, bucket=bucket, key=dest_path
              ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/mlflow/store/artifact/s3_artifact_repo.py", line 135, in _get_s3_client
    return _get_s3_client(
           ^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/mlflow/store/artifact/s3_artifact_repo.py", line 110, in _get_s3_client
    return _cached_get_s3_client(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/mlflow/store/artifact/s3_artifact_repo.py", line 69, in _cached_get_s3_client
    return boto3.client(
           ^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/boto3/__init__.py", line 92, in client
    return _get_default_session().client(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/boto3/__init__.py", line 80, in _get_default_session
    setup_default_session()
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/boto3/__init__.py", line 34, in setup_default_session
    DEFAULT_SESSION = Session(**kwargs)
                      ^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/boto3/session.py", line 88, in __init__
    self._setup_loader()
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/boto3/session.py", line 129, in _setup_loader
    self._loader = self._session.get_component('data_loader')
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/botocore/session.py", line 802, in get_component
    return self._components.get_component(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/botocore/session.py", line 1140, in get_component
    self._components[name] = factory()
                             ^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/botocore/session.py", line 199, in <lambda>
    lambda: create_loader(self.get_config_variable('data_path')),
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/botocore/session.py", line 323, in get_config_variable
    return self.get_component('config_store').get_config_variable(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/botocore/configprovider.py", line 472, in get_config_variable
    return provider.provide()
           ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/botocore/configprovider.py", line 678, in provide
    value = provider.provide()
            ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/botocore/configprovider.py", line 765, in provide
    scoped_config = self._session.get_scoped_config()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/mlxp-wrapper/lib/python3.11/site-packages/botocore/session.py", line 422, in get_scoped_config
    raise ProfileNotFound(profile=profile_name)
botocore.exceptions.ProfileNotFound: The config profile () could not be found

I've spent the whole week-end wrapping my head on the source of the error but can't find any. I tried deleting or creating aws config files, checked environment variables on remote host, etc. but nothing worked.

Any help would be appreciated!

I tried a simplified version of the initial problem, as well as manipulating config variables on remote EC2 instance.