Reputation: 479
I'm trying to log a python experiment with mlflow
on a remote disk that I mount automatically in /etc/fstab
(debian, smb). Logging params works perfectly fine, the artifact gets copied, but code throws an error about copying attributes of the artifact:
import mlflow # version==2.16.0
import os
mlflow.set_tracking_uri(os.environ["MLFLOW_DIR"])
mlflow.set_experiment("mlflow_bug")
mlflow.start_run()
mlflow.log_param("param1", 1) # this gets logged
with open("test.txt", "w") as f:
f.write("Hello world!")
mlflow.log_artifact("test.txt") # this gets copied, then error is thrown
mlflow.end_run()
gives me the following error:
---------------------------------------------------------------------------
PermissionError Traceback (most recent call last)
Cell In[4], line 7
5 with open("test.txt", "w") as f:
6 f.write("Hello world!")
----> 7 mlflow.log_artifact("test.txt")
8 mlflow.end_run()
File ~/code/syn_decoder/.venv/lib/python3.11/site-packages/mlflow/tracking/fluent.py:1123, in log_artifact(local_path, artifact_path, run_id)
1095 """
1096 Log a local file or directory as an artifact of the currently active run. If no run is
1097 active, this method will create a new active run.
(...)
1120 mlflow.log_artifact(path)
1121 """
1122 run_id = run_id or _get_or_start_run().info.run_id
-> 1123 MlflowClient().log_artifact(run_id, local_path, artifact_path)
File ~/code/syn_decoder/.venv/lib/python3.11/site-packages/mlflow/tracking/client.py:1923, in MlflowClient.log_artifact(self, run_id, local_path, artifact_path)
1919 if run_id.startswith(TRACE_REQUEST_ID_PREFIX):
1920 raise MlflowException(
1921 f"Invalid run id: {run_id}. `log_artifact` run id must map to a valid run."
1922 )
-> 1923 self._tracking_client.log_artifact(run_id, local_path, artifact_path)
File ~/code/syn_decoder/.venv/lib/python3.11/site-packages/mlflow/tracking/_tracking_service/client.py:822, in TrackingServiceClient.log_artifact(self, run_id, local_path, artifact_path)
820 artifact_repo.log_artifacts(local_path, path_name)
821 else:
--> 822 artifact_repo.log_artifact(local_path, artifact_path)
File ~/code/syn_decoder/.venv/lib/python3.11/site-packages/mlflow/store/artifact/local_artifact_repo.py:45, in LocalArtifactRepository.log_artifact(self, local_file, artifact_path)
43 mkdir(artifact_dir)
44 try:
---> 45 shutil.copy2(local_file, os.path.join(artifact_dir, os.path.basename(local_file)))
46 except shutil.SameFileError:
47 pass
File /usr/lib/python3.11/shutil.py:437, in copy2(src, dst, follow_symlinks)
435 dst = os.path.join(dst, os.path.basename(src))
436 copyfile(src, dst, follow_symlinks=follow_symlinks)
--> 437 copystat(src, dst, follow_symlinks=follow_symlinks)
438 return dst
File /usr/lib/python3.11/shutil.py:376, in copystat(src, dst, follow_symlinks)
374 st = lookup("stat")(src, follow_symlinks=follow)
375 mode = stat.S_IMODE(st.st_mode)
--> 376 lookup("utime")(dst, ns=(st.st_atime_ns, st.st_mtime_ns),
377 follow_symlinks=follow)
378 # We must copy extended attributes before the file is (potentially)
379 # chmod()'ed read-only, otherwise setxattr() will error with -EACCES.
380 _copyxattr(src, dst, follow_symlinks=follow)
PermissionError: [Errno 1] Operation not permitted
I'm executing this code as a user
being part of a group
that has read/write access to the logging folder. The test.txt
file get copied, but then the error is thrown.
> ls -l $MLFLOW_DIR/940396422779777099/487c714b29d441e1af3e3b962b108afc/artifacts/
-rw-rw-r-- 1 root group 12 Sep 12 14:17 test.txt
What am I doing wrong ?
Upvotes: 0
Views: 71