CODEWITHSUNDEEP
google-cloud-storagegoogle-compute-engine
user3644954
user3644954

Reputation: 23

gsutil cp error--device or resource busy

I've got an issue with gsutil cp. I'm getting a "OSError: [Errno 16] Device or resource busy" error. I'm using python 2.7.2. The gsutil output is below. I tried changing my temp directory and that didn't help.
Any idea what might be causing this error? I've tried this a bunch of times and sometimes the file gets uploaded in spite of the error message, but sometimes it does not.
Thanks! 

Fri May 16 14:05:50 ~ > gsutil cp hello.txt gs://cg2
Copying file://hello.txt [Content-Type=text/plain]...
Traceback (most recent call last):
  File "/nfs/apps/python/2.7.2/lib/python2.7/multiprocessing/util.py", line 261, in _run_finalizers
    finalizer()
  File "/nfs/apps/python/2.7.2/lib/python2.7/multiprocessing/util.py", line 200, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/nfs/apps/python/2.7.2/lib/python2.7/shutil.py", line 249, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/nfs/apps/python/2.7.2/lib/python2.7/shutil.py", line 247, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/ifs/scratch/c2b2/ngs_lab/db2175/TEMP/pymp-omqdNL/.nfs00000000250c4c2000003c71'
Traceback (most recent call last):
  File "/nfs/apps/python/2.7.2/lib/python2.7/multiprocessing/util.py", line 261, in _run_finalizers
    finalizer()
  File "/nfs/apps/python/2.7.2/lib/python2.7/multiprocessing/util.py", line 200, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/nfs/apps/python/2.7.2/lib/python2.7/shutil.py", line 249, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/nfs/apps/python/2.7.2/lib/python2.7/shutil.py", line 247, in rmtree
    os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '/ifs/scratch/c2b2/ngs_lab/db2175/TEMP/pymp-u61unF/.nfs00000000251f72fd00003c72'

Upvotes: 2

Views: 1819

Answers (1)

CharlesB
CharlesB

Reputation: 227

If you look at the gsutil cp command source code, you will see that some code paths invoke tempfile.NamedTemporaryFile. It looks like the setting of TEMP/TMPDIR/TMP in your environment points to an NFS directory, which sometimes causes an error when the temporary directory is removed. If you change your environment so that tempfiles are created on local disk, you should stop experiencing this issue.

Upvotes: 1

Related Questions