CODEWITHSUNDEEP
pythonsftppysftp
Amir Zare
Amir Zare

Reputation: 583

Python pysftp.put raises "No such file" exception although file is uploaded

I am using pysftp to connect to a server and upload a file.

cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
self.sftp = pysftp.Connection(host=self.serverConnectionAuth['host'], port=self.serverConnectionAuth['port'],
                              username=self.serverConnectionAuth['username'], password=self.serverConnectionAuth['password'], 
                              cnopts=cnopts)
self.sftp.put(localpath=self.filepath+filename, remotepath=filename)

Although it works correctly most of the times, sometimes it raises the exception attached bellow. The file is read and processed by another program running on the server, so I can see that, even at these times, the file is indeed uploaded and is not corrupted.

  File "E:\Anaconda\envs\py35\lib\site-packages\pysftp\__init__.py", line 364, in put
    confirm=confirm)
  File "E:\Anaconda\envs\py35\lib\site-packages\paramiko\sftp_client.py", line 727, in put
    return self.putfo(fl, remotepath, file_size, callback, confirm)
  File "E:\Anaconda\envs\py35\lib\site-packages\paramiko\sftp_client.py", line 689, in putfo
    s = self.stat(remotepath)
  File "E:\Anaconda\envs\py35\lib\site-packages\paramiko\sftp_client.py", line 460, in stat
    t, msg = self._request(CMD_STAT, path)
  File "E:\Anaconda\envs\py35\lib\site-packages\paramiko\sftp_client.py", line 780, in _request
    return self._read_response(num)
  File "E:\Anaconda\envs\py35\lib\site-packages\paramiko\sftp_client.py", line 832, in _read_response
    self._convert_status(msg)
  File "E:\Anaconda\envs\py35\lib\site-packages\paramiko\sftp_client.py", line 861, in _convert_status
    raise IOError(errno.ENOENT, text)
FileNotFoundError: [Errno 2] No such file

How can I prevent this exception?

Upvotes: 3

Views: 6138

Answers (2)

Martin Prikryl
Martin Prikryl

Reputation: 202168

From the described behaviour, I assume that the file is removed very shortly after it is uploaded by some server-side process.

By default pysftp.Connection.put verifies the upload by checking a size of the target file. If the server-side processes manages to remove the file too fast, reading the file size would fail.

You can disable the post-upload check by setting confirm parameter to False:

self.sftp.put(localpath=self.filepath+filename, remotepath=filename, confirm=False)

I believe the check is redundant anyway, see
How to perform checksums during a SFTP file transfer for data integrity?

For a similar question about Paramiko (which pysftp uses internally), see:
Paramiko put method throws "[Errno 2] File not found" if SFTP server has trigger to automatically move file upon upload

And actually, you should use Paramiko directly, as pysftp is a dead project. See pysftp vs. Paramiko.

Upvotes: 14

mRyan
mRyan

Reputation: 390

Also had this issue of the file automatically getting moved before paramiko could do an os.stat on the uploaded file and compare the local and uploaded file sizes.

@Martin_Prikryl solution works works fine for removing the error by passing in confirm=False when using sftp.put or sftp.putfo

If you want this check to still run to verify the file has been uploaded fully you can run something along these lines. For this to work you will need to know the moved file location and have the ability to read the file.

import os

sftp.putfo(source_file_object, destination_file, confirm=False)
upload_size = sftp.stat(moved_path).st_size
local_size = os.stat(source_file_object).st_size
if upload_size != local_size:
    raise IOError(
        "size mismatch in put!  {} != {}".format(upload_size, local_size)
    )

Both checks use os.stat

Upvotes: 0

Related Questions