CODEWITHSUNDEEP
apache-camelsftp
David
David

Reputation: 267

GenericFileOperationFailedException: Cannot change directory to: .. - when trying to SFTP 1G file

I am playing around with getting large files via SFTP and experimenting with the readLocks. When trying to get a 1G file (a large file taking many seconds to copy) I get the following exception. BTW, when I try a small file on the order of 1M, the SFTP works fine.

Is this a bug in Camel 2.12.1 or am I doing something wrong?

2013-12-13 14:30:12,964 [alhost/outbound]  INFO SftpConsumer                   - Connected and logged in to: sftp://tdr@localhost:22
2013-12-13 14:30:19,893 [alhost/outbound]  WARN SftpConsumer                   - Error processing file RemoteFile[Centos5.7-64-Base-GS-Image-UI.ova] due to Cannot change directory to: ... Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change directory to: ..]
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: ..
    at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:542)
    at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:530)
    at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:656)
    at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:594)
    at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:362)
    at org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:99)
    at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:201)
    at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:165)
    at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:187)
    at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:114)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)

My route looks like the following:

from("sftp://tdr@localhost/outbound?password=xxx&delete=true&readLock=changed&delay=10000&readLockCheckInterval=3000")
        .to("file:///home/cps/camel/input?flatten=true");

Upvotes: 2

Views: 8365

Answers (1)

David
David

Reputation: 267

After some googling, I found a recommendation to add stepwise=false to prevent the exception seen above.

After setting the stepwise=false, I then got an OutOfMemory exception and also added streamDownload=true.
The following route will now sftp my large file and I have proven that the readlocks work because I validated that the SFTP did not begin while I was copying the large file into the SFTP directory.

from("sftp://tdr@localhost/outbound?" +
        "streamDownload=true" +
        "&stepwise=false" +
        "&password=xxx&delete=true&readLock=changed&delay=5000&readLockCheckInterval=2000")
        .to("file:///home/cps/camel/input?" +
                "flatten=true");

Upvotes: 7

Related Questions