Jsch SFTP client unable to create new native thread

Question

I use Jsch as SFTP client to read and write XML files from a remote SFTP directory.

I use a 5 second job to check if new files available for drafts, after 30 or 40 min loop I get the following error

Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method) [rt.jar:1.7.0_65]
        at java.lang.Thread.start(Thread.java:714) [rt.jar:1.7.0_65]
        at com.jcraft.jsch.Session.connect(Session.java:528) [jsch-0.1.53.jar:]
        at com.jcraft.jsch.Session.connect(Session.java:183) [jsch-0.1.53.jar:]

This is the source code used to create connexion

 public InputStream getFile(String path){
    Session session = null;
    Channel channel = null;
    try {

      ChannelSftp sftp = openConnexion(session, channel);
      return sftp.get(path);

    } catch (SftpException e) {
      new RuntimeException("Error detected during get file from SFTP specific path : " + e.getMessage(), e);

    } finally {
      closeConnexion(session, channel);
    }
 }

 private ChannelSftp openConnexion(Session session, Channel channel) {
    try {
      JSch ssh = new JSch();
      session = ssh.getSession("user", "hostname", 22);
      session.setPassword("password");
      session.setConfig("StrictHostKeyChecking", "no");
      session.connect();
      channel = session.openChannel(SFTP_CHANNEL);
      channel.connect();
      ChannelSftp sftp = (ChannelSftp) channel;

      return sftp;

    } catch (JSchException e) {
      throw new RuntimeException("Error detected during open SFTP connexion : " + e.getMessage(), e);
    }
  }

  private void closeConnexion(Session session, Channel channel) {
    if (channel != null) {
      channel.disconnect();
    }
    if (session != null) {
      session.disconnect();
    }
  }

I tried to increase the size of JVM thread stack and also increase the limits of native process allowed by unix => same error.

I used the following command to do that :

ulimit -u unlimited

I tried to create a pool of jsch session, jsch session when it is not disconnected, it is unusable => "SFTP Error 4"

My job is runned into war deployed on jboss-as-7, this is the JVM option :

JAVA_OPTS="-Xms1024m -Xmx1024m -XX:MaxPermSize=256m -Xss1024k"

Do you have a suggestion for this kind of treatment?

Thank you !

Answer 1

The problem is that you're not closing the channel and session after each loop, which will leak at least the thread that's used to perform the download over SFTP.

The attempt to close the session and channel in the finally block, if it worked, would unfortunately invalidate the InputStream that you're trying to read from; preventing you from processing the file properly.

I'm going to refactor the code slightly, which should address the resource exhaustion issue, with comments:

// session and channel are at the object scope
Session session = null;
Channel channel = null;

public InputStream getFile(String path){
    // First, close any existing connections.
    try {
      closeConnexion();
    } catch (SftpException e) {
      // You can try to handle an issue here; but it's
      // probably not worth it
    }
    try {
      ChannelSftp sftp = openConnexion();
      return sftp.get(path);
    } catch (SftpException e) {
      new RuntimeException("Error detected during get file from SFTP specific path : " + e.getMessage(), e);

    } finally {
    }
 }

 private ChannelSftp openConnexion() {
    try {
      JSch ssh = new JSch();
      // use the object's session variable
      session = ssh.getSession("user", "hostname", 22);
      session.setPassword("password");
      session.setConfig("StrictHostKeyChecking", "no");
      session.connect();
      // use the object's channel object
      channel = session.openChannel(SFTP_CHANNEL);
      channel.connect();
      ChannelSftp sftp = (ChannelSftp) channel;

      return sftp;

    } catch (JSchException e) {
      throw new RuntimeException("Error detected during open SFTP connexion : " + e.getMessage(), e);
    }
  }

  private void closeConnexion() {
    // close object's channel and session
    if (channel != null) {
      channel.disconnect();
      channel = null;
    }
    if (session != null) {
      session.disconnect();
      session = null;
    }
  }

If I was to re-design this, I would return a container class rather than an InputStream that contained the channel, session and InputStream. The container class would have a 'close' method, which would close the InputStream, channel and session, and then I wouldn't store the channel and session in the object.