CODEWITHSUNDEEP
nodeshpcnextflow
juststuck
juststuck

Reputation: 656

How to make nextflow use tmp directory of specific node?

I have to run multiples nextflow pipeline in parallel, however, by doing so tmp/ directory becomes clogged up. Apparent reason is that while there are 30 different nodes available with large tmp/ directories, nextflow uses only the first one, which is filled pretty quickly so "not enough space" error is thrown out.

What i tried was running each nextflow pipe from one specific compute node, but how to make it use only tmp/ of this specific node?

Upvotes: 0

Views: 1776

Answers (1)

Steve
Steve

Reputation: 54502

The scratch directive will allow you to execute one or more of your processes in a temporary folder that is local to the execution node. If your temporary directory is defined by the $TMPDIR environment variable, you can use:

process myprocess {

  scratch true

  """
  <your script here>
  """
}

If your scratch directory is defined by some other variable, ensure it's wrapped in single quotes:

process myprocess {

  scratch '$TEMP_DIR'

  """
  <your script here>
  """
}

Upvotes: 4

Related Questions