CODEWITHSUNDEEP
nextflow
mehmet
mehmet

Reputation: 110

publish two separate result directories from nextflow process

I have a nextflow process that outputs a result folder and a log as below:

process test {
    label "x"

    input:
    path(somefile)

    output:
    path "test_results"
    path "test_log.log"
    path "something_for_next_process", emit: f

    shell:
    '''
    myshellcommand
    '''
}

And a config file to publish the results like below.

process
{
    withLabel : x
    {
        publishDir =
        [
            path: {"$params.outdir/"},
            pattern: "*_results",
            mode: 'copy',
            saveAs: {filename -> "${filename.split('_')[0]}/all_results/${filename.split("_")[1]}"}
        ]
    }
    withLabel : x
    {
        publishDir =
        [
            path: {"$params.outdir/"},
            pattern: "*.log",
            mode: 'copy',
            saveAs: {filename -> "${filename.split('_')[0]}/logs/${filename}"}
        ]
    }
}

I tried multiple combinations however, I can't get a label to publish its desired contents to two different folders. It always takes whichever is the last one in the publish config (in this case the logs). I know I can just put the publishdir options to the process in the workflow and that also works but I would like to do it through the config file. Is there a way I can get this to work?

Upvotes: 3

Views: 1019

Answers (2)

Steve
Steve

Reputation: 54502

You can specify the publishDir directive more than once in your nextflow.config by supplying a list of maps, for example:

process {

  withLabel: 'x' {

    publishDir = [
      [
        path: { "$params.outdir/" },
        pattern: "*_results",
        mode: 'copy',
        saveAs: { fn -> "${fn.split('_')[0]}/all_results/${fn.split("_")[1]}" }
      ],
      [
        path: { "$params.outdir/" },
        pattern: "*.log",
        mode: 'copy',
        saveAs: { fn -> "${fn.split('_')[0]}/logs/${fn}" }
      ]
    ]
  }
}

If you don't need the withName or withLabel process selectors, the publishDir directive can just be specified multiple times inside of your process definition, for example:

process test {

    publishDir(
        path: { "$params.outdir/" },
        pattern: "*_results",
        mode: 'copy',
        saveAs: { fn -> "${fn.split('_')[0]}/all_results/${fn.split("_")[1]}" }
    )
    publishDir(
        path: { "$params.outdir/" },
        pattern: "*.log",
        mode: 'copy',
        saveAs: { fn -> "${fn.split('_')[0]}/logs/${fn}" }
    )

    ...
}

Upvotes: 5

mribeirodantas
mribeirodantas

Reputation: 489

You can have multiple publishDir directives for the same process. No need to have the withLabel twice. If it was in the process block, it would be like the snippet below, for example.

process foo {
  publishDir "$params.outdir/$sampleId/counts", pattern: "*_counts.txt"
  publishDir "$params.outdir/$sampleId/outlooks", pattern: '*_outlook.txt'
  publishDir "$params.outdir/$sampleId/", pattern: '*.fq'

  input:

Check an example here.

Upvotes: 1

Related Questions