CODEWITHSUNDEEP
nextflow
matteok
matteok

Reputation: 2199

handle input from multiple processes example in nextflow dsl2

How do I go about defining a workflow that executes two initial processes in parallel and then handles both outputs of those processes in a third process? The simple examples I was able to find in tutorials always define sequential flows and often use stdin/stdout to transport information.

In order to illustrate what I want to achieve, here is a diagram of the DAG that I imagine:

DAG

I imagine the .nf file to look something like the following but cannot fill in the blanks

#!/usr/bin/env nextflow
nextflow.enable.dsl=2

process produceRandomX {

    output:
        x // what do I put in these places?

    """
    print $RANDOM
    """
}


process produceRandomY {

    output:
        y

    """
    print -$RANDOM
    """
}

process calculateSum {
    input:
        x
        y
    output:
        sum

    """
    print $x+$y
    """
}

process printResult {
    input:
        sum
    output:
        stdout

    """
    print $sum
    """
}

workflow {
    // syntax here is broken and just meant to illustrate what I want to achieve
    produceRandomX \
                     | calculateSum | printResult
    produceRandomY /
}

Upvotes: 1

Views: 733

Answers (1)

Steve
Steve

Reputation: 54402

Nextflow processes can define one or more input and output channels. The interaction between these, and ultimately the pipeline execution itself, is implicitly defined by these input and output declarations1. In the following example, produceRandomX and produceRandomY could be run in parallel (assuming there are sufficient system resources available) but even if they're not, calculateSum will wait until it receives a complete input configuration (i.e. until it receives a value from each input channel).

process produceRandomX {

    output:
    stdout

    """
    echo "\$RANDOM"
    """
}

process produceRandomY {

    output:
    stdout

    """
    echo "-\$RANDOM"
    """
}

process calculateSum {

    input:
    val x
    val y

    output:
    stdout

    """
    echo \$(( $x + $y ))
    """
}

workflow {

    x = produceRandomX()
    y = produceRandomY()

    sum = calculateSum( x, y )

    sum.view()
}

Results:

$ nextflow run -ansi-log false main.nf 
N E X T F L O W  ~  version 22.10.0
Launching `main.nf` [lonely_torvalds] DSL2 - revision: 3b752ca2cc
[07/8e1473] Submitted process > produceRandomX
[fd/d9a629] Submitted process > produceRandomY
[15/c88aac] Submitted process > calculateSum
-1331

Upvotes: 2

Related Questions