Combining multiple collectEntries in Groovy and Jenkins Pipeline

Question

I am trying to use multiple collectEntries in series in my Groovy script. Its better to see the code, right now I've got:

stage('Test') {
            // Reading content of the file
            def portsFileContent = readFile 'UsedPorts.txt'

            // Split the file by next line
            def ports = portsFileContent.split('\n')

            def steps = ports.collectEntries { port ->
                ["UI Test on port $port", {
                    sh "#!/bin/bash -lx \n startServerWithDifferentPort --params=port=$port"
                }]
            }
            parallel steps
        }

In the file "UsedPorts.txt" are different ports seperate by line break like:

4723
4733
4743

So this numbers getting stored in the variable ports and this variable is then used to start an instance of an server for each port. So in this case it would then start 3 different serverinstance by this command:

def steps = ports.collectEntries { port ->
                ["UI Test on port $port", {
                    sh "#!/bin/bash -lx \n startServerWithDifferentPort --params=port=$port"
                }]
            }
            parallel steps

Because of parallel steps its starting 3 instance of the server simultaneously with different ports.

Thats working fine, but I have another file and need to do the same again. So in my second file there are entries like:

name1
name2
name3

I created again a variable where I store my 3 entries:

def anotherFile = readFile 'Names.txt'
def names = anotherFile.split('\n')

This is what I tried:

stage('Test') {
            // Reading content of the file
            def portsFileContent = readFile 'UsedPorts.txt'

            // Split the file by next line
            def ports = portsFileContent.split('\n')

            // Do the same again
            def anotherFile = readFile 'Names.txt'
            def names = anotherFile.split('\n')

            def steps = ports.collectEntries, names.collectEntries { port, name ->
                ["UI Test on $name", {
                    sh "#!/bin/bash -lx \n someMoreShellStuff --params=port=$port"
                }]
            }
            parallel steps
        }

But I can not seperate my second collectEntries by comma, because it gives me a syntax error. And now my problem is, how I can use this variable in the same command. Is it even possible?

Thanks

Update #1

After using the answer of Szymon Stepniak my new code looks like this:

stage('Test') {
            // Reading content of the file
            def portsFileContent = readFile 'AppiumUsedPorts.txt'

            // Split the file by next line
            def ports = portsFileContent.split('\n')

            // Getting device IDs to get properties of device
            def deviceIDFileContent = readFile 'DeviceIDs.txt'
            def deviceIDs = deviceIDFileContent.split('\n')

            // Define port and id as an pair
            def pairs = (0..Math.min(ports.size(), deviceIDs.size())).collect { i -> [id: deviceIDs[i], port: ports[i]] }

            def steps = pairs.collectEntries { pair ->
                ["UI Test on ${pair.id}", {
                    sh "echo 'Running test with port ${pair.port}'"
                }]
            }
            parallel steps
        }

This is causing the error java.lang.ArrayIndexOutOfBoundsException

Update #2

Content of AppiumUsedPorts.txt:

4723
4733

Content of DeviceIDs.txt

5353352c
G000KU0663550R92

Answer 1

Looks like you want to zip elements from two lists - ports and names and use these pairs in creating steps for parallel execution. So assuming that ports and names contain something like:

def ports = [8080, 8081, 8082, 8083]
def names = ['Host A', 'Host B', 'Host C', 'Host D', 'Host E']

you need a list of pairs like:

def pairs = [[port: 8080, name: 'Host A'], [port: 8081, name: 'Host B'], [port: 8082, name: 'Host C'], [port:8083, 'Host D']]

I used two different size lists on purpose, to explain that result of zipping two lists is always the same size then the shortest list.

Groovy has a method GroovyCollections.transpose(List lists) that takes a list of lists (e.g. [[8080, 8081, 8082, 8083], ['Host A', 'Host B', 'Host C', 'Host D', 'Host E']]) and "zips" two lists together like:

[[8080, 'Host A'], [8081, 'Host B'], [8082, 'Host C'], [8083, 'Host D']]

but it wont work in Jenkins Pipeline - if you try to use it you will get:

org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods transpose java.util.List

Anyway you can simply do pretty the same using collect on range from 0 to min(ports.size(), names.size()) to create a list those pairs/maps. Take a look at following example:

node {
    stage('Test') {
        def ports = [8080, 8081, 8082, 8083]
        def names = ['Host A', 'Host B', 'Host C', 'Host D', 'Host E']

        def pairs = (0..<Math.min(ports.size(), names.size())).collect { i -> [name: names[i], port: ports[i]] }

        def steps = pairs.collectEntries { pair ->
            ["UI Test on ${pair.name}", {
                sh "echo 'Running test with port ${pair.port}'"
            }]
        }

        parallel steps
    }
}

In this example we transpose two lists into a list of maps like [port: ..., name: ...] and we call collectEntries on that list of maps to get both - port and name in the same execution step. Running this script in Jenkins Pipeline produces following output:

[Pipeline] {
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] parallel
[Pipeline] [UI Test on Host A] { (Branch: UI Test on Host A)
[Pipeline] [UI Test on Host B] { (Branch: UI Test on Host B)
[Pipeline] [UI Test on Host C] { (Branch: UI Test on Host C)
[Pipeline] [UI Test on Host D] { (Branch: UI Test on Host D)
[Pipeline] [UI Test on Host A] sh
[UI Test on Host A] [test-pipeline] Running shell script
[Pipeline] [UI Test on Host B] sh
[UI Test on Host B] [test-pipeline] Running shell script
[Pipeline] [UI Test on Host C] sh
[UI Test on Host C] [test-pipeline] Running shell script
[Pipeline] [UI Test on Host D] sh
[UI Test on Host A] + echo Running test with port 8080
[UI Test on Host A] Running test with port 8080
[UI Test on Host B] + echo Running test with port 8081
[UI Test on Host B] Running test with port 8081
[UI Test on Host D] [test-pipeline] Running shell script
[Pipeline] [UI Test on Host A] }
[UI Test on Host C] + echo Running test with port 8082
[UI Test on Host C] Running test with port 8082
[UI Test on Host D] + echo Running test with port 8083
[UI Test on Host D] Running test with port 8083
[Pipeline] [UI Test on Host B] }
[Pipeline] [UI Test on Host C] }
[Pipeline] [UI Test on Host D] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

Hope it helps.