CODEWITHSUNDEEP
jenkinsparallel-processingjenkins-pipelinejenkins-plugins
quangthang10d4
quangthang10d4

Reputation: 38

How to restrict parallel jobs to particular agents in Declarative Pipeline

I have 3 nodes: A, B, C

On each of these nodes I set up a Jenkins agent with its own root directory They all have the following label: test && database && mysql

I want to run a job in parallel on all 3 nodes, to clean the workspace folder on them To achieve that, I wrote this Jenkins script

def labels = "test && mysql && database"

def getNodesName(labels){
    def targets = []
    def nodes = Jenkins.instance.getLabel(labels).getNodes()
    for(node in nodes){
        targets.add(node.getNodeName())
    }
    return targets
}

def nodes = getNodesName(labels)

def cleanWSTasks(targets){
    tasks = [:]
    for(target in targets){
        tasks[target] = {
            node(target){
                script {
                    cleanWs()
                }
            }
        }
    }
    return tasks
}

pipeline{
    agent none
    
    stages{
        
        stage ('Clean Workspace'){
            steps{
                script{
                    parallel cleanWSTasks(nodes)
                }
            }
        }

    }
}

So I thought with node(target) in the cleanWsTasks function I already told Jenkins to restrict the execution of the task only on the particular target node I want. So that all 3 nodes will start cleaning their own workspaces at the same time.

However, what I see is that only 1 node picked up the task to cleanUp the workspace, and it does it 3 times.

For example, it shows:

Running on node A in ...

clean up workspace ..

Running on node A in ...

clean up workspace ..

Running on node A in ...

clean up workspace ..

What did I do wrong in my code? Please help.

Upvotes: 1

Views: 497

Answers (1)

Vasiliki Siakka
Vasiliki Siakka

Reputation: 1283

The node step is working correctly, the problem you're coming across has to do with how you're defining your tasks. In your for loop, you're assigning this closure:

          {
            node(target){
                script {
                    cleanWs()
                }
            }

to tasks[target].

The code inside the closure won't get evaluated until you execute the closure. So even though you assign node(target) inside the for loop, target's value won't get evaluated until parallel tasks runs, which is when the closure is executed. That happens after the for loop has finished running and so target's value is the name of the last node in your list of nodes.

An easy fix for this is to create a variable in your for loop that's equal to target and use that inside the closure, because you will force the evaluation of target to happen inside your for loop, instead of when the closure runs. That would look like this:

def cleanWSTasks(targets){
    tasks = [:]
    for(target in targets){
        def thisTarget = target
        tasks[thisTarget] = {
            node(thisTarget){
                script {
                    cleanWs()
                }
            }
        }
    }
    return tasks
}

Upvotes: 1

Related Questions