CODEWITHSUNDEEP
mavenjenkinsjunitworkflowjenkins-pipeline
jnikolic
jnikolic

Reputation: 21

Jenkins Workflow: How to execute integration tests in parallel

Hi all i have been working on tis issue for some time and i cant find solution becouse integration tests run each time in new directory called MyProject@1,2,3,4...

I have stated that i want them to run in specific directory but that is not helping. Here is example of my code.

parallel( FirstSuite: {
    node('master') {
         ws('/opt/.jenkins/workspace/MyProject') {
             sh 'sleep 0 && mvn -s settings.xml -f java/pom.xml integration-test -P dev  -Dmaven.repo.local=.repository -Dit.test=FirstSuite -Djava.io.tmpdir=java/target -Dlog.dir=java/target/log/FirstSuite'
         } 
     }
 }, 
 SecondSuite: {
     node('master') {
         ws('/opt/.jenkins/workspace/MyProject') {
             sh 'sleep 20 && mvn -s settings.xml -f java/pom.xml integration-test -P dev  -Dmaven.repo.local=.repository -Dit.test=SecondSuite -Djava.io.tmpdir=java/target -Dlog.dir=java/target/log/SecondSuite'
         }
     }
 },
 ThirdSuite: {
     node('master') {
         ws('/opt/.jenkins/workspace/MyProject') {
             sh 'sleep 40 && mvn -s settings.xml -f java/pom.xml integration-test -P dev  -Dmaven.repo.local=.repository -Dit.test=ThirdSuite -Djava.io.tmpdir=java/target -Dlog.dir=java/target/log/ThirdSuite' 
         }
     }
 },
 FourthSuite: {
     node('master') {
         ws('/opt/.jenkins/workspace/MyProject') {
             sh 'sleep 60 && mvn -s settings.xml -f java/pom.xml integration-test -P dev  -Dmaven.repo.local=.repository -Dit.test=FourthSuite -Djava.io.tmpdir=java/target -Dlog.dir=java/target/log/FourthSuite' 
         }
     }
 },
 FifthSuite: {
     node('master') {
         ws('/opt/.jenkins/workspace/MyProject') {
             sh 'sleep 80 && mvn -s settings.xml -f java/pom.xml integration-test -P dev  -Dmaven.repo.local=.repository -Dit.test=FifthSuite -Djava.io.tmpdir=java/target -Dlog.dir=java/target/log/FifthSuite' 
         }
     }
 },
 SextSuite: {
     node('master') {
         ws('/opt/.jenkins/workspace/MyProject') {
             sh 'sleep 100 && mvn -s settings.xml -f java/pom.xml integration-test -P dev  -Dmaven.repo.local=.repository -Dit.test=SextSuite -Djava.io.tmpdir=java/target -Dlog.dir=java/target/log/SextSuite' 
         }
     }
 },
 SeventhSuite: {
     node('master') {
         ws('/opt/.jenkins/workspace/MyProject') {
             sh 'sleep 120 && mvn -s settings.xml -f java/pom.xml integration-test -P dev  -Dmaven.repo.local=.repository -Dit.test=SeventhSuite -Djava.io.tmpdir=java/target -Dlog.dir=java/target/log/SeventhSuite' 
         }
     }
 },
 EighthSuite: {
     node('master') {
         ws('/opt/.jenkins/workspace/MyProject') {
             sh 'sleep 140 && mvn -s settings.xml -f java/pom.xml integration-test -P dev  -Dmaven.repo.local=.repository -Dit.test=EighthSuite -Djava.io.tmpdir=java/target -Dlog.dir=java/target/log/EighthSuite' 
         }
     }
 },)

But i am still getting errors in maven that pom is not readable, and that directory is missing becouse this ws command is not helping. My integration tests are executed in MyProject@2, @3, @4 and so on...

[FirstSuite] + mvn -s settings.xml -f java/pom.xml integration-test -P dev -Dmaven.repo.local=.repository -Dit.test=FirstSuite -Djava.io.tmpdir=java/target -Dlog.dir=java/target/FirstSuite
[FirstSuite] Java HotSpot(TM) Server VM warning: ignoring option MaxPermSize=384m; support was removed in 8.0
[FirstSuite] [INFO] Scanning for projects...
[FirstSuite] [ERROR] The build could not read 1 project -> [Help 1]
[PmClientTestsuite] [ERROR]   
[FirstSuite] [ERROR]   The project  (/opt/.jenkins/workspace/MyProject@16/java/pom.xml) has 1 error
[FirstSuite] [ERROR]     Non-readable POM /opt/.jenkins/workspace/MyProject@16/java/pom.xml: /opt/.jenkins/workspace/MyProject@16/java/pom.xml (Datei oder Verzeichnis nicht gefunden)
[FirstSuite] [ERROR] 
[FirstSuite] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[FirstSuite] [ERROR] Re-run Maven using the -X switch to enable full debug logging.
[FirstSuite] [ERROR] 
[FirstSuite] [ERROR] For more information about the errors and possible solutions, please read the following articles:
[FirstSuite] [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException

Upvotes: 1

Views: 1179

Answers (1)

izzekil
izzekil

Reputation: 6071

This is because you use ws step. Instead of it use dir which does not check uniqueness of the path.

From the Jenkins Pipeline Snippet Generator help hint about ws:

If concurrent builds ask for the same workspace, a directory with a suffix such as @2 may be locked instead. Currently there is no option to wait to lock the exact directory requested; if you need to enforce that behavior, you can either fail (error) when pwd indicates that you got a different directory, or you may enforce serial execution of this part of the build by some other means such as stage name: '…', concurrency: 1.

If you do not care about locking, just use the dir step to change current directory.

In addition I would suggest moving node out of parallel if your actual intention is to perform all execution streams on the same node. Like this:

node('master') {
    dir('/opt/.jenkins/workspace/MyProject') {
        parallel FirstSuite: { ... },
                 SecondSuite: { ... },
                 <... other suites ... >
    }
}

Upvotes: 1

Related Questions