CODEWITHSUNDEEP
makefileparallel-processing
Sharath BJ
Sharath BJ

Reputation: 1473

Build multiple packages in parallel

how to build packages in Parallel to reduce the time taken by make command(Golang)

my current directory is "thor"

inside my Makefile

build:
    go build -o thor
    go build thor/package1
    go build thor/scripts/package2

in the above case ,all the packages are stand alone processes which run independently . when i run make command each packages build one by one . so if each package build take 30 sec ,totally it takes 90 sec(30 sec * 3) . but if i can build these packages in parallel ,it will only take 30 sec.

in case of shell scripts this kind of cases can be handled by running each scripts in background with & and wait till the end of each scripts using wait

sample code

#!/bin/bash
echo "Starts"
./s1.sh &
./s2.sh &
./s3.sh &
wait
echo "ends"

In the above cases all the scripts s1.sh ,s2.sh ,s3.sh will run concurrently and wait till all the process finished.

So , can we do something like this in Makefile also :)

Upvotes: 0

Views: 2935

Answers (2)

Muhammad Soliman
Muhammad Soliman

Reputation: 23756

You can use gox that will parallelize builds for multiple platforms.

By default, Gox will parallelize based on the number of CPUs you have and build for every platform by default

Exactly the same as go build, build using

gox

If you want to build packages and sub-packages: 

gox ./...

Upvotes: 2

user2371524
user2371524

Reputation:

Instead of

build:
    go build -o thor
    go build thor/package1
    go build thor/scripts/package2

split this in three separate recipes, something like this

build: build1 build2 build3

build1:
    go build -o thor

build2:
    go build thor/package1

build3:
    go build thor/scripts/package2


.PHONY: build build1 build2 build3

Then call make with the -j option and you're done.

It would be better to have targets that correspond to files that are actually created by your go build command, together with a list of prerequisites. Then you don't need .PHONY and make can decide whether a rebuild is actually needed.

Upvotes: 6

Related Questions