CODEWITHSUNDEEP
asynchronousjuliadistributed
Huy Tran
Huy Tran

Reputation: 61

Julia @distributed: subsequent code run before all workers finish

I have been headbutting on a wall for a few days around this code:

using Distributed
using SharedArrays

# Dimension size
M=10;
N=100;

z_ijw = zeros(Float64,M,N,M)
z_ijw_tmp = SharedArray{Float64}(M*M*N)
i2s = CartesianIndices(z_ijw)

@distributed for iall=1:(M*M*N)
    # get index
    i=i2s[iall][1]
    j=i2s[iall][2]
    w=i2s[iall][3]
    # Assign function value
    z_ijw_tmp[iall]=sqrt(i+j+w) # Any random function would do
end

# Print the last element of the array
println(z_ijw_tmp[end])
println(z_ijw_tmp[end])
println(z_ijw_tmp[end])

The first printed out number is always 0, the second number is either 0 or 10.95... (sqrt of 120, which is correct). The 3rd is either 0 or 10.95 (if the 2nd is 0)

So it appears that the print code (@mainthread?) is allowed to run before all the workers finish. Is there anyway for the print code to run properly the first time (without a wait command)

Without multiple println, I thought it was a problem with scope and spend a few days reading about it @.@

Upvotes: 2

Views: 515

Answers (1)

carstenbauer
carstenbauer

Reputation: 10147

@distributed with a reducer function, i.e. @distributed (+), will be synced, whereas @distributed without a reducer function will be started asynchronously.

Putting a @sync in front of your @distributed should make the code behave the way you want it to.

This is also noted in the documentation here:

Note that without a reducer function, @distributed executes asynchronously, i.e. it spawns independent tasks on all available workers and returns immediately without waiting for completion. To wait for completion, prefix the call with @sync

Upvotes: 3

Related Questions