CODEWITHSUNDEEP
multithreadingparallel-processingjuliarandom-seed
AliG
AliG

Reputation: 303

Julia 1.5.2 Initiating random number selection in parallel

I am currently attempting to run my code in parallel. Part of my code (genetic algorithm) relies on the selection of random numbers. Therefore, I need to make sure that each thread used, have their own random number generator. I must also be able to replicate the results. Therefore, I have "borrowed" a function, which has the ability to set a random seed or create specific seeds for each thread.

https://discourse.julialang.org/t/how-do-i-deal-with-random-number-generation-when-multithreading/5636/2

In order to test if this was working correctly, I wanted to set the seeds equal for each thread and test to see if the same random numbers were generated. However, this was not the case!

I believe that I am missing something fundamental about the generation of the numbers. When creating the random number generators, I have used the same UInt32. I believe that they should therefore draw the same numbers, however the randjump may interfere with this. Perhaps the random number generators do not "jump" forward to the same point/state to ensure that each is far enough from each other in the cycle. Perhaps they share a state when doing it this way? Is there a way to control if the implementation is accurate? Perhaps there is a bigger error at play here?

clearconsole()
using Test
using Random
using Future
using Test

function set_rngs(random_seed)
    if random_seed == true
        rng_seed = Random.make_seed()
        rng = MersenneTwister(rng_seed)
        return Future.randjump(rng, Threads.nthreads());
    else
        rng_seed = UInt32[0xbd8e7dc4,0xbd8e7dc4];
        rng = Random.MersenneTwister(rng_seed)
        return Future.randjump(rng, Threads.nthreads());
    end
end

function random_test()
    tmp = Vector{Float64}(undef, 100)

    Threads.@threads for i in 1:100
        tmp[i] = rand()
    end
    return tmp
end

Random.seed!(set_rngs(false))
tester = random_test()


function testing_rand(tester::Vector{Float64})
    @testset "Testing if random numbers drawn by each processer are different" begin
        for i in 1:50
            @test tester[i] != tester[i+50]
        end
    end
end

testing_rand(tester)

Upvotes: 1

Views: 524

Answers (1)

Przemyslaw Szufel
Przemyslaw Szufel

Reputation: 42234

You basically do not need to use randjump because the MersenneTwister streams are not correlated.

Hence, the standard way to initiate the random streams for threads is the following:

using Random
const mts = MersenneTwister.(1:Threads.nthreads())

When writing a code you always access an approppiate random state via:

rand(mts[Threads.threadid()])

On some rare occasion you might indeed want to have different parts of the same random stream and need to perform randjump.

const mts2 = [Future.randjump(MersenneTwister(0), i) for i in 1:Threads.nthreads()]

The access pattern to the object is the same as above. Note that in each case you need to hold a separate random state for each thread. This approach is also recommended for performance reasons (rather than several threads competing for a single random state).

Upvotes: 2

Related Questions