CODEWITHSUNDEEP
julia
Chiara
Chiara

Reputation: 11

Using Distributed and SharedArrays in Julia

Using Julia 1.2. I'm running a fairly large problem concerning numerous prosumers on an electricity market and want to use distributed computing so that the solution time is faster. But I'm not exactly sure what the layout should be, I don't want the workers to be writing into the shared arrays one on top of the other. Also my code still runs extremely slowly...

Could someone check that the layout below is correct? 

 using Distributed
 n_workers = workers()
 if n_workers[end]<3
    addprocs(3)
 end


 @everywhere using JuMP, Gurobi, DataFrames, CSV, DelimitedFiles, SharedArrays

 ###I initialize many shared arrays into which I want my results to go into
 Results=SharedArray{Float64,2}(length(vec4), 38)
 Injection_IndPros=SharedArray{Float64,3}(length(vec4)*length(day_steps), length(time_steps), tot_consumers)  
 Injection_Prosumers=SharedArray{Float64,2}(length(vec4)*length(day_steps), length(time_steps))
 Prosumer_caps=SharedArray{Float64,2}(length(vec4)*4, tot_households)
 Year_behav=SharedArray{Float64,2}(length(vec4)*length(time_steps),6)

 @sync @distributed for iter4=1:length(vec4)

     ###Call in a function which returns results 

     ####Then I carry out some calculations on these results (here I also use normal matrices, I do not use @everywhere here)

     ####Then I put my calculated results into the shared arrays for e.g.
     Results[iter4,:]=[w_max_with[1]/8, w_max_inj[1]/8, obj, Net_injection, Invest_PV, Totcap_PV, Invest_PVINV, Totcap_PVINV, Totgen_PV, Maxgen_PV, Curtailment_PV, Invest_BAT, Totcap_BAT, Invest_BATINV, Totcap_BATINV, Invest_WIND, Totcap_WIND, Totgen_WIND, Invest_Conv_Base, cap_conv_opt[1], Invest_Conv_Mid, cap_conv_opt[2], Invest_Conv_Peak, cap_conv_opt[3],Totgen_CONVBASE, Totgen_CONVMID, Totgen_CONVPEAK, Totgen_CONV, Totgen_ALL, Genshare_PV, Genshare_WIND, Genshare_CONVBASE, Genshare_CONVMID, Genshare_CONVPEAK, Yearavg_elec_price, Max_inj_pos, Avg_INVtoPV_ratio, Avg_INVtoBAT_ratio]


end #here ends the distributed for loop

Upvotes: 1

Views: 539

Answers (1)

Przemyslaw Szufel
Przemyslaw Szufel

Reputation: 42194

When asking a question you should provide a minimal working example not just copy paste some of you production code.

Here is a typical pattern for SharedArrays usage similar to what you are likely trying to do:


using SharedArrays, Distributed

addprocs(3)

@everywhere using SharedArrays, Distributed

sx = SharedArray(Array(reshape(1:30,5,6)))

@everywhere function myjob(v)
    sum(v)
end

@sync @distributed for i in 1:5
    s = myjob(sx[i,1:end-1]) + 1000*myid()
    sx[i,end] = s
end

Upvotes: 1

Related Questions