CODEWITHSUNDEEP
julia
Cokes
Cokes

Reputation: 4093

Julia parallel for loop with two reductions

I would like to perform two reductions in a parallel for loop in Julia. I am trying to compute the error in a random forest inside the parallel for loop as each tree is built. Any ideas?

Current:

forest = @parallel (vcat) for i in 1:ntrees
    inds = rand(1:Nlabels, Nsamples)
    build_tree(labels[inds], features[inds,:], nsubfeatures)
end

What I want, intuitively is to do an addition inside this for loop as well to get the out of bag error. This is how I would wish for it to work:

forest, ooberror = @parallel (vcat, +) for i in 1:ntrees
    inds = rand(1:Nlabels, Nsamples)
    tree = build_tree(labels[inds], features[inds,:], nsubfeatures)
    error = geterror(ids, features, tree)
    (tree, error)
end

Upvotes: 4

Views: 485

Answers (1)

IainDunning
IainDunning

Reputation: 11664

Using a type might be best in terms of simplicity and clarity, e.g.

type Forest
  trees :: Vector
  error
end
join(a::Forest, b::Forest) = Forest(vcat(a.trees,b.trees), a.error+b.error)

#...

forest = @parallel (join) for i in 1:ntrees
    inds  = rand(1:Nlabels, Nsamples)
    tree  = build_tree(labels[inds], features[inds,:], nsubfeatures)
    error = geterror(ids, features, tree)
    Forest(tree, error)
end

Upvotes: 5

Related Questions