CODEWITHSUNDEEP
clojure
Josh Peterson
Josh Peterson

Reputation: 2329

Does Clojure use multiple threads in a map call?

I'm attempting to explore the behavior of a CPU-bound algorithm as it scales to multiple CPUs using Clojure. The algorithm takes a large sequence of consecutive integers as input, partitions the sequence into a given number of sub-sequences, then uses map to apply a function to each sub-sequence. Once the map function has completed, reduce is used to collect the results.

The full code is available on Github, but here is a sample:

(map computation-function (partitioning-function number-of-partitions input))

When I execute this code on a machine with twelve cores, I see most of the the cores in use, when I expect to see only one core in use.

Ideally, I would like to use pmap to use a given number of threads, but I am unable to cause the code to execute using only one thread.

So is Clojure spreading the computation across multiple CPUs? If so, is there anything that I can do to control this behavior?

Upvotes: 1

Views: 565

Answers (1)

JohnJ
JohnJ

Reputation: 4823

My understanding is that pmap uses multiple cores and map uses the current thread only. (There would be no point in having both functions in the library if both used all available cores.)

The following simple experiment shows that pmap uses separate threads and map does not:

(defn something-slow [x]
  (Thread/sleep 1000))

(map something-slow (range 5))
;; Takes 5 seconds
(pmap something-slow (range 5))
;; Takes 1 second

I do note that your GitHub code uses pmap in the example which runs in main-; if you change back to map does the parallelism persist?

Upvotes: 4

Related Questions