CODEWITHSUNDEEP
memoryclojure
Carcigenicate
Carcigenicate

Reputation: 45750

Program refuses to continually consume memory

I was trying this challenge, which requires you to write a program that continually consumes memory.

I figured this would be easy, so I wrote up:

(reduce conj [] (range))

Which basically adds numbers from an infinite range into a vector; theoretically forever.

The problem is, this jumps up to 3/4GB of memory, then just stops. The CPU is still running hard, but it refuses to grow any larger.

I decided to steal an idea from a Java answer and override finalize to create more objects every time one is deallocated:

(defrecord A []
  Object
  (finalize [this] (mapv (fn [_] (A.)) (range 5)))) ; Create 5 more

(mapv (fn [_] (A.)) (range))

But this stops growing around the same point as well.

How is this not exploding? Especially since I'm using a strict map, it should be retaining everything in memory shouldn't it? As far as it "knows", I'll want it to print the entire list at some point, so it needs to hold onto everything. Plus, if it was deallocating at some point, shouldn't the overridden finalize method overcome this?

Can anyone explain what's going on here? I wrote this on my break, so I may be overlooking something, but I can't see what. It was tested in Intellij/Cursive's REPL.

Upvotes: 4

Views: 81

Answers (1)

ClojureMostly
ClojureMostly

Reputation: 4713

Your JVM bascially spends the entire time garbage collecting. On my system the limit of the default heap size is ~8.7GB so depending what this gives you, your GC will be trying to stay below this even earlier:

java -XX:+PrintFlagsFinal -version | grep HeapSize

You can increase this with -Xmx JVM option.

You program will actually continue running and keep adding things to your vector. I recommend starting your JVM with the option :XX:+PrintGCDetails to see what's going on. After you get close to your heap limit the JVM will do Full GCs for seconds. Note, that if the JVM spends too much time for the GC then it will throw and OOM error.

To see that you're still allocating, just throw a (when (zero? (rem x 1e5)) (println x)) in:

java -server -Xmx2g -XX:+PrintGCDetails -cp clojure-1.8.0.jar clojure.main -e '(mapv #(cond-> % (zero? (rem % 1e4)) println) (range))'

Upvotes: 2

Related Questions