CODEWITHSUNDEEP
crystal-lang
HaydenJones
HaydenJones

Reputation: 13

How can I change this to not run out of memory?

So basically this is trying to generate a wordsquare of size N. It is a very naive algorithm, because it is bruting every combination(N) in my word list. However, I read the docs a bit and I thought that adding .each would make this not put all this on the heap? I'm not really sure how it all actually works, but it sure does seem like this should be working.

It works for N=2, but for anything higher than that:

GC Warning: Failed to expand heap by 8388608 bytes GC Warning: Failed to expand heap by 8388608 bytes GC Warning: Failed to expand heap by 65536 bytes GC Warning: Out of Memory! Heap size: 2967 MiB. Returning NULL! Invalid memory access (signal 11) at address 0xc [0x5d700] *CallStack::print_backtrace:Int32 +104 [0x5222c] __crystal_sigfault_handler +84

I am coming to crystal from Ruby, so it's not like I actually know what I'm doing here. Any tips would be appreciated here! I hope I made it clear enough exactly what I am trying to do here, but I can add more details if needed.

def wordsquare(n)
  words = File.read_lines("../txt/data/scrabble.txt").map{ |s| s.chomp }.select { |w| w.size == n }
  words
    .combinations(n).each
    .select do |combo|
      (0...n).all? { |idx| words.includes?(combo.map { |cmb| cmb[idx] }.join) }
    end
end

p wordsquare(3).first(1)

# why is it running out of memory even though it's an iterator?
# :(

Upvotes: 1

Views: 393

Answers (1)

Jonne Haß
Jonne Haß

Reputation: 4857

You want to use Array#each_combination, setting reuse to true for maximum memory savings. Do not keep around or modify the array handed to you in that case.

Upvotes: 1

Related Questions