CODEWITHSUNDEEP
ruby-on-railsherokusidekiqrmagick
josal
josal

Reputation: 452

Heroku worker dyno giving R14 errors - problems with memory management - rmagick memory limit options

I've tried to use a 2X worker dyno, waiting to not get R14 errors, but this is the result:

2013-06-04T13:03:30.723260+00:00 heroku[worker.1]: Process running mem=1047M(102.3%)
2013-06-04T13:03:30.723260+00:00 heroku[worker.1]: Error R14 (Memory quota exceeded)

However, the task for the worker is finished successfully.

Questions:

Thanks in advance

Upvotes: 5

Views: 2382

Answers (2)

Igor
Igor

Reputation: 1331

A little late for the party, but in my case the garbage collector was my savior. Just run GC.start after each job.

Upvotes: 0

josal
josal

Reputation: 452

I finally found a solution. The problem was with rmagick. It gets all the memory it sees available. It doesn't matter if you use a 2X worker dyno instead of a 1X one. It grows to the maximum available resources. So, we have to set a limit. But in my case not all the limits worked.

You can make a system call directly this way:

convert -limit memory 0 -limit map 0 list_of_input_files output_file

This way, you avoid to use the memory cache, going directly to the disk cache. This is the only way I've found to avoid R14 errors in heroku. With other combinations like -limit memory 32 -limit map 64 or similar, it always gave me errors. I took the idea from here.

Of course, you could always use the rmagick library with this lines, however it didn't work for me and I did use the syscall approach explained before:

Magick.limit_resource(:memory, 0)
Magick.limit_resource(:map, 0)

UPDATE: I've used the nice command to ensure more priority. It seems it improves, but eventually I'm getting R14 errors, but it's true they are not so often (even with the same file!).

Upvotes: 5

Related Questions