CODEWITHSUNDEEP
ruby-on-railsrubyherokumemory-managementdownload
taylorthurlow
taylorthurlow

Reputation: 3083

Streamed ZIP file download from Rails on Heroku causes increasing memory usage

I'm using zipline to generate and stream a ZIP file as a browser download. The ZIP generation and downloading happen simultaneously. Everything functions as expected but Heroku seems to use increasingly large amounts of memory as files are downloaded. The increase in memory consumption is generally close to, but not as large in size as the zip file being downloaded. The ZIP is compressed but the contents are highly compressed already so I don't think this difference is due to that. Consistently triggering a re-download of the same ZIP does repeatedly increase memory usage, but it does seem to trail off.

There is no passive memory leak, but at least as far as Heroku is reporting, it's clear that something about this process requests more memory and doesn't seem to ever release it to the OS (?).

Last 24 hours:

last 24 hours memory usage

Last 2 hours:

last 2 hours memory usage

Ruby metrics as reported by Heroku:

heroku ruby memory metrics

Virtually every sharp increase in the memory usage graphs is a ZIP download being triggered. I changed to a dyno with 1GB memory instead of 512MB to see if it would help or release memory but it did not.

Ruby free memory slots and heap objects do not seem to change at all. I don't know if this is a flaw in the beta metrics plugin on Heroku but I don't know how to investigate that.

The zipline gem uses ZipTricks as a back-end, and I've re-implemented what I already had done with ZipTricks directly and found the exact same problem.

Upvotes: 3

Views: 296

Answers (1)

iyoda
iyoda

Reputation: 49

https://github.com/schneems/puma_worker_killer Why don't you try this? Memory fragmentation seems to progress even when memory release is performed.

Upvotes: 2

Related Questions