CODEWITHSUNDEEP
ruby-on-railsherokumemory-leaksbackground-processheroku-toolbelt
Kelsey Hannan
Kelsey Hannan

Reputation: 2947

How to spin up Heroku One-Off Dynos from within my app?

I have Sidekiq background jobs in my Heroku Rails app suffering from huge, persistent memory leaks (900mb or higher). These memory leaks stay within my Worker Dynos after these tasks get run, resulting in them triggering many R14 and even R15 errors within my Worker dyno unless I or Heroku restarts my worker dyno (such as after 24 hours).

One solution that has worked to reduce the impact of these memory leaks has been to move our rake tasks over to Heroku scheduler, where we can benefit from Heroku spinning up One-Off dynos with their own separate memory and process to do each job for us before spinning down again. For scheduled tasks this has given us a lot of breathing room to isolate the impact of these memory leaks since each one is prevented from affecting others.

However, many of our memory intensive background jobs cannot be moved to Heroku scheduler because they are triggered as a result of what people do within our app.

How can I move app-triggered background jobs to Heroku One-Off Dynos?

Upvotes: 3

Views: 1095

Answers (1)

Michał Młoźniak
Michał Młoźniak

Reputation: 5556

The best way to do that IHMO is to use Heroku API. This method has many advantages:

  • you will get api response if dyno was started or not
  • you can select dyno size, so it can be different that your main app and it can be set kind of dynamically. I can imagine a use case when you start dyno of different size depending on your task size
  • you can pass additional ENV variables, so you can treat it like arguments
  • you can set time to live

Here is sample request from the docs (it is missing authentication token):

curl -n -X POST https://api.heroku.com/apps/$APP_ID_OR_NAME/dynos \
  -d '{
    "attach": true,
    "command": "bash",
    "env": {
      "COLUMNS": "80",
      "LINES": "24"
    },
    "force_no_tty": null,
    "size": "standard-1X",
    "type": "run",
    "time_to_live": 1800
  }' \
-H "Content-Type: application/json" \
-H "Accept: application/vnd.heroku+json; version=3"

One possible disadvantage is that you need to pass authentication token to your application via ENV variable, but there is no other way I think. With different solutions you will still have to pass authentication token.

Upvotes: 3

Related Questions