CODEWITHSUNDEEP
ruby-on-railsruby-on-rails-3twitterherokumemcached
Undistraction
Undistraction

Reputation: 43401

Twitter Rate Limits for Site hosted on Heroku

I have a rails application hosted on Heroku using a custom Domain. The application displays the latest Tweet by a single user at the top of each page. To avoid hitting Twitter rate limits (150 requests p/h) I have the application cache the search result in memcache (Dalli) with a 5 minute expiration. This works nicely, ensuring the application only ever makes 12 requests per hour. If it was highly trafficked this solution might be problematic, but as it is I think its fine.

Despite the fact I am well below the rate limits, my site periodically goes down and looking at my server logs it is because I have exceeded Twitter's rate limits.

Is this something to do with Heroku? What else could be causing it? Is it something to do with shared IP addresses?

Logs:

2012-02-20T22:09:52+00:00 app[web.1]: Rendered pages/home.html.erb within layouts/application (0.8ms)
2012-02-20T22:09:52+00:00 app[web.1]: Rendered layouts/_header.html.erb (336.2ms)
2012-02-20T22:09:52+00:00 app[web.1]: Completed 500 Internal Server Error in 339ms
2012-02-20T22:09:52+00:00 app[web.1]: ActionView::Template::Error (Rate limit exceeded. Clients may not make more than 150 requests per hour.):
2012-02-20T22:09:52+00:00 app[web.1]:     5:             <p class="latest-tweet">
2012-02-20T22:09:52+00:00 app[web.1]: 
2012-02-20T22:09:52+00:00 app[web.1]:     7:                 <% if !latest_tweet%>
2012-02-20T22:09:52+00:00 app[web.1]:     8:                     <% latest_tweet = Twitter.user_timeline("sometwitterusername").first.text %>
2012-02-20T22:09:52+00:00 app[web.1]:     6:                 <% latest_tweet = Rails.cache.read "latest_tweet" %>
2012-02-20T22:09:52+00:00 app[web.1]:     9:                     <% Rails.cache.write("latest_tweet", latest_tweet, :expires_in => 5.minutes) %>
2012-02-20T22:09:52+00:00 app[web.1]:     10:                 <% end %>
2012-02-20T22:09:52+00:00 app[web.1]:     11:                 <%= latest_tweet %>
2012-02-20T22:09:52+00:00 app[web.1]:   app/views/layouts/_header.html.erb:8:in `_app_views_layouts__header_html_erb___4472938277532005844_37037700'
2012-02-20T22:09:52+00:00 app[web.1]:   app/views/layouts/application.html.erb:14:in `_app_views_layouts_application_html_erb__4101970984987800094_41561940'

Upvotes: 1

Views: 998

Answers (2)

Tom van der Woerdt
Tom van der Woerdt

Reputation: 29985

Pretty simple. If you share an IP with multiple other developers, you'll share the rate limits as well.

I recommend using authentication to raise this to 350 requests per hour, removing the IP-based limits. This'll simply give you 350 requests, regardless of the developers you share the IP with.

https://dev.twitter.com/docs/rate-limiting

Upvotes: 6

Neil Middleton
Neil Middleton

Reputation: 22240

Have you looked at using the Apigee add-on?

https://addons.heroku.com/apigee

This gives you increased rate limits, plus some other bells and whistles.

Upvotes: 1

Related Questions