Christoffer
Christoffer

Reputation: 2411

Rails gem GoogleAjax. How can I perform more searches?

Using: Rails 3.1.1

I am using the googleajax gem to perform Google-searches in a script with severals of thousands searches.

After some 20 searches or so, I need to have a rescue that waits and retries since it seems like you cannot perform more than a certain amount of searches in a row. After approximately one minute the retry makes the search continue for 10 more searches. The result is that it takes about one minute to perform 10 searches, which makes the script incredibly slow.

It seems likely that Google has a block in the amount of searches one can perform (based on ip? based on googleajax referrer?) but is there a way around it?

What can I do to be able to perform Google searches through the googleajax gem without having to pause and wait all the time? What alternatives do I have?

The code (with unimportant parts cut out):

            begin
              puts "Searching with " + gsquery
                results = GoogleAjax::Search.web(gsquery)[:results]
                if results.count > 0
                  puts "#{results.count} results found for #{page.name}. Registering the connection!"
                end
            rescue
                puts "Try again in 3 sec"
                sleep 3
                retry
            rescue Timeout::Error 
              puts "Timeout Error, sleep 15 sec"
              sleep 15
              retry
            end

Upvotes: 1

Views: 518

Answers (2)

user3226080
user3226080

Reputation: 1

Ive found this neat little gem to be quite handy in my latest project. Ruby - Google Search API

Here is a simple use case for searching for an image. This basically states that if the item's name does not equal an empty string, return the search of the first 5 images using the item's name. If the item's name is equal to a empty string and thus being nil, do nothing.

- if item.name != "" 
  - Google::Search::Image.new(:query => item.name).first(5).each do |image|
    = image_tag(image.uri)

Upvotes: 0

Chris Bailey
Chris Bailey

Reputation: 4136

Sorry, but I think you're out of luck. GoogleAjax uses the now deprecated web search API (it's been deprecated for over a year now), which may disappear at any point in the future, making the gem useless. Secondly, both the web search API and it's replacement are limited to a maximum number of queries a day, beyond which the service will just stop responding - it's 100 queries a day for the custom search API. To get more than that you'll have to pay (the rate is $5 / 1000 searches). The rate limit is based on the number of queries associated with a single API key.

I'd suggest that you:

  1. Use the google-api-client gem instead of GoogleAjax (it uses the Custom Web Search API which replaces the web search API)
  2. Get an API key for the custom search API using Google's API console
  3. Consider enabling billing. Half a cent per search is not terrible, and for several thousand searches will only cost you $10

Upvotes: 2

Related Questions