CODEWITHSUNDEEP
rubyweb-crawlerhttp-redirect
Nick
Nick

Reputation: 241

How do I get the destination URL of a shortened URL using Ruby?

How do I take this URL http://t.co/yjgxz5Y and get the destination URL which is http://nickstraffictricks.com/4856_how-to-rank-1-in-google/

Upvotes: 14

Views: 4646

Answers (4)

ericteubert
ericteubert

Reputation: 4591

For resolving redirects you should use a HEAD request to avoid downloading the whole response body (imagine resolving a URL to an audio or video file).

Working example using the Faraday gem:

require 'faraday'
require 'faraday_middleware'

def resolve_redirects(url)
    response = fetch_response(url, method: :head)
    if response
        return response.to_hash[:url].to_s
    else
        return nil
    end
end

def fetch_response(url, method: :get)
    conn = Faraday.new do |b|
        b.use FaradayMiddleware::FollowRedirects;
        b.adapter :net_http
    end
    return conn.send method, url
rescue Faraday::Error, Faraday::Error::ConnectionFailed => e
    return nil
end

puts resolve_redirects("http://cre.fm/feed/m4a") # http://feeds.feedburner.com/cre-podcast

Upvotes: 3

the Tin Man
the Tin Man

Reputation: 160553

I've used open-uri for this, because it's nice and simple. It will retrieve the page, but will also follow multiple redirects:

require 'open-uri'

final_uri = ''
open('http://t.co/yjgxz5Y') do |h|
  final_uri = h.base_uri
end
final_uri # => #<URI::HTTP:0x00000100851050 URL:http://nickstraffictricks.com/4856_how-to-rank-1-in-google/>

The docs show a nice example for using the lower-level Net::HTTP to handle redirects.

require 'net/http'
require 'uri'

def fetch(uri_str, limit = 10)
  # You should choose better exception.
  raise ArgumentError, 'HTTP redirect too deep' if limit == 0

  response = Net::HTTP.get_response(URI.parse(uri_str))
  case response
  when Net::HTTPSuccess     then response
  when Net::HTTPRedirection then fetch(response['location'], limit - 1)
  else
    response.error!
  end
end

puts fetch('http://www.ruby-lang.org')

Of course this all breaks down if the page isn't using a HTTP redirect. A lot of sites use meta-redirects, which you have to handle by retrieving the URL from the meta tag, but that's a different question.

Upvotes: 8

Mladen Jablanović
Mladen Jablanović

Reputation: 44080

require 'net/http'
require 'uri'

Net::HTTP.get_response(URI.parse('http://t.co/yjgxz5Y'))['location']
# => "http://nickstraffictricks.com/4856_how-to-rank-1-in-google/"

Upvotes: 24

Spyros
Spyros

Reputation: 48596

You would have to follow the redirect. I think that would help :

http://shadow-file.blogspot.com/2009/03/handling-http-redirection-in-ruby.html

Upvotes: 1

Related Questions