CODEWITHSUNDEEP
ruby
Asaf
Asaf

Reputation: 3105

Can't get page source code with ruby

The ruby code that should get the source code of a given website doesn't work. It says that the site does not exist.

require 'uri'
require 'net/http'
uri = URI.parse("http://nice.com/careers?category=170")
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Get.new(uri.request_uri)
response=http.request(request)
@data = response.body

It does work with other sites.

  1. Why?
  2. how to fix?

Upvotes: 1

Views: 52

Answers (1)

Cody Caughlan
Cody Caughlan

Reputation: 32758

I always like to check stuff using curl on the command line.

In this case it turns out a request to http://nice.com/careers?category=170 gives you a HTTP 301 redirect to the same domain but with www.:

curl -I "http://nice.com/careers?category=170"
HTTP/1.1 301 Moved Permanently
Date: Sun, 30 Mar 2014 20:50:00 GMT
Server: Apache
Location: http://www.nice.com/careers?category=170

So update your code to to use the www.nice.com domain.

Also, you can implement logic to follow redirects, like in this answer:

https://stackoverflow.com/a/6934503/25398

Upvotes: 2

Related Questions