CODEWITHSUNDEEP
rubyhttpartypry
Jay
Jay

Reputation: 86

HTTParty request returns 404 code

I'm sending an HTTP request with the HTTParty Ruby gem with the following code:

require 'httparty'
require 'pry'

page = HTTParty.get('http://www.cubuffs.com/')

binding.pry

You can verify that the URL is valid. When exploring the results with Pry, I get the following:

[1] pry(main)> page
=> nil
[2] pry(main)> page.code
=> 404
[3] pry(main)> page.response
=> #<Net::HTTPNotFound 404 Not Found readbody=true>

I'm pretty sure nothing is wrong with my code, because I can substitute other URLs and they work as expected. For some reason, URLs from this domain return a 404 code. Any ideas what is wrong here and how to fix it?

Upvotes: 1

Views: 883

Answers (1)

Sunil D.
Sunil D.

Reputation: 18193

The owner of that site is checking the User-Agent from the browser, and doesn't like the one that HTTParty is using. You can get the page by including a user agent header from a browser, here is the one from Chrome:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36

Modify your code as follows:

require 'httparty'
user_agent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36'

page = HTTParty.get('http://www.cubuffs.com/', headers: {"User-Agent": user_agent})

Upvotes: 3

Related Questions