CODEWITHSUNDEEP
ruby-on-railsrubyruby-on-rails-4mechanizemechanize-ruby
Severin
Severin

Reputation: 8588

Mechanize::ResponseReadError - Content-Length does not match response body length

I have encountered this error while requesting a page with Mechanize:

Mechanize::ResponseReadError
Content-Length (17317) does not match response body length (17070) (Mechanize::ResponseReadError)

Any thoughts on why this occurs and how I could get about fixing it are much appreciated!

Upvotes: 2

Views: 1436

Answers (2)

Paritosh Piplewar
Paritosh Piplewar

Reputation: 8132

It occurs because Content-Length header is not equal to the size of response-body length.

Check the below specs taken by mechanize gem. it will raise same error. 

  def test_response_read_content_length_mismatch
    def @res.content_length() 5 end
    def @res.read_body() yield 'part' end

    e = assert_raises Mechanize::ResponseReadError do
      @agent.response_read @res, @req, @uri
    end

    assert_equal 'Content-Length (5) does not match response body length (4)' \
      ' (Mechanize::ResponseReadError)', e.message
  end

Upvotes: 0

i-blis
i-blis

Reputation: 3179

It happens that sites return a wrong content length-value. Catch the error and force page parsing.

agent = Mechanize.new
begin
  page = agent.get 'http://bad.com'
rescue Mechanize::ResponseReadError => e
  page = e.force_parse
end

You can also set agent.ignore_bad_chunking to true — but then beware of possible silent content loss.

Upvotes: 1

Related Questions