Rails/unicode issue

Question

I have a bit of my Ruby/Rails (Ruby 2.0.0p195, Rails 3.2.13) project that works as a proxy; that is, you pass it a URL, it goes out and fetches the page, and presents it to you. This generally works as expected, but it seems to munge certain characters (such as è).

A simplified version of the controller is this:

class HomeController < ApplicationController
  def geoproxy
    require 'net/http'
    require 'timeout'

    rawurl = CGI::unescape(params[:url])

    fixedurl = rawurl.gsub('\\', '%5C')   # Escape backslashes... why oh why???!?
    r = nil;

    status = 200
    content_type = ''

    begin
      Timeout::timeout(15) {        # Time, in seconds

        if request.get? then
          res = Net::HTTP.get_response(URI.parse(fixedurl))

          status = res.code    # If there was an  error, pass that code back to our caller
          @page = res.body.encode('UTF-8')
          content_type = res['content-type']    
        end
      }

    rescue Timeout::Error
      @page = "TIMEOUT"
      status = 504    # 504 Gateway Timeout  We're the gateway, we timed out.  Seems logical.
    end

    render :layout => false, :status => status, :content_type => content_type
  end
end

The corresponding view is quite simple:

<%= raw @page %>

When I use this proxy to fetch XML containing an è (for example), I get the following error:

Encoding::UndefinedConversionError in HomeController#geoproxy
"\xE8" from ASCII-8BIT to UTF-8

This error occurs at the following line:

@page = res.body.encode('UTF-8')

If I remove the .encode(), the error is resolved, but my XML contains a placeholder instead of the è.

How can I get my project to display the XML properly?

Answer 1

Could you check if the following code works for you? I was able to fix similar problem of mine with it.

@page = res.body.force_encoding('Windows-1254').encode('UTF-8')