CODEWITHSUNDEEP
ruby-on-railsruby
Carsen
Carsen

Reputation: 183

how to render special characters properly in ruby on rails?

I get characters like ’ … – “ †‘ from DB. The table from which i am fetching is in latin1 character set. I need to show these characters properly. How to do this in Ruby on rails? Is there a function or piece of code which will replace these characters with correct ones?

Upvotes: 1

Views: 2181

Answers (2)

srghma
srghma

Reputation: 5353

I have tried all encodings intil I found the right one

text.encode('windows-1250').force_encoding("UTF-8")
text.encode('utf-7').force_encoding("UTF-8")
text.encode('ibm852').force_encoding("UTF-8")
text.encode('shift_jis').force_encoding("UTF-8")
text.encode('iso-2022-jp').force_encoding("UTF-8")
text.encode("Windows-1252").force_encoding("UTF-8")
text.encode("latin1").force_encoding("UTF-8")
text.encode("ISO-8859-1").force_encoding("UTF-8")
text.encode("ISO-8859-2").force_encoding("UTF-8")
text.encode("ISO-8859-3").force_encoding("UTF-8")
text.encode("ISO-8859-4").force_encoding("UTF-8")
text.encode("ISO-8859-5").force_encoding("UTF-8")
text.encode("ISO-8859-6").force_encoding("UTF-8")
text.encode("ISO-8859-7").force_encoding("UTF-8")
text.encode("ISO-8859-8").force_encoding("UTF-8")
text.encode("ISO-8859-9").force_encoding("UTF-8")
text.encode("ISO-8859-10").force_encoding("UTF-8")
text.encode("ISO-8859-11").force_encoding("UTF-8")
text.encode("ISO-8859-12").force_encoding("UTF-8")
text.encode("ISO-8859-13").force_encoding("UTF-8")
text.encode("ISO-8859-14").force_encoding("UTF-8")
text.encode("ISO-8859-15").force_encoding("UTF-8")

then I've created map of invalid characters and replaced them using script (inspired by https://markmcb.com/2011/11/07/replacing-with-utf-8-characters-in-ruby-on-rails/ )

def fix(text)
  replacements = [
    ['–',           "—"],
    ["—",           "–"],
    ["‘",           "‘"],
    ['…',           '…'],
    ['’',           '’'],
    ['“',           '“'],
    [/â€[[:cntrl:]]/, '”'],
    ['â€?',           '”'],
    ['”',           '”'],
    ['“',           '“'],
    ['
',           '—'], # not sure about this one
    ['″',           '″'],
    ['‎',           ''], # emtpy str
    [' ',           ''], # emtpy str
    [' ',           ''], # emtpy str
    ['​',           ''], # emtpy str
    ['â€',           ''], # emtpy str
    ["â€s'",           ''], # emtpy str
  ]

  new_text = text
  replacements.each { |set| new_text = new_text.gsub(set[0], set[1]) }
  new_text
end

# rails automatically will check if publication was changed and won't save if it wasn't changed
Publication.where('content like ?', "%â€%").find_each do |publication|
  publication.title         = fix(publication.title)
  publication.content       = fix(publication.content)
  publication.short_content = fix(publication.short_content)
  publication.save!
end

until Publication.where('content like ?', "%â€%").count was equal to 0

Upvotes: 1

Linuxios
Linuxios

Reputation: 35788

You probably need to set the encoding of the DB string. Try the encode method of String:

dbstr.encode("iso-8859-1")

There are plenty of other encodings if ISO 8859 1 doesn't work for you. If the users browser doesn't support the right encoding, there are options you can pass to encode to get it to replace unknowns with ?s, etc.

Upvotes: 1

Related Questions