incompatible character encodings: ASCII-8BIT and UTF-8

Question

I use Ruby 1.9.2 and Rails 3.0.5

I have the following error:

incompatible character encodings: ASCII-8BIT and UTF-8

It has nothing to do with the database i think.

The error is happinging on this line in a view (just a div haml call):

#content

full stack:

    ActionView::Template::Error (incompatible character encodings: ASCII-8BIT and UTF-8):
        21:                     -flash.each do |name, msg|
        22:                         =content_tag :div, msg, :id => "flash_#{name}"
        23:                         %div.clear                      
        24:                     #content                                        
        25:                         = yield
        26:             = render :partial => "layouts/grid_right" if render_grid_right?
        27:             = render :partial => "layouts/footer"
      app/views/layouts/application.html.haml:24:in `_app_views_layouts_application_html_haml___4380000789490545718_2180251300_2717546578298801795'
      actionpack (3.0.5) lib/action_view/template.rb:135:in `block in render'
      activesupport (3.0.5) lib/active_support/notifications.rb:54:in `instrument'
      actionpack (3.0.5) lib/action_view/template.rb:127:in `render'
      actionpack (3.0.5) lib/action_view/render/layouts.rb:80:in `_render_layout'
      actionpack (3.0.5) lib/action_view/render/rendering.rb:62:in `block in _render_template'
      activesupport (3.0.5) lib/active_support/notifications.rb:52:in `block in instrument'
      activesupport (3.0.5) lib/active_support/notifications/instrumenter.rb:21:in `instrument'
      activesupport (3.0.5) lib/active_support/notifications.rb:52:in `instrument'
      actionpack (3.0.5) lib/action_view/render/rendering.rb:56:in `_render_template'
      actionpack (3.0.5) lib/action_view/render/rendering.rb:26:in `render'
      haml (3.0.25) lib/haml/helpers/action_view_mods.rb:13:in `render_with_haml'
      actionpack (3.0.5) lib/abstract_controller/rendering.rb:115:in `_render_template'
      actionpack (3.0.5) lib/abstract_controller/rendering.rb:109:in `render_to_body'
      actionpack (3.0.5) lib/action_controller/metal/renderers.rb:47:in `render_to_body'
      actionpack (3.0.5) lib/action_controller/metal/compatibility.rb:55:in `render_to_body'
      actionpack (3.0.5) lib/abstract_controller/rendering.rb:102:in `render_to_string'
      actionpack (3.0.5) lib/abstract_controller/rendering.rb:93:in `render'
      actionpack (3.0.5) lib/action_controller/metal/rendering.rb:17:in `render'
      actionpack (3.0.5) lib/action_controller/metal/instrumentation.rb:40:in `block (2 levels) in render'
      activesupport (3.0.5) lib/active_support/core_ext/benchmark.rb:5:in `block in ms'
      /Users/michaelkoper/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/benchmark.rb:309:in `realtime'
      activesupport (3.0.5) lib/active_support/core_ext/benchmark.rb:5:in `ms'
      actionpack (3.0.5) lib/action_controller/metal/instrumentation.rb:40:in `block in render'
      actionpack (3.0.5) lib/action_controller/metal/instrumentation.rb:78:in `cleanup_view_runtime'
      activerecord (3.0.5) lib/active_record/railties/controller_runtime.rb:15:in `cleanup_view_runtime'
      actionpack (3.0.5) lib/action_controller/metal/instrumentation.rb:39:in `render'
      actionpack (3.0.5) lib/action_controller/metal/implicit_render.rb:10:in `default_render'
      actionpack (3.0.5) lib/action_controller/metal/mime_responds.rb:261:in `block in retrieve_response_from_mimes'
      actionpack (3.0.5) lib/action_controller/metal/mime_responds.rb:192:in `call'
      actionpack (3.0.5) lib/action_controller/metal/mime_responds.rb:192:in `respond_to'
      app/controllers/home_controller.rb:9:in `index'
      actionpack (3.0.5) lib/action_controller/metal/implicit_render.rb:4:in `send_action'
      actionpack (3.0.5) lib/abstract_controller/base.rb:150:in `process_action'
      actionpack (3.0.5) lib/action_controller/metal/rendering.rb:11:in `process_action'
      actionpack (3.0.5) lib/abstract_controller/callbacks.rb:18:in `block in process_action'
      activesupport (3.0.5) lib/active_support/callbacks.rb:445:in `_run__3968431659371141392__process_action__3163094469870857953__callbacks'
      activesupport (3.0.5) lib/active_support/callbacks.rb:409:in `_run_process_action_callbacks'
      activesupport (3.0.5) lib/active_support/callbacks.rb:93:in `run_callbacks'
      actionpack (3.0.5) lib/abstract_controller/callbacks.rb:17:in `process_action'
      actionpack (3.0.5) lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'
      activesupport (3.0.5) lib/active_support/notifications.rb:52:in `block in instrument'
      activesupport (3.0.5) lib/active_support/notifications/instrumenter.rb:21:in `instrument'
      activesupport (3.0.5) lib/active_support/notifications.rb:52:in `instrument'
      actionpack (3.0.5) lib/action_controller/metal/instrumentation.rb:29:in `process_action'
      actionpack (3.0.5) lib/action_controller/metal/rescue.rb:17:in `process_action'
      actionpack (3.0.5) lib/abstract_controller/base.rb:119:in `process'
      actionpack (3.0.5) lib/abstract_controller/rendering.rb:41:in `process'
      actionpack (3.0.5) lib/action_controller/metal.rb:138:in `dispatch'
      actionpack (3.0.5) lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
      actionpack (3.0.5) lib/action_controller/metal.rb:178:in `block in action'
      actionpack (3.0.5) lib/action_dispatch/routing/route_set.rb:62:in `call'
      actionpack (3.0.5) lib/action_dispatch/routing/route_set.rb:62:in `dispatch'
      actionpack (3.0.5) lib/action_dispatch/routing/route_set.rb:27:in `call'
      rack-mount (0.6.13) lib/rack/mount/route_set.rb:148:in `block in call'
      rack-mount (0.6.13) lib/rack/mount/code_generation.rb:93:in `block in recognize'
      rack-mount (0.6.13) lib/rack/mount/code_generation.rb:68:in `optimized_each'
      rack-mount (0.6.13) lib/rack/mount/code_generation.rb:92:in `recognize'
      rack-mount (0.6.13) lib/rack/mount/route_set.rb:139:in `call'
      actionpack (3.0.5) lib/action_dispatch/routing/route_set.rb:492:in `call'
      haml (3.0.25) lib/sass/plugin/rack.rb:41:in `call'
      actionpack (3.0.5) lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
      actionpack (3.0.5) lib/action_dispatch/middleware/head.rb:14:in `call'
      rack (1.2.1) lib/rack/methodoverride.rb:24:in `call'
      actionpack (3.0.5) lib/action_dispatch/middleware/params_parser.rb:21:in `call'
      actionpack (3.0.5) lib/action_dispatch/middleware/flash.rb:182:in `call'
      actionpack (3.0.5) lib/action_dispatch/middleware/session/abstract_store.rb:149:in `call'
      actionpack (3.0.5) lib/action_dispatch/middleware/cookies.rb:302:in `call'
      activerecord (3.0.5) lib/active_record/query_cache.rb:32:in `block in call'
      activerecord (3.0.5) lib/active_record/connection_adapters/abstract/query_cache.rb:28:in `cache'
      activerecord (3.0.5) lib/active_record/query_cache.rb:12:in `cache'
      activerecord (3.0.5) lib/active_record/query_cache.rb:31:in `call'
      activerecord (3.0.5) lib/active_record/connection_adapters/abstract/connection_pool.rb:354:in `call'
      actionpack (3.0.5) lib/action_dispatch/middleware/callbacks.rb:46:in `block in call'
      activesupport (3.0.5) lib/active_support/callbacks.rb:415:in `_run_call_callbacks'
      actionpack (3.0.5) lib/action_dispatch/middleware/callbacks.rb:44:in `call'
      rack (1.2.1) lib/rack/sendfile.rb:107:in `call'
      actionpack (3.0.5) lib/action_dispatch/middleware/remote_ip.rb:48:in `call'
      actionpack (3.0.5) lib/action_dispatch/middleware/show_exceptions.rb:47:in `call'
      railties (3.0.5) lib/rails/rack/logger.rb:13:in `call'
      rack (1.2.1) lib/rack/runtime.rb:17:in `call'
      activesupport (3.0.5) lib/active_support/cache/strategy/local_cache.rb:72:in `call'
      rack (1.2.1) lib/rack/lock.rb:11:in `block in call'
      <internal:prelude>:10:in `synchronize'
      rack (1.2.1) lib/rack/lock.rb:11:in `call'
      actionpack (3.0.5) lib/action_dispatch/middleware/static.rb:30:in `call'
      railties (3.0.5) lib/rails/application.rb:168:in `call'
      railties (3.0.5) lib/rails/application.rb:77:in `method_missing'
      railties (3.0.5) lib/rails/rack/log_tailer.rb:14:in `call'
      rack (1.2.1) lib/rack/content_length.rb:13:in `call'
      rack (1.2.1) lib/rack/chunked.rb:15:in `call'
      rack (1.2.1) lib/rack/handler/mongrel.rb:67:in `process'
      mongrel (1.2.0.pre2) lib/mongrel.rb:165:in `block in process_client'
      mongrel (1.2.0.pre2) lib/mongrel.rb:164:in `each'
      mongrel (1.2.0.pre2) lib/mongrel.rb:164:in `process_client'
      mongrel (1.2.0.pre2) lib/mongrel.rb:291:in `block (2 levels) in run'

My gems:

        Using rake (0.8.7) 
        Using RedCloth (4.2.2) 
        Using abstract (1.0.0) 
        Using activesupport (3.0.5) 
        Using builder (2.1.2) 
        Using i18n (0.5.0) 
        Using activemodel (3.0.5) 
        Using erubis (2.6.6) 
        Using rack (1.2.1) 
        Using rack-mount (0.6.13) 
        Using rack-test (0.5.7) 
        Using tzinfo (0.3.24) 
        Using actionpack (3.0.5) 
        Using mime-types (1.16) 
        Using polyglot (0.3.1) 
        Using treetop (1.4.9) 
        Using mail (2.2.15) 
        Using actionmailer (3.0.5) 
        Using arel (2.0.9) 
        Using activerecord (3.0.5) 
        Using activeresource (3.0.5) 
        Using authlogic (2.1.6) 
        Using xml-simple (1.0.14) 
        Using aws-s3 (0.6.2) 
        Using block_helpers (0.3.3) 
        Using bundler (1.0.10) 
        Using diff-lcs (1.1.2) 
        Using json (1.4.6) 
        Using gherkin (2.3.4) 
        Using term-ansicolor (1.0.5) 
        Using cucumber (0.10.0) 
        Using cucumber-rails (0.3.2) 
        Using daemons (1.0.10) 
        Using database_cleaner (0.6.5) 
        Using factory_girl (1.3.3) 
        Using faker (0.9.5) 
        Using formtastic (1.2.3) 
        Using gem_plugin (0.2.3) 
        Using haml (3.0.25) 
        Using thor (0.14.6) 
        Using railties (3.0.5) 
        Using rails (3.0.5) 
        Using kaminari (0.10.4) 
        Using mongrel (1.2.0.pre2) 
        Using mysql2 (0.2.6) 
        Using nokogiri (1.4.4) 
        Using paperclip (2.3.8) 
        Using rspec-core (2.5.1) 
        Using rspec-expectations (2.5.0) 
        Using rspec-mocks (2.5.0) 
        Using rspec (2.5.0) 
        Using yard (0.6.4) 
        Using pickle (0.4.4) 
        Using populator (1.0.0) 
        Using rspec-rails (2.5.0) 
        Using webrat (0.7.3) 

Answer 1

If the cause of the error is a render block, this works:

<%= (
  render "{SOME_TEMPLATE}", some_variable: some_variable
).force_encoding("UTF-8") %>

I have split the lines to show the brackets and the force encoding

Answer 2

For prevent an error "can't modify frozen string" for encoding a varible you can use: var.dup.force_encoding(Encoding::ASCII_8BIT) or var.dup.force_encoding(Encoding::UTF_8)

Answer 3

I got the same cryptic error message from Rails 4.1, Ruby 2.3.3 in a recent project, stacktrace originating in layout application.html.haml

After a wild goose chase, the culprit was a UTF-8 character which recently had been added to the footer of all pages. For some weird reason the error would only show up intermittently.

Replacing the UTF-8 character with the corresponding HTML escape sequence &#xHHHH; solved the issue.

I hope this saves other people some time in the future..

Answer 4

you can force UTF8 with force_encoding(Encoding::UTF_8):

Example:

<%= yield.force_encoding(Encoding::UTF_8) %>

Answer 5

ASCII-8BIT is Ruby's description for characters above the normal 0-0x7f ASCII character-set, and that are single-byte characters. Typically that would be something like ISO-8859-1, or one of its siblings.

If you can identify which character is causing the problem, then you can tell Ruby 1.9.2 to convert between the character set of that character to UTF-8.

James Grey wrote a series of blogs talking about these sort of problems and how to deal with them. I'd recommend going through them.

incompatible character encodings: ASCII-8BIT and UTF-8

That typically happens because you are trying to concatenate two strings, and one contains characters that do not map to the character-set of the other string. There are characters in ISO-8859-1 that do not have equivalents in UTF-8, and vice-versa and how to handle string joining with those incompatibilities requires the programmer to step in.

Answer 6

i had a similiar problem and the gem string-scrub automagically fixed it for me. https://github.com/hsbt/string-scrub If the given string contains an invalid byte sequence then that invalid byte sequence is replaced with the unicode replacement character (�) and a new string is returned.

Answer 7

The problem was the use of incorrect quotes around the iOS version. Make sure all your quotes are ' and not ‘ or ’.

https://github.com/CocoaPods/CocoaPods/issues/829

Answer 8

I had the same problem when parsing CSV files on Ruby 1.9.2 that were correctly parsed on Ruby 1.8. I found the answer here. When opening the CSV file with Ruby CSV module it is necessary to specify UTF-8 enconding as following:

CSV.foreach("file.txt", encoding: "UTF-8") do |row|
   # foo and bar correctly encoded
   foo, bar, ... = row
end

Answer 9

I encountered the error while migrating an app from Ruby 1.8.7 to 1.9.3 and it only occured in production. It turned out that I had some leftovers in my Memcache store. The now encoding sensitive Ruby 1.9.3 version of my app tried to mix old ASCII-8BIT values with new UTF-8.

It was as simple as flushing the cache to fix it for me.

Answer 10

Just for the record: for me it turned out that it was the gem called 'mysql' ... obviously this is working with US-ASCII 8 bit by default. So changing it to the gem called mysql2 (the 2 is the important point here) solved all of my issues.

I looked @ the gem list posted above - Michael Koper has obviously mysql2 installed but I posted this in case someone has this issue as well .. (took me some time to figure out).

If you dislike this answer please comment and I will delete it.

P.S: German umlauts (ä,ö and ü) screwed it out with mysql

Answer 11

I installed gem package mysql2.

gem install mysql2

and then I changed the adapter in mysql2 in database.yml.

Answer 12

I solved it by following these steps:

• Make sure config.encoding = "utf-8" is in the application.rb file.
• Make sure you are using the 'mysql2' gem.
• Put # encoding: utf-8 at the top of file containing UTF-8 characters.

• Above the <App Name>::Application.initialize! line in the environment.rb file, add following two lines:

  Encoding.default_external = Encoding::UTF_8
  Encoding.default_internal = Encoding::UTF_8
  

http://rorguide.blogspot.com/2011/06/incompatible-character-encodings-ascii.html

Answer 13

For Haml put an encoding hint:

-# coding: UTF-8

on the top left of the Haml page.

Answer 14

I have a suspicion that you either copy/pasted a part of your Haml template into the file, or you're working with a non-Unicode/non-UTF-8 friendly editor.

See if you can recreate that file from the scratch in a UTF-8 friendly editor. There are plenty for any platform and see whether this fixes your problem. Start by erasing the line with #content and retyping it manually.

Answer 15

it's very strange that I met this problem because I forgot specify the 'type' parameter. e.g.:

add_column :cms_push_msgs, :android_title

which should be:

add_column :cms_push_msgs, :android_content, :string 

Answer 16

The creation of pdf-documents with the rails-latex-gem lead to a similar problem. I solved this by modifying layouts/application.pdf.erb to

\begin{document}

<%= yield.force_encoding("UTF-8") %>


\end{document}

Answer 17

I had a similar issue on a custom CoffeeScript file. I solved it by changing the endline encoding from "Unix/Linux" to "Mac OS Classic"

Answer 18

Try to find the exact line which causing this problem and then enforce UTF8 coding, this solution worked for me.

title.to_s.force_encoding("UTF-8")

Answer 19

I've experienced similar problem. Although I had have UTF-8 encodings solved (with mysql2 and Encoding.default_external = Encoding::UTF_8 ...) incompatible character encodings: UTF-8 and ASCII-8BIT arose when I used incorrect helper parameters e.g. f.button :submit, "Zrušiť" - works perfectly but f.button "Zrušiť"- throws encoding error.