Rails middleware to encode characters and avoid exploding controllers

char_converter.rb

# config/initializers/char_converter.rb
require 'uri'

module Support
  class CharConverter

    SANITIZE_ENV_KEYS = [
      "HTTP_COOKIE",     # bad cookie encodings kill rack: https://github.com/rack/rack/issues/225
      "HTTP_REFERER",
      "PATH_INFO",
      "QUERY_STRING",
      "REQUEST_PATH",
      "REQUEST_URI",
    ]

    def initialize(app)
      @app = app
    end

    def call(env)
      @app.call(sanitize_env(env))
    end

    def sanitize_env(env)
      SANITIZE_ENV_KEYS.each do |key|
        next unless value = env[key]
        value = sanitize_string(URI.decode(value))
        env[key] = URI.encode(value)
      end
      env
    end

    def sanitize_string(string)
      return string unless string.is_a? String

      # Try it as UTF-8 directly
      cleaned = string.dup.force_encoding('UTF-8')
      if cleaned.valid_encoding?
        cleaned
      else
        # Some of it might be old Windows code page
        string.encode(Encoding::UTF_8, Encoding::Windows_1250)
      end
    rescue EncodingError
      # Force it to UTF-8, throwing out invalid bits
      string.encode!('UTF-8', invalid: :replace, undef: :replace)
    end

  end
end

Rails.application.config.middleware.insert_before(0, Support::CharConverter)

Great solution. Thanks

just for cross reference, because you did not fork the gist: https://gist.github.com/1336754

there is a mayor flaw in this implementation, as values from HTTP_COOKIE should not be url-encoded. i updated my version recently.

btw it's possible to fork gists instead to copy it.

https://github.com/singlebrook/utf8-cleaner

There is one problem I have, when I used this module. My session data is reseting!

@entity1991 you could skip session data