Last active
August 29, 2015 14:01
-
-
Save limhoff-r7/329aa446944a63a10dd1 to your computer and use it in GitHub Desktop.
Encoding::UndefinedConversionError when msf/sanity is required and sets encodings to binary and something explicitly sets the external encoding to utif-8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Encoding.default_external: UTF-8 | |
Encoding.default_internal: | |
Encoding.default_external: ASCII-8BIT | |
Encoding.default_internal: ASCII-8BIT | |
Setting no encoding on file | |
f.external_encoding: ASCII-8BIT | |
f.internal_encoding: | |
Contents: امثلة | |
Only setting external encoding on file | |
f.external_encoding: UTF-8 | |
f.internal_encoding: ASCII-8BIT | |
U+0627 from UTF-8 to ASCII-8BIT (Encoding::UndefinedConversionError) | |
Setting external and internal encoding on file | |
i18n-fail.rb:21: warning: Ignoring internal encoding utf-8: it is identical to external encoding utf-8 | |
f.external_encoding: UTF-8 | |
f.internal_encoding: | |
Contents: امثلة | |
Encoding.default_external: UTF-8 | |
Encoding.default_internal: UTF-8 | |
Only setting external encoding on file | |
f.external_encoding: UTF-8 | |
f.internal_encoding: | |
Contents: امثلة |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# -*- encoding: utf-8 -*- | |
# | |
# CONSTANTS | |
# | |
PATH = 'arabic.txt' | |
# | |
# Methods | |
# | |
def read_arabic(encoding=nil) | |
mode = "r" | |
if encoding | |
mode = "#{mode}:#{encoding}" | |
end | |
begin | |
contents = File.open(PATH, mode) do |f| | |
puts "f.external_encoding: #{f.external_encoding}" | |
puts "f.internal_encoding: #{f.internal_encoding}" | |
f.read | |
end | |
rescue Encoding::UndefinedConversionError => error | |
puts "#{error} (#{error.class})" | |
else | |
puts "Contents: #{contents}" | |
end | |
end | |
# | |
# Script | |
# | |
puts "Encoding.default_external: #{Encoding.default_external}" | |
puts "Encoding.default_internal: #{Encoding.default_internal}" | |
File.open(PATH, 'w') do |f| | |
# 'examples' in Arabic | |
# @see https://github.com/cucumber/gherkin/blob/ae88a979d0f29c25f6c0c25198d770a45d96d2ba/lib/gherkin/i18n.json#L37 | |
f.puts('امثلة') | |
end | |
# @see https://github.com/rapid7/metasploit-framework/blob/3a3d038904feb9729281af43e5ef97d2530d59aa/lib/msf/sanity.rb#L44 | |
Encoding.default_external = Encoding.default_internal = "binary" | |
puts | |
puts "Encoding.default_external: #{Encoding.default_external}" | |
puts "Encoding.default_internal: #{Encoding.default_internal}" | |
puts | |
puts 'Setting no encoding on file' | |
read_arabic | |
puts | |
puts 'Only setting external encoding on file' | |
read_arabic('utf-8') | |
puts | |
puts 'Setting external and internal encoding on file' | |
read_arabic('utf-8:utf-8') | |
Encoding.default_external = Encoding.default_internal = 'utf-8' | |
puts | |
puts "Encoding.default_external: #{Encoding.default_external}" | |
puts "Encoding.default_internal: #{Encoding.default_internal}" | |
puts | |
puts 'Only setting external encoding on file' | |
read_arabic('utf-8') |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment