Method to help understand encoding issues in ruby 1.9

debug_string.rb

def debug_string input_string
  input_string.each_char do |c|  
    puts c
    c.each_byte do |b|
      hex =  "0x#{b.to_s(16).upcase}"
      nom1 = b.to_s(2)[0..3]
      nom2 = b.to_s(2)[4..7]
      puts "#{hex} #{b}  #{nom1} #{nom2}" 
    end
    puts " "
  end 
end

Peering into Japanese and Korean strings

pry(main)> debug_string "今日は"

今
0xE4 228 1110 0100
0xBB 187 1011 1011
0x8A 138 1000 1010

日
0xE6 230 1110 0110
0x97 151 1001 0111
0xA5 165 1010 0101

は
0xE3 227 1110 0011
0x81 129 1000 0001
0xAF 175 1010 1111

# With Korean a single grapheme or what we normally call a character 
#can be made up of multiple glyphs
#Here is the breakdown of 'kam' 캄 which is made of 
k=ㄱ, a=ㅏ and m=ㅁ.
 
pry(main)> debug_string "ㄱ" #g or k
ㄱ
0xE3 227 1110 0011
0x84 132 1000 0100
0xB1 177 1011 0001
 
=> "ㄱ"

pry(main)> debug_string "ㅏ" #a
ㅏ
0xE3 227 1110 0011
0x85 133 1000 0101
0x8F 143 1000 1111
 
=> "ㅏ"

pry(main)> debug_string "ㅁ"  #m
ㅁ
0xE3 227 1110 0011
0x85 133 1000 0101
0x81 129 1000 0001
 
=> "ㅁ"

pry(main)> debug_string "가" #ka
가
0xEA 234 1110 1010
0xB0 176 1011 0000
0x80 128 1000 0000
 
=> "가"
pry(main)> debug_string "감" #kam
감
0xEA 234 1110 1010
0xB0 176 1011 0000
0x90 144 1001 0000
 
=> "감"

Korean Character translation