Skip to content

Instantly share code, notes, and snippets.

@rgarner
Last active August 29, 2015 14:24
Show Gist options
  • Save rgarner/0473a9747e2bb65d619b to your computer and use it in GitHub Desktop.
Save rgarner/0473a9747e2bb65d619b to your computer and use it in GitHub Desktop.
Guess the encoding of a line-based resource on the web and print it to STDERR, followed by the contents
#!/usr/bin/env ruby
require 'rubygems'
require 'open-uri'
require 'csv'
require 'charlock_holmes'
def read(url)
contents = File.read(open(url))
detection = CharlockHolmes::EncodingDetector.detect(contents)
STDERR.puts detection[:encoding]
utf8_encoded_content = CharlockHolmes::Converter.convert contents, detection[:encoding], 'UTF-8'
CSV.new(utf8_encoded_content, headers: :first_row).each do |line|
puts line
end
end
ARGV.length == 1 or abort 'No file argument given'
read(ARGV[0])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment