Skip to content

Instantly share code, notes, and snippets.

@morhekil
Last active April 17, 2017 23:01
Show Gist options
  • Save morhekil/8383627 to your computer and use it in GitHub Desktop.
Save morhekil/8383627 to your computer and use it in GitHub Desktop.
json_file_gz codec for Logstash, to handle gzipped json files handled to it as file names in the input stream. Details: http://speakmy.name/2014/01/13/gzipped-json-files-and-logstash/
# encoding: utf-8
require 'logstash/codecs/base'
class LogStash::Codecs::JsonFileGz < LogStash::Codecs::Base
config_name 'json_file_gz'
milestone 1
public
def register
require 'zlib'
end
def decode(path)
begin
json_data = Zlib::GzipReader.open(path) { |f| f.read }
rescue Zlib::GzipFile::Error => e
@logger.info('Gzip failure, skipped', :error => e, :data => json_data)
end
begin
yield LogStash::Event.new(JSON.parse(json_data)) if json_data
rescue JSON::ParserError => e
@logger.info('JSON parse failure. Falling back to plain-text', :error => e, :data => json_data)
yield LogStash::Event.new('message' => json_data)
end
end # def decode
def encode(data)
raise NotImplementedError
end # def encode
end # class LogStash::Codecs::JsonFileGz
@shurane
Copy link

shurane commented Jul 21, 2014

Man this would be a really cool pull request against Logstash... especially if they ended up merging it in.

@federicoaaguirre
Copy link

Hi there.! It doesn't work for me :( it says: "Couldn't find any codec plugin named 'json_file_gz'. Are you sure this is correct? Trying to load the json_file_gz codec plugin resulted in this error: no such file to load -- logstash/codecs/json_file_gz". I've my logstash installed @ /opt/logstash.!
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment