Last active
April 17, 2017 23:01
-
-
Save morhekil/8383627 to your computer and use it in GitHub Desktop.
json_file_gz codec for Logstash, to handle gzipped json files handled to it as file names in the input stream. Details: http://speakmy.name/2014/01/13/gzipped-json-files-and-logstash/
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# encoding: utf-8 | |
require 'logstash/codecs/base' | |
class LogStash::Codecs::JsonFileGz < LogStash::Codecs::Base | |
config_name 'json_file_gz' | |
milestone 1 | |
public | |
def register | |
require 'zlib' | |
end | |
def decode(path) | |
begin | |
json_data = Zlib::GzipReader.open(path) { |f| f.read } | |
rescue Zlib::GzipFile::Error => e | |
@logger.info('Gzip failure, skipped', :error => e, :data => json_data) | |
end | |
begin | |
yield LogStash::Event.new(JSON.parse(json_data)) if json_data | |
rescue JSON::ParserError => e | |
@logger.info('JSON parse failure. Falling back to plain-text', :error => e, :data => json_data) | |
yield LogStash::Event.new('message' => json_data) | |
end | |
end # def decode | |
def encode(data) | |
raise NotImplementedError | |
end # def encode | |
end # class LogStash::Codecs::JsonFileGz |
Hi there.! It doesn't work for me :( it says: "Couldn't find any codec plugin named 'json_file_gz'. Are you sure this is correct? Trying to load the json_file_gz codec plugin resulted in this error: no such file to load -- logstash/codecs/json_file_gz". I've my logstash installed @ /opt/logstash.!
Thanks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Man this would be a really cool pull request against Logstash... especially if they ended up merging it in.