Skip to content

Instantly share code, notes, and snippets.

@sonots
Last active July 11, 2017 02:23
Show Gist options
  • Save sonots/5b75236cf4db51731f80b6f221272d60 to your computer and use it in GitHub Desktop.
Save sonots/5b75236cf4db51731f80b6f221272d60 to your computer and use it in GitHub Desktop.

https://github.com/sonots/embulk-filter-timestamp_format/tree/master/bench

🍣  $ embulk run -I lib bench/config_java.yml
2017-07-10 17:58:32.524 +0900: Embulk v0.8.27
2017-07-10 17:58:33.709 +0900 [INFO] (0001:transaction): Loaded plugin embulk/filter/timestamp_format from a load path
2017-07-10 17:58:33.727 +0900 [INFO] (0001:transaction): Listing local files at directory 'bench' filtering filename by prefix 'dummy'
2017-07-10 17:58:33.729 +0900 [INFO] (0001:transaction): "follow_symlinks" is set false. Note that symbolic links to directories are skipped.
2017-07-10 17:58:33.734 +0900 [INFO] (0001:transaction): Loading files [bench/dummy.csv]
2017-07-10 17:58:33.792 +0900 [INFO] (0001:transaction): Using local thread executor with max_threads=16 / output tasks 8 = input tasks 1 * 8
2017-07-10 17:58:33.797 +0900 [INFO] (0001:transaction): {done:  0 / 1, running: 0}
2017-07-10 17:58:35.313 +0900 [INFO] (0001:transaction): {done:  1 / 1, running: 0}
2017-07-10 17:58:35.318 +0900 [INFO] (main): Committed.
2017-07-10 17:58:35.318 +0900 [INFO] (main): Next config diff: {"in":{"last_path":"bench/dummy.csv"},"out":{}}
🍣  $ embulk run -I lib bench/config_jruby.yml
2017-07-10 17:58:48.439 +0900: Embulk v0.8.27
2017-07-10 17:58:49.653 +0900 [INFO] (0001:transaction): Loaded plugin embulk/filter/timestamp_format from a load path
2017-07-10 17:58:49.674 +0900 [INFO] (0001:transaction): Listing local files at directory 'bench' filtering filename by prefix 'dummy'
2017-07-10 17:58:49.676 +0900 [INFO] (0001:transaction): "follow_symlinks" is set false. Note that symbolic links to directories are skipped.
2017-07-10 17:58:49.682 +0900 [INFO] (0001:transaction): Loading files [bench/dummy.csv]
2017-07-10 17:58:49.757 +0900 [INFO] (0001:transaction): Using local thread executor with max_threads=16 / output tasks 8 = input tasks 1 * 8
2017-07-10 17:58:49.764 +0900 [INFO] (0001:transaction): {done:  0 / 1, running: 0}
2017-07-10 17:59:42.032 +0900 [INFO] (0001:transaction): {done:  1 / 1, running: 0}
2017-07-10 17:59:42.038 +0900 [INFO] (main): Committed.
2017-07-10 17:59:42.038 +0900 [INFO] (main): Next config diff: {"in":{"last_path":"bench/dummy.csv"},"out":{}}

jruby timestamp parser is not fast yet ... why?

@sonots
Copy link
Author

sonots commented Jul 11, 2017

How to run bench:

git clone [email protected]:sonots/embulk-filter-timestamp_format.git
cd embulk-filter-timestamp_format
ruby bench/gen_dummy.rb
./gradlew classpath
embulk run -I lib bench/config_jruby.yml

@sonots
Copy link
Author

sonots commented Jul 11, 2017

org.embulk.spi.time.JRubyTimeParserHelperFactory is not modified?

@sonots
Copy link
Author

sonots commented Jul 11, 2017

ah, https://github.com/sonots/embulk-filter-timestamp_format/blob/e3151b13da7f16ba021b3b7f150dc00bd44ff98d/src/main/java/org/embulk/filter/timestamp_format/TimestampParser.java#L68

JRubyTimeParserHelperFactory helperFactory = (JRubyTimeParserHelperFactory) jruby.runScriptlet("Embulk::Java::TimeParserHelper::Factory.new");

@sonots
Copy link
Author

sonots commented Jul 11, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment