pjson benchmarks

benchmark.patch

From a74d2f6d2337d28a97d48aeb06299269bac6354d Mon Sep 17 00:00:00 2001
From: Yannick Scherer <[email protected]>
Date: Sun, 31 Aug 2014 15:51:53 +0200
Subject: [PATCH] benchmark adjustments.

---
 src/main/bench/pjson/parse_bench.clj | 37 +++++++++++++++++-------------------
 1 file changed, 17 insertions(+), 20 deletions(-)

diff --git a/src/main/bench/pjson/parse_bench.clj b/src/main/bench/pjson/parse_bench.clj
index a8e0dd1..72c0926 100644
--- a/src/main/bench/pjson/parse_bench.clj
+++ b/src/main/bench/pjson/parse_bench.clj
@@ -7,33 +7,30 @@
   (:use perforate.core))
 
 (defgoal json-parse "JSON Parse Benchmark"
-         :setup (fn [] (let [msg (-> "test-resources/msg.json" slurp )]
-                            [(.getBytes msg "UTF-8") msg])))
+  :setup (fn [] (let [msg (-> "test-resources/msg.json" slurp )]
+                  [(.getBytes msg "UTF-8") msg])))
 
 (defcase json-parse :pjson
-         [^"[B" bts _]
-         (dotimes [i 100000]
-                  (asObj bts)))
-
+  [^"[B" bts _]
+  (dotimes [i 100]
+    (pr-str (asObj bts))))
 
 (defcase json-parse :boon
-         [_ ^String msg]
-         (dotimes [i 100000]
-                  (JsonFactory/fromJson msg)))
-
+  [_ ^String msg]
+  (dotimes [i 100]
+    (pr-str (JsonFactory/fromJson msg))))
 
 (defcase json-parse :data.json
-         [_ ^String msg]
-         (dotimes [i 100000]
-                  (data-json/read-str msg)))
+  [_ ^String msg]
+  (dotimes [i 100]
+    (pr-str (data-json/read-str msg))))
 
 (defcase json-parse :clj-json
-         [_ ^String msg]
-         (dotimes [i 100000]
-                  (clj-json/parse-string msg)))
-
+  [_ ^String msg]
+  (dotimes [i 100]
+    (pr-str (clj-json/parse-string msg))))
 
 (defcase json-parse :cheshire
-         [_ ^String msg]
-         (dotimes [i 100000]
-                  (cheshire/parse-string msg)))
\ No newline at end of file
+  [_ ^String msg]
+  (dotimes [i 100]
+    (pr-str (cheshire/parse-string msg))))
-- 
2.0.0

result.txt

$ lein perforate
======================
Reflection warning, perforate/core.clj:13:8 - reference to field hashCode can't be resolved.
Reflection warning, perforate/core.clj:13:22 - reference to field hashCode can't be resolved.
Reflection warning, pjson/parse_bench.clj:11:20 - call to method getBytes can't be resolved (target class is unknown).
Benchmarking the following goals: 
json-parse
WARNING: Final GC required 1.2361350759045642 % of runtime
Goal:  JSON Parse Benchmark
-----
Case:  :clj-json
Evaluation count : 2520 in 60 samples of 42 calls.
             Execution time mean : 24.687519 ms
    Execution time std-deviation : 1.081020 ms
   Execution time lower quantile : 23.951998 ms ( 2.5%)
   Execution time upper quantile : 27.206570 ms (97.5%)
                   Overhead used : 1.765620 ns

Found 5 outliers in 60 samples (8.3333 %)
        low-severe       1 (1.6667 %)
        low-mild         4 (6.6667 %)
 Variance from outliers : 30.3047 % Variance is moderately inflated by outliers

Case:  :data.json
Evaluation count : 2040 in 60 samples of 34 calls.
             Execution time mean : 29.645788 ms
    Execution time std-deviation : 310.300684 µs
   Execution time lower quantile : 29.191410 ms ( 2.5%)
   Execution time upper quantile : 30.299750 ms (97.5%)
                   Overhead used : 1.765620 ns

Case:  :pjson
Evaluation count : 2700 in 60 samples of 45 calls.
             Execution time mean : 22.693678 ms
    Execution time std-deviation : 238.598282 µs
   Execution time lower quantile : 22.331043 ms ( 2.5%)
   Execution time upper quantile : 23.088354 ms (97.5%)
                   Overhead used : 1.765620 ns

Case:  :cheshire
Evaluation count : 2580 in 60 samples of 43 calls.
             Execution time mean : 24.125483 ms
    Execution time std-deviation : 832.205041 µs
   Execution time lower quantile : 23.434231 ms ( 2.5%)
   Execution time upper quantile : 25.749352 ms (97.5%)
                   Overhead used : 1.765620 ns

Found 4 outliers in 60 samples (6.6667 %)
        low-severe       1 (1.6667 %)
        low-mild         3 (5.0000 %)
 Variance from outliers : 20.6393 % Variance is moderately inflated by outliers

Case:  :boon
Evaluation count : 2640 in 60 samples of 44 calls.
             Execution time mean : 24.013051 ms
    Execution time std-deviation : 713.018664 µs
   Execution time lower quantile : 23.061248 ms ( 2.5%)
   Execution time upper quantile : 25.753707 ms (97.5%)
                   Overhead used : 1.765620 ns

Found 3 outliers in 60 samples (5.0000 %)
        low-severe       2 (3.3333 %)
        low-mild         1 (1.6667 %)
 Variance from outliers : 17.3406 % Variance is moderately inflated by outliers