Clojure Map Traversal Benchmark

benchmark-map-traversal.clj

(defonce bench-output (atom {}))

(defn make-map [gen-zipmap-seq]
  (zipmap (gen-zipmap-seq) (gen-zipmap-seq)))

(defn benchmark-map-traversal [m]
  (reset! bench-output {})
  (System/gc)
  (println "Benchmarking .forEachRemaining :::")
  (criterium.core/quick-bench
    (do
      (swap! bench-output dissoc :forEachRemaining)
      (let [itr (.keyIterator ^clojure.lang.IMapIterable m)
            arr (java.util.ArrayList.)]
        (.forEachRemaining
          itr
          (fn [x] (.add ^java.util.ArrayList arr x)))
        (swap! bench-output assoc :forEachRemaining arr))))
  (System/gc)
  (println "Benchmarking (doall (keys m)) :::")
  (criterium.core/quick-bench
    (do
      (swap! bench-output dissoc :doall-keys-m)
      (let [output (doall (keys m))]
        (swap! bench-output assoc :doall-keys-m output))))
  (System/gc)
  (println "Benchmarking KeySeq reduce :::")
  (criterium.core/quick-bench
    (do
      (swap! bench-output dissoc :KeySeq-reduce)
      (let [output (reduce
                     (fn [^java.util.ArrayList accum x]
                       (.add accum x)
                       accum)
                     (java.util.ArrayList.)
                     (keys m))]
        (swap! bench-output assoc :KeySeq-reduce output))))
  (apply = (vals @bench-output)))

(comment
  ;map with numbers
  (benchmark-map-traversal (make-map (fn [] (repeatedly 1000000 #(rand-int Integer/MAX_VALUE)))))
  ;map with random-uuid's
  (benchmark-map-traversal (make-map (fn [] (repeatedly 10000000 random-uuid)))))


;; Results 
;; Apple MBP M2 Max 64GB
;; JVM 24
;; Clojure 1.12.2
;(benchmark-map-traversal (make-map (fn [] (repeatedly 1000000 #(rand-int Integer/MAX_VALUE)))))

Benchmarking .forEachRemaining :::
Evaluation count : 12 in 6 samples of 2 calls.
             Execution time mean : 74.724221 ms
    Execution time std-deviation : 9.987542 ms
   Execution time lower quantile : 70.011395 ms ( 2.5%)
   Execution time upper quantile : 92.008934 ms (97.5%)
                   Overhead used : 1.273186 ns

Found 1 outliers in 6 samples (16.6667 %)
	low-severe	 1 (16.6667 %)
 Variance from outliers : 31.7136 % Variance is moderately inflated by outliers
Benchmarking (doall (keys m)) :::
Evaluation count : 12 in 6 samples of 2 calls.
             Execution time mean : 69.649107 ms
    Execution time std-deviation : 7.719125 ms
   Execution time lower quantile : 63.257249 ms ( 2.5%)
   Execution time upper quantile : 82.625515 ms (97.5%)
                   Overhead used : 1.273186 ns

Found 1 outliers in 6 samples (16.6667 %)
	low-severe	 1 (16.6667 %)
 Variance from outliers : 30.9776 % Variance is moderately inflated by outliers
Benchmarking KeySeq reduce :::
Evaluation count : 12 in 6 samples of 2 calls.
             Execution time mean : 68.371128 ms
    Execution time std-deviation : 630.732771 µs
   Execution time lower quantile : 67.699291 ms ( 2.5%)
   Execution time upper quantile : 69.089976 ms (97.5%)
                   Overhead used : 1.273186 ns
=> true