Skip to content

Instantly share code, notes, and snippets.

@Velrok
Created June 28, 2013 22:15
Show Gist options
  • Save Velrok/5888568 to your computer and use it in GitHub Desktop.
Save Velrok/5888568 to your computer and use it in GitHub Desktop.
I indendet to see how to use fold to utilize pralell processing just to see that its prallel without the use of fold and pmap :/
(ns reducers.core
;(:require [clojure.core.reducers :as r])
)
(def N (iterate inc 0))
(def finish 6e6)
; this is just for testing if a random map function will stop the parelleltiy
(defn only-even [x]
(if (even? x)
x
0))
(time (reduce + (map only-even (take finish N)))) ; "Elapsed time: 50830.559 msecs" 999999000000
@djui
Copy link

djui commented Jun 29, 2013

When using pmap my time extends from 3175ms to 9117ms.

@Velrok
Copy link
Author

Velrok commented Jun 29, 2013

Craig noted that the CPU usage I'm seeing might be from the GC collecting all the created wrapper objects.

pmap is only fast if the amount of work for each map run is bigger that the overhead. In this case its not true. One would normally use chunking before the pmap. However I have not done this and thus can not give a more in depth answer.

@thebusby
Copy link

thebusby commented Jul 2, 2013

This may be a bit better for comparison,
(time (->> (range 6e6) vec (filter even?) (reduce +))) ;; "Elapsed time: 1894.056685 msecs" 8999997000000
(time (->> (range 6e6) vec (r/filter even?) (r/fold +))) ;; "Elapsed time: 1697.588728 msecs" 8999997000000

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment