Skip to content

Instantly share code, notes, and snippets.

@viktorklang

viktorklang/Instructions.txt

Created October 8, 2017 13:01
Show Gist options
  • Save viktorklang/1c2c9e213bce6159d4114e14c6bc1f06 to your computer and use it in GitHub Desktop.
Save viktorklang/1c2c9e213bce6159d4114e14c6bc1f06 to your computer and use it in GitHub Desktop.
Download ZIP
Try the latest Scala Future impl!
Raw
Instructions.txt
First: Close all other applications, so that they don't interfere with the benchmark.
Second: Make sure that your computer is adequately cooled during the benchmark so it doesn't start thermal throttling.
Third: clone https://github.com/viktorklang/scala-futures/
Fourth: git checkout wip-optimizations-√
Fifth: start sbt
> project benches
> jmh:runMain scala.future.BenchRunner -p impl=stdlib,improved -p pool=fjp -p threads=1 -i 10 -wi 15 -f1 -t1 .TransformationBenchmark*
> jmh:runMain scala.future.BenchRunner -p impl=stdlib,improved -p pool=fjp -p threads=1 -i 10 -wi 15 -f1 -t1 .CallbackBenchmark*
> jmh:runMain scala.future.BenchRunner -p impl=stdlib,improved -p pool=fjp -p threads=1 -i 10 -wi 15 -f1 -t1 .CompletionWithBenchmark*
> jmh:runMain scala.future.BenchRunner -p impl=stdlib,improved -p pool=fjp -p threads=1 -i 10 -wi 15 -f1 -t1 .CompletionBenchmark*
Last, but not least, report back the results of the benches, as a comment on this gist!
@jlprat
Copy link

jlprat commented Oct 10, 2017

i7 32GB DDR4 @ 2133MHz

[info] Result "scala.future.TransformationBenchmark.transformation_8192":
[info]   14.740 ±(99.9%) 4.271 ops/us [Average]
[info]   (min, avg, max) = (11.641, 14.740, 18.341), stdev = 2.825
[info]   CI (99.9%): [10.469, 19.012] (assumes normal distribution)
[info]
[info]
[info] # Run complete. Total time: 00:05:57
[info]
[info] Benchmark                                      (impl)  (pool)  (threads)   Mode  Cnt   Score   Error   Units
[info] TransformationBenchmark.transformation_1       stdlib     fjp          1  thrpt   10   0.155 ± 0.003  ops/us
[info] TransformationBenchmark.transformation_1     improved     fjp          1  thrpt   10   0.170 ± 0.051  ops/us
[info] TransformationBenchmark.transformation_1024    stdlib     fjp          1  thrpt   10   6.901 ± 0.141  ops/us
[info] TransformationBenchmark.transformation_1024  improved     fjp          1  thrpt   10  16.790 ± 2.441  ops/us
[info] TransformationBenchmark.transformation_16      stdlib     fjp          1  thrpt   10   1.848 ± 0.052  ops/us
[info] TransformationBenchmark.transformation_16    improved     fjp          1  thrpt   10   2.230 ± 0.071  ops/us
[info] TransformationBenchmark.transformation_2       stdlib     fjp          1  thrpt   10   0.308 ± 0.003  ops/us
[info] TransformationBenchmark.transformation_2     improved     fjp          1  thrpt   10   0.326 ± 0.049  ops/us
[info] TransformationBenchmark.transformation_4       stdlib     fjp          1  thrpt   10   0.624 ± 0.084  ops/us
[info] TransformationBenchmark.transformation_4     improved     fjp          1  thrpt   10   0.636 ± 0.082  ops/us
[info] TransformationBenchmark.transformation_64      stdlib     fjp          1  thrpt   10   4.399 ± 0.063  ops/us
[info] TransformationBenchmark.transformation_64    improved     fjp          1  thrpt   10   6.745 ± 0.231  ops/us
[info] TransformationBenchmark.transformation_8192    stdlib     fjp          1  thrpt   10   7.391 ± 0.193  ops/us
[info] TransformationBenchmark.transformation_8192  improved     fjp          1  thrpt   10  14.740 ± 4.271  ops/us
[info]  scala.future.TransformationBenchmark.transformation_1024_impl=improved_pool=fjp_threads=1       17      ops/us
[info]  scala.future.TransformationBenchmark.transformation_1024_impl=stdlib_pool=fjp_threads=1 7       ops/us
[info]  scala.future.TransformationBenchmark.transformation_16_impl=improved_pool=fjp_threads=1 2       ops/us
[info]  scala.future.TransformationBenchmark.transformation_16_impl=stdlib_pool=fjp_threads=12ops/us
[info]  scala.future.TransformationBenchmark.transformation_1_impl=improved_pool=fjp_threads=10 ops/us
[info]  scala.future.TransformationBenchmark.transformation_1_impl=stdlib_pool=fjp_threads=1    0ops/us
[info]  scala.future.TransformationBenchmark.transformation_2_impl=improved_pool=fjp_threads=10 ops/us
[info]  scala.future.TransformationBenchmark.transformation_2_impl=stdlib_pool=fjp_threads=1    0ops/us
[info]  scala.future.TransformationBenchmark.transformation_4_impl=improved_pool=fjp_threads=11 ops/us
[info]  scala.future.TransformationBenchmark.transformation_4_impl=stdlib_pool=fjp_threads=1    1ops/us
[info]  scala.future.TransformationBenchmark.transformation_64_impl=improved_pool=fjp_threads=1 7       ops/us
[info]  scala.future.TransformationBenchmark.transformation_64_impl=stdlib_pool=fjp_threads=14ops/us
[info]  scala.future.TransformationBenchmark.transformation_8192_impl=improved_pool=fjp_threads=1       15      ops/us
[info]  scala.future.TransformationBenchmark.transformation_8192_impl=stdlib_pool=fjp_threads=1 7       ops/us

[info] Result "scala.future.CallbackBenchmark.onComplete_8192":
[info]   49.861 ±(99.9%) 5.720 ops/us [Average]
[info]   (min, avg, max) = (47.079, 49.861, 56.413), stdev = 3.783
[info]   CI (99.9%): [44.141, 55.580] (assumes normal distribution)
[info]
[info]
[info] # Run complete. Total time: 00:05:54
[info]
[info] Benchmark                            (impl)  (pool)  (threads)   Mode  Cnt   Score   Error   Units
[info] CallbackBenchmark.onComplete_1       stdlib     fjp          1  thrpt   10  21.110 ± 0.342  ops/us
[info] CallbackBenchmark.onComplete_1     improved     fjp          1  thrpt   10  25.511 ± 0.166  ops/us
[info] CallbackBenchmark.onComplete_1024    stdlib     fjp          1  thrpt   10  38.052 ± 0.090  ops/us
[info] CallbackBenchmark.onComplete_1024  improved     fjp          1  thrpt   10  52.904 ± 0.573  ops/us
[info] CallbackBenchmark.onComplete_16      stdlib     fjp          1  thrpt   10  36.827 ± 0.270  ops/us
[info] CallbackBenchmark.onComplete_16    improved     fjp          1  thrpt   10  38.935 ± 0.344  ops/us
[info] CallbackBenchmark.onComplete_2       stdlib     fjp          1  thrpt   10  27.046 ± 0.223  ops/us
[info] CallbackBenchmark.onComplete_2     improved     fjp          1  thrpt   10  34.077 ± 0.381  ops/us
[info] CallbackBenchmark.onComplete_4       stdlib     fjp          1  thrpt   10  31.698 ± 0.237  ops/us
[info] CallbackBenchmark.onComplete_4     improved     fjp          1  thrpt   10  33.668 ± 0.265  ops/us
[info] CallbackBenchmark.onComplete_64      stdlib     fjp          1  thrpt   10  38.727 ± 0.461  ops/us
[info] CallbackBenchmark.onComplete_64    improved     fjp          1  thrpt   10  51.312 ± 0.235  ops/us
[info] CallbackBenchmark.onComplete_8192    stdlib     fjp          1  thrpt   10  38.602 ± 0.251  ops/us
[info] CallbackBenchmark.onComplete_8192  improved     fjp          1  thrpt   10  49.861 ± 5.720  ops/us
[info]  scala.future.CallbackBenchmark.onComplete_1024_impl=improved_pool=fjp_threads=1 53      ops/us
[info]  scala.future.CallbackBenchmark.onComplete_1024_impl=stdlib_pool=fjp_threads=1   38      ops/us
[info]  scala.future.CallbackBenchmark.onComplete_16_impl=improved_pool=fjp_threads=1   39      ops/us
[info]  scala.future.CallbackBenchmark.onComplete_16_impl=stdlib_pool=fjp_threads=1     37      ops/us
[info]  scala.future.CallbackBenchmark.onComplete_1_impl=improved_pool=fjp_threads=1    26      ops/us
[info]  scala.future.CallbackBenchmark.onComplete_1_impl=stdlib_pool=fjp_threads=1      21      ops/us
[info]  scala.future.CallbackBenchmark.onComplete_2_impl=improved_pool=fjp_threads=1    34      ops/us
[info]  scala.future.CallbackBenchmark.onComplete_2_impl=stdlib_pool=fjp_threads=1      27      ops/us
[info]  scala.future.CallbackBenchmark.onComplete_4_impl=improved_pool=fjp_threads=1    34      ops/us
[info]  scala.future.CallbackBenchmark.onComplete_4_impl=stdlib_pool=fjp_threads=1      32      ops/us
[info]  scala.future.CallbackBenchmark.onComplete_64_impl=improved_pool=fjp_threads=1   51      ops/us
[info]  scala.future.CallbackBenchmark.onComplete_64_impl=stdlib_pool=fjp_threads=1     39      ops/us
[info]  scala.future.CallbackBenchmark.onComplete_8192_impl=improved_pool=fjp_threads=1 50      ops/us
[info]  scala.future.CallbackBenchmark.onComplete_8192_impl=stdlib_pool=fjp_threads=1   39      ops/us

[info] Result "scala.future.CompletionWithBenchmark.tryCompleteWith_8192":
[info]   7.556 ±(99.9%) 0.035 ops/us [Average]
[info]   (min, avg, max) = (7.525, 7.556, 7.598), stdev = 0.023
[info]   CI (99.9%): [7.522, 7.591] (assumes normal distribution)
[info] 
[info] 
[info] # Run complete. Total time: 00:11:55
[info] 
[info] Benchmark                                       (impl)  (result)   Mode  Cnt  Score   Error   Units
[info] CompletionWithBenchmark.tryCompleteWith_1       stdlib   success  thrpt   10  3.465 ± 0.020  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_1       stdlib   failure  thrpt   10  3.025 ± 0.022  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_1     improved   success  thrpt   10  4.846 ± 0.040  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_1     improved   failure  thrpt   10  4.038 ± 0.035  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_1024    stdlib   success  thrpt   10  4.644 ± 0.057  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_1024    stdlib   failure  thrpt   10  4.329 ± 0.030  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_1024  improved   success  thrpt   10  7.660 ± 0.050  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_1024  improved   failure  thrpt   10  7.724 ± 0.045  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_16      stdlib   success  thrpt   10  4.618 ± 0.040  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_16      stdlib   failure  thrpt   10  4.105 ± 0.014  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_16    improved   success  thrpt   10  7.344 ± 0.042  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_16    improved   failure  thrpt   10  7.380 ± 0.030  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_2       stdlib   success  thrpt   10  4.040 ± 0.018  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_2       stdlib   failure  thrpt   10  3.443 ± 0.035  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_2     improved   success  thrpt   10  5.969 ± 0.034  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_2     improved   failure  thrpt   10  5.169 ± 0.037  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_4       stdlib   success  thrpt   10  4.240 ± 0.024  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_4       stdlib   failure  thrpt   10  3.743 ± 0.031  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_4     improved   success  thrpt   10  6.788 ± 0.037  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_4     improved   failure  thrpt   10  6.189 ± 0.059  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_64      stdlib   success  thrpt   10  4.714 ± 0.041  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_64      stdlib   failure  thrpt   10  4.090 ± 0.027  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_64    improved   success  thrpt   10  7.521 ± 0.044  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_64    improved   failure  thrpt   10  7.393 ± 0.032  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_8192    stdlib   success  thrpt   10  4.630 ± 0.045  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_8192    stdlib   failure  thrpt   10  4.345 ± 0.032  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_8192  improved   success  thrpt   10  7.599 ± 0.023  ops/us
[info] CompletionWithBenchmark.tryCompleteWith_8192  improved   failure  thrpt   10  7.556 ± 0.035  ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_1024_impl=improved_result=failure	8	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_1024_impl=improved_result=success	8	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_1024_impl=stdlib_result=failure	4	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_1024_impl=stdlib_result=success	5	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_16_impl=improved_result=failure	7	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_16_impl=improved_result=success	7	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_16_impl=stdlib_result=failure	4	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_16_impl=stdlib_result=success	5	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_1_impl=improved_result=failure	4	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_1_impl=improved_result=success	5	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_1_impl=stdlib_result=failure	3	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_1_impl=stdlib_result=success	3	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_2_impl=improved_result=failure	5	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_2_impl=improved_result=success	6	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_2_impl=stdlib_result=failure	3	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_2_impl=stdlib_result=success	4	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_4_impl=improved_result=failure	6	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_4_impl=improved_result=success	7	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_4_impl=stdlib_result=failure	4	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_4_impl=stdlib_result=success	4	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_64_impl=improved_result=failure	7	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_64_impl=improved_result=success	8	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_64_impl=stdlib_result=failure	4	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_64_impl=stdlib_result=success	5	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_8192_impl=improved_result=failure	8	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_8192_impl=improved_result=success	8	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_8192_impl=stdlib_result=failure	4	ops/us
[info] 	scala.future.CompletionWithBenchmark.tryCompleteWith_8192_impl=stdlib_result=success	5	ops/us

[info] Result "scala.future.CompletionBenchmark.tryComplete_8192":
[info]   687.949 ±(99.9%) 0.456 ops/us [Average]
[info]   (min, avg, max) = (687.437, 687.949, 688.570), stdev = 0.301
[info]   CI (99.9%): [687.493, 688.405] (assumes normal distribution)
[info] 
[info] 
[info] # Run complete. Total time: 00:11:48
[info] 
[info] Benchmark                               (impl)  (result)   Mode  Cnt    Score    Error   Units
[info] CompletionBenchmark.tryComplete_1       stdlib   success  thrpt   10   33.644 ±  0.110  ops/us
[info] CompletionBenchmark.tryComplete_1       stdlib   failure  thrpt   10   17.823 ±  0.110  ops/us
[info] CompletionBenchmark.tryComplete_1     improved   success  thrpt   10   31.379 ±  0.155  ops/us
[info] CompletionBenchmark.tryComplete_1     improved   failure  thrpt   10   19.834 ±  0.121  ops/us
[info] CompletionBenchmark.tryComplete_1024    stdlib   success  thrpt   10  378.529 ±  2.911  ops/us
[info] CompletionBenchmark.tryComplete_1024    stdlib   failure  thrpt   10   39.280 ±  0.331  ops/us
[info] CompletionBenchmark.tryComplete_1024  improved   success  thrpt   10  826.627 ± 10.282  ops/us
[info] CompletionBenchmark.tryComplete_1024  improved   failure  thrpt   10  820.172 ±  5.641  ops/us
[info] CompletionBenchmark.tryComplete_16      stdlib   success  thrpt   10  251.321 ±  1.813  ops/us
[info] CompletionBenchmark.tryComplete_16      stdlib   failure  thrpt   10   36.846 ±  0.224  ops/us
[info] CompletionBenchmark.tryComplete_16    improved   success  thrpt   10  289.859 ±  0.921  ops/us
[info] CompletionBenchmark.tryComplete_16    improved   failure  thrpt   10  223.847 ±  1.592  ops/us
[info] CompletionBenchmark.tryComplete_2       stdlib   success  thrpt   10   51.896 ±  0.144  ops/us
[info] CompletionBenchmark.tryComplete_2       stdlib   failure  thrpt   10   24.898 ±  0.301  ops/us
[info] CompletionBenchmark.tryComplete_2     improved   success  thrpt   10   51.125 ±  0.127  ops/us
[info] CompletionBenchmark.tryComplete_2     improved   failure  thrpt   10   36.193 ±  0.439  ops/us
[info] CompletionBenchmark.tryComplete_4       stdlib   success  thrpt   10   94.477 ±  0.316  ops/us
[info] CompletionBenchmark.tryComplete_4       stdlib   failure  thrpt   10   30.565 ±  0.197  ops/us
[info] CompletionBenchmark.tryComplete_4     improved   success  thrpt   10   97.251 ±  0.335  ops/us
[info] CompletionBenchmark.tryComplete_4     improved   failure  thrpt   10   69.855 ±  0.128  ops/us
[info] CompletionBenchmark.tryComplete_64      stdlib   success  thrpt   10  355.271 ±  4.243  ops/us
[info] CompletionBenchmark.tryComplete_64      stdlib   failure  thrpt   10   38.420 ±  0.216  ops/us
[info] CompletionBenchmark.tryComplete_64    improved   success  thrpt   10  575.297 ±  5.536  ops/us
[info] CompletionBenchmark.tryComplete_64    improved   failure  thrpt   10  501.714 ±  6.928  ops/us
[info] CompletionBenchmark.tryComplete_8192    stdlib   success  thrpt   10  382.756 ±  1.102  ops/us
[info] CompletionBenchmark.tryComplete_8192    stdlib   failure  thrpt   10   41.690 ±  2.324  ops/us
[info] CompletionBenchmark.tryComplete_8192  improved   success  thrpt   10  687.765 ±  1.608  ops/us
[info] CompletionBenchmark.tryComplete_8192  improved   failure  thrpt   10  687.949 ±  0.456  ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_1024_impl=improved_result=failure	820	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_1024_impl=improved_result=success	827	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_1024_impl=stdlib_result=failure	39	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_1024_impl=stdlib_result=success	379	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_16_impl=improved_result=failure	224	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_16_impl=improved_result=success	290	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_16_impl=stdlib_result=failure	37	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_16_impl=stdlib_result=success	251	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_1_impl=improved_result=failure	20	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_1_impl=improved_result=success	31	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_1_impl=stdlib_result=failure	18	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_1_impl=stdlib_result=success	34	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_2_impl=improved_result=failure	36	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_2_impl=improved_result=success	51	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_2_impl=stdlib_result=failure	25	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_2_impl=stdlib_result=success	52	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_4_impl=improved_result=failure	70	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_4_impl=improved_result=success	97	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_4_impl=stdlib_result=failure	31	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_4_impl=stdlib_result=success	94	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_64_impl=improved_result=failure	502	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_64_impl=improved_result=success	575	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_64_impl=stdlib_result=failure	38	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_64_impl=stdlib_result=success	355	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_8192_impl=improved_result=failure	688	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_8192_impl=improved_result=success	688	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_8192_impl=stdlib_result=failure	42	ops/us
[info] 	scala.future.CompletionBenchmark.tryComplete_8192_impl=stdlib_result=success	383	ops/us

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment