Benchmarking very fast functions with benchee

fast.exs

range = 1..10
integer1 = :rand.uniform(100)
integer2 = :rand.uniform(100)

Benchee.run(
  %{
    "Integer addition (wrong)" => fn -> 1 + 1 end,
    "Integer addition" => fn -> integer1 + integer2 end,
    "String concatention (wrong)" => fn -> "1" <> "1" end,
    "adding a head to an array (wrong)" => fn -> [1 | [1]] end,
    "++ array concat (wrong)" => fn -> [1] ++ [1] end,
    "noop" => fn -> 0 end,
    "Enum.map(10)" => fn -> Enum.map(range, fn i -> i end) end
  },
  time: 1,
  warmup: 1,
  memory_time: 1,
  formatters: [{Benchee.Formatters.Console, extended_statistics: true}]
)

output

tobi@speedy:~/github/benchee$ mix run samples/fast_functions.exs 
Operating System: Linux
CPU Information: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
Number of Available Cores: 8
Available memory: 15.61 GB
Elixir 1.8.0
Erlang 21.2.2

Benchmark suite executing with the following configuration:
warmup: 1 s
time: 1 s
memory time: 1 s
parallel: 1
inputs: none specified
Estimated total run time: 21 s


Benchmarking ++ array concat (wrong)...
Benchmarking Enum.map(10)...
Benchmarking Integer addition...
Benchmarking Integer addition (wrong)...
Benchmarking String concatention (wrong)...
Benchmarking adding a head to an array (wrong)...
Benchmarking noop...

Name                                        ips        average  deviation         median         99th %
String concatention (wrong)            814.22 M        1.23 ns  ±2868.77%           0 ns          21 ns
++ array concat (wrong)                749.67 M        1.33 ns  ±2705.83%           0 ns          23 ns
noop                                   639.06 M        1.56 ns  ±2388.12%           0 ns          35 ns
adding a head to an array (wrong)      553.47 M        1.81 ns  ±2228.78%           0 ns          29 ns
Integer addition (wrong)               544.93 M        1.84 ns  ±2803.80%           0 ns          31 ns
Integer addition                       179.88 M        5.56 ns   ±737.19%           4 ns          39 ns
Enum.map(10)                             2.30 M      435.06 ns  ±2872.78%         356 ns         667 ns

Comparison: 
String concatention (wrong)            814.22 M
++ array concat (wrong)                749.67 M - 1.09x slower
noop                                   639.06 M - 1.27x slower
adding a head to an array (wrong)      553.47 M - 1.47x slower
Integer addition (wrong)               544.93 M - 1.49x slower
Integer addition                       179.88 M - 4.53x slower
Enum.map(10)                             2.30 M - 354.23x slower

Extended statistics: 

Name                                      minimum        maximum    sample size                     mode
String concatention (wrong)                  0 ns        9219 ns         1.54 M                     0 ns
++ array concat (wrong)                      0 ns       17501 ns         1.54 M                     0 ns
noop                                         0 ns        9220 ns         1.53 M                     0 ns
adding a head to an array (wrong)            0 ns       23216 ns         1.54 M                     0 ns
Integer addition (wrong)                     0 ns       16040 ns         1.52 M                     0 ns
Integer addition                             0 ns        9818 ns         1.52 M                     4 ns
Enum.map(10)                               335 ns     7385903 ns       952.30 K                   354 ns

Memory usage statistics:

Name                                 Memory usage
String concatention (wrong)                   0 B
++ array concat (wrong)                       0 B
noop                                          0 B
adding a head to an array (wrong)             0 B
Integer addition (wrong)                      0 B
Integer addition                              0 B
Enum.map(10)                                424 B

**All measurements for memory usage were the same**