Benchmark for comparing python 3.12 and 3.13 on list comprehensions with a filter

bench_list_comprehensions.py

import sys
from time import time

N_repeats = 100


def func(foo: list[int]):
    return [x for x in foo if x % 2 == 0]


print(f"Running on Python {sys.version}")
for N in (100, 1_000, 10_000, 100_000, 1_000_000):
    foo = list(range(N))
    times: list[float] = []

    for _ in range(N_repeats):
        t1 = time()
        func(foo)
        t2 = time()
        times.append(t2 - t1)

    comprehension_time = min(times)
    time_per_iter_ns = comprehension_time * 1e9 / N
    print(f"{time_per_iter_ns:.2f}ns on N={N:_}")

Benchmarks for second revision of the above script:

Running on Python 3.12.7 (main, Oct  2 2024, 15:45:55) [Clang 18.1.8 ]
16.69ns on N=100
18.84ns on N=1_000
15.88ns on N=10_000
14.26ns on N=100_000
14.70ns on N=1_000_000

Running on Python 3.13.0 (main, Oct 16 2024, 08:05:40) [Clang 18.1.8 ]
9.54ns on N=100
17.88ns on N=1_000
15.81ns on N=10_000
14.14ns on N=100_000
14.46ns on N=1_000_000

Here are mine, expanded to include 3.11 and a compiled free-threading 3.13t build:

❯ for py in python3.11 python3.12 python3.13 python3.13t; do eval "$py test_loops.py"; done
Running on Python 3.11.10 (main, Sep  7 2024, 01:03:31) [Clang 16.0.0 (clang-1600.0.26.3)]
28.61ns on N=100
35.76ns on N=1_000
27.68ns on N=10_000
23.89ns on N=100_000
24.19ns on N=1_000_000
Running on Python 3.12.7 (main, Oct  1 2024, 02:05:46) [Clang 16.0.0 (clang-1600.0.26.3)]
16.69ns on N=100
21.70ns on N=1_000
21.58ns on N=10_000
21.83ns on N=100_000
22.01ns on N=1_000_000
Running on Python 3.13.0 (main, Oct  7 2024, 05:02:14) [Clang 16.0.0 (clang-1600.0.26.3)]
26.23ns on N=100
32.66ns on N=1_000
32.31ns on N=10_000
32.18ns on N=100_000
32.94ns on N=1_000_000
Running on Python 3.13.0 experimental free-threading build (main, Oct 23 2024, 21:45:14) [Clang 16.0.0 (clang-1600.0.26.3)]
26.23ns on N=100
28.85ns on N=1_000
29.37ns on N=10_000
29.73ns on N=100_000
30.78ns on N=1_000_000

I also ran them with Hyperfine:

❯ hyperfine -w 10 -r 10 "python3.11 test_loops.py" "python3.12 test_loops.py" "python3.13 test_loops.py" "python3.13t test_loops.py"
Benchmark 1: python3.11 test_loops.py
  Time (mean ± σ):      2.893 s ±  0.070 s    [User: 2.836 s, System: 0.056 s]
  Range (min … max):    2.786 s …  3.012 s    10 runs

Benchmark 2: python3.12 test_loops.py
  Time (mean ± σ):      2.517 s ±  0.009 s    [User: 2.457 s, System: 0.059 s]
  Range (min … max):    2.494 s …  2.525 s    10 runs

Benchmark 3: python3.13 test_loops.py
  Time (mean ± σ):      3.734 s ±  0.033 s    [User: 3.675 s, System: 0.057 s]
  Range (min … max):    3.694 s …  3.786 s    10 runs

Benchmark 4: python3.13t test_loops.py
  Time (mean ± σ):      3.484 s ±  0.035 s    [User: 3.461 s, System: 0.021 s]
  Range (min … max):    3.454 s …  3.570 s    10 runs

Summary
  python3.12 test_loops.py ran
    1.15 ± 0.03 times faster than python3.11 test_loops.py
    1.38 ± 0.01 times faster than python3.13t test_loops.py
    1.48 ± 0.01 times faster than python3.13 test_loops.py