A simple Python benchmark

bench.py

from __future__ import print_function
from math import sin, cos, radians
import timeit

'''
A simple Python benchmark.

Results on an overclocked AMD FX-8150 Eight-Core CPU @ 3.0 GHz, and
an Intel Core i5-2410M CPU @ 2.30GHz.

$ python -OO bench.py
1.99843406677 2.00139904022 2.0145778656
2.38226699829 2.38675498962 2.38853287697

$ python3 -OO bench.py
2.2073315899979207 2.2098999509980786 2.222747125000751
2.273064840992447  2.274112678001984 2.2759074380010134

$ pypy -OO bench.py
0.245079994202 0.24707698822  0.247714996338
0.241708040237 0.242873907089 0.245008945465

$ pypy3 -OO bench.py
1.1291401386260986 1.1360960006713867 1.1375579833984375
1.2108190059661865 1.2172389030456543 1.2178328037261963

'''


def bench():
    product = 1.0
    for counter in range(1, 1000, 1):
        for dex in list(range(1, 360, 1)):
            angle = radians(dex)
            product *= sin(angle)**2 + cos(angle)**2
    return product

if __name__ == '__main__':
    result = timeit.repeat('bench.bench()', setup='import bench', number=10, repeat=10)
    result = list(sorted(result))
    print(*result[:3])

This doesn't work?

Apple M1 MacBook Air 16GB gets:
python -OO bench.py
0.645364999771 0.645573139191 0.646193981171
python3 -OO bench.py
0.6115190000000004 0.6118404580000001 0.611904

with 13 apps in the background :)

AMD Ryzen 7 3700x with 48GB RAM:
1.8907193999999992 1.8923012000000021 1.8995879999999996
with 9 apps in the background

I'm 3X faster? Which python version did you run?

Snapdragon 865+ (Samsung Galaxy Tab S7+, Ubuntu 20.04 on Termux in Android with Samsung Dex, Python 3.8)
0.9339403639896773 0.9342242699931376 0.9347601039917208
I am not sure which apps are in the background. Anyway, results are pretty consistent.

Why the sudden interest on this benchmark?

Apple MacBook Pro 15" (2015) 2.5 GHz Quad-Core Intel Core i7 16GB Ram

python -OO bench.py
1.1667073040000004 1.1672709139999995 1.1685495430000001

python3 -OO bench.py
1.2099393779999996 1.2139072740000003 1.2139641090000008

pypy -OO bench.py
0.13471456800016313 0.13493599199955497 0.1350344459997359

pypy3 -OO bench.py
0.13697827399937523 0.13717118099975778 0.13746063700000377

Jupyter %timeit function:
115 ms ± 2.06 ms per loop (mean ± std. dev. of 10 runs, 10 loops each)

I'm surprised that such an old machine is doing so well. However, only 2 apps in background, and CPU usage was idling at 8-10% prior to the test.

Apple M1 Pro 16GB Ram gets:
Python 3.9.7

✗ python -OO bench.py
0.6188454170000002 0.6192598749999999 0.6194524169999998

Not pypy available build for M1 yet (at least in homebrew).

@apalala Not sure about everyone else, but I needed an easy way to compare certain applications performance on x86 vs ARM

heh )
Nexus 5 - Postmarket OS
31.562464889000694 31.56695511900034 31.588837952996982
Asus A53 Intel i7
1.829460040997219 1.8857796539996343 1.8871137909991376
its only sitgle threads ?

Macbook Air from early 2015

$ python3.9 bench.py
1.6820437410000002 1.6992794439999992 1.7123132529999996

$ python3.10 bench.py
1.4265755990054458 1.4323118559550494 1.4479744010022841

$ python3.11 bench.py
1.2640831180033274 1.268143099034205 1.282965452002827

$ pypy3 bench.py
0.13054357696091756 0.13099552801577374 0.1329099930007942

E3 1230 V2:
PS G:> python -OO .\bench.py
3.2582996999999994 3.3040652 3.321830900000002

R2S Soft Router:
$ python -OO bench.py
13.691393590997905 13.70187233100296 13.703434251016006

Baidu Yun 1C2G VM:
root@lsizINeXaH:~# python3 -OO bench.py
1.8363916339876596 1.8446048560144845 1.8466756780107971

Mac Mini Intel [email protected]
wangxudeMini:workfolder wangxu$ python3 -OO bench.py
3.1054392130000004 3.1057074900000003 3.386463592000002

Mi 11 Ultra Snapdragon 888:
TBD

I was curious about the impact of QEMU on M1 Macbooks with x86 Docker images, so I ran this benchmark.

Specs:

• My device is 2021 14" MBP with M1 Pro and 32GB RAM.
• My Python version is 3.10.
• Docker has 11GB RAM and 3 cores available.

Below are the results:

• Running natively: 0.6509173749946058 0.6511464999057353 0.6511690000770614
• Running in a native arm64 container: 0.5683689580037026 0.5695775839994894 0.5701702090009348
• Running in an x86 emulated container: 14.544861840011436 14.5703257569985 14.61461971499375

So, as expected, the emulated container is much slower than the native ones, but I couldn't really figure out what was the deal with the Docker container being faster than running the script directly.

if someone is interested: running the benchmark on pyodide (python inbrowser, Node.js, based on WebAssembly), it runs about 1.3 sec per iteration. That is about twice as native. (MBA M1, firefox)

Looks interesting. I tested Rossetta compiled python (anaconda) and native python (miniforge)

Specs:

• Macbook pro M1 Max
• Anaconda python version is: 3.9.7, miniforge is 3.9.10

Anaconda:
1.3440638339999995 1.345124833 1.345285625

Miniforge:
0.637413166 0.6387493339999999 0.638863084

∩( ・ω・)∩

Apple Mac mini 8G anaconda Python 3.9
0.6556459589999992 0.6564532079999998 0.6569619580000001

I also tested it with my ASUS Laptop
ASUS ZX53V i5-7300HQ 12G 2133
2.1350787999999987 2.1404666 2.142507799999999

That's amazing!

1.0594426930183545 1.062258291989565 1.0630121920257807
On Gitpod
CPU: AMD EPYC 7B13
Arch: x86_64
OS: Linux

Maybe the list() on line 33 produces variations that depend on the memory manager or system load?

The benchmark could use just the range(), or precompute the list.

...I did the optimization, and it had no effect on my numbers.

def bench():
    product = 1.0
    dexes = list(range(1, 360, 1))
    for counter in range(1, 1000, 1):
        for dex in dexes:
            angle = radians(dex)
            product *= sin(angle)**2 + cos(angle)**2
    return product

$ python3.10 -OO bench.py
1.2302762420004 1.2326733869995223 1.23436711600516

$ pypy3 -OO bench.py
0.1486122600035742 0.15070434800873045 0.15452972600178327

Comparing my old Intel MacBook to new Apple Silicon MacBook:

Both running on Python 3.9.12 in a fresh conda environment

Apple MacBook Pro 15" (2015) 2.5 GHz Quad-Core Intel Core i7 16GB Ram

$ python -OO bench.py
1.3066859839999996 1.315500633000001 1.3444327110000005

$ pypy -OO bench.py
0.13471456800016313 0.13493599199955497 0.1350344459997359

Jupyter %timeit function:
129 ms ± 615 µs per loop (mean ± std. dev. of 10 runs, 10 loops each)

Apple MacBook Pro 16" (2021) M1 Max 10-core 64GB RAM

$ python -OO bench.py
0.60054525 0.6010592090000002 0.6021369170000002

$ pypy -OO bench.py (pypy nightly build for Apple Silicon)
0.03519466705620289 0.035204792162403464 0.03520554187707603

Jupyter %timeit function:
60.7 ms ± 1.22 ms per loop (mean ± std. dev. of 10 runs, 10 loops each)

Acer Aspire A315, AMD A6 dual-core 2.9GHz, 8GB ddr4 ram, AMD R5 GPU 2GB gddr5, SSD SATA, Zorin os 16.1 pro (Linux kernel 5.15):
1.9391337739998562 1.939522442999987 1.9427987970002505

M1 Pro (2021)

Intel binaries through Rosetta 2

Python 3.10.8, osx-64 through Rosetta 2 (conda-forge)
1.580 1.589 1.591

Python 3.11.0, osx-64 through Rosetta 2 (conda-forge)
1.517 1.517 1.521

Python 3.11.2, osx-64 (binary from https://www.python.org/ftp/python/3.11.2/python-3.11.2-macos11.pkg)
1.302 1.386 1.299

Python 3.12.0a5, osx-64 through Rosetta 2 (binary from https://www.python.org/downloads/release/python-3120a5/)
1.326 1.295 1.270

Native binaries

Python 3.10.9, osx-arm64 (conda-forge)
0.762 0.762 0.769

Python 3.11.0, osx-arm64 (conda-forge)
0.637 0.637 0.637

Python 3.11.2, osx-arm64 (binary from https://www.python.org/ftp/python/3.11.2/python-3.11.2-macos11.pkg)
0.599 0.598 0.599

Python 3.12.0a5, osx-arm64 (binary from https://www.python.org/downloads/release/python-3120a5/)
0.602 0.602 0.602

Intel Pypy through Rosetta 2

pypy 3.9 (7.3.11), osx-64 through Rosetta 2 (conda-forge)
0.159 0.160 0.160

@corneliusroemer, Why not a Python 3.12.x test too?

Raspberry Pi 4B (4GB Ram): Python 3.9.2
pi17@rpi17:~ $ python3 bench.py
3.983417603000021 4.000531667000018 4.001489117999995
pi17@rpi17:~ $ pypy bench.py
0.395509004593 0.395509958267 0.395529985428

MacBook Pro 2015

$ python3.12 bench.py
1.1071506180014694 1.1081515140103875 1.1114070440089563

$ pypy3
Python 3.10.12 (af44d0b8114cb82c40a07bb9ee9c1ca8a1b3688c, Jun 25 2023, 12:41:21)
[PyPy 7.3.12 with GCC Apple LLVM 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>>

$ pypy3 -O bench.py
0.1369127530051628 0.13725044101011008 0.13727515301434323

Macbook Air M2 2023

$ python3.11 -O bench.py
0.47316745800344506 0.47342624999873806 0.4737870000026305

$ python3.12 -O bench.py
0.5029823330041836 0.5032062500031316 0.5033741669976735

$ pypy3 -O bench.py
0.03534275000129128 0.03537720799795352 0.0354161660070531

Raspberry Pi 5 Model B Rev 1.0

Official python docker image 3.13.0:

$ python
Python 3.13.0 (main, Oct  8 2024, 00:55:30) [GCC 13.2.1 20240309] on linux

$ docker run --rm -it -v $(pwd):/bench python:3.13-alpine python /bench/bench.py
1.379287590039894 1.3798810039879754 1.3804634480038658

Custom python 3.13.0 compiled with --enable-experimental-jit:

$ python
Python 3.13.0 (main, Oct  9 2024, 01:04:00) [GCC 14.2.0] on linux

$ docker run --rm -it -v $(pwd):/bench my_custom_python /root/.pyenv/versions/3.13.0/bin/python /bench/bench.py
1.1263249380281195 1.1263620959362015 1.1266266249585897

Not bad.

MacBook Air M2 2023

$ python3.13 --version
Python 3.13.1

$ python3.13 bench.py
0.5406068330048583 0.5409433339955285 0.5412480410013814

$ python3.13 -OO bench.py
0.5364589579985477 0.5369448330020532 0.5370732499868609