Skip to content

Instantly share code, notes, and snippets.

@nelhage
Created February 11, 2025 19:38
Show Gist options
  • Save nelhage/0b1cf340dcf86d42f8c4814338cc883e to your computer and use it in GitHub Desktop.
Save nelhage/0b1cf340dcf86d42f8c4814338cc883e to your computer and use it in GitHub Desktop.
LTO+PGO pyperformance run

Benchmarks with tag 'apps':

Benchmark 2025-02-11_15-23-computed-goto-247b50dec8af 2025-02-11_16-19-tailcall-10c138c179ba 2025-02-11_16-15-computed-goto-nomerge-3178d094669b
2to3 295 ms 288 ms: 1.02x faster 293 ms: 1.01x faster
docutils 2.86 sec 2.83 sec: 1.01x faster not significant
html5lib 64.3 ms 63.9 ms: 1.01x faster 64.9 ms: 1.01x slower
Geometric mean (ref) 1.01x faster 1.00x slower

Benchmarks with tag 'math':

Benchmark 2025-02-11_15-23-computed-goto-247b50dec8af 2025-02-11_16-19-tailcall-10c138c179ba 2025-02-11_16-15-computed-goto-nomerge-3178d094669b
float 73.4 ms 66.3 ms: 1.11x faster 68.2 ms: 1.08x faster
nbody 108 ms 89.1 ms: 1.21x faster 93.4 ms: 1.15x faster
pidigits 274 ms 274 ms: 1.00x faster 274 ms: 1.00x faster
Geometric mean (ref) 1.10x faster 1.07x faster

Benchmarks with tag 'regex':

Benchmark 2025-02-11_15-23-computed-goto-247b50dec8af 2025-02-11_16-19-tailcall-10c138c179ba 2025-02-11_16-15-computed-goto-nomerge-3178d094669b
regex_compile 145 ms 133 ms: 1.09x faster 142 ms: 1.02x faster
regex_dna 218 ms 218 ms: 1.00x faster 216 ms: 1.01x faster
regex_effbot 2.92 ms 2.94 ms: 1.01x slower 2.89 ms: 1.01x faster
regex_v8 26.3 ms 25.8 ms: 1.02x faster 25.6 ms: 1.03x faster
Geometric mean (ref) 1.03x faster 1.02x faster

Benchmarks with tag 'serialize':

Benchmark 2025-02-11_15-23-computed-goto-247b50dec8af 2025-02-11_16-19-tailcall-10c138c179ba 2025-02-11_16-15-computed-goto-nomerge-3178d094669b
json_dumps 12.4 ms 11.9 ms: 1.04x faster 12.1 ms: 1.02x faster
json_loads 26.8 us 25.9 us: 1.04x faster 26.6 us: 1.01x faster
pickle 12.7 us 12.7 us: 1.00x slower 12.6 us: 1.01x faster
pickle_dict 32.0 us 31.3 us: 1.02x faster 31.1 us: 1.03x faster
pickle_list 4.82 us 4.74 us: 1.02x faster 4.70 us: 1.02x faster
pickle_pure_python 353 us 315 us: 1.12x faster 337 us: 1.05x faster
tomli_loads 2.33 sec 2.01 sec: 1.16x faster 2.13 sec: 1.09x faster
unpickle 13.8 us 13.7 us: 1.01x faster 14.0 us: 1.01x slower
unpickle_list 4.59 us 4.54 us: 1.01x faster 4.68 us: 1.02x slower
unpickle_pure_python 235 us 204 us: 1.15x faster 210 us: 1.12x faster
xml_etree_parse 164 ms 162 ms: 1.01x faster 162 ms: 1.01x faster
xml_etree_iterparse 109 ms 104 ms: 1.05x faster 104 ms: 1.05x faster
xml_etree_generate 89.7 ms 81.5 ms: 1.10x faster 84.7 ms: 1.06x faster
xml_etree_process 64.1 ms 56.8 ms: 1.13x faster 60.2 ms: 1.06x faster
Geometric mean (ref) 1.06x faster 1.04x faster

Benchmarks with tag 'startup':

Benchmark 2025-02-11_15-23-computed-goto-247b50dec8af 2025-02-11_16-19-tailcall-10c138c179ba 2025-02-11_16-15-computed-goto-nomerge-3178d094669b
python_startup 14.5 ms 14.5 ms: 1.01x faster 14.6 ms: 1.00x slower
python_startup_no_site 10.3 ms 10.3 ms: 1.00x faster 10.3 ms: 1.00x slower
Geometric mean (ref) 1.00x faster 1.00x slower

Benchmarks with tag 'template':

Benchmark 2025-02-11_15-23-computed-goto-247b50dec8af 2025-02-11_16-19-tailcall-10c138c179ba 2025-02-11_16-15-computed-goto-nomerge-3178d094669b
django_template 40.4 ms 37.3 ms: 1.08x faster 39.2 ms: 1.03x faster
genshi_text 24.9 ms 21.6 ms: 1.15x faster 24.0 ms: 1.04x faster
genshi_xml 55.2 ms 49.8 ms: 1.11x faster 54.6 ms: 1.01x faster
mako 11.9 ms 11.0 ms: 1.09x faster 11.7 ms: 1.02x faster
Geometric mean (ref) 1.11x faster 1.02x faster

All benchmarks:

Benchmark 2025-02-11_15-23-computed-goto-247b50dec8af 2025-02-11_16-19-tailcall-10c138c179ba 2025-02-11_16-15-computed-goto-nomerge-3178d094669b
2to3 295 ms 288 ms: 1.02x faster 293 ms: 1.01x faster
async_generators 442 ms 439 ms: 1.01x faster 436 ms: 1.01x faster
asyncio_tcp_ssl 1.61 sec 1.60 sec: 1.00x faster not significant
chaos 62.2 ms 58.8 ms: 1.06x faster 59.0 ms: 1.05x faster
comprehensions 17.2 us 15.4 us: 1.12x faster 16.3 us: 1.06x faster
bench_mp_pool 284 ms 215 ms: 1.32x faster not significant
bench_thread_pool 939 us 869 us: 1.08x faster 934 us: 1.01x faster
coroutines 22.0 ms 21.3 ms: 1.03x faster 21.1 ms: 1.04x faster
coverage 852 ms 819 ms: 1.04x faster 812 ms: 1.05x faster
crypto_pyaes 80.9 ms 74.4 ms: 1.09x faster 75.3 ms: 1.07x faster
dask 341 ms 333 ms: 1.02x faster not significant
deepcopy 285 us 268 us: 1.06x faster 277 us: 1.03x faster
deepcopy_reduce 2.96 us 2.78 us: 1.07x faster 2.89 us: 1.03x faster
deepcopy_memo 31.9 us 27.3 us: 1.17x faster 28.7 us: 1.11x faster
deltablue 3.34 ms 3.10 ms: 1.08x faster 3.26 ms: 1.02x faster
django_template 40.4 ms 37.3 ms: 1.08x faster 39.2 ms: 1.03x faster
docutils 2.86 sec 2.83 sec: 1.01x faster not significant
dulwich_log 64.7 ms 61.3 ms: 1.06x faster 64.4 ms: 1.01x faster
fannkuch 446 ms 370 ms: 1.21x faster 396 ms: 1.13x faster
float 73.4 ms 66.3 ms: 1.11x faster 68.2 ms: 1.08x faster
create_gc_cycles 1.90 ms 1.95 ms: 1.02x slower not significant
gc_traversal 4.34 ms 4.64 ms: 1.07x slower 4.70 ms: 1.08x slower
generators 33.0 ms 27.7 ms: 1.19x faster 31.6 ms: 1.04x faster
genshi_text 24.9 ms 21.6 ms: 1.15x faster 24.0 ms: 1.04x faster
genshi_xml 55.2 ms 49.8 ms: 1.11x faster 54.6 ms: 1.01x faster
go 137 ms 123 ms: 1.12x faster 131 ms: 1.04x faster
hexiom 6.91 ms 5.89 ms: 1.17x faster 6.32 ms: 1.09x faster
html5lib 64.3 ms 63.9 ms: 1.01x faster 64.9 ms: 1.01x slower
json_dumps 12.4 ms 11.9 ms: 1.04x faster 12.1 ms: 1.02x faster
json_loads 26.8 us 25.9 us: 1.04x faster 26.6 us: 1.01x faster
logging_format 6.75 us 6.58 us: 1.03x faster 6.63 us: 1.02x faster
logging_silent 92.2 ns 83.6 ns: 1.10x faster 87.4 ns: 1.06x faster
logging_simple 6.13 us 5.97 us: 1.03x faster 6.01 us: 1.02x faster
mako 11.9 ms 11.0 ms: 1.09x faster 11.7 ms: 1.02x faster
mdp 2.67 sec not significant 2.65 sec: 1.01x faster
meteor_contest 134 ms 132 ms: 1.01x faster 134 ms: 1.00x slower
nbody 108 ms 89.1 ms: 1.21x faster 93.4 ms: 1.15x faster
nqueens 96.6 ms 91.0 ms: 1.06x faster 93.7 ms: 1.03x faster
pathlib 15.0 ms 14.4 ms: 1.04x faster 14.7 ms: 1.02x faster
pickle 12.7 us 12.7 us: 1.00x slower 12.6 us: 1.01x faster
pickle_dict 32.0 us 31.3 us: 1.02x faster 31.1 us: 1.03x faster
pickle_list 4.82 us 4.74 us: 1.02x faster 4.70 us: 1.02x faster
pickle_pure_python 353 us 315 us: 1.12x faster 337 us: 1.05x faster
pidigits 274 ms 274 ms: 1.00x faster 274 ms: 1.00x faster
pprint_safe_repr 891 ms 789 ms: 1.13x faster 838 ms: 1.06x faster
pprint_pformat 1.81 sec 1.61 sec: 1.13x faster 1.71 sec: 1.06x faster
pyflate 457 ms 410 ms: 1.11x faster 425 ms: 1.07x faster
python_startup 14.5 ms 14.5 ms: 1.01x faster 14.6 ms: 1.00x slower
python_startup_no_site 10.3 ms 10.3 ms: 1.00x faster 10.3 ms: 1.00x slower
raytrace 277 ms 264 ms: 1.05x faster 267 ms: 1.04x faster
regex_compile 145 ms 133 ms: 1.09x faster 142 ms: 1.02x faster
regex_dna 218 ms 218 ms: 1.00x faster 216 ms: 1.01x faster
regex_effbot 2.92 ms 2.94 ms: 1.01x slower 2.89 ms: 1.01x faster
regex_v8 26.3 ms 25.8 ms: 1.02x faster 25.6 ms: 1.03x faster
richards 47.3 ms 42.0 ms: 1.13x faster 43.9 ms: 1.08x faster
richards_super 53.2 ms 47.9 ms: 1.11x faster 50.1 ms: 1.06x faster
scimark_fft 323 ms 294 ms: 1.10x faster 288 ms: 1.12x faster
scimark_lu 109 ms 94.2 ms: 1.15x faster 97.6 ms: 1.11x faster
scimark_monte_carlo 70.6 ms 59.0 ms: 1.20x faster 61.9 ms: 1.14x faster
scimark_sor 123 ms 96.7 ms: 1.27x faster 101 ms: 1.22x faster
scimark_sparse_mat_mult 4.83 ms 4.21 ms: 1.15x faster 4.33 ms: 1.11x faster
spectral_norm 87.0 ms 79.8 ms: 1.09x faster 80.9 ms: 1.08x faster
sqlglot_normalize 119 ms 115 ms: 1.04x faster 118 ms: 1.01x faster
sqlglot_optimize 59.6 ms 57.8 ms: 1.03x faster 58.7 ms: 1.01x faster
sqlglot_parse 1.39 ms 1.27 ms: 1.10x faster 1.33 ms: 1.05x faster
sqlglot_transpile 1.76 ms 1.65 ms: 1.07x faster 1.71 ms: 1.03x faster
sqlite_synth 2.78 us 2.72 us: 1.02x faster 2.77 us: 1.00x faster
sympy_expand 500 ms 482 ms: 1.04x faster 503 ms: 1.01x slower
sympy_integrate 23.2 ms 22.5 ms: 1.03x faster 23.1 ms: 1.01x faster
sympy_sum 154 ms 148 ms: 1.04x faster 154 ms: 1.00x faster
sympy_str 296 ms 283 ms: 1.04x faster 295 ms: 1.00x faster
telco 8.64 ms 8.36 ms: 1.03x faster 8.49 ms: 1.02x faster
tomli_loads 2.33 sec 2.01 sec: 1.16x faster 2.13 sec: 1.09x faster
typing_runtime_protocols 173 us 168 us: 1.03x faster 170 us: 1.02x faster
unpack_sequence 64.1 ns 54.5 ns: 1.18x faster 63.4 ns: 1.01x faster
unpickle 13.8 us 13.7 us: 1.01x faster 14.0 us: 1.01x slower
unpickle_list 4.59 us 4.54 us: 1.01x faster 4.68 us: 1.02x slower
unpickle_pure_python 235 us 204 us: 1.15x faster 210 us: 1.12x faster
xml_etree_parse 164 ms 162 ms: 1.01x faster 162 ms: 1.01x faster
xml_etree_iterparse 109 ms 104 ms: 1.05x faster 104 ms: 1.05x faster
xml_etree_generate 89.7 ms 81.5 ms: 1.10x faster 84.7 ms: 1.06x faster
xml_etree_process 64.1 ms 56.8 ms: 1.13x faster 60.2 ms: 1.06x faster
Geometric mean (ref) 1.07x faster 1.04x faster

Benchmark hidden because not significant (2): asyncio_tcp, asyncio_websockets

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment