Skip to content

Instantly share code, notes, and snippets.

@mattip
Last active February 7, 2019 08:34
Show Gist options
  • Save mattip/05951f44e41224e36f00b62ebb4e2444 to your computer and use it in GitHub Desktop.
Save mattip/05951f44e41224e36f00b62ebb4e2444 to your computer and use it in GitHub Desktop.
PyPy 2 utf8 and cpython speed comparisons

I ran the python performance suite (not our benchmark suite) 3 times for pypy2 v7.0 and pypy2 unicode-utf8

for i in seq 1 3; do pyperformance run --python=$python -o $prefix_$i.json; done

python -mperf compare_to python2_1.json pypy2_*.json --min-speed 5 --table

Benchmark pypy2_1 pypy2utf8_1 pypy2utf8_2 pypy2utf8_3
2to3 896 ms 972 ms: 1.09x slower (+9%) 978 ms: 1.09x slower (+9%) 978 ms: 1.09x slower (+9%)
django_template 17.0 ms 9.98 ms: 1.70x faster (-41%) 10.0 ms: 1.70x faster (-41%) 10.0 ms: 1.70x faster (-41%)
dulwich_log 72.5 ms 65.8 ms: 1.10x faster (-9%) 64.7 ms: 1.12x faster (-11%) 64.5 ms: 1.12x faster (-11%)
python_startup 13.8 ms 13.1 ms: 1.05x faster (-5%) not significant not significant
regex_v8 10.4 ms 9.39 ms: 1.11x faster (-10%) 9.40 ms: 1.10x faster (-9%) 9.50 ms: 1.09x faster (-9%)
spambayes 27.8 ms 25.4 ms: 1.09x faster (-9%) not significant not significant
sqlalchemy_declarative 86.2 ms not significant not significant 82.0 ms: 1.05x faster (-5%)
sympy_integrate 16.4 ms 17.5 ms: 1.07x slower (+7%) not significant not significant
sympy_sum 51.0 ms not significant 44.7 ms: 1.14x faster (-12%) not significant
sympy_str 49.9 ms not significant not significant 52.7 ms: 1.06x slower (+6%)
xml_etree_parse 310 ms 289 ms: 1.08x faster (-7%) 289 ms: 1.08x faster (-7%) 285 ms: 1.09x faster (-8%)
xml_etree_iterparse 120 ms 110 ms: 1.09x faster (-8%) 109 ms: 1.10x faster (-9%) 109 ms: 1.10x faster (-9%)
xml_etree_process 33.1 ms 31.0 ms: 1.07x faster (-6%) 30.8 ms: 1.07x faster (-7%) 31.2 ms: 1.06x faster (-6%)

Not significant (46): chameleon; chaos; crypto_pyaes; deltablue; fannkuch; float; go; hexiom; hg_startup; html5lib; json_dumps; json_loads; logging_format; logging_silent; logging_simple; meteor_contest; nbody; nqueens; pathlib; pickle; pickle_dict; pickle_list; pickle_pure_python; pidigits; pyflate; python_startup_no_site; raytrace; regex_compile; regex_dna; regex_effbot; scimark_fft; scimark_lu; scimark_monte_carlo; scimark_sor; scimark_sparse_mat_mult; spectral_norm; sqlalchemy_imperative; sqlite_synth; sympy_expand; telco; tornado_http; unpack_sequence; unpickle; unpickle_list; unpickle_pure_python; xml_etree_generate Ignored benchmarks (3) of pypy2_1.json: genshi_text, genshi_xml, mako Ignored benchmarks (2) of pypy2utf8_2.json: genshi_text, genshi_xml

I ran the python performance suite (not our benchmark suite) 3 times for python2 (whatevr is in ubuntu 18.04) and pypy2 v7.0

for i in seq 1 3; do pyperformance run --python=$python -o $prefix_$i.json; done

python -mperf compare_to python2_1.json pypy2_*.json --min-speed 10 --table

Benchmark python2_1 pypy2_1 pypy2_2 pypy2_3
2to3 330 ms 896 ms: 2.72x slower (+172%) 895 ms: 2.72x slower (+172%) 896 ms: 2.72x slower (+172%)
chameleon 11.7 ms 3.85 ms: 3.03x faster (-67%) 3.89 ms: 3.00x faster (-67%) 3.91 ms: 2.98x faster (-66%)
chaos 98.8 ms 3.27 ms: 30.20x faster (-97%) 3.30 ms: 29.94x faster (-97%) 3.28 ms: 30.14x faster (-97%)
crypto_pyaes 76.2 ms 5.62 ms: 13.56x faster (-93%) 5.62 ms: 13.55x faster (-93%) 5.65 ms: 13.49x faster (-93%)
deltablue 7.54 ms 217 us: 34.81x faster (-97%) 218 us: 34.66x faster (-97%) 218 us: 34.61x faster (-97%)
django_template 147 ms 17.0 ms: 8.66x faster (-88%) 17.0 ms: 8.65x faster (-88%) 17.2 ms: 8.55x faster (-88%)
dulwich_log 87.7 ms 72.5 ms: 1.21x faster (-17%) 72.3 ms: 1.21x faster (-18%) 72.4 ms: 1.21x faster (-17%)
fannkuch 409 ms 90.2 ms: 4.53x faster (-78%) 90.1 ms: 4.54x faster (-78%) 90.1 ms: 4.54x faster (-78%)
float 115 ms 17.9 ms: 6.42x faster (-84%) 17.7 ms: 6.49x faster (-85%) 17.9 ms: 6.41x faster (-84%)
genshi_text 33.2 ms 25.3 ms: 1.31x faster (-24%) 23.7 ms: 1.40x faster (-29%) 25.7 ms: 1.29x faster (-23%)
genshi_xml 71.5 ms 37.6 ms: 1.90x faster (-47%) 37.0 ms: 1.93x faster (-48%) 39.3 ms: 1.82x faster (-45%)
go 208 ms 41.8 ms: 4.98x faster (-80%) 41.9 ms: 4.97x faster (-80%) 41.9 ms: 4.96x faster (-80%)
hexiom 9.79 ms 1.22 ms: 8.03x faster (-88%) 1.22 ms: 8.04x faster (-88%) 1.22 ms: 8.01x faster (-88%)
hg_startup 46.4 ms 113 ms: 2.44x slower (+144%) 113 ms: 2.44x slower (+144%) 113 ms: 2.44x slower (+144%)
html5lib 136 ms 23.6 ms: 5.75x faster (-83%) 24.2 ms: 5.61x faster (-82%) 23.7 ms: 5.72x faster (-83%)
json_dumps 11.1 ms 3.70 ms: 3.00x faster (-67%) 3.70 ms: 3.00x faster (-67%) 3.73 ms: 2.98x faster (-66%)
json_loads 49.4 us 12.2 us: 4.04x faster (-75%) 12.6 us: 3.92x faster (-74%) 12.2 us: 4.05x faster (-75%)
logging_format 12.7 us 2.09 us: 6.09x faster (-84%) 2.10 us: 6.04x faster (-83%) 2.07 us: 6.13x faster (-84%)
logging_silent 291 ns 0.10 ns: 2829.40x faster (-100%) 0.10 ns: 2836.66x faster (-100%) 0.10 ns: 2834.99x faster (-100%)
logging_simple 10.3 us 1.46 us: 7.06x faster (-86%) 1.45 us: 7.11x faster (-86%) 1.46 us: 7.06x faster (-86%)
mako 35.9 ms 10.6 ms: 3.38x faster (-70%) 10.4 ms: 3.45x faster (-71%) 10.6 ms: 3.38x faster (-70%)
meteor_contest 96.3 ms 42.8 ms: 2.25x faster (-56%) 43.1 ms: 2.23x faster (-55%) 43.3 ms: 2.22x faster (-55%)
nbody 125 ms 19.3 ms: 6.49x faster (-85%) 19.3 ms: 6.49x faster (-85%) 19.3 ms: 6.50x faster (-85%)
nqueens 90.9 ms 17.9 ms: 5.07x faster (-80%) 17.9 ms: 5.07x faster (-80%) 18.0 ms: 5.06x faster (-80%)
pathlib 20.3 ms 17.4 ms: 1.17x faster (-15%) 16.9 ms: 1.20x faster (-17%) 16.9 ms: 1.21x faster (-17%)
pickle 17.3 us 30.7 us: 1.78x slower (+78%) 30.8 us: 1.78x slower (+78%) 30.4 us: 1.76x slower (+76%)
pickle_dict 31.3 us 145 us: 4.62x slower (+362%) 145 us: 4.63x slower (+363%) 145 us: 4.65x slower (+365%)
pickle_list 6.00 us 20.6 us: 3.43x slower (+243%) 20.3 us: 3.39x slower (+239%) 20.6 us: 3.44x slower (+244%)
pickle_pure_python 368 us 30.5 us: 12.08x faster (-92%) 30.7 us: 11.99x faster (-92%) 30.8 us: 11.95x faster (-92%)
pidigits 181 ms 136 ms: 1.33x faster (-25%) 136 ms: 1.34x faster (-25%) 136 ms: 1.33x faster (-25%)
pyflate 588 ms 146 ms: 4.02x faster (-75%) 148 ms: 3.98x faster (-75%) 145 ms: 4.06x faster (-75%)
python_startup 5.12 ms 13.8 ms: 2.70x slower (+170%) 14.1 ms: 2.75x slower (+175%) 13.9 ms: 2.71x slower (+171%)
python_startup_no_site 3.89 ms 12.9 ms: 3.32x slower (+232%) 13.0 ms: 3.35x slower (+235%) 12.9 ms: 3.32x slower (+232%)
raytrace 523 ms 7.38 ms: 70.84x faster (-99%) 7.40 ms: 70.66x faster (-99%) 7.40 ms: 70.67x faster (-99%)
regex_compile 182 ms 29.1 ms: 6.24x faster (-84%) 29.0 ms: 6.27x faster (-84%) 28.9 ms: 6.28x faster (-84%)
regex_dna 149 ms 68.2 ms: 2.18x faster (-54%) 68.4 ms: 2.18x faster (-54%) 68.6 ms: 2.17x faster (-54%)
regex_effbot 2.83 ms 1.51 ms: 1.88x faster (-47%) 1.51 ms: 1.87x faster (-47%) 1.51 ms: 1.88x faster (-47%)
regex_v8 29.3 ms 10.4 ms: 2.82x faster (-65%) 9.58 ms: 3.06x faster (-67%) 10.4 ms: 2.82x faster (-65%)
scimark_fft 286 ms 6.25 ms: 45.80x faster (-98%) 6.23 ms: 45.92x faster (-98%) 6.22 ms: 46.03x faster (-98%)
scimark_lu 251 ms 1.08 ms: 232.78x faster (-100%) 1.07 ms: 234.17x faster (-100%) 1.07 ms: 234.12x faster (-100%)
scimark_monte_carlo 101 ms 2.39 ms: 42.40x faster (-98%) 2.39 ms: 42.45x faster (-98%) 2.39 ms: 42.29x faster (-98%)
scimark_sor 206 ms 529 us: 389.09x faster (-100%) 530 us: 388.47x faster (-100%) 530 us: 388.28x faster (-100%)
scimark_sparse_mat_mult 3.54 ms 80.8 us: 43.86x faster (-98%) 81.0 us: 43.76x faster (-98%) 81.0 us: 43.75x faster (-98%)
spambayes 75.4 ms 27.8 ms: 2.71x faster (-63%) 26.6 ms: 2.83x faster (-65%) 29.3 ms: 2.57x faster (-61%)
spectral_norm 112 ms 3.66 ms: 30.60x faster (-97%) 3.67 ms: 30.50x faster (-97%) 3.67 ms: 30.54x faster (-97%)
sqlite_synth 2.44 us 2.10 us: 1.16x faster (-14%) 2.10 us: 1.16x faster (-14%) 2.09 us: 1.17x faster (-14%)
sympy_expand 765 ms 103 ms: 7.40x faster (-86%) 103 ms: 7.45x faster (-87%) 104 ms: 7.38x faster (-86%)
sympy_integrate 26.8 ms 16.4 ms: 1.64x faster (-39%) 16.7 ms: 1.60x faster (-38%) 16.3 ms: 1.65x faster (-39%)
sympy_sum 186 ms 51.0 ms: 3.64x faster (-73%) 52.2 ms: 3.56x faster (-72%) 52.8 ms: 3.52x faster (-72%)
sympy_str 459 ms 49.9 ms: 9.19x faster (-89%) 50.4 ms: 9.12x faster (-89%) 49.2 ms: 9.34x faster (-89%)
telco 303 ms 8.43 ms: 35.99x faster (-97%) 8.50 ms: 35.69x faster (-97%) 8.44 ms: 35.94x faster (-97%)
tornado_http 203 ms 139 ms: 1.46x faster (-32%) 133 ms: 1.53x faster (-34%) 138 ms: 1.47x faster (-32%)
unpack_sequence 60.8 ns 109 ns: 1.79x slower (+79%) 110 ns: 1.82x slower (+82%) 110 ns: 1.81x slower (+81%)
unpickle 12.7 us 28.0 us: 2.21x slower (+121%) 28.0 us: 2.20x slower (+120%) 27.9 us: 2.20x slower (+120%)
unpickle_list 6.59 us 10.9 us: 1.66x slower (+66%) 11.0 us: 1.67x slower (+67%) 10.9 us: 1.65x slower (+65%)
unpickle_pure_python 178 us 28.3 us: 6.27x faster (-84%) 28.0 us: 6.35x faster (-84%) 27.9 us: 6.36x faster (-84%)
xml_etree_parse 100 ms 310 ms: 3.09x slower (+209%) 315 ms: 3.14x slower (+214%) 312 ms: 3.10x slower (+210%)
xml_etree_iterparse 86.7 ms 120 ms: 1.38x slower (+38%) 118 ms: 1.37x slower (+37%) 118 ms: 1.36x slower (+36%)
xml_etree_generate 142 ms 45.7 ms: 3.11x faster (-68%) 46.3 ms: 3.07x faster (-67%) 46.0 ms: 3.09x faster (-68%)
xml_etree_process 95.5 ms 33.1 ms: 2.89x faster (-65%) 33.1 ms: 2.89x faster (-65%) 33.2 ms: 2.88x faster (-65%)

Ignored benchmarks (3) of python2_1.json: richards, sqlalchemy_declarative, sqlalchemy_imperative Ignored benchmarks (2) of pypy2_1.json: sqlalchemy_declarative, sqlalchemy_imperative Ignored benchmarks (1) of pypy2_2.json: sqlalchemy_declarative Ignored benchmarks (1) of pypy2_3.json: sqlalchemy_imperative

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment