Skip to content

Instantly share code, notes, and snippets.

@arnaud-lb
Last active April 12, 2024 11:29
Show Gist options
  • Save arnaud-lb/5240aaa866ec6f662863dd5fe2bd10e9 to your computer and use it in GitHub Desktop.
Save arnaud-lb/5240aaa866ec6f662863dd5fe2bd10e9 to your computer and use it in GitHub Desktop.

glibc vs musl benchmark

PHP on glibc (on Debian bookworm) versus musl (on Alpine 3.19, musl 1.2.4). We use slightly modified php containers: https://github.com/arnaud-lb/docker-php/tree/musl-benchmark (non-stripped php, added apache-zts variant).

php-cgi

Running Symfony Demo (/blog/en/) with php-cgi -T10,50. 10 warmup requests, 50 measured requests, executed 5 times. We measure the mean of the total time taken by the 50 requests.

Opcache is enabled and Symfony is in production mode.

Non-ZTS (lower is better) :

- bookworm: php:8.3-bookworm-dbgsym php-cgi
- alpine: php:8.3-alpine3.19-dbgsym php-cgi
- alpine-mimalloc: php:8.3-alpine3.19-dbgsym env LD_PRELOAD=/usr/lib/libmimalloc.so.2 php-cgi
- alpine-jemalloc: php:8.3-alpine3.19-dbgsym env LD_PRELOAD=/usr/lib/libjemalloc.so.2 php-cgi

bookworm:         mean:  0.5663;  stddev:  0.0013;  diff:  -0.00%
alpine:           mean:  0.6509;  stddev:  0.0029;  diff:  +14.93%
alpine-mimalloc:  mean:  0.6337;  stddev:  0.0021;  diff:  +11.89%
alpine-jemalloc:  mean:  0.6335;  stddev:  0.0008;  diff:  +11.86%

ZTS (lower is better) :

- bookworm-zts: php:8.3-bookworm-zts-dbgsym php-cgi
- alpine-zts: php:8.3-alpine3.19-zts-dbgsym php-cgi

bookworm-zts:  mean:  0.6069;  stddev:  0.0034;  diff:  +0.00%
alpine-zts:    mean:  0.7075;  stddev:  0.0026;  diff:  +16.57%

Apache ZTS

Running Symfony Demo (/blog/en/) in mod_php ZTS, with ab -n 20000 -c 100 (lower is better) :

bookworm:           0.849000; +0.00%
alpine:             0.995000; +17.20%
alpine-mimalloc:    0.991000; +16.73%
alpine-jemalloc:    0.980000; +15.43%

Franken PHP

Running the frankenphp-demo, slightly modified (frankenphp, frankenphp-demo), with k6 run --insecure-skip-tls-verify -u 100 -d 30s script.js:

bookworm:                                                                               
Metric             Count      Rate  Average  Maximum  Median  Minimum  90th_Percentile  95th_Percentile
http_req_duration  -          -     70.57    1058.36  63.70   6.34     98.91            102.74

alpine:                                                                                 
Metric             Count      Rate  Average  Maximum  Median  Minimum  90th_Percentile  95th_Percentile
http_req_duration  -          -     98.52    529.25   90.75   4.90     124.51           127.77

alpine-jemalloc:                                                            
Metric             Count      Rate  Average  Maximum  Median  Minimum  90th_Percentile  95th_Percentile
http_req_duration  -          -     101.94   885.45   97.48   6.70     125.27           128.50

Notes

PHP under musl is ~15% slower. Using mimalloc or jemalloc under musl is slightly less slow.

perf shows no low hanging fruits. memcpy takes considerably more time in musl than in glic builds (10% in musl builds).

Reproducing

The benchmark can be reproduced by using the linked repositories.

Executed on Intel i9-12900KS, turbo disabled, randomize_va_space disabled:

sudo sh -c '
    for f in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do echo performance > "$f"; done
    echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo
    echo 0 > /proc/sys/kernel/randomize_va_space
'

docker-php containers are built with docker build -t php:8.3-... in subdirectories: https://github.com/arnaud-lb/docker-php/tree/musl-benchmark/8.3, e.g.:

cd docker-php/8.3/$os/cli && docker build -t php:8.3-$os-cli .

apache-zts containers were started with

docroot=/var/www/html # or /var/www/localhost/htdocs for alpine
docker run -v "$(pwd)/symfony-demo:$docroot" -d --name musl-bench-apache-zts php:8.3-bookworm-apache-zts-dbgsym

frankenphp containers are built with PHP_VERSION=8.3 docker buildx bake --load --set '*.platform=linux/amd64' default in https://github.com/arnaud-lb/frankenphp/tree/musl-benchmark, and started with

docker run \
    -e FRANKENPHP_CONFIG="worker ./public/index.php" \
    -v $PWD:/app \
    -p 80:80 -p 443:443/tcp -p 443:443/udp \
    --name FrankenPHP-demo \
    dunglas/frankenphp

LD_PRELOAD is used to enable mimalloc or jemalloc in specified benchmarks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment