Testing DDEV performance

tests.md

MacBook Pro (14-inch, 2021), macOS 12.3, M1Pro, 16GB, Docker Desktop 4.6.0

Started with D9 Quickstart

Almost all tests took longer on first run. I ran the tests 4 times and dropped the first.

ddev

Test script: time ( ddev drush si demo_umami -y && ddev drush cr && ddev drush uli )

virtualization.framework (new), mutagen off, virtiofs off

0.43s user 0.46s system 1% cpu 1:14.90 total
0.41s user 0.43s system 1% cpu 1:03.97 total
0.41s user 0.43s system 1% cpu 1:05.46 total

hypervisor.framework (old), mutagen off, virtiofs off

0.38s user 0.43s system 0% cpu 1:48.78 total
0.39s user 0.45s system 0% cpu 1:47.44 total
0.39s user 0.45s system 0% cpu 1:48.79 total

virtualization.framework, mutagen ON, virtiofs OFF

0.36s user 0.32s system 3% cpu 21.925 total
0.36s user 0.35s system 3% cpu 21.634 total
0.37s user 0.35s system 3% cpu 21.091 total

virtualization.framework, mutagen off, virtiofs ON (slightly modified script with 'exec' to get it to work)

time ( ddev exec drush si demo_umami -y && ddev exec drush cr && ddev exec drush )

0.34s user 0.37s system 2% cpu 28.071 total
0.34s user 0.38s system 2% cpu 28.186 total
0.34s user 0.35s system 2% cpu 28.554 total

docksal

Test command: time ( fin drush si demo_umami -y && fin drush cr && fin drush uli )

virtualization.framework, mutagen off, virtiofs off

0.38s user 0.58s system 0% cpu 2:04.54 total
0.38s user 0.54s system 1% cpu 1:28.88 total
0.39s user 0.63s system 1% cpu 1:26.75 total

virtualization.framework, mutagen off, virtiofs ON

0.39s user 0.56s system 1% cpu 1:10.06 total
0.37s user 0.52s system 1% cpu 1:07.94 total
0.37s user 0.53s system 1% cpu 1:09.98 total

virtualization.framework, mutagen ON, virtiofs off using nicoschi/mutagen-for-docksal

0.39s user 0.56s system 1% cpu 1:01.46 total
0.37s user 0.55s system 1% cpu 1:03.89 total
0.38s user 0.55s system 1% cpu 1:03.26 total

While this is interesting, the drush si doesn't really benchmark the file system. A better performance benchmark would be to do rm -rf vendor && composer install within the docker container. The problem with nfs in the past was with handling hundreds of small files, like with vendor or node_modules.

Thanks @mike-potter, that's a great suggestion. I'll try to add it to the mix when I have a moment. I also got a request to test with https://github.com/abiosoft/colima.

ddev composer install is crazy fast with mutagen, and even faster if the packages are already cached in the global (all projects) cache. The reason is that it all happens completely inside the container on a native linux filesystem... and only after that is synced up via mutagen to the host.

I would love to see an easy scripted way to run a d9 web install. I've always just done it using a stopwatch and counting the module install section.

I tested on MacBookPro (2019 i7) + Drupal 9.3.8
ddev 1.19.0 + Colima + Mutagen

time ( drush si demo_umami -y && drush cr && drush uli )
45.52 secs

time ( rm -rf vendor && composer install )
Times ranged between 3.5 - 5.5 seconds

Using Outline Design System - https://github.com/phase2/outline
time ( rm -rf node_modules && yarn install )

1m19.158s
1m08.448s
1m14.587s

@rfay has a script to more consistently measure things: https://github.com/drud/ddev-puppeteer

I also did a blog about the comparison, getting old now: https://ddev.com/ddev-local/docker-desktop-and-colima-benchmarking-on-macos/