multi-wp-env.md

Status: Draft (The final form of this may move to a blog post or a full repo.)

This describes how to set up the repos for WordPress/wordpress-develop, WordPress/gutenberg, and WordPress/performance in one single place all using their own built-in environments (in Docker). Importantly, the latter two repos (for plugins) are configured to use the core files from the repos for both their development and tests environments.

Benefits:

1. WordPress core files are shared across all environments, including when running unit tests. So if you run PHPUnit tests for Gutenberg or Performance Lab, any core changes you've made will be reflected. This resolves a big painpoint with trying out core patches in plugin unit tests.
2. Plugins, mu-plugins, and themes and are shared across all environments, although you can add additional environment-specific extensions in your .wp-env.override.json. To add a new plugin or theme, you can just install it in WordPress via any of the environments. Or you can clone a new repo into the themes/plugins directories.
3. The same wp-content/debug.log is shared by all environment (by default).
4. You can run the tests for each project in their own respective built-in environments.

The repos for gutenberg and performance are cloned into the src/wp-content/plugins directory of the wordpress-develop repo.

Set up WordPress core (WordPress/wordpress-develop)

Clone repo and enter:

git clone https://github.com/WordPress/wordpress-develop.git
cd wordpress-develop
wordpress_develop_dir=$(pwd)

Add remote to my fork, including setting my fork as the origin remote for pushing:

git remote add westonruter https://github.com/westonruter/wordpress-develop.git
git remote set-url --push origin https://github.com/westonruter/wordpress-develop.git

Install dependencies and build:

npm install
composer install
npm run build:dev

Bonus: Add a PHPStan config (cf. #61175):

cat << PHPSTAN > phpstan.neon
includes:
        - phar://phpstan.phar/conf/bleedingEdge.neon
parameters:
        level: 9
        treatPhpDocTypesAsCertain: false
        paths:
                - .
        ignoreErrors:
                -
                        identifier: missingType.return
                        path: *
PHPSTAN

Set up plugins

From the wordpress-develop directory:

cd src/wp-content/plugins
plugins_dir=$(pwd)

Add Gutenberg (WordPress/gutenberg)

Clone repo and enter:

cd "$plugins_dir"
git clone https://github.com/WordPress/gutenberg.git
cd gutenberg

Add an .wp-env.override.json that sets unique port numbers for this environment while also reusing core files from wordpress-develop:

cat << 'WPENVOVERRIDE' > .wp-env.override.json
{
  "$schema": "./schemas/json/wp-env.json",
  "core": "../../../",
  "port": 8891,
  "testsPort": 8991,
  "env": {
    "development": {
      "phpmyadminPort": 9001
    },
    "tests": {
      "mappings": {
        "/wordpress-phpunit": "../../../../tests/phpunit"
      }
    }
  }
}
WPENVOVERRIDE

Install dependencies and build:

npm install
composer install
npm run build

Add Performance Lab (WordPress/performance)

Clone repo and enter:

cd "$plugins_dir"
git clone https://github.com/WordPress/performance.git
cd performance

As with Gutenberg, add an .wp-env.override.json that sets unique port numbers for this environment while also reusing core files from wordpress-develop. This also adds mappings so the misc plugins (extensions for Optimization Detective) are available in the wp-env environment for the performance repo:

cat << 'WPENVOVERRIDE' > .wp-env.override.json
{
  "$schema": "https://schemas.wp.org/trunk/wp-env.json",
  "core": "../../../",
  "port": 8890,
  "testsPort": 8990,
  "env": {
    "development": {
      "phpmyadminPort": 9000
    },
    "tests": {
      "mappings": {
        "/wordpress-phpunit": "../../../../tests/phpunit"
      }
    }
  }
}
WPENVOVERRIDE

Install dependencies and build:

npm install
composer install
npm run build

Add symlinks so the Performance Lab plugins are available in wordpress-develop:

cd "$plugins_dir"
for plugin in $(ls performance/plugins); do ln -s "performance/plugins/$plugin"; done

Add Misc Themes and Plugins

Clone these repos into src/wp-content/plugins as desired:

cd "$plugins_dir"
repos=(
  od-admin-ui
  od-content-visibility
  od-default-disabled
  od-dev-mode
  od-intrinsic-dimensions
  od-store-query-vars
  od-store-user-agent
)
for repo in "${repos[@]}"; do
  git clone https://github.com/westonruter/$repo.git
done

Start environments for the first time

First, start up the wordpress-develop environment:

cd "$wordpress_develop_dir"
npm run env:start
npm run env:install

This causes the wp-config.php file to be emitted at the wordpress-develop repo root.

Note that at this point there is no wp-config.php file in the src directory.

This is the tricky part, to get multiple wp-env instances to use the same wp-config.php while also preventing wordpress-develop from using it.

First, start up the Gutenberg environment:

cd "$plugins_dir/gutenberg"
npm run wp-env start

Expected output:

WordPress development site started at http://localhost:8891
WordPress test site started at http://localhost:8991
MySQL is listening on port 58145
MySQL for automated testing is listening on port 58249
phpMyAdmin started at http://localhost:9001

At this point you'll be able to successfully navigate to the Gutenberg development environment, but if you try going to the wordpress-development environment, you'll see an error from WordPress:

Warning: mysqli_real_connect(): (HY000/1045): Access denied for user 'example username'@'172.18.0.3' (using password: YES) in /var/www/src/wp-includes/class-wpdb.php on line 1988

This is because when starting wp-env for Gutenberg, it created a wp-config.php in the src directory of wordpress-develop, and WordPress defaults to using the wp-config.php at its root directory before looking one directory above, which is what wordpress-develop normally does. The wp-config.php generated by wp-env comes from wp-config-docker.php in docker-library/wordpress, and then wp-env uses WP-CLI to set the WP constants to be whatever is located in the .wp-env.json/.wp-env.override.json. For example:

wp-config.php

<?php
/**
 * The base configuration for WordPress
 *
 * The wp-config.php creation script uses this file during the installation.
 * You don't have to use the website, you can copy this file to "wp-config.php"
 * and fill in the values.
 *
 * This file contains the following configurations:
 *
 * * Database settings
 * * Secret keys
 * * Database table prefix
 * * ABSPATH
 *
 * This has been slightly modified (to read environment variables) for use in Docker.
 *
 * @link https://developer.wordpress.org/advanced-administration/wordpress/wp-config/
 *
 * @package WordPress
 */

// IMPORTANT: this file needs to stay in-sync with https://github.com/WordPress/WordPress/blob/master/wp-config-sample.php
// (it gets parsed by the upstream wizard in https://github.com/WordPress/WordPress/blob/f27cb65e1ef25d11b535695a660e7282b98eb742/wp-admin/setup-config.php#L356-L392)

// a helper function to lookup "env_FILE", "env", then fallback
if (!function_exists('getenv_docker')) {
	// https://github.com/docker-library/wordpress/issues/588 (WP-CLI will load this file 2x)
	function getenv_docker($env, $default) {
		if ($fileEnv = getenv($env . '_FILE')) {
			return rtrim(file_get_contents($fileEnv), "\r\n");
		}
		else if (($val = getenv($env)) !== false) {
			return $val;
		}
		else {
			return $default;
		}
	}
}

// ** Database settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
define( 'DB_NAME', getenv_docker('WORDPRESS_DB_NAME', 'wordpress') );

/** Database username */
define( 'DB_USER', getenv_docker('WORDPRESS_DB_USER', 'example username') );

/** Database password */
define( 'DB_PASSWORD', getenv_docker('WORDPRESS_DB_PASSWORD', 'example password') );

/**
 * Docker image fallback values above are sourced from the official WordPress installation wizard:
 * https://github.com/WordPress/WordPress/blob/1356f6537220ffdc32b9dad2a6cdbe2d010b7a88/wp-admin/setup-config.php#L224-L238
 * (However, using "example username" and "example password" in your database is strongly discouraged.  Please use strong, random credentials!)
 */

/** Database hostname */
define( 'DB_HOST', getenv_docker('WORDPRESS_DB_HOST', 'mysql') );

/** Database charset to use in creating database tables. */
define( 'DB_CHARSET', getenv_docker('WORDPRESS_DB_CHARSET', 'utf8') );

/** The database collate type. Don't change this if in doubt. */
define( 'DB_COLLATE', getenv_docker('WORDPRESS_DB_COLLATE', '') );

/**#@+
 * Authentication unique keys and salts.
 *
 * Change these to different unique phrases! You can generate these using
 * the {@link https://api.wordpress.org/secret-key/1.1/salt/ WordPress.org secret-key service}.
 *
 * You can change these at any point in time to invalidate all existing cookies.
 * This will force all users to have to log in again.
 *
 * @since 2.6.0
 */
define( 'AUTH_KEY',         getenv_docker('WORDPRESS_AUTH_KEY',         'de8724d4375446bc55cab44d0e30dd5fd889c3fd') );
define( 'SECURE_AUTH_KEY',  getenv_docker('WORDPRESS_SECURE_AUTH_KEY',  '26a684919680c6f85faa6e665a382fc653da2032') );
define( 'LOGGED_IN_KEY',    getenv_docker('WORDPRESS_LOGGED_IN_KEY',    'e039bb46676f860f8ba591250c3d1f0ff0afa405') );
define( 'NONCE_KEY',        getenv_docker('WORDPRESS_NONCE_KEY',        '8095fdb8ffd5974f45018fae0833f19fa67f21ce') );
define( 'AUTH_SALT',        getenv_docker('WORDPRESS_AUTH_SALT',        '25587d086964742511fff7f9cf9c75fcf5c71bff') );
define( 'SECURE_AUTH_SALT', getenv_docker('WORDPRESS_SECURE_AUTH_SALT', 'b8b12ced9f4c27288ef2db5d43e489dd9dea1a41') );
define( 'LOGGED_IN_SALT',   getenv_docker('WORDPRESS_LOGGED_IN_SALT',   '87ecae825ca29c34d93b9517597aa8c9b67cfb7b') );
define( 'NONCE_SALT',       getenv_docker('WORDPRESS_NONCE_SALT',       '3d450ea8b90a58a3ba04aff962cde33e326ceadd') );
// (See also https://wordpress.stackexchange.com/a/152905/199287)

/**#@-*/

/**
 * WordPress database table prefix.
 *
 * You can have multiple installations in one database if you give each
 * a unique prefix. Only numbers, letters, and underscores please!
 *
 * At the installation time, database tables are created with the specified prefix.
 * Changing this value after WordPress is installed will make your site think
 * it has not been installed.
 *
 * @link https://developer.wordpress.org/advanced-administration/wordpress/wp-config/#table-prefix
 */
$table_prefix = getenv_docker('WORDPRESS_TABLE_PREFIX', 'wp_');

/**
 * For developers: WordPress debugging mode.
 *
 * Change this to true to enable the display of notices during development.
 * It is strongly recommended that plugin and theme developers use WP_DEBUG
 * in their development environments.
 *
 * For information on other constants that can be used for debugging,
 * visit the documentation.
 *
 * @link https://developer.wordpress.org/advanced-administration/debug/debug-wordpress/
 */
define( 'FS_METHOD', 'direct' );
define( 'SCRIPT_DEBUG', true );
define( 'WP_ENVIRONMENT_TYPE', 'local' );
define( 'WP_PHP_BINARY', 'php' );
define( 'WP_TESTS_EMAIL', '[email protected]' );
define( 'WP_TESTS_TITLE', 'Test Blog' );
define( 'WP_TESTS_DOMAIN', 'localhost:8891' );
define( 'WP_SITEURL', 'http://localhost:8891' );
define( 'WP_HOME', 'http://localhost:8891' );
define( 'WP_DEBUG', true );

/* Add any custom values between this line and the "stop editing" line. */

// If we're behind a proxy server and using HTTPS, we need to alert WordPress of that fact
// see also https://wordpress.org/support/article/administration-over-ssl/#using-a-reverse-proxy
if (isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && strpos($_SERVER['HTTP_X_FORWARDED_PROTO'], 'https') !== false) {
	$_SERVER['HTTPS'] = 'on';
}
// (we include this by default because reverse proxying is extremely common in container environments)

if ($configExtra = getenv_docker('WORDPRESS_CONFIG_EXTRA', '')) {
	eval($configExtra);
}

/* That's all, stop editing! Happy publishing. */

/** Absolute path to the WordPress directory. */
if ( ! defined( 'ABSPATH' ) ) {
	define( 'ABSPATH', __DIR__ . '/' );
}

/** Sets up WordPress vars and included files. */
require_once ABSPATH . 'wp-settings.php';

Most notably:

define( 'WP_TESTS_DOMAIN', 'localhost:8891' );
define( 'WP_SITEURL', 'http://localhost:8891' );
define( 'WP_HOME', 'http://localhost:8891' );

If the Gutenberg environment is currently running (in which the .wp-env.override.json has a port set to 8891), and then I now try starting the Performance Lab environment (in which the .wp-env.override.json has a port set to 8890), then wp-env updates the constants used in the wp-config.php created by wp-env in Gutenberg to use the values for the Performance Lab environment:

define( 'WP_TESTS_DOMAIN', 'localhost:8890' );
define( 'WP_SITEURL', 'http://localhost:8890' );
define( 'WP_HOME', 'http://localhost:8890' );

The result is attempting to access the Gutenberg environment (http://localhost:8891) will cause WordPress to do a canonical redirection (with 301 Moved Permanently) to the Performance Lab environment (http://localhost:8890), meaning you can't access both environments at the same time (besides the fact that the wordpress-develop environment is also broken right now).

So here is the solution I've come up with to ensure that these three environments are all able to run concurrently.

1️⃣ First, open the src/wp-config.php file, and add the following PHP code to the top of after the file header comment:

// Load the wp-config.php for wordpress-develop when the current request is coming from that environment.
if ( basename( __DIR__ ) === 'src' && file_exists( __DIR__ . '/../wp-config.php' ) ) {
	require __DIR__ . '/../wp-config.php';
	return;
}

At this point, the wordpress-develop environment (http://localhost:8889/) works again! 🎉

2️⃣ Second, we need to dynamically load the correct config constants for the current wp-env environment (e.g. Gutenberg or Performance Lab). This can be done be adding the following code right after the the if statement added above:

( function () {
	$env_config = array(
		'FS_METHOD'           => 'direct',
		'SCRIPT_DEBUG'        => true,
		'WP_DEBUG'            => true,
		'WP_DEBUG_LOG'        => true,
		'WP_DEVELOPMENT_MODE' => 'all',
		'WP_ENVIRONMENT_TYPE' => 'local',
		'WP_HOME'             => 'http://localhost:' . $_SERVER['SERVER_PORT'],
		'WP_PHP_BINARY'       => 'php',
		'WP_SITEURL'          => 'http://localhost:' . $_SERVER['SERVER_PORT'],
		'WP_TESTS_DOMAIN'     => 'localhost:' . $_SERVER['SERVER_PORT'],
		'WP_TESTS_EMAIL'      => '[email protected]',
		'WP_TESTS_TITLE'      => 'Test Blog',
	);
	if ( isset( $_SERVER['SERVER_PORT'] ) ) {
		foreach ( glob( __DIR__ . '/wp-content/plugins/*' ) as $plugin_dir ) {
			$env_port            = null;
			$plugin_config       = array();
			$plugin_env_base     = array();
			$plugin_env_override = array();
			if ( file_exists( $plugin_dir . '/.wp-env.json' ) ) {
				$plugin_env_base = json_decode( file_get_contents( $plugin_dir . '/.wp-env.json' ), true );
			}
			if ( file_exists( $plugin_dir . '/.wp-env.override.json' ) ) {
				$plugin_env_override = json_decode( file_get_contents( $plugin_dir . '/.wp-env.override.json' ), true );
			}
			foreach ( array( $plugin_env_base, $plugin_env_override ) as $plugin_env ) {
				if ( isset( $plugin_env['env']['development']['port'] ) ) {
					$env_port = $plugin_env['env']['development']['port'];
				} elseif ( isset( $plugin_env['port'] ) ) {
					$env_port = $plugin_env['port'];
				}
				if ( isset( $plugin_env['config'] ) ) {
					$plugin_config = array_merge( $plugin_config, $plugin_env['config'] );
				}
				if ( isset( $plugin_env['env']['development']['config'] ) ) {
					$plugin_config = array_merge( $plugin_config, $plugin_env['env']['development']['config'] );
				}
			}

			if ( $env_port && $env_port === (int) $_SERVER['SERVER_PORT'] ) {
				$env_config = array_merge( $env_config, $plugin_config );
				break;
			}
		}
	}

	foreach ( $env_config as $key => $value ) {
		define( $key, $value );
	}
} )();

3️⃣ Finally, in order to prevent PHP from warning about the config constants already being defined, you can wrap the original constants in if ( false ) {:

+ if ( false ) {
  define( 'FS_METHOD', 'direct' );
  define( 'SCRIPT_DEBUG', true );
  define( 'WP_ENVIRONMENT_TYPE', 'local' );
  define( 'WP_PHP_BINARY', 'php' );
  define( 'WP_TESTS_EMAIL', '[email protected]' );
  define( 'WP_TESTS_TITLE', 'Test Blog' );
  define( 'WP_TESTS_DOMAIN', 'localhost:8890' );
  define( 'WP_SITEURL', 'http://localhost:8890' );
  define( 'WP_HOME', 'http://localhost:8890' );
  define( 'WP_DEVELOPMENT_MODE', 'plugin' );
  define( 'WP_DEBUG', true );
+ }

This is because wp-env will continue use WP-CLI to add/update/remove these constants. So this hack ensures that the wp config commands will succeed, by ultimately be ingored in favor of the logic added to dynamically read them from the .wp-env.json/.wp-env.override.json files.

Warning

Running npm run env:install will cause this file to overwritten. You should copy this to wp-config-multi-env.php and then copy it back to wp-config.php after doing a re-installation.

Starting all environments

cd "$wordpress_develop_dir"
npm run env:start

cd "$plugins_dir/gutenberg"
npm run wp-env start

cd "$plugins_dir/performance"
npm run wp-env start

Running tests in all environments

Warning

Running npm run test:e2e currently causes the wordpress-develop DB to be reset. Proposed fix in WordPress/gutenberg#70280

cd "$wordpress_develop_dir"
npm run test:php
npm run test:e2e
npm run test:visual

cd "$plugins_dir/gutenberg"
npm run test:unit:php:base
npm run test:unit
npm run test:e2e # TODO: Error: The plugin "gutenberg-test-plugin-disables-the-css-animations" isn't installed. Fix in https://github.com/WordPress/gutenberg/pull/70280

cd "$plugins_dir/performance"
npm run test-php

Configuring PhpStorm

You can continue to have separate PhpStorm projects for all three repos.

TODO: Pointing to core files when opening Gutenberg or Performance Lab projects.