SDC environment with mediawiki-docker-dev

README.md

Basic Docker Dev for Structured Data on Commons with mediawiki-docker-dev

This gist adapts mediawiki-docker-dev for use in Structured Data on Commons project. If you spin up the app this way it lives at http://default.web.mw.localhost:8080.

Initial Setup

Clone mediawiki and mediawiki-docker-dev in folders next to eachother locally.

Replace the mediawiki-docker-dev/docker-compose.yml with the compose file attached - it includes a stock ElasticSearch image.

Edit the local.env in mediawiki-docker-dev as per instructions to indicate the relative location of the mediawiki codebase.

At this point, manual steps to configure extensions in the mediawiki folder are required.

Configuration

You will need a skin and extensions. Extensions are listed in order they should be set up, * indicates required for SDC*.

Extensions can be cloned down into the mediawiki/extensions folder and installed according to (linked) instructions. To clone via SSH use git clone ssh://[email protected]:29418/mediawiki/extensions/EXTENSION_NAME.

Skins:

• Vector*

Extensions:

• Scribunto
• TemplateStyles
• CommonsMetadata
• MultimediaViewer *
• UploadWizard
• UniversalLanguageSelector*
• Elastica *
• CirrusSearch *
• Wikibase* (follow instructions for setting up repo and client)
• WikibaseCirrusSearch*
• WikibaseMediaInfo*

This gist has an in-depth guide to installing more troublesome extensions. Make sure you follow the install instructions for each extension completely.

Managing php dependencies

You will have to install composer on your machine locally or else from docker.

To gather dependencies composer install (or docker run -it --rm --user $(id -u):$(id -g) -v ~/.composer:/tmp -v $(pwd):/app docker.io/composer install --no-dev for using docker) inside mediawiki/extensions/wikibase as well as at the root level of mediawiki to sure php dependencies set up properly. Reference the composer-lock.json if you get any strange errors. It may be that you need to run composer install commands twice - the first time to get the wikibase/composer-merge-plugin and most of the libraries then the second time to get those dependencies which are added in by the merge plugin. You may also have to usecomposer update to adjust anything out of sync.

Changes required in LocalSettings**

mediawiki-docker-dev imports LocalSettings.php from the mediawiki folder (and combines them with settings alreayd declared) Attached here are the likely changes in mediawiki/LocalSettings.php to have the SDC setup running properly.

Useful Scripts (run from the root of mediawiki-docker-dev )

• docker-compose exec "web" can be used for running any arbitrary command including maintenance scripts in your mediawiki docker environment. For example,docker-compose exec "web" php /var/www/mediawiki/maintenance/update.php --wiki default --quick
• ./bash allows you to shell into the docker "web" container
• ./create initializes spin up containers. If you want to see extended feedback, run docker-compose up after the containers been built to restart the container and see logs.

docker-compose.yml

version: '2'

services:

  db-master:
    image: ${DB}:latest
    environment:
      - MYSQL_ROOT_PASSWORD=toor
    volumes:
      - sql-data-master:/var/lib/mysql
      - ./config/mysql/master:/tmp/mwdd
    entrypoint: "/tmp/mwdd/entrypoint.sh"
    command: "mysqld"

  db-slave:
    image: ${DB}:latest
    environment:
      - MYSQL_ROOT_PASSWORD=toor
    volumes:
      - sql-data-slave:/var/lib/mysql
      - ./config/mysql/slave:/tmp/mwdd
    entrypoint: "/tmp/mwdd/entrypoint.sh"
    command: "mysqld"

  db-configure:
    image: ${DB}:latest
    environment:
      - "MYSQL_SLAVE_PASSWORD=toor"
      - "MYSQL_MASTER_PASSWORD=toor"
      - "MYSQL_ROOT_PASSWORD=toor"
      - "MYSQL_REPLICATION_USER=repl"
      - "MYSQL_REPLICATION_PASSWORD=repl"
    depends_on:
     - db-master
     - db-slave
    volumes:
      - ./scripts/mysql_connector.sh:/tmp/mysql_connector.sh
      - ./scripts/wait-for-it.sh:/tmp/wait-for-it.sh
    command: /bin/bash -x /tmp/mysql_connector.sh

  phpmyadmin:
    image: phpmyadmin/phpmyadmin
    environment:
     - PMA_USER:root
     - PMA_PASSWORD:toor
     - PMA_HOSTS=db-master,db-slave
     - PMA_ARBITRARY=1
     - VIRTUAL_HOST=phpmyadmin.mw.localhost
    volumes:
      - ./config/phpmyadmin/config.user.inc.php:/etc/phpmyadmin/config.user.inc.php

  elasticsearch:
    image: elasticsearch:5.6
    volumes: # Persist ES data in seperate "esdata" volume
      - elasticdata:/usr/share/elasticsearch/data
    environment:
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - discovery.type=single-node
    ports:
      - "9300:9300"
      - "9200:9200"
    networks:
      default:
        aliases:
          - elasticsearch.svc

  web:
    image: webdevops/${PHPORHHVM}-${WEBSERVER}${DOCKERPHPDEV}:${RUNTIMEVERSION}
    environment:
     - WEB_DOCUMENT_ROOT=/var/www
     # Used by various maintenance scripts to find MediaWiki.
     # Also required for /var/www/index.php - https://phabricator.wikimedia.org/T153882
     - MW_INSTALL_PATH=/var/www/mediawiki
     - VIRTUAL_HOST=*.web.mw.localhost
     - PHP_DEBUGGER=xdebug
     - XDEBUG_REMOTE_AUTOSTART=1
     - XDEBUG_REMOTE_HOST=${IDELOCALHOST}
     - XDEBUG_REMOTE_PORT=9000
     - XDEBUG_REMOTE_CONNECT_BACK=0
     - PHP_IDE_CONFIG=serverName=docker
     # elastic
     - MW_ELASTIC_HOST=elasticsearch.svc
     - MW_ELASTIC_PORT=9200
    depends_on:
     - db-configure
     - elasticsearch
    volumes:
     - "${DOCKER_MW_PATH}:/var/www/mediawiki"
     - ./config/mediawiki:/var/www/mediawiki/.docker:ro
     - ./config/hhvm/php.ini:/etc/hhvm/php.ini:ro
     - ./config/hhvm/server.ini:/etc/hhvm/server.ini:ro
     - ./scripts/wait-for-it.sh:/srv/wait-for-it.sh:ro
     - mw-images:/var/www/mediawiki/images/docker

  graphite-statsd:
    image: hopsoft/graphite-statsd
    environment:
     - VIRTUAL_HOST=graphite.mw.localhost
    #  allow runnning composer as root user for docker
    # - COMPOSER_ALLOW_SUPERUSER=1
    volumes:
     - graphite-data:/opt/graphite/storage

  nginx-proxy:
    image: jwilder/nginx-proxy
    environment:
     - VIRTUAL_HOST=proxy.mw.localhost
    ports:
      - "${DOCKER_MW_PORT}:80"
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro

volumes:
  elasticdata:
  sql-data-master:
  sql-data-slave:
  graphite-data:
  mw-images:

LocalSettings.php

<?php
require_once __DIR__ . '/.docker/LocalSettings.php';

wfLoadExtension( 'CommonsMetadata' );
wfLoadExtension( 'Elastica' );
wfLoadExtension( 'MultimediaViewer' );
wfLoadExtension( 'ParserFunctions' );
wfLoadExtension( 'Scribunto' );
wfLoadExtension( 'TemplateStyles' );
wfLoadExtension( 'UniversalLanguageSelector' );
wfLoadExtension( 'UploadWizard' );
wfLoadSkin( 'Vector' );

// Cirrus Search
require_once( "$IP/extensions/CirrusSearch/CirrusSearch.php" );
$wgCirrusSearchServers = [ "elasticsearch.svc" ];
$wgSearchType = "CirrusSearch";

// Upload Wizard
$wgUseInstantCommons = false;
$wgExtensionFunctions[] = function() {
	$GLOBALS['wgUploadNavigationUrl'] = SpecialPage::getTitleFor( 'UploadWizard' )->getLocalURL();
	return true;
};
$wgEnableUploads = true;
$wgUseImageMagick = true;
$wgUploadWizardConfig['wikibase']['enabled'] = true;
$wgApiFrameOptions = 'SAMEORIGIN';

// Wikibase and WikibaseMediaInfo
$wgEnableWikibaseRepo = true;
$wgEnableWikibaseClient = true;
require_once "$IP/extensions/Wikibase/repo/Wikibase.php";
require_once "$IP/extensions/Wikibase/repo/ExampleSettings.php";
require_once "$IP/extensions/Wikibase/client/WikibaseClient.php";
require_once "$IP/extensions/Wikibase/client/ExampleSettings.php";


// WikibaseMediaInfo
wfLoadExtension( 'WikibaseCirrusSearch' );
wfLoadExtension( 'WikibaseMediaInfo' );
$wgMediaInfoEnableFilePageDepicts = true;
$wgMediaInfoProperties = [
	'depicts' => 'P1',
];

$wgDepictsQualifierProperties = [
	'features' =>  'P2',
	'color' =>  'P3',
	'wears' =>  'P4',
	'part' =>  'P5',
	'inscription' =>  'P6',
	'symbolizes' =>  'P7',
	'position' =>  'P8',
	'quantity' =>  'P9',
];

// Other Extensions
$wgMediaViewerEnableByDefault = true;
$wgPFEnableStringFunctions = true;
$wgScribuntoDefaultEngine = 'luastandalone';