Skip to content

Instantly share code, notes, and snippets.

@ljfranklin

ljfranklin/mysql_docker_sadness.md

Last active August 2, 2017 22:57
Show Gist options
  • Save ljfranklin/4a28740fbdfd86b896529f70be254768 to your computer and use it in GitHub Desktop.
Save ljfranklin/4a28740fbdfd86b896529f70be254768 to your computer and use it in GitHub Desktop.
Download ZIP
Repro steps: MySQL service won't start in docker unless I touch all its files...
Raw
mysql_docker_sadness.md

Docker version affected:

  • ERROR: Docker version 17.06.0-ce on OSX host
  • ERROR: Concourse CI container (Linux based executing with runC)
  • OK: Docker version 17.06.0-ce on Ubuntu 16.04 host

Dockerfile:

FROM debian:jessie

RUN echo 'mysql-server mysql-server/root_password password password' | debconf-set-selections \
    && echo 'mysql-server mysql-server/root_password_again password password' | debconf-set-selections \
    && apt-get update \
    && apt-get -y install mysql-server

Steps to reproduce:

docker build -t mysql-fails .
docker run -it mysql-fails /bin/bash -c 'service mysql start'
[FAIL] Starting MySQL database server: mysqld . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . failed!

But if you touch some files...

docker run -it mysql-fails /bin/bash -c 'find /var/lib/mysql/mysql -exec touch -c -a {} + && service mysql start'
[ ok ] Starting MySQL database server: mysqld ..
[info] Checking for tables which need an upgrade, are corrupt or were
not closed cleanly..

By changing the access time on all the files under /var/lib/mysql/mysql/, the service magically starts?

MySQL error log:

170802 20:49:58 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
170802 20:49:58 [Note] Plugin 'FEDERATED' is disabled.
170802 20:49:58 InnoDB: The InnoDB memory heap is disabled
170802 20:49:58 InnoDB: Mutexes and rw_locks use GCC atomic builtins
170802 20:49:58 InnoDB: Compressed tables use zlib 1.2.8
170802 20:49:58 InnoDB: Using Linux native AIO
170802 20:49:58 InnoDB: Initializing buffer pool, size = 128.0M
170802 20:49:58 InnoDB: Completed initialization of buffer pool
InnoDB: The first specified data file ./ibdata1 did not exist:
InnoDB: a new database to be created!
170802 20:49:58  InnoDB: Setting file ./ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
170802 20:49:58  InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 5 MB
InnoDB: Database physically writes the file full: wait...
170802 20:49:58  InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: 127 rollback segment(s) active.
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
170802 20:49:58  InnoDB: Waiting for the background threads to start
170802 20:49:59 InnoDB: 5.5.57 started; log sequence number 0
170802 20:49:59  InnoDB: Starting shutdown...
170802 20:50:00  InnoDB: Shutdown completed; log sequence number 1595675
170802 20:50:00 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
170802 20:50:00 [Note] Plugin 'FEDERATED' is disabled.
170802 20:50:00 InnoDB: The InnoDB memory heap is disabled
170802 20:50:00 InnoDB: Mutexes and rw_locks use GCC atomic builtins
170802 20:50:00 InnoDB: Compressed tables use zlib 1.2.8
170802 20:50:00 InnoDB: Using Linux native AIO
170802 20:50:00 InnoDB: Initializing buffer pool, size = 128.0M
170802 20:50:00 InnoDB: Completed initialization of buffer pool
170802 20:50:00 InnoDB: highest supported file format is Barracuda.
170802 20:50:00  InnoDB: Waiting for the background threads to start
170802 20:50:01 InnoDB: 5.5.57 started; log sequence number 1595675
170802 20:50:01  InnoDB: Starting shutdown...
170802 20:50:02  InnoDB: Shutdown completed; log sequence number 1595675
170802 20:50:02 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
170802 20:50:02 [Note] Plugin 'FEDERATED' is disabled.
170802 20:50:02 InnoDB: The InnoDB memory heap is disabled
170802 20:50:02 InnoDB: Mutexes and rw_locks use GCC atomic builtins
170802 20:50:02 InnoDB: Compressed tables use zlib 1.2.8
170802 20:50:02 InnoDB: Using Linux native AIO
170802 20:50:02 InnoDB: Initializing buffer pool, size = 128.0M
170802 20:50:02 InnoDB: Completed initialization of buffer pool
170802 20:50:02 InnoDB: highest supported file format is Barracuda.
170802 20:50:02  InnoDB: Waiting for the background threads to start
170802 20:50:03 InnoDB: 5.5.57 started; log sequence number 1595675
ERROR: 1064  You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'ALTER TABLE user ADD column Show_view_priv enum('N','Y') CHARACTER SET utf8 NOT ' at line 1
170802 20:50:03 [ERROR] Aborting

170802 20:50:03  InnoDB: Starting shutdown...
170802 20:50:04  InnoDB: Shutdown completed; log sequence number 1595675
170802 20:50:04 [Note] /usr/sbin/mysqld: Shutdown complete

170802 20:50:04 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
170802 20:50:04 [Note] Plugin 'FEDERATED' is disabled.
170802 20:50:04 InnoDB: The InnoDB memory heap is disabled
170802 20:50:04 InnoDB: Mutexes and rw_locks use GCC atomic builtins
170802 20:50:04 InnoDB: Compressed tables use zlib 1.2.8
170802 20:50:04 InnoDB: Using Linux native AIO
170802 20:50:04 InnoDB: Initializing buffer pool, size = 128.0M
170802 20:50:04 InnoDB: Completed initialization of buffer pool
170802 20:50:04 InnoDB: highest supported file format is Barracuda.
170802 20:50:04  InnoDB: Waiting for the background threads to start
170802 20:50:05 InnoDB: 5.5.57 started; log sequence number 1595675
170802 20:50:05  InnoDB: Starting shutdown...
170802 20:50:06  InnoDB: Shutdown completed; log sequence number 1595675
170802 20:50:06 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
170802 20:50:06 [Note] Plugin 'FEDERATED' is disabled.
170802 20:50:06 InnoDB: The InnoDB memory heap is disabled
170802 20:50:06 InnoDB: Mutexes and rw_locks use GCC atomic builtins
170802 20:50:06 InnoDB: Compressed tables use zlib 1.2.8
170802 20:50:06 InnoDB: Using Linux native AIO
170802 20:50:06 InnoDB: Initializing buffer pool, size = 128.0M
170802 20:50:06 InnoDB: Completed initialization of buffer pool
170802 20:50:06 InnoDB: highest supported file format is Barracuda.
170802 20:50:06  InnoDB: Waiting for the background threads to start
170802 20:50:07 InnoDB: 5.5.57 started; log sequence number 1595675
ERROR: 1050  Table 'plugin' already exists
170802 20:50:07 [ERROR] Aborting

170802 20:50:07  InnoDB: Starting shutdown...
170802 20:50:08  InnoDB: Shutdown completed; log sequence number 1595675
170802 20:50:08 [Note] /usr/sbin/mysqld: Shutdown complete

170802 21:26:28 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
170802 21:26:28 [Note] Plugin 'FEDERATED' is disabled.
/usr/sbin/mysqld: Got error 140 from storage engine
170802 21:26:28 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
170802 21:26:28 InnoDB: The InnoDB memory heap is disabled
170802 21:26:28 InnoDB: Mutexes and rw_locks use GCC atomic builtins
170802 21:26:28 InnoDB: Compressed tables use zlib 1.2.8
170802 21:26:28 InnoDB: Using Linux native AIO
170802 21:26:28 InnoDB: Initializing buffer pool, size = 128.0M
170802 21:26:28 InnoDB: Completed initialization of buffer pool
170802 21:26:28 InnoDB: highest supported file format is Barracuda.
170802 21:26:28  InnoDB: Waiting for the background threads to start
170802 21:26:29 InnoDB: 5.5.57 started; log sequence number 1595675
170802 21:26:29 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
170802 21:26:29 [Note]   - '127.0.0.1' resolves to '127.0.0.1';
170802 21:26:29 [Note] Server socket created on IP: '127.0.0.1'.
170802 21:26:29 [ERROR] Fatal error: Can't open and lock privilege tables: Got error 140 from storage engine

MySQL happy log:

170802 20:49:58 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
170802 20:49:58 [Note] Plugin 'FEDERATED' is disabled.
170802 20:49:58 InnoDB: The InnoDB memory heap is disabled
170802 20:49:58 InnoDB: Mutexes and rw_locks use GCC atomic builtins
170802 20:49:58 InnoDB: Compressed tables use zlib 1.2.8
170802 20:49:58 InnoDB: Using Linux native AIO
170802 20:49:58 InnoDB: Initializing buffer pool, size = 128.0M
170802 20:49:58 InnoDB: Completed initialization of buffer pool
InnoDB: The first specified data file ./ibdata1 did not exist:
InnoDB: a new database to be created!
170802 20:49:58  InnoDB: Setting file ./ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
170802 20:49:58  InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 5 MB
InnoDB: Database physically writes the file full: wait...
170802 20:49:58  InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: 127 rollback segment(s) active.
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
170802 20:49:58  InnoDB: Waiting for the background threads to start
170802 20:49:59 InnoDB: 5.5.57 started; log sequence number 0
170802 20:49:59  InnoDB: Starting shutdown...
170802 20:50:00  InnoDB: Shutdown completed; log sequence number 1595675
170802 20:50:00 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
170802 20:50:00 [Note] Plugin 'FEDERATED' is disabled.
170802 20:50:00 InnoDB: The InnoDB memory heap is disabled
170802 20:50:00 InnoDB: Mutexes and rw_locks use GCC atomic builtins
170802 20:50:00 InnoDB: Compressed tables use zlib 1.2.8
170802 20:50:00 InnoDB: Using Linux native AIO
170802 20:50:00 InnoDB: Initializing buffer pool, size = 128.0M
170802 20:50:00 InnoDB: Completed initialization of buffer pool
170802 20:50:00 InnoDB: highest supported file format is Barracuda.
170802 20:50:00  InnoDB: Waiting for the background threads to start
170802 20:50:01 InnoDB: 5.5.57 started; log sequence number 1595675
170802 20:50:01  InnoDB: Starting shutdown...
170802 20:50:02  InnoDB: Shutdown completed; log sequence number 1595675
170802 20:50:02 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
170802 20:50:02 [Note] Plugin 'FEDERATED' is disabled.
170802 20:50:02 InnoDB: The InnoDB memory heap is disabled
170802 20:50:02 InnoDB: Mutexes and rw_locks use GCC atomic builtins
170802 20:50:02 InnoDB: Compressed tables use zlib 1.2.8
170802 20:50:02 InnoDB: Using Linux native AIO
170802 20:50:02 InnoDB: Initializing buffer pool, size = 128.0M
170802 20:50:02 InnoDB: Completed initialization of buffer pool
170802 20:50:02 InnoDB: highest supported file format is Barracuda.
170802 20:50:02  InnoDB: Waiting for the background threads to start
170802 20:50:03 InnoDB: 5.5.57 started; log sequence number 1595675
ERROR: 1064  You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'ALTER TABLE user ADD column Show_view_priv enum('N','Y') CHARACTER SET utf8 NOT ' at line 1
170802 20:50:03 [ERROR] Aborting

170802 20:50:03  InnoDB: Starting shutdown...
170802 20:50:04  InnoDB: Shutdown completed; log sequence number 1595675
170802 20:50:04 [Note] /usr/sbin/mysqld: Shutdown complete

170802 20:50:04 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
170802 20:50:04 [Note] Plugin 'FEDERATED' is disabled.
170802 20:50:04 InnoDB: The InnoDB memory heap is disabled
170802 20:50:04 InnoDB: Mutexes and rw_locks use GCC atomic builtins
170802 20:50:04 InnoDB: Compressed tables use zlib 1.2.8
170802 20:50:04 InnoDB: Using Linux native AIO
170802 20:50:04 InnoDB: Initializing buffer pool, size = 128.0M
170802 20:50:04 InnoDB: Completed initialization of buffer pool
170802 20:50:04 InnoDB: highest supported file format is Barracuda.
170802 20:50:04  InnoDB: Waiting for the background threads to start
170802 20:50:05 InnoDB: 5.5.57 started; log sequence number 1595675
170802 20:50:05  InnoDB: Starting shutdown...
170802 20:50:06  InnoDB: Shutdown completed; log sequence number 1595675
170802 20:50:06 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
170802 20:50:06 [Note] Plugin 'FEDERATED' is disabled.
170802 20:50:06 InnoDB: The InnoDB memory heap is disabled
170802 20:50:06 InnoDB: Mutexes and rw_locks use GCC atomic builtins
170802 20:50:06 InnoDB: Compressed tables use zlib 1.2.8
170802 20:50:06 InnoDB: Using Linux native AIO
170802 20:50:06 InnoDB: Initializing buffer pool, size = 128.0M
170802 20:50:06 InnoDB: Completed initialization of buffer pool
170802 20:50:06 InnoDB: highest supported file format is Barracuda.
170802 20:50:06  InnoDB: Waiting for the background threads to start
170802 20:50:07 InnoDB: 5.5.57 started; log sequence number 1595675
ERROR: 1050  Table 'plugin' already exists
170802 20:50:07 [ERROR] Aborting

170802 20:50:07  InnoDB: Starting shutdown...
170802 20:50:08  InnoDB: Shutdown completed; log sequence number 1595675
170802 20:50:08 [Note] /usr/sbin/mysqld: Shutdown complete

170802 21:28:13 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
170802 21:28:13 [Note] Plugin 'FEDERATED' is disabled.
170802 21:28:13 InnoDB: The InnoDB memory heap is disabled
170802 21:28:13 InnoDB: Mutexes and rw_locks use GCC atomic builtins
170802 21:28:13 InnoDB: Compressed tables use zlib 1.2.8
170802 21:28:13 InnoDB: Using Linux native AIO
170802 21:28:13 InnoDB: Initializing buffer pool, size = 128.0M
170802 21:28:13 InnoDB: Completed initialization of buffer pool
170802 21:28:13 InnoDB: highest supported file format is Barracuda.
170802 21:28:13  InnoDB: Waiting for the background threads to start
170802 21:28:14 InnoDB: 5.5.57 started; log sequence number 1595675
170802 21:28:14 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
170802 21:28:14 [Note]   - '127.0.0.1' resolves to '127.0.0.1';
170802 21:28:14 [Note] Server socket created on IP: '127.0.0.1'.
170802 21:28:14 [Note] Event Scheduler: Loaded 0 events
170802 21:28:14 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.5.57-0+deb8u1'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  (Debian)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment