Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save OliPelz/4cf1ea585fe0a8a3c24ba711d5bccb89 to your computer and use it in GitHub Desktop.
Save OliPelz/4cf1ea585fe0a8a3c24ba711d5bccb89 to your computer and use it in GitHub Desktop.
How to create bigger/huge Docker images (>100gb) in CentOS 7.x (using Docker 1.12.5)
The default Docker daemon on CentOS 7.x from the official package repository has two very annoying limitations when trying to create bigger images.
First is the default base image is 10gb space and second there is another barrier with the thin layer which is set to 100gb.
I needed a Docker image containing a big database which was about 120gb.
When I COPY'ed the huge database in the Dockerfile to build the image, I got a
No space left on device
error when I was hitting the 10gb barrier. This can be controlled by the Docker Base device size which is set by default to
10gb and can be controlled by the dockerd daemon parameter dm.basesize.
After using a systemd drop-in config file with the changed dockerd parameter it went past this barrier but again after hitting
a COPY amount of 100gb I got more weired errors:
ApplyLayer exit status 1 stdout: stderr: input/output error
and
docker: Error response from daemon: devmapper: Thin Pool has 0 free data blocks which is less than minimum required
so I learned that I had to expand the thin layer. Sadly, you have to wipe your Docker data dir first :( in order to do so.
So here I provide the files and stuff I did in the end to make things work!
Please note that in this example I have changed my default Docker data dir to /data/docker-root from /var/lib/docker because of free space issues on my server!
mkdir -p /etc/systemd/system/docker.service.d
echo "[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --storage-opt dm.basesize=200G -g /data/docker-root" > /etc/systemd/system/docker.service.d/docker.conf
# reload
systemctl daemon-reload
systemctl restart docker
# first wipe off your docker data dir - handle with care - i am not responsible for your data loss
# please note: this is not the default docker data dir ;)
systemctl stop docker
rm -rf /data/docker-root/*
mkdir /data/docker-root/devicemapper/devicemapper -p
# create a new thin layer of 200GB which is double the size of default Docker installation on CentOS 7
dd if=/dev/zero of=/data/docker-root/devicemapper/devicemapper/data bs=1G count=0 seek=200
# this initalizes everything
systemctl start docker
docker info | grep -i space
# output should now be 200gb instead of 100gb
Now everything works and I was able to create huge images >100gb!!!
Here are some useful links I used to accomplish the above:
https://linuxconfig.org/how-to-move-docker-s-default-var-lib-docker-to-another-directory-on-ubuntu-debian-linux
http://www.projectatomic.io/blog/2016/03/daemon_option_basedevicesize/
https://jpetazzo.github.io/2014/01/29/docker-device-mapper-resize/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment