Last active
June 5, 2019 12:14
-
-
Save OliPelz/4cf1ea585fe0a8a3c24ba711d5bccb89 to your computer and use it in GitHub Desktop.
How to create bigger/huge Docker images (>100gb) in CentOS 7.x (using Docker 1.12.5)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The default Docker daemon on CentOS 7.x from the official package repository has two very annoying limitations when trying to create bigger images. | |
First is the default base image is 10gb space and second there is another barrier with the thin layer which is set to 100gb. | |
I needed a Docker image containing a big database which was about 120gb. | |
When I COPY'ed the huge database in the Dockerfile to build the image, I got a | |
No space left on device | |
error when I was hitting the 10gb barrier. This can be controlled by the Docker Base device size which is set by default to | |
10gb and can be controlled by the dockerd daemon parameter dm.basesize. | |
After using a systemd drop-in config file with the changed dockerd parameter it went past this barrier but again after hitting | |
a COPY amount of 100gb I got more weired errors: | |
ApplyLayer exit status 1 stdout: stderr: input/output error | |
and | |
docker: Error response from daemon: devmapper: Thin Pool has 0 free data blocks which is less than minimum required | |
so I learned that I had to expand the thin layer. Sadly, you have to wipe your Docker data dir first :( in order to do so. | |
So here I provide the files and stuff I did in the end to make things work! | |
Please note that in this example I have changed my default Docker data dir to /data/docker-root from /var/lib/docker because of free space issues on my server! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mkdir -p /etc/systemd/system/docker.service.d | |
echo "[Service] | |
ExecStart= | |
ExecStart=/usr/bin/dockerd --storage-opt dm.basesize=200G -g /data/docker-root" > /etc/systemd/system/docker.service.d/docker.conf | |
# reload | |
systemctl daemon-reload | |
systemctl restart docker |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# first wipe off your docker data dir - handle with care - i am not responsible for your data loss | |
# please note: this is not the default docker data dir ;) | |
systemctl stop docker | |
rm -rf /data/docker-root/* | |
mkdir /data/docker-root/devicemapper/devicemapper -p | |
# create a new thin layer of 200GB which is double the size of default Docker installation on CentOS 7 | |
dd if=/dev/zero of=/data/docker-root/devicemapper/devicemapper/data bs=1G count=0 seek=200 | |
# this initalizes everything | |
systemctl start docker | |
docker info | grep -i space | |
# output should now be 200gb instead of 100gb | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Now everything works and I was able to create huge images >100gb!!! | |
Here are some useful links I used to accomplish the above: | |
https://linuxconfig.org/how-to-move-docker-s-default-var-lib-docker-to-another-directory-on-ubuntu-debian-linux | |
http://www.projectatomic.io/blog/2016/03/daemon_option_basedevicesize/ | |
https://jpetazzo.github.io/2014/01/29/docker-device-mapper-resize/ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment