Docker image build speedup tricks

README.md

Docker image build speedup tricks

I experimented with a few tactics on how to speed up my docker builds. This guide is mainly for debian-based distros, but it's possible to tweak this approach to support other distros as well.

Use squid-deb-proxy locally

docker build -t my/squid-deb-proxy -f Dockerfile.squid-deb-proxy .
docker volume create squid-deb-proxy
docker network create squid
docker run --rm -d --network squid \
  --volume squid-deb-proxy:/data \
  --name squid-deb-proxy my/squid-deb-proxy

Once you did that, you can have the squid-deb-proxy cache deb files across docker builds.

Example:

docker build \
  --network squid \
  --build-arg http_proxy=http://squid-deb-proxy:8000 \
  -t my/example .

On first build packages will be downloaded from the internet, and cached in squid. On later builds only squid will be involved unless the package is outdated and a new one exists or the cache gets expired.

For me the squid-deb-proxy approach brought the biggest speedup to my docker builds.

eatmydata

This approach makes filesystem writes unsafe but improves their speed. In general when your data is important (eg. non-reproducible) you shouldn't use this. For builds however, this is a really nice tool in the toolbox to improve performance.

This is how you would improve performance of your docker build with eatmydata:

docker build  --network squid \
  --build-arg http_proxy=http://squid-deb-proxy:8000 \
  -t my/eatmydata-test \
  -f Dockerfile.eatmydata .

With ubuntu:focal I got about 20% of speed improvements by introducing eatmydata next to the squid solution.

One thing to note here is that ubuntu docker images already force unsafe IO for DPKG so that it doesn't do fsync. That setting can be found in /etc/dpkg/dpkg.cfg.d/docker-apt-speedup

It is still curious though that I could still achieve performance improvements with eatmydata, showing that there are still safe-io bottlenecks in apt package installs.

Dockerfile.eat-my-data

FROM ubuntu:focal

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update \
	&& apt-get install -y eatmydata \
	&& eatmydata apt-get install -y \
    openjdk-11-jdk awscli azure-cli # just some big packages to notice the speedup \
	&& rm -rf /var/lib/apt/lists/*

Dockerfile.squid-deb-proxy

FROM ubuntu:focal

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update \
	&& apt-get install -y squid-deb-proxy \
	&& rm -rf /var/lib/apt/lists/*

# Point cache directory to /data
# Redirect logs to stdout for the container
RUN ln -sf /data /var/cache/squid-deb-proxy \
  && ln -sf /dev/stdout /var/log/squid-deb-proxy/access.log \
  && ln -sf /dev/stdout /var/log/squid-deb-proxy/store.log \
  && ln -sf /dev/stdout /var/log/squid-deb-proxy/cache.log

VOLUME ["/data"]
EXPOSE 8000

ADD entrypoint-squid-deb-proxy.sh /entrypoint.sh
CMD ["/entrypoint.sh"]

entrypoint-squid-deb-proxy.sh

#!/bin/bash
chown -R proxy.proxy /data
chown proxy.proxy /dev/stdout

source /usr/share/squid-deb-proxy/init-common.sh
pre_start
post_start
exec /usr/sbin/squid -N -f /etc/squid-deb-proxy/squid-deb-proxy.conf

An easier way to ensure that eatmydata is loaded is to use LD_PRELOAD.

eg add following to Dockerfile:

ENV LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libeatmydata.so

Note: ENV LD_PRELOAD=libeatmydata.so is also likely to work, and has the bonus works regardless of architecture used.