Yarn cache compatible Dockerfile

README.md

Yarn Dockerfile

An yarn cache compatible Dockerfile, for building node.js images faster.

Usage

Clone this gist into your project root, and add it to your source control. Change the image service-name:latest tag to your project name in the Dockerfile and build.sh files. Then, always build your image using the build.sh script.

$ chmod +x ./build.sh
$ ./build.sh

build.sh

#!/bin/bash

if [ ! -f .yarn-cache.tgz ]; then
  echo "+ build: Init empty .yarn-cache.tgz"
  tar cvzf .yarn-cache.tgz --files-from /dev/null
fi

docker build -t service-name:latest .

docker run \
  --rm \
  --entrypoint cat \
  service-name:latest \
  /tmp/yarn.lock > /tmp/yarn.lock

if ! diff -q yarn.lock /tmp/yarn.lock > /dev/null  2>&1; then
  echo "+ build: Saving Yarn cache"

  docker run \
    --rm \
    --entrypoint tar \
    service-name:latest \
    czf - /root/.yarn-cache/ > .yarn-cache.tgz

  echo "+ build: Saving yarn.lock"

  cp /tmp/yarn.lock yarn.lock
fi

Dockerfile

FROM alpine

RUN apk add --update --no-cache nodejs
RUN npm i -g yarn

ADD package.json yarn.lock /tmp/
ADD .yarn-cache.tgz /

RUN cd /tmp && yarn
RUN mkdir -p /service && cd /service && ln -s /tmp/node_modules

COPY . /service
WORKDIR /service

ENV FORCE_COLOR=1

ENTRYPOINT ["npm"]
CMD ["start"]

Here is a setup I configured without the need for a local build.sh script to build your image. This will not build node dependencies in to the image, but instead preserve them on a mounted volume:

docker-compose.yml

version: '3.7'

services:
  web:
    build: .
    volumes:
    - yarn_cache:/yarn
    - node_modules_cache:/app/node_modules

volumes:
  yarn_cache:
  node_modules_cache:

Dockerfile

FROM node:8.15.0-alpine

ENV YARN_CACHE_FOLDER /yarn
WORKDIR /app/

# Add entrypoint script to handle yarn cache
RUN mkdir -p $YARN_CACHE_FOLDER
COPY ./modules/docker-entrypoint.sh /
RUN chmod +x /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]

ADD . .

CMD ["yarn", "start"]

docker-entrypoint.sh

#!/bin/sh

yarn config set cache-folder $YARN_CACHE_FOLDER
cd $CUSTOMIZATIONS_PATH
(yarn check --integrity && yarn check --verify-tree) || yarn install --frozen-lockfile
cd $DEVENV_PATH

# Call command issued to the docker service
exec "$@"

Great ! Thanks

@etgrieco, thanks for sharing. It works for me. One limit in the solution is that IDE can't use node modules(for type hints or other function) because the modules do not exist in the project folder.

My workaround is replacing this line: - node_modules_cache:/app/node_modules with - .:/app:cached, and node modules show in the project folder. Doing this works fine with IDE, but another problem comes - every time I run yarn add <package>, it seems to reinstall all packages without using Yarn cache.

Do you have any idea about this problem? Maybe I missed something or use this in the wrong way.

FYI, I don't use docker compose. I use docker CLI instead:

$ docker volume create yarn_cache
$ docker build -f Dockerfile.dev -t create-react-app-project:dev .
$ docker run --rm -v ${PWD}:/app:cached -v yarn_cache:/yarn_cache create-react-app-project:dev /bin/sh -c "yarn install"
$ docker run --rm -itd -v ${PWD}:/app:ro,cached -v yarn_cache:/yarn_cache -p 3000:3000 -p 35729:35729 create-react-app-project:dev
$ docker run --rm -v ${PWD}:/app:cached -v yarn_cache:/yarn_cache create-react-app-project:dev /bin/sh -c "yarn add|remove <package name>"

In case someone arrives here wondering why yarn install doesn't even cache locally, check out your COPY commands beforehand, and split them in two:

WORKDIR /app
COPY package.json yarn.lock .yarnrc ./
RUN yarn --pure-lockfile --non-interactive
COPY . ./

If you just do COPY . ./, it's very very likely something has changed in your project, and caching will never work.

In case someone arrives here wondering why yarn install doesn't even cache locally, check out your COPY commands beforehand, and split them in two:

WORKDIR /app
COPY package.json yarn.lock .yarnrc ./
RUN yarn --pure-lockfile --non-interactive
COPY . ./

If you just do COPY . ./, it's very very likely something has changed in your project, and caching will never work.

If you are also wondering why your container doesn't work after building on OSX, its because yarn has issues symlinking on OSX/BSD

@etgrieco The provided code works. On the initial container start up the dependencies need to be downloaded and then are stored into the volume. When the container it is stoped and started again, dependencies does not need to re-installed but yarn take some time to "Link the dependencies", and this happens on every container restart. Any idea about that ? It take a good amount of time in my end (~20 seconds to link the dependencies).

@etgrieco The provided code works. On the initial container start up the dependencies need to be downloaded and then are stored into the volume. When the container it is stoped and started again, dependencies does not need to re-installed but yarn take some time to "Link the dependencies", and this happens on every container restart. Any idea about that ? It take a good amount of time in my end (~20 seconds to link the dependencies).

I haven't used this docker-based workflow in a while, but hopefully others who ran into your issue can chime in!

@etgrieco Ah, okay. I am developing a Next.js app, and its dependency installation is quite annoying. Without such a "trick", when I change something in package.json, it takes over 150 + seconds just for the yarn install step when a rebuild the Docker Image. So, any trick/optimization technique would be greatly appreciated!