Although @setup/node as a built-in cache option, it lacks an opportunity regarding cache persistence. Depending on usage, the action below might give you faster installs and potentially reduce carbon emissions (♻️🌳❤️).
Yarn 3+/4+ with nodeLinker: node-modules. (Not using yarn ? see the corresponding pnpm 7/8+ action gist)
.
└── .github
├── actions
│ └── yarn-nm-install/action.yml (composite action)
└── workflows
└── ci.yml (uses: ./.github/actions/yarn-nm-install)
Create a file in .github/actions/yarn-nm-install/action.yml and paste
########################################################################################
# "yarn install" composite action for yarn 3/4+ and "nodeLinker: node-modules" #
#--------------------------------------------------------------------------------------#
# Requirement: @setup/node should be run before #
# #
# Usage in workflows steps: #
# #
# - name: 📥 Monorepo install #
# uses: ./.github/actions/yarn-nm-install #
# with: #
# enable-corepack: false # (default = 'false') #
# cwd: ${{ github.workspace }}/apps/my-app # (default = '.') #
# cache-prefix: add cache key prefix # (default = 'default') #
# cache-node-modules: false # (default = 'false') #
# cache-install-state: false # (default = 'false') #
# #
# Reference: #
# - latest: https://gist.github.com/belgattitude/042f9caf10d029badbde6cf9d43e400a #
# #
# Versions: #
# - 1.2.1 - 25-05-2024 - fix a missing action/cache not upraded to v4 #
# - 1.2.0 - 01-05-2024 - action/cache upraded to v4 #
# - 1.1.0 - 22-07-2023 - Option to enable npm global cache folder. #
# - 1.0.4 - 15-07-2023 - Fix corepack was always enabled. #
# - 1.0.3 - 05-07-2023 - YARN_ENABLE_MIRROR to false (speed up cold start) #
# - 1.0.2 - 02-06-2023 - install-state default to false #
# - 1.0.1 - 29-05-2023 - cache-prefix doc #
# - 1.0.0 - 27-05-2023 - new input: cache-prefix #
########################################################################################
name: 'Monorepo install (yarn)'
description: 'Run yarn install with node_modules linker and cache enabled'
inputs:
cwd:
description: "Changes node's process.cwd() if the project is not located on the root. Default to process.cwd()"
required: false
default: '.'
cache-prefix:
description: 'Add a specific cache-prefix'
required: false
default: 'default'
cache-npm-cache:
description: 'Cache npm global cache folder often used by node-gyp, prebuild binaries (invalidated on lock/os/node-version)'
required: false
default: 'true'
cache-node-modules:
description: 'Cache node_modules, might speed up link step (invalidated lock/os/node-version/branch)'
required: false
default: 'false'
cache-install-state:
description: 'Cache yarn install state, might speed up resolution step when node-modules cache is activated (invalidated lock/os/node-version/branch)'
required: false
default: 'false'
enable-corepack:
description: 'Enable corepack'
required: false
default: 'true'
runs:
using: 'composite'
steps:
- name: ⚙️ Enable Corepack
if: inputs.enable-corepack == 'true'
shell: bash
working-directory: ${{ inputs.cwd }}
run: corepack enable
- name: ⚙️ Expose yarn config as "$GITHUB_OUTPUT"
id: yarn-config
shell: bash
working-directory: ${{ inputs.cwd }}
env:
YARN_ENABLE_GLOBAL_CACHE: 'false'
run: |
echo "CACHE_FOLDER=$(yarn config get cacheFolder)" >> $GITHUB_OUTPUT
echo "CURRENT_NODE_VERSION="node-$(node --version)"" >> $GITHUB_OUTPUT
echo "CURRENT_BRANCH=$(echo ${GITHUB_REF#refs/heads/} | sed -r 's,/,-,g')" >> $GITHUB_OUTPUT
echo "NPM_GLOBAL_CACHE_FOLDER=$(npm config get cache)" >> $GITHUB_OUTPUT
- name: ♻️ Restore yarn cache
uses: actions/cache@v4
id: yarn-download-cache
with:
path: ${{ steps.yarn-config.outputs.CACHE_FOLDER }}
key: yarn-download-cache-${{ inputs.cache-prefix }}-${{ hashFiles(format('{0}/yarn.lock', inputs.cwd), format('{0}/.yarnrc.yml', inputs.cwd)) }}
restore-keys: |
yarn-download-cache-${{ inputs.cache-prefix }}-
- name: ♻️ Restore node_modules
if: inputs.cache-node-modules == 'true'
id: yarn-nm-cache
uses: actions/cache@v4
with:
path: ${{ inputs.cwd }}/**/node_modules
key: yarn-nm-cache-${{ inputs.cache-prefix }}-${{ runner.os }}-${{ steps.yarn-config.outputs.CURRENT_NODE_VERSION }}-${{ steps.yarn-config.outputs.CURRENT_BRANCH }}-${{ hashFiles(format('{0}/yarn.lock', inputs.cwd), format('{0}/.yarnrc.yml', inputs.cwd)) }}
- name: ♻️ Restore global npm cache folder
if: inputs.cache-npm-cache == 'true'
id: npm-global-cache
uses: actions/cache@v4
with:
path: ${{ steps.yarn-config.outputs.NPM_GLOBAL_CACHE_FOLDER }}
key: npm-global-cache-${{ inputs.cache-prefix }}-${{ runner.os }}-${{ steps.yarn-config.outputs.CURRENT_NODE_VERSION }}-${{ hashFiles(format('{0}/yarn.lock', inputs.cwd), format('{0}/.yarnrc.yml', inputs.cwd)) }}
- name: ♻️ Restore yarn install state
if: inputs.cache-install-state == 'true' && inputs.cache-node-modules == 'true'
id: yarn-install-state-cache
uses: actions/cache@v4
with:
path: ${{ inputs.cwd }}/.yarn/ci-cache
key: yarn-install-state-cache-${{ inputs.cache-prefix }}-${{ runner.os }}-${{ steps.yarn-config.outputs.CURRENT_NODE_VERSION }}-${{ steps.yarn-config.outputs.CURRENT_BRANCH }}-${{ hashFiles(format('{0}/yarn.lock', inputs.cwd), format('{0}/.yarnrc.yml', inputs.cwd)) }}
- name: 📥 Install dependencies
shell: bash
working-directory: ${{ inputs.cwd }}
run: yarn install --immutable --inline-builds
env:
# Overrides/align yarnrc.yml options (v3, v4) for a CI context
YARN_ENABLE_GLOBAL_CACHE: 'false' # Use local cache folder to keep downloaded archives
YARN_ENABLE_MIRROR: 'false' # Prevent populating global cache for caches misses (local cache only)
YARN_NM_MODE: 'hardlinks-local' # Reduce node_modules size
YARN_INSTALL_STATE_PATH: '.yarn/ci-cache/install-state.gz' # Might speed up resolution step when node_modules present
# Other environment variables
HUSKY: '0' # By default do not run HUSKY installTo use it in the workflows
steps:
- uses: actions/checkout@v4
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
- name: 📥 Monorepo install
uses: ./.github/actions/yarn-nm-installBe sure that your .yarnrc.yml sets the nodeLinker: node-modules parameter:
nodeLinker: node-modules
#compressionLevel: 0 # Will give 10%-30% install speed up, but takes more space locally
# This line can be ommited if corepack is enabled (requires the packageManager field in package.json)
yarnPath: .yarn/releases/yarn-3.6.0.cjs # or 4.0.0-rc.45 (rc's seems quite stable imho)... - name: 📥 Monorepo install
uses: ./.github/actions/yarn-nm-install
with:
cwd: '.'
enable-corepack: false
cache-node-modules: true
cache-install-state: true | Parameter | Default | Comment |
|---|---|---|
| cwd | '.' | Run the install in a specific folder. |
| enable-corepack | false |
Activate corepack. |
| cache-prefix | default |
Allows to have multiple distinct install. |
| cache-node-modules | false |
Cache node-modules (only for exceptional use-cases) |
| cache-install-state | false |
Only useful is cache-node-modules is activated |
This action always caches the yarn config get cacheFolder to avoid fetching archives
from the npm repository. Depending on the number of your dependencies, this generally gives
a 2x overall improvement. It affects the yarn fetch step and protects from npm outages as well. An example
of speed gain could be:
| CI Scenario | Install | CI fetch cache | Total | Cache size | CI persist cache |
|---|---|---|---|---|---|
| yarn4 with cache | 34s | 3s | 37s | 201Mb | (±5s) |
| yarn4 without cache | 83s | N/A | 83s | N/A | N/A |
Link: https://github.com/belgattitude/compare-package-managers#-install-speed
In some circumstances, you might archieve better install time by caching the node_modules folder as well.
This will impact the yarn link step. The link step is where yarn runs postinstalls (node-gyp, download binaries...).
But be aware that the time saved by doing this creates an overhead for the cache fetch/compression/persist (@action/cache
has more to deal with). Use this option with care and in exceptional circumstances (ie you have multiple dependendent steps that run install). Also remember that the node_modules folder does not have the same portability/reliability than the recommended yarn cache folder. To get an idea of the performance gains, see this table:
| Command | Mean [s] | Min [s] | Max [s] | Relative |
|---|---|---|---|---|
yarnCache:on - installState:off - nm:off |
22.717 ± 0.225 | 22.510 | 22.957 | 5.95 ± 0.25 |
yarnCache:on - installState:on - nm:off |
21.350 ± 0.216 | 21.185 | 21.595 | 5.59 ± 0.24 |
yarnCache:on - installState:off - nm:on |
11.676 ± 0.041 | 11.647 | 11.722 | 3.06 ± 0.13 |
yarnCache:on - installState:on - nm:on |
3.820 ± 0.156 | 3.645 | 3.946 | 1.00 |
Link: https://github.com/belgattitude/compare-package-managers#compare-yarn-options
On install, when only few deps changed
Cost of action/cache compression
When a PR is closed or merged the best is to remove install cache rather than letting github reach the max (10GB) and prune.
Here's an example (feel free to adapt if you need to preserse some things, ie gh actions-cache list -R $REPO -B $BRANCH | cut -f 1 | grep yarn will only clear yarn related caches)
# https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries
name: Cleanup caches for closed branches
on:
pull_request:
types:
- closed
workflow_dispatch:
jobs:
cleanup:
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Cleanup
run: |
gh extension install actions/gh-actions-cache
REPO=${{ github.repository }}
BRANCH="refs/pull/${{ github.event.pull_request.number }}/merge"
echo "Fetching list of cache key"
cacheKeysForPR=$(gh actions-cache list -R $REPO -B $BRANCH | cut -f 1 )
## Setting this to not fail the workflow while deleting cache keys.
set +e
echo "Deleting caches..."
for cacheKey in $cacheKeysForPR
do
gh actions-cache delete $cacheKey -R $REPO -B $BRANCH --confirm
done
echo "Done"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Thanks for the great gist @belgattitude After upgrading to
[email protected]I'm getting the following error: