Local caching for GitHub Actions self hosted runner using Squid Proxy

README.md

One of the biggest issues with using a self hosted GitHub runner is that actions that require downloading large amounts of data will bottleneck at the network. actions/cache does not support locally caching objects and artifacts stored on GitHub's servers will require a lot of bandwidth to fetch on every job. We can, however, set up a content proxy using Squid with SSL bumping to locally cache requests from jobs.

This is for Squid 7.6 on macOS with Homebrew.

Install

git clone https://gist.github.com/30c5c96d7575efd1d2a2db5e3def0815.git squid-cache
cd squid-cache
bash deploy.sh

deploy.sh builds the patched Squid through a local tap, generates the SSL-bump CA and DH parameters, writes the config and store ID helper, initialises the cache, sets up daily log rotation and starts the service. It is safe to re-run: certificates and a squid.conf you have edited are left alone unless you pass --force-certs / --force-config. See bash deploy.sh --help.

When it finishes it prints the lines to add to each runner's .env:

http_proxy=http://127.0.0.1:3128
https_proxy=http://127.0.0.1:3128
NODE_EXTRA_CA_CERTS=/opt/homebrew/etc/squid/squid-self-signed.pem
SSL_CERT_FILE=/opt/homebrew/etc/squid/ca-bundle.pem
REQUESTS_CA_BUNDLE=/opt/homebrew/etc/squid/ca-bundle.pem
CURL_CA_BUNDLE=/opt/homebrew/etc/squid/ca-bundle.pem

NODE_EXTRA_CA_CERTS gets the bare CA because Node adds it to its own roots. The other three get ca-bundle.pem (public roots plus the bumping CA, written by deploy.sh) because they replace the default bundle. Pointed at the bare CA those clients would trust Squid and nothing else, and would reject every host Squid does not bump — Python in particular fails with CERTIFICATE_VERIFY_FAILED: unable to get local issuer certificate. Rebuild the bundle after brew upgrade ca-certificates.

The rest of this document explains what gets set up and why.

Patching Squid

A major challenge is that actions/cache uses Azure storage APIs which makes HTTP range requests. While Squid supports range requests, it is not good at caching them. There is an option, range_offset_limit none which, according to the documentation:

A size of 'none' causes Squid to always fetch the object from the beginning so it may cache the result. (2.0 style)

Three things stop that from working. squid-7.6.patch fixes them, and the formula embeds it so brew install applies it automatically.

1. Azure's x-ms-range header. Azure Storage clients put their byte range in a proprietary x-ms-range header. Squid does not recognise it, so it forwards the header upstream and caches at best the single range the client asked for — and it never learns the request was ranged at all, which also costs it the range_offset_limit exemption in CheckQuickAbortIsReasonable(). The patch rewrites x-ms-range into a standard Range header in clientInterpretRequestHeaders(), before anything else looks at it, so range parsing, range_offset_limit, quick_abort, 206 assembly and upstream Range suppression all work unmodified. Values Squid cannot parse are forwarded untouched for the origin to interpret.

2. The pinned server connection. With ssl_bump, the to-origin connection is pinned to the client connection, and ConnStateData::swanSong() closes it unconditionally — upstream marks that line XXX: Closing pinned conn is too harsh: The Client may want to continue!. With range_offset_limit none that aborts the whole-object download the moment the client has its range, so nothing is ever cached. The patch hands a busy connection over instead of closing it and lets quick_abort decide whether the orphaned transaction should continue, since it already understands range_offset_limit. Plain HTTP is unaffected; this only matters once SSL bumping is in play.

3. Content-Length on a 304. Azure answers revalidation with 304 Not Modified plus Content-Length: 0, describing the empty 304 rather than the stored representation, which RFC 9110 §8.6 forbids. Squid merges it into the cached reply, after which every hit returns 200 with an empty body. The patch keeps the stored Content-Length, which matches the stored body whether or not the origin is compliant.

Store ID Helper Program

GitHub releases and Actions cache blobs are fetched with GET requests carrying authentication parameters in the query string. A helper program maps the URL to a store ID so that Squid sees the same object requested with different signatures as one object. github_store_id_helper.py also does some GNOME mirror mapping that you can remove if you only need to cache GitHub objects.

Do not pin a storage account. GitHub moves Actions storage between accounts, so the helper matches any of them:

STRIP_PARAMS = [
  re.compile(r'^https://[a-z0-9]+\.blob\.core\.windows\.net(:[0-9]+)?/actions-cache/'),
  re.compile(r'^https://release-assets\.githubusercontent\.com(:[0-9]+)?/github-production-release-asset/'),
  ...
]

The Azure rule is scoped to the actions-cache container, whose blob names are content-addressed and immutable; actions-results is upload-only in practice and is deliberately left alone. Release assets are matched on both the current and legacy hostnames.

Two details that will bite you if you write your own helper:

• Squid also sends store ID lookups for CONNECT, where the "URL" is a bare host:port with no ://. Detecting the concurrency channel ID by looking for :// therefore misfires, the helper answers without the channel prefix, and Squid dies with assertion failed: helper.cc: skip == 0 && eom == nullptr. Detect it by shape — a leading integer — and echo it on every reply including BH.
• PURGE must be normalised too. If the helper only answers for GET/HEAD, Squid looks up the un-normalised URL and every purge of a signed URL returns 404.

Squid Configuration

The configuration is largely inspired from this blog post which details setting up SSL bump for caching large downloads. squid.conf is the full version; here are the important parts.

SSL Bump

http_port 127.0.0.1:3128 tcpkeepalive=60,30,3 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=20MB tls-cert=/opt/homebrew/etc/squid/squid-self-signed.crt tls-key=/opt/homebrew/etc/squid/squid-self-signed.key cipher=HIGH:MEDIUM:!LOW:!RC4:!SEED:!IDEA:!3DES:!MD5:!EXP:!PSK:!DSS options=NO_TLSv1,NO_SSLv3 tls-dh=prime256v1:/opt/homebrew/etc/squid/squid-self-signed_dhparam.pem

acl step1 at_step SslBump1
acl github_controlplane ssl::server_name_regex \.actions\.githubusercontent\.com$
acl github_git ssl::server_name github.com

sslcrtd_program /opt/homebrew/opt/squid/libexec/security_file_certgen -s /opt/homebrew/var/logs/ssl_db -M 20MB
sslcrtd_children 5
ssl_bump peek step1
ssl_bump splice github_controlplane
ssl_bump splice github_git
ssl_bump stare all
sslproxy_cert_error deny all

Do not put SINGLE_DH_USE or SINGLE_ECDH_USE in options=: Squid 7 does not have SINGLE_ECDH_USE and OpenSSL 3 does not have SSL_OP_SINGLE_DH_USE, so they log ERROR: Unsupported TLS option. They are no-ops on modern OpenSSL anyway.

Two host groups are spliced because nothing on them is cacheable, so bumping would only add certificate generation and latency to the runner's critical path:

• The Actions control plane — job broker long-polls, token exchange, health checks, result submission. Match it by domain rather than enumerating hostnames; GitHub adds new ones regularly, and everything cacheable lives on blob.core.windows.net or *.githubusercontent.com instead.
• github.com, which serves only /info/refs (no-cache, must-revalidate), git-upload-pack (a POST), and /releases/download/ redirects whose signed target changes on every request. The release bytes come from release-assets.githubusercontent.com, a different host that stays bumped and cached.

Splicing only works if every client trusts the public roots as well as the bumping CA — see ca-bundle.pem above.

Note that Squid compiles ACL and refresh_pattern regexes with POSIX regcomp(3). \d and \w are not supported; spell character classes out.

macOS: shared memory segment names

Squid mixes a hash of the pid filename into POSIX shared memory segment names. The result, /squid-XXXX-tls_session_cache.shm, is 33 characters, and Darwin caps shm_open() names at PSHMNAMLEN = 31. Any TLS port therefore aborts at startup:

FATAL: Ipc::Mem::Segment::create failed to shm_open(/squid-3BNP-tls_session_cache.shm): (63) File name too long

Disabling the shared TLS session cache avoids the segment entirely:

sslproxy_session_cache_size 0

The cost is TLS session resumption between the runner and Squid, which is negligible when the runner holds a few long-lived connections. If you would rather keep it, shorten the service name with squid -n sq instead — but then every squid -k ... invocation needs the same flag.

Collapsed Forwarding

If one request is currently being cached and another request is made to the same object, we want to stall the second request until the first one finished. Usually, this isn't good for performance, but when we are exclusively caching large downloads, this will reduce a lot of redundant downloads.

collapsed_forwarding on

FD Limit

On macOS, the default FD limit (256) is too low.

max_filedescriptors 4096

Cache settings

The helper lets Squid recognise different GET requests as the same object. One process serves many concurrent lookups because the helper answers on a channel ID, so a handful of children is plenty.

store_id_program /opt/homebrew/etc/squid/github_store_id_helper.py
store_id_children 10 startup=2 idle=2 concurrency=10

Each object is limited to 2000 MB and the total cache to 100000 MB; adjust to taste. Actions cache objects run to several hundred megabytes.

maximum_object_size 2000 MB
cache_dir aufs /opt/homebrew/var/cache/squid 100000 16 256

Refresh patterns for the Azure blobs, release assets and action tarballs. The overrides ensure these are cached regardless of the HTTP response, which is fine because the objects have unique immutable IDs in the URL. override-lastmod is as important as override-expire: Azure sends these blobs with a Last-Modified but no Cache-Control, so Squid otherwise falls back to the last-modified factor — freshness of 20% of the object's age — and a cache blob read minutes after it was uploaded goes stale within seconds, making every subsequent range request revalidate. With store_id_program in use these patterns match the store ID — the query-stripped URL — not the original request URL.

refresh_pattern -i ^https://[a-z0-9]+\.blob\.core\.windows\.net(:[0-9]+)?/actions-cache/	1440	20%	10080	ignore-reload ignore-no-store ignore-private override-expire override-lastmod
refresh_pattern -i ^https://release-assets\.githubusercontent\.com(:[0-9]+)?/github-production-release-asset/	1440	20%	10080	ignore-reload ignore-no-store ignore-private override-expire override-lastmod
refresh_pattern -i ^https://codeload\.github\.com(:[0-9]+)?/.*/(tar\.gz|zip)/	1440	20%	10080	ignore-reload ignore-no-store ignore-private override-expire override-lastmod

As detailed above, this requires a patched Squid to work. We want range downloads to cache the entire object.

acl azure_storage dstdomain .blob.core.windows.net
acl github_release_assets dstdomain release-assets.githubusercontent.com
range_offset_limit -1 azure_storage
range_offset_limit -1 github_release_assets

And do not abandon a download just because the client that started it went away — finishing it puts the object in the cache for the next job. Uncachable and private responses are still aborted, because CheckQuickAbortIsReasonable() checks those first.

quick_abort_min -1 KB

PURGE

acl PURGE method PURGE

That line is not decoration. cache_cf.cc derives Config2.onoff.enable_purge from the number of ACLs mentioning the PURGE method, so without it every PURGE returns 403 — even from localhost. Authorisation comes from the ordinary http_access allow localhost rule; a dedicated http_access allow PURGE localhost placed after http_access deny all would be unreachable.

Log rotation

Homebrew installs none, and macOS newsyslog needs root, so org.squid-cache.logrotate.plist runs squid -k rotate daily from a user LaunchAgent. With logfile_rotate 5 in squid.conf this bounds the log directory at a few tens of megabytes.

What this will not cache

• git clone / git fetch over HTTPS. git-upload-pack is a POST with a dynamically negotiated pack; no proxy can cache it. A git mirror is the only fix.
• codeload.github.com action tarballs. Squid will not store a response to a request carrying Authorization unless the response is Cache-Control: public/must-revalidate (http.cc, RFC 9111 §3.5), and the runner sends a token to codeload even for public actions. No refresh_pattern option overrides this, and request_header_access Authorization deny does not help either — request->flags.auth is set from the client's headers in clientInterpretRequestHeaders(), long before the server-side header filter runs.

Checking that it works

squid -k parse                 # only the four "violates HTTP" warnings are expected
awk '{print $4}' /opt/homebrew/var/logs/access.log | sort | uniq -c | sort -rn | head

TCP_HIT / TCP_MEM_HIT should be a large share of requests. If everything is TCP_MISS, the store ID helper is probably returning ERR — check that the hostnames in STRIP_PARAMS still match what your runners actually fetch:

awk '{print $7}' /opt/homebrew/var/logs/access.log | cut -d/ -f3 | sort | uniq -c | sort -rn | head

deploy.sh

#!/bin/bash
#
# Set up a patched Squid caching proxy for self-hosted GitHub Actions runners.
#
#   curl -fsSL https://gist.githubusercontent.com/osy/30c5c96d7575efd1d2a2db5e3def0815/raw/deploy.sh | bash
#
# or, to read it first (recommended):
#
#   git clone https://gist.github.com/30c5c96d7575efd1d2a2db5e3def0815.git squid-cache
#   cd squid-cache && bash deploy.sh
#
# Safe to re-run: existing certificates and an existing squid.conf are left
# alone unless you ask for them to be replaced.
#
#   --force-config   overwrite squid.conf and the store ID helper
#   --force-certs    regenerate the CA and DH parameters (invalidates the CA
#                    already trusted by your runners)
#   --skip-install   don't touch Homebrew; only refresh config/certs/service
#   --uninstall      stop the service and remove what this script installed
#
set -euo pipefail

GIST_ID=30c5c96d7575efd1d2a2db5e3def0815
GIST_REPO="https://gist.github.com/${GIST_ID}.git"
TAP=osy/local
FORMULA=squid
ASSETS=(squid.rb squid.conf github_store_id_helper.py org.squid-cache.logrotate.plist)

CERT_DAYS=3650
DH_BITS=2048

force_config=0
force_certs=0
skip_install=0
uninstall=0
for arg in "$@"; do
    case "$arg" in
        --force-config) force_config=1 ;;
        --force-certs)  force_certs=1 ;;
        --skip-install) skip_install=1 ;;
        --uninstall)    uninstall=1 ;;
        -h|--help)      awk 'NR>1 && /^#/ {sub(/^# ?/,""); print; next} NR>1 {exit}' "$0"; exit 0 ;;
        *) echo "unknown option: $arg (try --help)" >&2; exit 2 ;;
    esac
done

say()  { printf '\033[1;34m==>\033[0m %s\n' "$*"; }
warn() { printf '\033[1;33mwarning:\033[0m %s\n' "$*" >&2; }
die()  { printf '\033[1;31merror:\033[0m %s\n' "$*" >&2; exit 1; }

# ---------------------------------------------------------------- preflight --

[ "$(uname -s)" = "Darwin" ] || die "this script targets macOS"
[ "$(id -u)" != "0" ] || die "do not run as root; Homebrew refuses to work under sudo"
command -v brew >/dev/null || die "Homebrew is required: https://brew.sh"
command -v openssl >/dev/null || die "openssl not found"

# SQUID_PREFIX exists so the script can be exercised against a scratch
# directory; normally the Homebrew prefix is the right answer.
PREFIX="${SQUID_PREFIX:-$(brew --prefix)}"
ETC="$PREFIX/etc"
CONFDIR="$ETC/squid"
VAR="$PREFIX/var"
LOGS="$VAR/logs"
CACHE="$VAR/cache/squid"
AGENTS="$HOME/Library/LaunchAgents"
PLIST_LABEL=org.squid-cache.logrotate
PLIST="$AGENTS/$PLIST_LABEL.plist"

# --------------------------------------------------------------- uninstall --

if [ "$uninstall" = 1 ]; then
    say "Stopping service"
    brew services stop "$FORMULA" 2>/dev/null || true
    if [ -f "$PLIST" ]; then
        launchctl unload "$PLIST" 2>/dev/null || true
        rm -f "$PLIST"
        say "Removed $PLIST"
    fi
    brew unpin "$FORMULA" 2>/dev/null || true
    say "Leaving $CONFDIR, $CACHE and the Homebrew package in place."
    say "Remove them yourself if you want a clean slate:"
    echo "    brew uninstall $FORMULA && brew untap $TAP"
    echo "    rm -rf $CONFDIR $CACHE"
    exit 0
fi

# ------------------------------------------------------------------ assets --

# Run from a checkout if the files are next to us; otherwise pull the gist.
here="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" 2>/dev/null && pwd || true)"
SRC=""
if [ -n "$here" ] && [ -f "$here/squid.rb" ] && [ -f "$here/squid.conf" ]; then
    SRC="$here"
    say "Using assets from $SRC"
else
    SRC="$(mktemp -d)"
    trap 'rm -rf "$SRC"' EXIT
    say "Fetching assets from gist $GIST_ID"
    if command -v git >/dev/null && git clone --depth 1 -q "$GIST_REPO" "$SRC/gist" 2>/dev/null; then
        SRC="$SRC/gist"
    else
        warn "git clone failed; falling back to raw downloads"
        for f in "${ASSETS[@]}"; do
            curl -fsSL -o "$SRC/$f" \
                "https://gist.githubusercontent.com/osy/$GIST_ID/raw/$f" \
                || die "could not download $f"
        done
    fi
fi
for f in "${ASSETS[@]}"; do
    [ -f "$SRC/$f" ] || die "missing asset: $f"
done

# ----------------------------------------------------------------- install --

if [ "$skip_install" = 1 ]; then
    say "Skipping Homebrew install (--skip-install)"
elif [ -n "${SQUID_PREFIX:-}" ]; then
    die "SQUID_PREFIX is set; re-run with --skip-install (it only makes sense for testing)"
else
    if ! brew tap | grep -qx "$TAP"; then
        say "Creating local tap $TAP"
        brew tap-new "$TAP" --no-git >/dev/null
    fi
    tapdir="$(brew --repository)/Library/Taps/${TAP%/*}/homebrew-${TAP#*/}"
    mkdir -p "$tapdir/Formula"
    cp "$SRC/squid.rb" "$tapdir/Formula/$FORMULA.rb"
    say "Installed formula into $tapdir/Formula/$FORMULA.rb"

    # Whatever is installed now gets replaced, so there is no need to tell
    # homebrew-core's bottle apart from an older build of this formula.
    if brew list --versions "$FORMULA" >/dev/null 2>&1; then
        say "Removing the currently installed squid"
        brew services stop "$FORMULA" 2>/dev/null || true
        brew unpin "$FORMULA" 2>/dev/null || true
        brew uninstall "$FORMULA"
    fi

    say "Building squid from source (this takes a few minutes)"
    brew install "$TAP/$FORMULA"
    # Without this a later "brew upgrade" swaps in homebrew-core's unpatched
    # bottle and range caching stops working.
    brew pin "$FORMULA"

    if ! strings "$(brew --prefix "$FORMULA")/sbin/squid" | grep -q "x-ms-range"; then
        die "the installed squid does not contain the patch; check the build output"
    fi
    say "Patched squid $(squid -v 2>/dev/null | head -1 | sed 's/.*Version //') installed"
fi

# ------------------------------------------------------------- directories --

say "Creating directories"
mkdir -p "$CONFDIR" "$LOGS" "$CACHE" "$AGENTS"

# ------------------------------------------------------------ certificates --

CA_KEY="$CONFDIR/squid-self-signed.key"
CA_CRT="$CONFDIR/squid-self-signed.crt"
CA_PEM="$CONFDIR/squid-self-signed.pem"
DHPARAM="$CONFDIR/squid-self-signed_dhparam.pem"

if [ "$force_certs" = 1 ] || [ ! -f "$CA_CRT" ] || [ ! -f "$CA_KEY" ]; then
    if [ -f "$CA_CRT" ]; then
        warn "replacing the existing CA; every runner must be given the new one"
        cp "$CA_CRT" "$CA_CRT.$(date +%Y%m%d%H%M%S).bak"
    fi
    say "Generating the SSL-bump CA (valid $CERT_DAYS days)"
    openssl req -new -newkey rsa:2048 -sha256 -days "$CERT_DAYS" -nodes -x509 \
        -keyout "$CA_KEY" -out "$CA_CRT" \
        -subj "/CN=Squid Runner Cache CA/O=$(hostname -s)" \
        -addext "basicConstraints=critical,CA:TRUE" \
        -addext "keyUsage=critical,keyCertSign,cRLSign,digitalSignature" 2>/dev/null
    # NODE_EXTRA_CA_CERTS wants the certificate on its own, in PEM form.
    cp "$CA_CRT" "$CA_PEM"
    chmod 600 "$CA_KEY"
    chmod 644 "$CA_CRT" "$CA_PEM"
else
    say "Keeping the existing CA ($CA_CRT)"
fi

if [ "$force_certs" = 1 ] || [ ! -f "$DHPARAM" ]; then
    say "Generating $DH_BITS-bit DH parameters (slow, up to a couple of minutes)"
    openssl dhparam -outform PEM -out "$DHPARAM" "$DH_BITS" 2>/dev/null
    chmod 644 "$DHPARAM"
else
    say "Keeping the existing DH parameters"
fi

# Combined trust store: the public roots plus our bumping CA.
#
# NODE_EXTRA_CA_CERTS *adds* to Node's built-in roots, so Node is happy with the
# bare CA. Python's SSL_CERT_FILE / REQUESTS_CA_BUNDLE and curl's CURL_CA_BUNDLE
# *replace* the default bundle, so pointing those at the bare CA leaves the
# client trusting Squid and nothing else -- it then rejects every host Squid
# does not bump, with "unable to get local issuer certificate". Point them here.
#
# Rebuilt on every run so it picks up ca-certificates updates from Homebrew.
CA_BUNDLE="$CONFDIR/ca-bundle.pem"
base_roots=""
for candidate in \
    "$PREFIX/etc/ca-certificates/cert.pem" \
    "$(python3 -m certifi 2>/dev/null || true)" \
    "$(openssl version -d 2>/dev/null | sed 's/.*"\(.*\)"/\1/')/cert.pem"
do
    if [ -n "$candidate" ] && [ -f "$candidate" ]; then base_roots="$candidate"; break; fi
done
if [ -n "$base_roots" ]; then
    cat "$base_roots" "$CA_CRT" > "$CA_BUNDLE"
    chmod 644 "$CA_BUNDLE"
    say "Built $CA_BUNDLE (public roots from $base_roots + the bumping CA)"
else
    warn "no public CA bundle found; skipping $CA_BUNDLE"
    warn "clients using SSL_CERT_FILE/REQUESTS_CA_BUNDLE will only trust bumped hosts"
fi

# ------------------------------------------------------------------ config --

install_file() {  # install_file <src> <dst> <mode> <what>
    local src="$1" dst="$2" mode="$3" what="$4"
    if [ -f "$dst" ] && [ "$force_config" != 1 ]; then
        if cmp -s "$src" "$dst"; then
            say "$what already up to date"
        else
            cp "$dst" "$dst.$(date +%Y%m%d%H%M%S).bak"
            warn "$dst differs from the gist version; kept yours, saved a .bak"
            warn "re-run with --force-config to overwrite it"
        fi
        return
    fi
    [ -f "$dst" ] && cp "$dst" "$dst.$(date +%Y%m%d%H%M%S).bak"
    install -m "$mode" "$src" "$dst"
    say "Installed $what"
}

# squid.conf ships with /opt/homebrew paths; rewrite them for this machine.
conf_tmp="$(mktemp)"
sed "s|/opt/homebrew|$PREFIX|g" "$SRC/squid.conf" > "$conf_tmp"
install_file "$conf_tmp" "$ETC/squid.conf" 644 "squid.conf"
rm -f "$conf_tmp"

install_file "$SRC/github_store_id_helper.py" "$CONFDIR/github_store_id_helper.py" 755 "store ID helper"

# ---------------------------------------------------------------- ssl_db ----

SSL_DB="$LOGS/ssl_db"
CERTGEN="$PREFIX/opt/squid/libexec/security_file_certgen"
if [ -x "$CERTGEN" ]; then
    if [ ! -d "$SSL_DB" ]; then
        say "Initialising the generated-certificate database"
        "$CERTGEN" -c -s "$SSL_DB" -M 20MB >/dev/null
    else
        say "Certificate database already present"
    fi
else
    warn "$CERTGEN not found; skipping ssl_db init (run again after installing squid)"
fi

# ----------------------------------------------------------- cache + start --

if command -v squid >/dev/null && [ "$skip_install" != 1 ]; then
    say "Validating the configuration"
    if squid -k parse 2>&1 | grep -E "^.*(ERROR|FATAL)" ; then
        die "squid -k parse reported errors (see above)"
    fi

    if [ ! -d "$CACHE/00" ]; then
        say "Building the cache directory structure"
        squid -z --foreground >/dev/null 2>&1 || true
    fi

    say "Starting squid"
    brew services restart "$FORMULA" >/dev/null
    sleep 3
    if squid -k check 2>/dev/null; then
        say "squid is running"
    else
        warn "squid did not come up; check $LOGS/cache.log"
    fi
fi

# ------------------------------------------------------------ log rotation --

say "Installing daily log rotation ($PLIST_LABEL)"
[ -f "$PLIST" ] && launchctl unload "$PLIST" 2>/dev/null || true
sed "s|@PREFIX@|$PREFIX|g" "$SRC/org.squid-cache.logrotate.plist" > "$PLIST"
chmod 644 "$PLIST"
launchctl load "$PLIST" 2>/dev/null || warn "launchctl load failed; load $PLIST by hand"

# -------------------------------------------------------------------- done --

cat <<EOF

$(printf '\033[1;32mDone.\033[0m') Point each runner at the proxy by adding these to its .env:

    http_proxy=http://127.0.0.1:3128
    https_proxy=http://127.0.0.1:3128
    NODE_EXTRA_CA_CERTS=$CA_PEM
    SSL_CERT_FILE=$CA_BUNDLE
    REQUESTS_CA_BUNDLE=$CA_BUNDLE
    CURL_CA_BUNDLE=$CA_BUNDLE

NODE_EXTRA_CA_CERTS gets the bare CA because Node adds it to its own roots.
The other three get the combined bundle because they replace the default one:
point them at the bare CA and the client will trust Squid and nothing else,
breaking every host Squid does not bump.

The proxy listens on 127.0.0.1 and [::1] only. If a runner is on another host,
change the http_port lines in $ETC/squid.conf and firewall the port
accordingly -- there is no authentication.

Useful commands:

    brew services restart squid          # after editing squid.conf
    squid -k parse                       # check the config
    tail -f $LOGS/access.log
    awk '{print \$4}' $LOGS/access.log | sort | uniq -c | sort -rn | head

The CA expires $(date -v+${CERT_DAYS}d '+%Y-%m-%d' 2>/dev/null || echo "in $CERT_DAYS days"); re-run with --force-certs to replace it
(every runner then needs the new $CA_PEM).
EOF

github_store_id_helper.py

#!/usr/bin/env python3

import re
import sys

# URLs whose query string carries authentication/signature material rather than
# object identity. Stripping it lets Squid recognise the same object across
# differently-signed URLs, which is the whole point of running a StoreID helper.
#
# These are patterns, not prefixes, so the Azure entry matches any storage
# account: GitHub moves Actions storage between accounts, and pinning one
# disables caching the day it changes. It is scoped to the actions-cache
# container, whose blob names are content-addressed and immutable;
# actions-results is upload-only in practice and is left alone.
# "(:\d+)?" because Squid keeps a non-default port in the URL it hands us.
STRIP_PARAMS = [
  re.compile(r'^https://[a-z0-9]+\.blob\.core\.windows\.net(:\d+)?/actions-cache/'),
  re.compile(r'^https://release-assets\.githubusercontent\.com(:\d+)?/github-production-release-asset/'),
  re.compile(r'^https://objects\.githubusercontent\.com(:\d+)?/github-production-release-asset-'),
  re.compile(r'^https://storage\.googleapis\.com(:\d+)?/chrome-infra-packages/store/SHA256/'),
]

GNOME_PROJECTS = [
  'glib',
  'json-glib',
  'libsoup',
  'phodav',
]

def stripParams(url):
  idx = url.find('?')
  if idx < 0:
    return None
  else:
    return url[0:idx]

def parseGnomeMirror(url, project):
  match = re.match(r'^https?:\/\/.*\/' + project + r'\/([\d\.]+)\/(' + project + r'-[\d\.]+\.tar\.\w+)', url)
  if not match:
    return None
  else:
    version = match.group(1)
    file = match.group(2)
    return f'https://download.gnome.org/sources/{project}/{version}/{file}'

# PURGE is included so cache maintenance can address an object by its real URL.
# Without it the helper returns ERR for PURGE, Squid looks up the unnormalised
# URL, and every purge of a signed URL returns 404.
CACHE_METHODS = ('GET', 'HEAD', 'PURGE')

def parseUrl(url, method=None):
  if method != None and method not in CACHE_METHODS:
    return None
  for candidate in STRIP_PARAMS:
    if candidate.match(url):
      return stripParams(url)
  for project in GNOME_PROJECTS:
    storeID = parseGnomeMirror(url, project)
    if storeID != None:
      return storeID

def splitChannelID(line):
  """Split a request line into (channel ID or None, remaining fields).

  Squid sends "[channel-ID ]URL" followed by store_id_extras, which defaults to
  "%>a/%>A %un %>rm myip=%la myport=%lp". The channel ID is present only when
  store_id_children is configured with concurrency=.

  Detect it by shape -- a bare integer first field. Testing the second field for
  "://" does not work: Squid also looks up CONNECT requests, whose "URL" is a
  bare "host:port". Getting this wrong under concurrency= makes the helper reply
  without a channel ID, which kills Squid with
  "assertion failed: helper.cc: skip == 0 && eom == nullptr".
  """
  parts = line.split(' ')
  if len(parts) > 1 and parts[0].isdigit():
    return (parts[0], parts[1:])
  return (None, parts)

def parseLine(line):
  (channelID, parts) = splitChannelID(line)
  url = parts[0]
  method = parts[3] if len(parts) > 3 else None
  return (channelID, parseUrl(url, method))

def main():
  for line in sys.stdin:
    line = line.strip()
    # Determine the channel ID outside the try: a reply must carry it even when
    # the request could not be parsed at all.
    (channelID, _) = splitChannelID(line)
    try:
      (_, storeID) = parseLine(line)
      if storeID == None:
        result = "ERR"
      else:
        result = "OK store-id=" + storeID
    except Exception:
      result = 'BH'
    if channelID != None:
      result = channelID + " " + result
    sys.stdout.write(result + '\n')
    sys.stdout.flush()

if __name__ == '__main__':
  sys.exit(main())

org.squid-cache.logrotate.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<!--
  Rotates Squid's access.log and cache.log daily.

  Homebrew installs no log rotation and macOS newsyslog needs root, so run
  "squid -k rotate" from a user LaunchAgent alongside the brew service. Squid
  keeps logfile_rotate old copies (5, set in squid.conf).

  deploy.sh installs this for you, rewriting @PREFIX@ for your Homebrew prefix.
  To install by hand:

    sed "s|@PREFIX@|$(brew --prefix)|g" org.squid-cache.logrotate.plist \
      > ~/Library/LaunchAgents/org.squid-cache.logrotate.plist
    launchctl load ~/Library/LaunchAgents/org.squid-cache.logrotate.plist
-->
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>org.squid-cache.logrotate</string>

    <key>ProgramArguments</key>
    <array>
        <string>@PREFIX@/opt/squid/sbin/squid</string>
        <string>-k</string>
        <string>rotate</string>
    </array>

    <!-- 04:15 daily, outside typical CI hours -->
    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>4</integer>
        <key>Minute</key>
        <integer>15</integer>
    </dict>

    <key>RunAtLoad</key>
    <false/>

    <key>StandardOutPath</key>
    <string>@PREFIX@/var/logs/logrotate.out</string>
    <key>StandardErrorPath</key>
    <string>@PREFIX@/var/logs/logrotate.err</string>
</dict>
</plist>

squid-7.6.patch

Squid 7.6: cache GitHub Actions / Azure Storage downloads behind ssl-bump
=========================================================================

Three fixes, all needed before a range-heavy workload (actions/cache, GitHub
release assets) can be cached by a bumping proxy.

1. src/client_side_request.cc -- interpret Azure's "x-ms-range" header

   Azure Storage clients put their byte range in a proprietary x-ms-range
   header. Squid does not recognise it, so it forwards the header upstream and
   caches at best the single range the client asked for. It also never learns
   the request was ranged, which costs it the range_offset_limit exemption in
   CheckQuickAbortIsReasonable().

   The header is rewritten into a standard Range header in
   clientInterpretRequestHeaders(), before anything else looks at it, so range
   parsing, range_offset_limit, quick_abort, 206 assembly and upstream Range
   suppression all work unmodified. Values Squid cannot parse are forwarded
   untouched, leaving their meaning to the origin server.

2. src/client_side.cc, src/client_side.h -- do not close a pinned connection a
   Client is still using

   With ssl_bump the to-origin connection is pinned to the client connection,
   and ConnStateData::swanSong() closed it unconditionally; upstream marks that
   line "XXX: Closing pinned conn is too harsh: The Client may want to
   continue!". Under "range_offset_limit none" it aborts the whole-object
   download the moment the client has its range, so nothing is ever cached.

   A connection currently borrowed by a server-side transaction is handed over
   rather than closed. Whether the orphaned transaction continues is
   quick_abort's decision (CheckQuickAbortIsReasonable), which already
   understands range_offset_limit; when it says no, StoreEntry::abort() tears
   the connection down through FwdState::HandleStoreAbort(). The transaction
   closes the socket itself when it finishes, in
   HttpStateData::processReplyBody().

3. src/HttpHeader.cc -- ignore Content-Length when applying a 304

   Azure answers revalidation with "304 Not Modified" plus "Content-Length: 0",
   describing the empty 304 rather than the stored representation, which RFC
   9110 Section 8.6 forbids. Squid merges it into the cached reply, after which
   every hit returns 200 with an empty body. The stored Content-Length is kept
   instead: it matches the stored body whether or not the origin is compliant.

   Also fixes HttpHeader::putRange(), which omitted the "bytes=" range-unit and
   so emitted a Range header Squid itself cannot parse.

--- a/src/HttpHeader.cc
+++ b/src/HttpHeader.cc
@@ -260,7 +260,14 @@
     return
         // TODO: Consider updating Vary headers after comparing the magnitude of
         // the required changes (and/or cache losses) with compliance gains.
-        (id == Http::HdrType::VARY);
+        (id == Http::HdrType::VARY) ||
+        // RFC 9110 Section 8.6 forbids a 304 Content-Length that disagrees with
+        // the length of the stored representation, but some origins (notably
+        // Azure Storage) send "Content-Length: 0" describing the empty 304
+        // itself. Applying that would make the cached response claim a body
+        // length it does not have, so keep the length we actually stored: it
+        // matches the stored body in both the compliant and the buggy case.
+        (id == Http::HdrType::CONTENT_LENGTH);
 }
 
 void
@@ -1045,6 +1052,9 @@
     /* pack into mb */
     MemBuf mb;
     mb.init();
+    /* HttpHdrRange packs a bare range-set; the field also needs its range-unit
+     * (RFC 9110 Section 14.2), without which Squid cannot re-parse what it wrote */
+    mb.append("bytes=", 6);
     range->packInto(&mb);
     /* put */
     addEntry(new HttpHeaderEntry(Http::HdrType::RANGE, SBuf(), mb.buf));
--- a/src/client_side.cc
+++ b/src/client_side.cc
@@ -598,8 +598,16 @@
     terminateAll(ERR_NONE, LogTagsErrors());
     checkLogging();
 
-    // XXX: Closing pinned conn is too harsh: The Client may want to continue!
-    unpinConnection(true);
+    // Closing a pinned connection that a Client is still using would abort that
+    // server-side transaction, even when Squid wants it to finish: with
+    // "range_offset_limit none", for example, the transaction keeps downloading
+    // a whole object into the cache after the client got the range it asked
+    // for. Whether an orphaned transaction should continue is quick_abort's
+    // decision (see CheckQuickAbortIsReasonable()); when it says no,
+    // StoreEntry::abort() closes this connection via FwdState::HandleStoreAbort().
+    // Either way the Client owns the connection now and closes it when done
+    // (see HttpStateData::processReplyBody()), so hand it over instead.
+    unpinConnection(!pinnedConnectionIsBusy());
 
     Server::swanSong();
 
@@ -3874,6 +3882,15 @@
     // there is no point since the client connection is now gone
     HttpRequestPointer requestPointer = request;
     throw ErrorState::NewForwarding(ERR_CANNOT_FORWARD, requestPointer, ale);
+}
+
+bool
+ConnStateData::pinnedConnectionIsBusy() const
+{
+    // startPinnedConnectionMonitoring() runs only while the connection is idle
+    // and borrowPinnedConnection() stops it, so a missing readHandler means the
+    // connection has been handed to a Client (or is about to be).
+    return Comm::IsConnOpen(pinning.serverConnection) && !pinning.readHandler;
 }
 
 void
--- a/src/client_side.h
+++ b/src/client_side.h
@@ -196,6 +196,11 @@
     /// Undo pinConnection() and, optionally, close the pinned connection.
     void unpinConnection(const bool andClose);
 
+    /// Whether a server-side transaction is currently using the pinned
+    /// connection. We only monitor a pinned connection while it sits idle, so
+    /// an open-but-unmonitored one has been borrowed by a Client.
+    bool pinnedConnectionIsBusy() const;
+
     /// \returns validated pinned to-server connection, stopping its monitoring
     /// \throws a newly allocated ErrorState if validation fails
     static Comm::ConnectionPointer BorrowPinnedConnection(HttpRequest *, const AccessLogEntryPointer &);
--- a/src/client_side_request.cc
+++ b/src/client_side_request.cc
@@ -40,6 +40,7 @@
 #include "http.h"
 #include "http/Stream.h"
 #include "HttpHdrCc.h"
+#include "HttpHeaderRange.h"
 #include "HttpReply.h"
 #include "HttpRequest.h"
 #include "internal.h"
@@ -75,6 +76,8 @@
 #include "ssl/support.h"
 #endif
 
+#include <memory>
+
 #if FOLLOW_X_FORWARDED_FOR
 
 #if !defined(SQUID_X_FORWARDED_FOR_HOP_MAX)
@@ -883,6 +886,44 @@
     }
 }
 
+/// Proprietary request header used by Azure Storage clients (e.g. the Azure
+/// SDK behind actions/cache) to express a byte range. Azure honours it in
+/// preference to the standard Range header when a request carries both.
+static const SBuf AzureRangeHeaderName("x-ms-range");
+
+/// Rewrites an Azure x-ms-range request header into an equivalent standard
+/// Range header so that the rest of Squid -- range parsing, range_offset_limit,
+/// the quick_abort exemption for full-object downloads, 206 assembly and
+/// upstream Range suppression -- handles these requests like any other range
+/// request.
+///
+/// A value Squid cannot parse is forwarded untouched, leaving its meaning to
+/// the origin server. Squid then keeps managing any Range header as before,
+/// which is the best available outcome: origins that ignore x-ms-range still
+/// get a whole-object request they can be cached from, and origins that honour
+/// it would have overridden that Range header anyway.
+/// \returns whether request headers were modified
+static bool
+clientInterpretAzureRange(HttpHeader &hdr)
+{
+    String value;
+    if (!hdr.hasNamed(AzureRangeHeaderName, &value))
+        return false;
+
+    const std::unique_ptr<HttpHdrRange> parsed(HttpHdrRange::ParseCreate(&value));
+    if (!parsed) {
+        debugs(85, 3, "forwarding unparsable " << AzureRangeHeaderName << ": " << value);
+        return false;
+    }
+
+    // Azure gives x-ms-range priority over Range, so drop any competing spec.
+    hdr.delByName(AzureRangeHeaderName);
+    hdr.delById(Http::HdrType::REQUEST_RANGE);
+    hdr.putRange(parsed.get()); // also removes any old Range header
+    debugs(85, 3, "interpreting " << AzureRangeHeaderName << " as Range: " << value);
+    return true;
+}
+
 static void
 clientInterpretRequestHeaders(ClientHttpRequest * http)
 {
@@ -925,6 +966,11 @@
 
     /* ignore range header in non-GETs or non-HEADs */
     if (request->method == Http::METHOD_GET || request->method == Http::METHOD_HEAD) {
+        // Any range parsed by HttpRequest::hdrCacheInit() predates this
+        // rewrite and now describes a header we just removed.
+        if (clientInterpretAzureRange(*req_hdr))
+            request->ignoreRange("x-ms-range supersedes Range");
+
         // XXX: initialize if we got here without HttpRequest::parseHeader()
         if (!request->range)
             request->range = req_hdr->getRange();
@@ -949,6 +995,7 @@
     else {
         req_hdr->delById(Http::HdrType::RANGE);
         req_hdr->delById(Http::HdrType::REQUEST_RANGE);
+        req_hdr->delByName(AzureRangeHeaderName);
         request->ignoreRange("neither HEAD nor GET");
     }
 

squid.conf

#
# Caching proxy for self-hosted GitHub Actions runners.
#

acl SSL_ports port 443
acl Safe_ports port 80		# http
acl Safe_ports port 21		# ftp
acl Safe_ports port 443		# https
acl Safe_ports port 70		# gopher
acl Safe_ports port 210		# wais
acl Safe_ports port 1025-65535	# unregistered ports
acl Safe_ports port 280		# http-mgmt
acl Safe_ports port 488		# gss-http
acl Safe_ports port 591		# filemaker
acl Safe_ports port 777		# multiling http
acl CONNECT method CONNECT

# Squid enables PURGE only if some ACL names the method: cache_cf.cc derives
# Config2.onoff.enable_purge from the count of PURGE ACLs. Without this line
# every PURGE returns 403. Authorisation comes from "http_access allow
# localhost" below.
acl PURGE method PURGE

acl intermediate_fetching transaction_initiator certificate-fetching
http_access allow intermediate_fetching

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
http_access deny to_localhost

# The built-in localhost ACL covers both 127.0.0.1 and ::1; do not redefine it.
# This also authorises PURGE.
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all

http_port 127.0.0.1:3128 tcpkeepalive=60,30,3 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=20MB tls-cert=/opt/homebrew/etc/squid/squid-self-signed.crt tls-key=/opt/homebrew/etc/squid/squid-self-signed.key cipher=HIGH:MEDIUM:!LOW:!RC4:!SEED:!IDEA:!3DES:!MD5:!EXP:!PSK:!DSS options=NO_TLSv1,NO_SSLv3 tls-dh=prime256v1:/opt/homebrew/etc/squid/squid-self-signed_dhparam.pem
http_port [::1]:3128 tcpkeepalive=60,30,3 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=20MB tls-cert=/opt/homebrew/etc/squid/squid-self-signed.crt tls-key=/opt/homebrew/etc/squid/squid-self-signed.key cipher=HIGH:MEDIUM:!LOW:!RC4:!SEED:!IDEA:!3DES:!MD5:!EXP:!PSK:!DSS options=NO_TLSv1,NO_SSLv3 tls-dh=prime256v1:/opt/homebrew/etc/squid/squid-self-signed_dhparam.pem

acl step1 at_step SslBump1

# Hosts to tunnel rather than decrypt. Nothing on them is cacheable, so bumping
# would only add certificate generation and latency to the runner's critical
# path.
#
# The Actions control plane -- job broker long-polls, token exchange, health
# checks, result submission -- is matched by domain because GitHub adds
# hostnames under it regularly. Everything cacheable lives on
# blob.core.windows.net or *.githubusercontent.com instead.
#
# github.com serves only /info/refs (no-cache, must-revalidate),
# git-upload-pack (a POST), and /releases/download/ redirects whose signed
# target changes on every request. The release bytes come from
# release-assets.githubusercontent.com, which is bumped and cached.
#
# Splicing requires every client to trust the public roots as well as this CA;
# see the note on ca-bundle.pem in the README.
#
# Squid compiles these with POSIX regcomp(3): use [0-9], not \d or \w.
acl github_controlplane ssl::server_name_regex \.actions\.githubusercontent\.com$
acl github_git ssl::server_name github.com

sslcrtd_program /opt/homebrew/opt/squid/libexec/security_file_certgen -s /opt/homebrew/var/logs/ssl_db -M 20MB
sslcrtd_children 5
ssl_bump peek step1
ssl_bump splice github_controlplane
ssl_bump splice github_git
ssl_bump stare all
sslproxy_cert_error deny all

# macOS caps shm_open() names at PSHMNAMLEN (31) characters, and Squid's shared
# TLS session cache segment is "/squid-XXXX-tls_session_cache.shm" (33). Any
# TLS port aborts at startup without this.
sslproxy_session_cache_size 0

# Stall a second request for an object already being fetched rather than
# fetching it twice. Worth it when the objects are large.
collapsed_forwarding on

# Maps differently-signed URLs for the same object onto one cache key. The
# helper answers on a channel ID, so a few children with concurrency= serve
# many parallel lookups.
store_id_program /opt/homebrew/etc/squid/github_store_id_helper.py
store_id_children 10 startup=2 idle=2 concurrency=10

maximum_object_size 2000 MB
cache_dir aufs /opt/homebrew/var/cache/squid 100000 16 256

# Leave coredumps in the first cache dir
coredump_dir /opt/homebrew/var/cache/squid

# Fetch and cache the whole object even when the client asks for a byte range.
# Actions cache blobs arrive as long runs of sequential range requests; without
# this every job re-downloads the object from Azure. Needs the patched Squid so
# the background download survives the client going away.
acl azure_storage dstdomain .blob.core.windows.net
acl github_release_assets dstdomain release-assets.githubusercontent.com
range_offset_limit -1 azure_storage
range_offset_limit -1 github_release_assets

# Finish a download whose client walked away: it lands in the cache for the
# next job. Uncachable and private responses are still aborted, because
# CheckQuickAbortIsReasonable() checks those first.
quick_abort_min -1 KB

# The macOS default of 256 is far too low.
max_filedescriptors 4096

# Keeps error pages and Via headers deterministic.
visible_hostname runner-cache

# The native "squid" format plus the error code, so a failed transaction says
# why in the log instead of only "503". The first ten fields are unchanged, so
# anything parsing the standard format still works.
logformat squid-err %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt %err_code/%err_detail

# Set explicitly so these cannot disagree with the rotation LaunchAgent.
access_log /opt/homebrew/var/logs/access.log squid-err
cache_log /opt/homebrew/var/logs/cache.log

# "squid -k rotate" (org.squid-cache.logrotate.plist) keeps this many old copies.
logfile_rotate 5

#
# Add any of your own refresh_pattern entries above these.
#

# These match the *store ID* -- the query-stripped URL -- not the request URL,
# because store_id_program is in use. "(:[0-9]+)?" tolerates a non-default port.
#
# override-lastmod matters as much as override-expire here. Azure sends these
# blobs with a Last-Modified but no Cache-Control, so Squid falls back to the
# last-modified factor: freshness = 20% of the object's age. A cache blob read
# minutes after it was uploaded is therefore stale within seconds, and every
# subsequent range request revalidates. override-lastmod enforces min instead,
# which is what we want for content whose URL already identifies it uniquely.

# Actions cache blobs. Match any storage account: GitHub moves Actions storage
# between accounts, and pinning one silently disables caching when it changes.
# Blob names under actions-cache are content-addressed and immutable.
refresh_pattern -i ^https://[a-z0-9]+\.blob\.core\.windows\.net(:[0-9]+)?/actions-cache/	1440	20%	10080	ignore-reload ignore-no-store ignore-private override-expire override-lastmod

# Release assets, on both the current and the legacy hostname.
refresh_pattern -i ^https://release-assets\.githubusercontent\.com(:[0-9]+)?/github-production-release-asset/	1440	20%	10080	ignore-reload ignore-no-store ignore-private override-expire override-lastmod
refresh_pattern -i ^https://objects\.githubusercontent\.com(:[0-9]+)?/github-production-release-asset-	1440	20%	10080	ignore-reload ignore-no-store ignore-private override-expire override-lastmod

# Action tarballs, pinned by commit SHA and therefore immutable.
refresh_pattern -i ^https://codeload\.github\.com(:[0-9]+)?/.*/(tar\.gz|zip)/	1440	20%	10080	ignore-reload ignore-no-store ignore-private override-expire override-lastmod

refresh_pattern -i ^https?://storage\.googleapis\.com/	1440	20%	10080	ignore-reload ignore-no-store ignore-private override-expire override-lastmod

# Source tarballs from the upstream mirrors the build pulls from.
refresh_pattern -i \.(gz|xz|bz2|tar|zip)$	1440	20%	10080	ignore-reload ignore-no-store ignore-private override-expire override-lastmod

refresh_pattern ^ftp:		1440	20%	10080
refresh_pattern ^gopher:	1440	0%	1440
refresh_pattern .		0	20%	4320

# Squid will not store a response to a request carrying Authorization unless it
# is Cache-Control: public/must-revalidate (RFC 9111 3.5), and no
# refresh_pattern option overrides that. The runner sends a token to codeload
# even for public actions, so those tarballs are relayed but not cached.

# Headroom for large artifact uploads.
client_request_buffer_max_size 1 GB

squid.rb

class Squid < Formula
  desc "Advanced proxy caching server for HTTP, HTTPS, FTP, and Gopher"
  homepage "https://www.squid-cache.org/"
  url "https://github.com/squid-cache/squid/releases/download/SQUID_7_6/squid-7.6.tar.bz2"
  sha256 "29e6d2fcffbbbff0052c5a6a24a09f93c9b934fa95c0629ef2251e64ff8ff8da"
  license "GPL-2.0-or-later"

  # Upstream sometimes creates releases that use a stable tag (e.g., `v1.2.3`)
  # but are labeled as "pre-release" on GitHub, so it's necessary to use the
  # `GithubLatest` strategy.
  livecheck do
    url :stable
    regex(/^SQUID[._-]v?(\d+(?:[._]\d+)+)$/i)
    strategy :github_latest do |json, regex|
      json["tag_name"]&.[](regex, 1)&.tr("_", ".")
    end
  end

  # No `bottle do` block on purpose: homebrew-core's bottles are built from
  # unpatched sources, and Homebrew would pour one instead of applying the
  # patch below. Its absence forces a source build.

  head do
    url "https://github.com/squid-cache/squid.git", branch: "master"

    depends_on "autoconf" => :build
    depends_on "automake" => :build
    depends_on "libtool" => :build
  end

  depends_on "openssl@3"

  uses_from_macos "libxcrypt"

  def install
    # https://stackoverflow.com/questions/20910109/building-squid-cache-on-os-x-mavericks
    ENV.append "LDFLAGS", "-lresolv"

    # For --disable-eui, see:
    # https://www.squid-cache.org/mail-archive/squid-users/201304/0040.html
    args = %W[
      --localstatedir=#{var}
      --sysconfdir=#{etc}
      --enable-ssl
      --enable-ssl-crtd
      --disable-eui
      --with-included-ltdl
      --with-gnutls=no
      --with-nettle=no
      --with-openssl
      --enable-delay-pools
      --enable-disk-io=yes
      --enable-removal-policies=yes
      --enable-storeio=yes
    ]

    args << "--enable-pf-transparent" if OS.mac?

    system "./bootstrap.sh" if build.head?
    system "./configure", *args, *std_configure_args
    system "make", "install"
  end

  service do
    run [opt_sbin/"squid", "-N", "-d 1"]
    keep_alive true
    working_dir var
    log_path var/"log/squid.log"
    error_log_path var/"log/squid.log"
  end

  test do
    assert_match version.to_s, shell_output("#{sbin}/squid -v")

    pid = spawn sbin/"squid"

    begin
      sleep 2
      system sbin/"squid", "-k", "check"
    ensure
      system sbin/"squid", "-k", "interrupt"
      Process.wait(pid)
    end
  end

  patch :DATA
end
__END__
Squid 7.6: cache GitHub Actions / Azure Storage downloads behind ssl-bump
=========================================================================

Three fixes, all needed before a range-heavy workload (actions/cache, GitHub
release assets) can be cached by a bumping proxy.

1. src/client_side_request.cc -- interpret Azure's "x-ms-range" header

   Azure Storage clients put their byte range in a proprietary x-ms-range
   header. Squid does not recognise it, so it forwards the header upstream and
   caches at best the single range the client asked for. It also never learns
   the request was ranged, which costs it the range_offset_limit exemption in
   CheckQuickAbortIsReasonable().

   The header is rewritten into a standard Range header in
   clientInterpretRequestHeaders(), before anything else looks at it, so range
   parsing, range_offset_limit, quick_abort, 206 assembly and upstream Range
   suppression all work unmodified. Values Squid cannot parse are forwarded
   untouched, leaving their meaning to the origin server.

2. src/client_side.cc, src/client_side.h -- do not close a pinned connection a
   Client is still using

   With ssl_bump the to-origin connection is pinned to the client connection,
   and ConnStateData::swanSong() closed it unconditionally; upstream marks that
   line "XXX: Closing pinned conn is too harsh: The Client may want to
   continue!". Under "range_offset_limit none" it aborts the whole-object
   download the moment the client has its range, so nothing is ever cached.

   A connection currently borrowed by a server-side transaction is handed over
   rather than closed. Whether the orphaned transaction continues is
   quick_abort's decision (CheckQuickAbortIsReasonable), which already
   understands range_offset_limit; when it says no, StoreEntry::abort() tears
   the connection down through FwdState::HandleStoreAbort(). The transaction
   closes the socket itself when it finishes, in
   HttpStateData::processReplyBody().

3. src/HttpHeader.cc -- ignore Content-Length when applying a 304

   Azure answers revalidation with "304 Not Modified" plus "Content-Length: 0",
   describing the empty 304 rather than the stored representation, which RFC
   9110 Section 8.6 forbids. Squid merges it into the cached reply, after which
   every hit returns 200 with an empty body. The stored Content-Length is kept
   instead: it matches the stored body whether or not the origin is compliant.

   Also fixes HttpHeader::putRange(), which omitted the "bytes=" range-unit and
   so emitted a Range header Squid itself cannot parse.

--- a/src/HttpHeader.cc
+++ b/src/HttpHeader.cc
@@ -260,7 +260,14 @@
     return
         // TODO: Consider updating Vary headers after comparing the magnitude of
         // the required changes (and/or cache losses) with compliance gains.
-        (id == Http::HdrType::VARY);
+        (id == Http::HdrType::VARY) ||
+        // RFC 9110 Section 8.6 forbids a 304 Content-Length that disagrees with
+        // the length of the stored representation, but some origins (notably
+        // Azure Storage) send "Content-Length: 0" describing the empty 304
+        // itself. Applying that would make the cached response claim a body
+        // length it does not have, so keep the length we actually stored: it
+        // matches the stored body in both the compliant and the buggy case.
+        (id == Http::HdrType::CONTENT_LENGTH);
 }
 
 void
@@ -1045,6 +1052,9 @@
     /* pack into mb */
     MemBuf mb;
     mb.init();
+    /* HttpHdrRange packs a bare range-set; the field also needs its range-unit
+     * (RFC 9110 Section 14.2), without which Squid cannot re-parse what it wrote */
+    mb.append("bytes=", 6);
     range->packInto(&mb);
     /* put */
     addEntry(new HttpHeaderEntry(Http::HdrType::RANGE, SBuf(), mb.buf));
--- a/src/client_side.cc
+++ b/src/client_side.cc
@@ -598,8 +598,16 @@
     terminateAll(ERR_NONE, LogTagsErrors());
     checkLogging();
 
-    // XXX: Closing pinned conn is too harsh: The Client may want to continue!
-    unpinConnection(true);
+    // Closing a pinned connection that a Client is still using would abort that
+    // server-side transaction, even when Squid wants it to finish: with
+    // "range_offset_limit none", for example, the transaction keeps downloading
+    // a whole object into the cache after the client got the range it asked
+    // for. Whether an orphaned transaction should continue is quick_abort's
+    // decision (see CheckQuickAbortIsReasonable()); when it says no,
+    // StoreEntry::abort() closes this connection via FwdState::HandleStoreAbort().
+    // Either way the Client owns the connection now and closes it when done
+    // (see HttpStateData::processReplyBody()), so hand it over instead.
+    unpinConnection(!pinnedConnectionIsBusy());
 
     Server::swanSong();
 
@@ -3874,6 +3882,15 @@
     // there is no point since the client connection is now gone
     HttpRequestPointer requestPointer = request;
     throw ErrorState::NewForwarding(ERR_CANNOT_FORWARD, requestPointer, ale);
+}
+
+bool
+ConnStateData::pinnedConnectionIsBusy() const
+{
+    // startPinnedConnectionMonitoring() runs only while the connection is idle
+    // and borrowPinnedConnection() stops it, so a missing readHandler means the
+    // connection has been handed to a Client (or is about to be).
+    return Comm::IsConnOpen(pinning.serverConnection) && !pinning.readHandler;
 }
 
 void
--- a/src/client_side.h
+++ b/src/client_side.h
@@ -196,6 +196,11 @@
     /// Undo pinConnection() and, optionally, close the pinned connection.
     void unpinConnection(const bool andClose);
 
+    /// Whether a server-side transaction is currently using the pinned
+    /// connection. We only monitor a pinned connection while it sits idle, so
+    /// an open-but-unmonitored one has been borrowed by a Client.
+    bool pinnedConnectionIsBusy() const;
+
     /// \returns validated pinned to-server connection, stopping its monitoring
     /// \throws a newly allocated ErrorState if validation fails
     static Comm::ConnectionPointer BorrowPinnedConnection(HttpRequest *, const AccessLogEntryPointer &);
--- a/src/client_side_request.cc
+++ b/src/client_side_request.cc
@@ -40,6 +40,7 @@
 #include "http.h"
 #include "http/Stream.h"
 #include "HttpHdrCc.h"
+#include "HttpHeaderRange.h"
 #include "HttpReply.h"
 #include "HttpRequest.h"
 #include "internal.h"
@@ -75,6 +76,8 @@
 #include "ssl/support.h"
 #endif
 
+#include <memory>
+
 #if FOLLOW_X_FORWARDED_FOR
 
 #if !defined(SQUID_X_FORWARDED_FOR_HOP_MAX)
@@ -883,6 +886,44 @@
     }
 }
 
+/// Proprietary request header used by Azure Storage clients (e.g. the Azure
+/// SDK behind actions/cache) to express a byte range. Azure honours it in
+/// preference to the standard Range header when a request carries both.
+static const SBuf AzureRangeHeaderName("x-ms-range");
+
+/// Rewrites an Azure x-ms-range request header into an equivalent standard
+/// Range header so that the rest of Squid -- range parsing, range_offset_limit,
+/// the quick_abort exemption for full-object downloads, 206 assembly and
+/// upstream Range suppression -- handles these requests like any other range
+/// request.
+///
+/// A value Squid cannot parse is forwarded untouched, leaving its meaning to
+/// the origin server. Squid then keeps managing any Range header as before,
+/// which is the best available outcome: origins that ignore x-ms-range still
+/// get a whole-object request they can be cached from, and origins that honour
+/// it would have overridden that Range header anyway.
+/// \returns whether request headers were modified
+static bool
+clientInterpretAzureRange(HttpHeader &hdr)
+{
+    String value;
+    if (!hdr.hasNamed(AzureRangeHeaderName, &value))
+        return false;
+
+    const std::unique_ptr<HttpHdrRange> parsed(HttpHdrRange::ParseCreate(&value));
+    if (!parsed) {
+        debugs(85, 3, "forwarding unparsable " << AzureRangeHeaderName << ": " << value);
+        return false;
+    }
+
+    // Azure gives x-ms-range priority over Range, so drop any competing spec.
+    hdr.delByName(AzureRangeHeaderName);
+    hdr.delById(Http::HdrType::REQUEST_RANGE);
+    hdr.putRange(parsed.get()); // also removes any old Range header
+    debugs(85, 3, "interpreting " << AzureRangeHeaderName << " as Range: " << value);
+    return true;
+}
+
 static void
 clientInterpretRequestHeaders(ClientHttpRequest * http)
 {
@@ -925,6 +966,11 @@
 
     /* ignore range header in non-GETs or non-HEADs */
     if (request->method == Http::METHOD_GET || request->method == Http::METHOD_HEAD) {
+        // Any range parsed by HttpRequest::hdrCacheInit() predates this
+        // rewrite and now describes a header we just removed.
+        if (clientInterpretAzureRange(*req_hdr))
+            request->ignoreRange("x-ms-range supersedes Range");
+
         // XXX: initialize if we got here without HttpRequest::parseHeader()
         if (!request->range)
             request->range = req_hdr->getRange();
@@ -949,6 +995,7 @@
     else {
         req_hdr->delById(Http::HdrType::RANGE);
         req_hdr->delById(Http::HdrType::REQUEST_RANGE);
+        req_hdr->delByName(AzureRangeHeaderName);
         request->ignoreRange("neither HEAD nor GET");
     }
 

Is there a docker image somewhere that would make it very easy to be used?