git checkout master # setup Dockerfile
docker-compose build # uses cache (expected)
git checkout HEAD^1 # removes some line from Dockerfile
docker-compose build # uses cache (expected)
git checkout master # adds back same line to Dockerfile
docker-compose build --no-cache # does not use input from the cache (expected)
git checkout HEAD^1 # removes same line from Dockerfile
docker-compose build # has unexpected cache miss
possibly related? moby/moby#3199
"Use the most recent image from the build cache" is implemented here: https://github.com/docker/docker/blob/415dd8688618ac2b3872c6a5b5af3afcef2c3e10/daemon/daemon.go#L1363
"Use the most recent image from the build cache" will fail because the last known good build may have used a different image. It's not enough to sort by build cache item recency. Cache miss event should cause the builder to backtrack and try the next most recent available matching image, until it exhausts all available image options, at which point it should backtrack to the previous Dockerfile step and repeat the exhaustive search.
"Use the most recent image from the build cache" will fail after using --no-cache because "last known most recently matching image" will naturally not have any relevant child images in the build cache. The "last known most recently matching image" will be whatever image was recently created during --no-cache, and therefore have none of the children from previous cached builds. Meanwhile the build cache is still present and available and has useful things inside.
.... turns out that I am not using "--no-cache" anyway, but still seeing lots of cache misses.
There should be more tools to display the status of the docker build cache. There should also be a tool to do a "dry run" of a docker build, which would display information about the docker build cache, without actually manipulating the docker build cache and without running an actual build.
"faster cache miss detection"
moby/moby#16821
"output reason for docker cache invalidation"
moby/moby#9294
Reported this upstream to docker, moby/moby#20304
"docker build --no-cache does not consistently invalidate the cache for subsequent builds"
moby/moby#3199
"use most recent image from the cache"
moby/moby#3375
moby/moby#3417
cache busting behavior problems?
moby/moby#9880
moby/moby#9973