Standalone script that mimics GitHub Actions' hashFiles helper

hash-files.md

hash-files.py

Standalone script that mimics GitHub Actions' hashFiles helper (used by actions/setup-python, and frequently with actions/cache)

Upstream hashFiles source is here, this script was written while debugging setup-python#919.

Examples

git clone https://gist.github.com/ryan-williams/2e616363c68072bf67cad675fcb18f3b hashfiles-gist
export PATH=$PATH:$(pwd)/hashfiles-gist

hash-files.py test.txt
# 87183f21076ad2e29f35c62196150ed17c6bdd890bf2617188143ef9d63bec96

hash-files.py test*.txt
# 1a4d74031d9ffdb900f56611fd112163d8abdd4d913eb071e899ae48d8d92617

hash-files.py test1.txt test2.txt  # same as above
# 1a4d74031d9ffdb900f56611fd112163d8abdd4d913eb071e899ae48d8d92617

It just sha256sums all provided files, then sha256sums those hashes (as binary data), outputting the result as a hex string.

You can also directly check a specific version of a file:

hash-files.py <(curl -s https://raw.githubusercontent.com/single-cell-data/TileDB-SOMA/274c6f49766e19fa645d93cc99e213d0dcca372f/apis/python/setup.py)
# 0bc0ed4dc8ce9f9b48ebc42fa10052206992eb7a7705c7a75ec8fcbe15e3e65f

which matches the hash used by setup-python in this GHA:

Cache restored from key: setup-python-Linux-22.04-Ubuntu-python-3.11.9-pip-0bc0ed4dc8ce9f9b48ebc42fa10052206992eb7a7705c7a75ec8fcbe15e3e65f

hash-files.py

#!/usr/bin/env python
import hashlib
import sys


sha256 = hashlib.sha256()

for path in sys.argv[1:]:
    file_sha256 = hashlib.sha256()
    with open(path, "rb") as f:
        for byte_block in iter(lambda: f.read(4096), b""):
            file_sha256.update(byte_block)
    sha256.update(file_sha256.digest())

print(sha256.hexdigest())

test1.txt

1
2
3
4
5
6
7
8
9
10

test2.txt

11
12
13
14
15
16
17
18
19
20