possible for incorrect selection of cached python hook environments across projects (~/.cache/pre-commit/abcdCachedHookDir) that use different python versions
suppose you have 2 projects, project-a and project-b, configured for python3.7 and python3.10 respecitvely.
Each project has been configured with an identical .pre-commit-config.yaml file:
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.2.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-filesrev: v3.2.0 is harded coded to a crusty version in pre-commit (current is v4.3.0).
This setup actually works fine for this pre-commit-config.yaml for most if not all cases, the issue is more likely to be encountered
for configurations with things like black (psf/black#3144) where the python version used by the tool matters.
Going through my notes I'm guessing this to be the cause of "heisenbug" problems with pre-commit when trying to help other/new devs setup their
own environments and where deleting ~/.cache/pre-commit and starting fresh solves the problem.
More likely to occur when:
- working with many repos (>100, mainly aws-lambda python projects on random fucking versions)
- Needing to support M1 and Intel mac?
- Custom python precommit hooks
- Dealing with and wanting to avoid homebrew at all costs 🤮
- pyenv 😑
Can this be done while keeping the file the same? Keeping
Setting the default language version in project-b in the pre-commit-config file correctly(?) creates an additional virtualvenv
# .pre-commit-config.yaml
default_language_version:
python: python3.10(venv) deric4@Derics-MacBook-Pro nsfw-packer-images % ls -la ~/.cache/pre-commit/repokq61vkic/py_env-python3*
/Users/deric4/.cache/pre-commit/repokq61vkic/py_env-python3:
total 24
drwxr-xr-x 8 deric4 staff 256 Jun 30 20:49 .
drwx------ 21 deric4 staff 672 Jun 30 20:52 ..
lrwxr-xr-x 1 deric4 staff 93 Jun 30 20:49 .Python -> /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/Python3
-rw-r--r-- 1 deric4 staff 40 Jun 30 20:49 .gitignore
-rw-r--r-- 1 deric4 staff 31 Jun 30 20:49 .install_state_v1
drwxr-xr-x 50 deric4 staff 1600 Jun 30 20:49 bin
drwxr-xr-x 3 deric4 staff 96 Jun 30 20:49 lib
-rw-r--r-- 1 deric4 staff 528 Jun 30 20:49 pyvenv.cfg
# !!! doesn't exist until specifying `default_language_version`
/Users/deric4/.cache/pre-commit/repokq61vkic/py_env-python3.10:
total 24
drwxr-xr-x 7 deric4 staff 224 Jun 30 20:52 .
drwx------ 21 deric4 staff 672 Jun 30 20:52 ..
-rw-r--r-- 1 deric4 staff 40 Jun 30 20:52 .gitignore
-rw-r--r-- 1 deric4 staff 31 Jun 30 20:52 .install_state_v1
drwxr-xr-x 50 deric4 staff 1600 Jun 30 20:52 bin
drwxr-xr-x 3 deric4 staff 96 Jun 30 20:52 lib
-rw-r--r-- 1 deric4 staff 390 Jun 30 20:52 pyvenv.cfgalso doesn't create namespace/recognize(?) for when using python3.10-intel
$ python3.10-intel64 -c 'import platform; print(f"{platform.machine()}\n{platform.version()}")'
x86_64
Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000$ python3.10 -c 'import platform; print(f"{platform.machine()}\n{platform.version()}")'
arm64
Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000This project gets created first
$ python3.7 -m venv venv
$ source venv/bin/activate
$ pip install pre-commit
$ pre-commit install --install-hooksDuring the setup for project-a, pre-commit/pre-commit-hooks is downloaded and installed to ~/.cache/pre-commit/repoXXXXXX with its own virtualenv.
During the setup of project-b however, the installation of hooks is essentially instant since pre-commit sees all the hooks are already installed (based only on repo and rev?)
$ python3.10 -m venv venv
$ source venv/bin/activate
$ pip install pre-commit
$ pre-commit install --install-hooks$GIT_DIR/.git/hooks/pre-commit
#!/usr/bin/env bash
# File generated by pre-commit: https://pre-commit.com
# ID: 138fd403232d2ddd5efb44317e38bf03
# start templated
INSTALL_PYTHON=/Users/deric4/projects/github.com/deric4/testing/venv/bin/python3
ARGS=(hook-impl --config=.pre-commit-config.yaml --hook-type=pre-commit)
# end templated
HERE="$(cd "$(dirname "$0")" && pwd)"
ARGS+=(--hook-dir "$HERE" -- "$@")
if [ -x "$INSTALL_PYTHON" ]; then
exec "$INSTALL_PYTHON" -mpre_commit "${ARGS[@]}"
elif command -v pre-commit > /dev/null; then
exec pre-commit "${ARGS[@]}"
else
echo '`pre-commit` not found. Did you forget to activate your virtualenv?' 1>&2
exit 1
fiwould be nice if pre-commit allowed GIT_TRACE* and GIT_CURL_VERBOSE
similar: pre-commit/pre-commit#2420
psf/black#751 psf/black#3143 psf/black#3144
TODO
TODO
TODO
local system details
$ python3 -c 'import platform; print(platform.platform(), platform.version())'
macOS-12.3.1-arm64-arm-64bit Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000
$ system_profiler -detailLevel mini SPHardwareDataType
Hardware:
Hardware Overview:
Model Name: MacBook Pro
Model Identifier: MacBookPro18,2
Chip: Apple M1 Max
Total Number of Cores: 10 (8 performance and 2 efficiency)
Memory: 64 GB
System Firmware Version: 7459.101.3
OS Loader Version: 7459.101.3
$ pre-commit -V
pre-commit 2.19.0