Skip to content

Instantly share code, notes, and snippets.

@Himan10
Created April 20, 2026 03:22
Show Gist options
  • Select an option

  • Save Himan10/a00927fb1cb1cc19b1f7097cd24046f4 to your computer and use it in GitHub Desktop.

Select an option

Save Himan10/a00927fb1cb1cc19b1f7097cd24046f4 to your computer and use it in GitHub Desktop.
Supply Chain Security for npm & PyPI on GitLab

Supply Chain Security for npm & PyPI on GitLab


1. Threat Model

Supply chain attacks hit at four distinct points:

[1] Package manifest       — typosquatting, unpinned versions
[2] Package download       — compromised packages, MITM, dependency confusion
[3] CI/CD pipeline         — malicious install scripts, compromised runner images
[4] Runtime                — transitive dependency vulnerabilities, known CVEs

2. Package Manifest Hygiene

Python — requirements.txt

Pattern Safe? Reason
requests==2.31.0 Yes Exact pin
requests>=2.28.0 No Allows any future version
requests~=2.28.0 No Compatible release — allows patch upgrades
requests!=2.29.0 No Excludes one version, rest unrestricted
requests No No version constraint at all

Hash verification is required alongside pinning:

# Pinned but not hash-verified — insufficient
requests==2.31.0

# Pinned + hash-verified — safe
requests==2.31.0 \
    --hash=sha256:58cd2187... \
    --hash=sha256:942c5a7...

A package at ==2.31.0 can be replaced on PyPI (the author can delete and re-upload). Hash verification catches this even when the version number matches.

npm — package.json

Pattern Safe? Reason
"4.17.21" Yes Exact pin
"^4.17.21" No Allows minor + patch upgrades
"~4.17.21" No Allows patch upgrades
">=4.0.0" No Unrestricted upper bound
"*" or "latest" No Anything goes
"4.17.x" No Wildcard patch

3. Generating Hashes for requirements.txt

Option 1 — pip-tools (recommended)

pip install pip-tools

# requirements.in contains abstract deps (no versions/hashes)
pip-compile --generate-hashes requirements.in
# Outputs requirements.txt with exact versions + hashes for all deps including transitive

Option 2 — uv

uv pip compile --generate-hashes requirements.in -o requirements.txt

Option 3 — hashin (adds hashes to existing file in-place)

pip install hashin
hashin requests==2.31.0 -r requirements.txt
# Fetches ALL platform wheel hashes — important for cross-platform compatibility

Platform-Specific Wheel Problem

Binary packages like psycopg2-binary ship different compiled wheels per platform. Generating hashes on macOS produces macOS hashes — Linux CI will fail with a mismatch.

Fix: generate hashes inside a container matching your CI environment:

docker run --rm -v $(pwd):/app -w /app python:3.12-slim bash -c \
  "pip install pip-tools && \
   apt-get update -qq && apt-get install -y libpq-dev gcc && \
   pip-compile --generate-hashes requirements.in"

Or use hashin — it fetches hashes for all platform variants at once.

Verifying Hash Enforcement

pip install --require-hashes -r requirements.txt
# Fails immediately if any package hash does not match

4. Why package-lock.json Is Required

What package.json Cannot Do

package.json only records intent, not resolution. "lodash": "^4.17.0" means "lodash 4.17.0 or any compatible version." Two installs on different days can resolve to different versions. It also says nothing about transitive dependencies — the dependencies of your dependencies.

What the Lockfile Records

"node_modules/lodash": {
  "version": "4.17.21",
  "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz",
  "integrity": "sha512-v2kDE8NMa2xtoBMaFyBMgBO+..."
}
Field What it locks
version Exact version — not a range
resolved Exact URL the tarball must come from
integrity SHA-512 hash of the tarball content

Attacks It Defeats

Malicious new version published:

Attacker publishes lodash@4.17.22 with malware.
package.json says "^4.17.0" — 4.17.22 satisfies this.

npm install  → resolves to 4.17.22 → malware installed
npm ci       → lockfile says 4.17.21 → 4.17.22 rejected

Tarball replacement at the registry:

Attacker replaces the 4.17.21 tarball with malicious content.

npm ci → downloads tarball → computes SHA-512
SHA-512 doesn't match integrity field → installation fails

Transitive dependency compromise:

You depend on "express". Express depends on "qs".
Attacker publishes malicious qs@6.11.1.

Without lockfile: qs resolved silently → malware in build
With lockfile: qs@6.11.0 pinned with hash → 6.11.1 rejected

Dependency confusion:

Attacker publishes "@company/internal-lib" to public npm with higher version.

Without lockfile: public npm wins (higher version)
With lockfile + scoped .npmrc: resolved URL is your internal registry → public npm never consulted

npm ci vs npm install

Behaviour npm install npm ci
Reads lockfile Sometimes Always
Updates lockfile Yes Never
Fails on version mismatch No Yes
Deletes node_modules first No Yes — clean install
Verifies integrity hashes Partially Always

Always use npm ci in CI pipelines.

Lockfile Tampering Risk

An attacker with repo write access could modify package-lock.json to point to a malicious tarball with a matching hash. Protect against this with CODEOWNERS:

# CODEOWNERS
package-lock.json   @security-team
package.json        @security-team
requirements*.txt   @security-team
uv.lock             @security-team

5. Download-Time Defenses

Dependency Confusion

Internal packages named company-utils can be targeted by attackers who publish the same name to PyPI/npm at a higher version. The package manager resolves the public one.

pip fix: use --index-url (not --extra-index-url). --extra-index-url falls back to PyPI for packages not in your private registry — the confusion vector.

# pip.conf
[global]
index-url = https://your-nexus.internal/repository/pypi-proxy/simple/

npm fix: scope internal packages and lock the scope to your registry:

# .npmrc
@company:registry=https://gitlab.your-company.com/api/v4/packages/npm/
registry=https://registry.npmjs.org/

Private Registry / Proxy

Route all downloads through an internal proxy (Nexus, Artifactory, GitLab Package Registry):

  • Caches verified packages
  • Central blocklist for known-bad packages
  • Full audit trail of every download
  • License policy enforcement

6. CI/CD Pipeline Defenses

Runner Image Pinning by Digest

A Docker image tag (python:3.12) is a mutable pointer — it can be silently updated. A digest is the SHA-256 hash of the image manifest — immutable and content-addressed.

# Mutable — compromised tag update affects every build silently
image: python:3.12

# Immutable — exactly this image, forever
image: python:3.12@sha256:a7f8c9d4e2b13f6...

If the runner image is compromised, it controls the environment where pip install and npm install run — hash verification, SSL libs, pip itself. The image is the root of trust for the entire pipeline.

Get the digest:

docker pull python:3.12
docker inspect python:3.12 --format='{{index .RepoDigests 0}}'

Safe Install Commands in CI

install_python:
  script:
    - pip install
        --require-hashes
        --no-deps
        --no-build-isolation
        --only-binary=:all:
        -r requirements.txt

install_node:
  script:
    - npm ci
        --ignore-scripts
        --audit
        --strict-ssl
        --no-fund

--ignore-scripts on npm blocks postinstall/preinstall hooks — a common arbitrary code execution vector in malicious packages.

--only-binary=:all: on pip refuses to build any package from source — no untrusted C code compiled at install time.

Audit as a Blocking CI Step

security_audit:
  script:
    - pip install pip-audit
    - pip-audit -r requirements.txt --fail-on-vuln
    - npm audit --audit-level=high
  allow_failure: false

7. GitLab-Native Controls

# .gitlab-ci.yml
include:
  - template: Security/Dependency-Scanning.gitlab-ci.yml
  - template: Security/License-Scanning.gitlab-ci.yml
  - template: Security/Secret-Detection.gitlab-ci.yml
  - template: Security/SAST.gitlab-ci.yml
Template What it scans Tool
Dependency-Scanning Known CVEs in lockfiles (direct + transitive) Gemnasium / Trivy
License-Scanning GPL/LGPL licenses violating policy Gemnasium
Secret-Detection Hardcoded credentials, API keys Gitleaks
SAST Static vulnerabilities in your own code Semgrep / Bandit

GitLab Dependency Scanning reads resolved lockfiles — transitive dependencies are covered.


8. Vulnerability Databases & Delta Polling System

Data Sources

Source Covers Strength
OSV (osv.dev) PyPI, npm, Go, Rust, Maven Free, machine-readable, covers CVEs + Google-curated findings
GitHub Advisory DB PyPI, npm, Maven Powers Dependabot, GraphQL API
Aikido Intel PyPI, npm Catches actively malicious packages, fast (hours vs days)
NVD (NIST) All CVEs Canonical source — slow to update
OSS Index (Sonatype) PyPI, npm Free REST API, uses purl format

Aikido Intel vs OSV distinction: OSV tracks CVEs in legitimate packages. Aikido Intel also catches actively malicious packages — typosquatting, dependency confusion attacks, legitimate packages taken over by bad actors.

Twice-Daily Delta Polling Architecture

Every 12 hours:
  1. Fetch Aikido Intel — findings since last_polled_at timestamp only
  2. Compare new findings against current GitLab package inventory
  3. Report newly affected packages to teams
  4. Update last_polled_at

No database needed — only a persisted timestamp between runs.

Persist last-polled timestamp in a GitLab CI variable:

def get_last_polled(gitlab_token, project_id, var_name):
    resp = httpx.get(
        f"https://gitlab.com/api/v4/projects/{project_id}/variables/{var_name}",
        headers={"PRIVATE-TOKEN": gitlab_token}
    )
    if resp.status_code == 200:
        return resp.json()["value"]
    # First run — default to 12 hours ago
    return datetime.now(timezone.utc).replace(hour=0, minute=0).isoformat()

Fetch only new findings:

def fetch_new_findings(api_key, since):
    resp = httpx.get(
        "https://intel.aikido.dev/api/...",
        headers={"Authorization": f"Bearer {api_key}"},
        params={"since": since}
    )
    return resp.json().get("findings", [])

Build GitLab package inventory via API (no cloning needed):

def build_package_inventory(gl):
    inventory = []
    dep_files = {
        "requirements.txt": "pypi",
        "package-lock.json": "npm",
        "uv.lock": "pypi",
    }
    for project in gl.projects.list(all=True, archived=False):
        for filename, ecosystem in dep_files.items():
            try:
                raw = project.files.get(filename, ref="main").decode()
                packages = parse_lockfile(raw, ecosystem)
                for pkg in packages:
                    inventory.append({
                        "project": project.path_with_namespace,
                        "ecosystem": ecosystem,
                        "file": filename,
                        **pkg
                    })
            except gitlab.exceptions.GitlabGetError:
                pass
    return inventory

GitLab scheduled pipeline — runs at 08:00 and 20:00 UTC:

aikido_delta_scan:
  image: python:3.12@sha256:...
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
  script:
    - pip install --require-hashes -r requirements.txt
    - python scripts/poll_and_compare.py

9. Detecting Unpinned / Unsafe Packages

Python — What to Check

Check Signal
No == operator Unpinned version
No --hash=sha256: Hash verification absent
--trusted-host in CI SSL verification disabled
--extra-index-url Dependency confusion vector
git+https:// or https://*.tar.gz Direct URL — no hash verification possible
pip install -e . in CI Editable install bypasses lockfile
Missing --no-deps with --require-hashes Transitive deps bypass hash check

npm — What to Check

Check Signal
^, ~, >=, *, latest in version Unpinned range
package-lock.json missing or in .gitignore Reproducible installs impossible
npm install in CI (not npm ci) Lockfile not enforced
Missing --ignore-scripts Install hooks execute freely
git+https:// or github: dependency Bypasses integrity verification
file: dependency Not reproducible in CI
strict-ssl=false in .npmrc SSL disabled

Automated Check Script

import re, json, sys
from pathlib import Path

UNSAFE_NPM_PREFIXES = re.compile(r'^[\^~><=*]|^latest$|^x$')
GIT_URL_PATTERN     = re.compile(r'git\+https?://|github:|gitlab:')

def check_python_requirements(path):
    issues = []
    lines = path.read_text().splitlines()
    has_any_hash = any("--hash=" in l for l in lines)

    for i, line in enumerate(lines, 1):
        line = line.strip()
        if not line or line.startswith("#") or line.startswith("-r"):
            continue
        if "--trusted-host" in line:
            issues.append(f"Line {i}: --trusted-host disables SSL")
        if "--extra-index-url" in line:
            issues.append(f"Line {i}: --extra-index-url — dependency confusion risk")
        if GIT_URL_PATTERN.search(line):
            issues.append(f"Line {i}: Direct URL/VCS install — no hash verification")
            continue
        if line.startswith("-"):
            continue
        if "==" not in line:
            issues.append(f"Line {i}: Unpinned: {line}")
    if not has_any_hash:
        issues.append("No hashes present — run pip-compile --generate-hashes")
    return issues

def check_npm_package_json(path):
    issues = []
    pkg = json.loads(path.read_text())
    for section in ["dependencies", "devDependencies", "optionalDependencies"]:
        for name, version in pkg.get(section, {}).items():
            if UNSAFE_NPM_PREFIXES.match(version):
                issues.append(f"{section}/{name}: Unpinned '{version}'")
            if GIT_URL_PATTERN.search(version):
                issues.append(f"{section}/{name}: Git dependency — no integrity check")
            if version.startswith("file:"):
                issues.append(f"{section}/{name}: Local file dep — not reproducible")
    if not (path.parent / "package-lock.json").exists():
        issues.append("package-lock.json missing")
    return issues

def check_ci_script(path):
    issues = []
    content = path.read_text()
    if "npm install" in content and "npm ci" not in content:
        issues.append("Use 'npm ci' instead of 'npm install' in CI")
    if "npm" in content and "--ignore-scripts" not in content:
        issues.append("npm used without --ignore-scripts")
    if "--trusted-host" in content:
        issues.append("--trusted-host disables SSL verification")
    if "--extra-index-url" in content:
        issues.append("--extra-index-url — dependency confusion risk")
    if "pip install" in content and "--require-hashes" not in content:
        issues.append("pip install without --require-hashes")
    return issues

10. Hardening npm and pip

npm — .npmrc

# .npmrc (commit to repo root)

strict-ssl=true
ignore-scripts=true
package-lock=true
save=false
audit=true
engine-strict=true

# Lock internal scope to private registry
@your-company:registry=https://gitlab.your-company.com/api/v4/packages/npm/
registry=https://registry.npmjs.org/
Setting Attack Closed
strict-ssl=true MITM — HTTP fallback stripped
ignore-scripts=true postinstall arbitrary code execution
package-lock=true Lockfile drift
audit=true CVEs silently ignored during install
Scoped registry Dependency confusion for internal packages

pip — pip.conf

# pip.conf

[global]
index-url = https://your-nexus.internal/repository/pypi-proxy/simple/
require-hashes = true
cert = /etc/ssl/certs/ca-certificates.crt
timeout = 30
disable-pip-version-check = true

[install]
only-binary = :all:

uv — uv.toml

[pip]
index-url = "https://your-nexus.internal/repository/pypi-proxy/simple/"
require-hashes = true
strict = true
only-binary = [":all:"]

[tool.uv]
python-version = ">=3.12,<3.13"

only-binary = :all: refuses to build any package from source — no untrusted C code compiled at install time. If no pre-built wheel exists, installation fails rather than silently compiling.

Environment Variables (for pipelines where config files can't be modified)

# pip
export PIP_REQUIRE_HASHES=1
export PIP_ONLY_BINARY=":all:"
export PIP_INDEX_URL="https://your-nexus.internal/..."

# npm
export NPM_CONFIG_STRICT_SSL=true
export NPM_CONFIG_IGNORE_SCRIPTS=true
export NPM_CONFIG_AUDIT=true

Set these as GitLab group-level CI/CD variables — they apply to every pipeline across all projects without touching individual .gitlab-ci.yml files.


11. Consolidated Best Practices Checklist

Package manifest
  [ ] All versions pinned exactly (no ranges)
  [ ] Hashes in requirements.txt / integrity in package-lock.json
  [ ] Lockfiles committed to the repository
  [ ] Lockfiles in CODEOWNERS requiring security team approval
  [ ] Internal packages scoped and pinned to private registry
  [ ] No git+https:// or direct URL dependencies

CI pipeline
  [ ] npm ci (not npm install)
  [ ] pip install --require-hashes --no-deps --only-binary=:all:
  [ ] --ignore-scripts on npm installs
  [ ] Runner base images pinned by digest (not tag)
  [ ] pip-audit / npm audit --audit-level=high as blocking steps
  [ ] GitLab Dependency Scanning template included
  [ ] GitLab Secret Detection template included

Package manager config
  [ ] .npmrc: strict-ssl, ignore-scripts, package-lock, scoped registry
  [ ] pip.conf: index-url (not extra-index-url), require-hashes, only-binary
  [ ] No --trusted-host or strict-ssl=false anywhere

Registry
  [ ] Internal proxy for PyPI and npm
  [ ] --index-url only (no --extra-index-url) in CI

Monitoring
  [ ] Twice-daily delta poll against Aikido Intel (or OSV)
  [ ] Cross-reference new findings against GitLab package inventory
  [ ] Alert teams on newly affected packages
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment