Purpose: Exhaustive implementor-level reference for building, migrating, and maintaining Pulumi Python infrastructure projects using Nix for hermetic dependency management. Replaces pip, uv, virtualenv, and pyproject.toml-based dependency resolution with a single
python.withPackagesderivation that is reproducible, offline-capable, and free of PATH collisions.Audience: Any developer or AI agent with zero prior Nix, pip, uv, or Pulumi knowledge. Every concept is defined from first principles. Every code snippet is complete and copy-paste viable.
Scope: Greenfield implementation and migration from existing pip/uv projects.
- Concepts and Terminology
- Why This Migration Exists
- Architecture Overview
- File Layout
- Step-by-Step Implementation
- The mkPulumiPypiPackage Pattern
- Finding PyPI Wheel URLs and Hashes
- The PATH Collision Problem
- The Two-Pass Import Pattern
- The PULUMI_PYTHON_CMD Wrapper
- Pyright Integration
- Environment Variables Reference
- Migration Checklist: pip/uv → Nix
- Debugging
- Common Failure Modes
- Native (Non-Pure) Python Packages
- Adding a New Provider SDK
- Adding a New Runtime Dependency
- Updating Package Versions
- Offline and Air-Gapped Builds
- CI/CD Integration
- Complete Reference Implementation
| Term | Definition |
|---|---|
| pip | Python's default package installer. Downloads packages from PyPI, installs into a directory. |
| uv | A fast pip replacement written in Rust. Drop-in compatible with pip but 10-100x faster. |
| virtualenv / venv | An isolated Python environment with its own site-packages. Created by python -m venv .venv. |
| pyproject.toml | Modern Python project metadata file. Declares dependencies, build system, tool configs. |
| wheel | A pre-built Python package distribution format (.whl file). No compilation needed at install time. |
| sdist | A source distribution. Requires compilation at install time (may need C compiler, headers). |
| pure Python wheel | A wheel with tag py3-none-any — runs on any Python 3, any OS, any architecture. No native code. |
| native wheel | A wheel with tag like cp313-cp313-manylinux_2_17_x86_64 — compiled for specific Python version, OS, and CPU. |
| site-packages | Directory where Python finds installed packages. import foo searches here. |
| PyPI | Python Package Index (pypi.org). The public repository of Python packages. |
| Term | Definition |
|---|---|
| Nix | A purely functional package manager. Every package is a function of its inputs, producing a unique store path. |
| nixpkgs | The Nix package repository. Contains 100,000+ packages including Python packages under python3Packages.*. |
| derivation | A build recipe in Nix. Specifies inputs, build commands, and outputs. Produces a /nix/store/<hash>-<name> path. |
| flake | A Nix project with a flake.nix file declaring inputs (dependencies) and outputs (packages, shells, modules). |
| devshell | A development environment created by nix develop. Provides tools and environment variables without installation. |
| overlay | A function that modifies or extends the nixpkgs package set. Applied during import nixpkgs { overlays = [...]; }. |
| mkShell | Nix function for creating development shells. Accepts nativeBuildInputs (tools) and buildInputs (libraries). |
| python.withPackages | Nix function that creates a Python environment with specific packages available for import. Returns a single derivation. |
| buildPythonPackage | Nix function that builds a Python package from source or wheel. Used for packages not in nixpkgs. |
| fetchurl | Nix function that downloads a file from a URL and verifies its hash. Used to fetch wheels from PyPI. |
| symlinkJoin | Nix function that merges multiple derivations into one by symlinking their contents. Used to combine pulumi CLI components. |
| writeShellScriptBin | Nix function that creates a shell script in /nix/store as an executable. Used for wrapper scripts. |
| nativeBuildInputs | Packages available at build time and in the devshell PATH. This is where you put tools. |
| propagatedBuildInputs | Dependencies that are automatically available to downstream consumers. Used for Python package deps. |
| shellHook | Shell code executed when entering a nix develop shell. Used for PATH manipulation and env setup. |
| env | Attribute set of environment variables exported by mkShell. Set once, available in shell. |
| Term | Definition |
|---|---|
| Pulumi | Infrastructure as Code tool. Programs in Python/Go/TypeScript/etc. define cloud resources declaratively. |
| Pulumi CLI | The pulumi command-line binary (written in Go). Orchestrates deployments. |
| Language plugin | Pulumi component that interprets your program. pulumi-language-python runs Python programs. |
| Provider SDK | Python package that defines resources for a specific cloud/service (e.g., pulumi-kubernetes). |
| Stack | A Pulumi deployment instance with its own state and configuration (e.g., dev, staging, prod). |
| PULUMI_PYTHON_CMD | Environment variable telling Pulumi which Python interpreter to use. Overrides auto-detection. |
| Pulumi.yaml | Project configuration file. Declares runtime (python), main module path, and optional virtualenv path. |
-
Non-hermetic: pip fetches packages from PyPI at runtime. Different machines get different versions. Network failures break builds.
-
PATH collision: Nix devshells and pip virtualenvs both provide
python3binaries.mkShellprepends all packagebin/directories to PATH including transitive dependencies. Whenpython.withPackagesis used alongside a venv, twopython3binaries compete. The wrong one wins, andimport pulumifails. -
Pyright resolution failure: Pulumi's Go-based language plugin runs
python -m pyrightto typecheck. Ifpyrightis only a standalone Node binary on PATH (from nixpkgs), Pulumi can't find it. It must be pip-visible (installed in the Python environment as a package). -
Duplicate Python interpreters:
python.withPackagescreates a wrapper script (python3) that knows about installed packages AND includes the barepython3from the underlying derivation as a build input. mkShell puts both in PATH. The bare one has no packages. -
PULUMI_PYTHON_CMD mismatch: Pulumi auto-detects Python by searching PATH. If it finds the bare interpreter instead of the withPackages wrapper,
pulumi upfails withModuleNotFoundError: No module named 'pulumi'. -
Virtualenv path doubling: Pulumi resolves virtualenv paths differently during
pulumi install(relative toPulumi.yamldirectory) vspulumi up(relative tomain:directory). This causes the path to be concatenated twice, creating non-existent directories.
- Hermetic: All packages are content-addressed in
/nix/store. Identical inputs always produce identical outputs. - Offline-capable: After first build, all packages are cached locally. No network needed.
- Single interpreter: One
python.withPackagescall, onepython3binary, onesite-packages. - No venv: No
.venv/directory, noUV_PROJECT_ENVIRONMENT, noPULUMI_PYTHON_VIRTUALENV. - Reproducible:
flake.lockpins every input. Same lock file = same environment on every machine.
┌─────────────────────────────────────────────────────────────────────────────┐
│ Dependency Resolution Flow │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ pyproject.toml (OLD — reference only, not used by Nix) │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ [project.dependencies] │ │
│ │ pulumi >= 3.0 │ │
│ │ pulumi-kubernetes >= 4.0 │ │
│ │ pydantic >= 2.0 │ │
│ │ ... │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │ │
│ Audit + Classify │
│ │ │
│ ▼ │
│ pulumi.nix (NEW — Nix-native dependency specification) │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ pythonDeps = ps: [ │ │
│ │ ps.pulumi # from nixpkgs │ │
│ │ pulumiKubernetes # from PyPI wheel (not in nixpkgs) │ │
│ │ ps.pydantic # from nixpkgs │ │
│ │ pyrightPkg # from PyPI wheel (Pulumi requirement)│ │
│ │ ]; │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │ │
│ Merge into single env │
│ │ │
│ ▼ │
│ languages.nix (Aggregation layer) │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ pythonEnv = python313.withPackages (ps: [ │ │
│ │ ps.ipython # dev tools │ │
│ │ ps.ruff # linter │ │
│ │ ] ++ pulumiDeps.pythonDeps ps); # <-- Pulumi deps merged here │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │ │
│ Single derivation │
│ │ │
│ ▼ │
│ /nix/store/<hash>-python3-3.13.12-env/bin/python3 │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ This ONE binary can import: │ │
│ │ pulumi, pulumi_kubernetes, pulumi_tls, pydantic, pyright, │ │
│ │ ipython, ruff, bcrypt, semver, gitpython, kubernetes, ... │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │ │
│ Used by: │
│ │ │
│ ┌────────────┼────────────┐ │
│ ▼ ▼ ▼ │
│ devshell pulumi CLI pyright │
│ PATH wrapper typechecker │
│ (PULUMI_ (python -m │
│ PYTHON_CMD) pyright) │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
your-project/
├── flake.nix # Flake entry point
├── flake.lock # Pinned input versions
├── src/
│ ├── packages/
│ │ ├── pulumi.nix # Pulumi Python deps + CLI wrapper
│ │ ├── languages.nix # Aggregates all language packages
│ │ └── default.nix # Package set entry point
│ └── devshells/
│ ├── base.nix # Base devshell
│ ├── full.nix # All languages
│ └── default.nix # Shell exports
├── infrastructure/
│ └── pulumi/
│ ├── Pulumi.yaml # Pulumi project config
│ ├── src/
│ │ ├── __main__.py # Entry point
│ │ └── ... # Your IaC code
│ └── stacks/
│ ├── Pulumi.dev.yaml
│ └── Pulumi.prod.yaml
├── pyproject.toml # LEGACY — kept for reference only
└── .envrc # direnv configuration
Key principle: pyproject.toml becomes a reference document only. Nix is the source of truth for all Python dependencies. You may keep pyproject.toml for editor tooling (pyright config, ruff config) but its [project.dependencies] section is not used.
Start by extracting every Python dependency from your current project:
# From pyproject.toml
grep -A 100 '^\[project\]' pyproject.toml | grep -A 100 '^dependencies' | head -50
# From requirements.txt (if using pip)
cat requirements.txt
# From uv.lock (if using uv)
uv pip list --format=columns
# From an active venv
pip list --format=columns
# From Pulumi.yaml (virtualenv path)
grep virtualenv Pulumi.yamlCreate a list of every package with its version:
pulumi==3.192.0
pulumi-kubernetes==4.23.0
pulumi-tls==5.2.1
pulumi-docker-build==0.0.14
pulumi-cloudflare==6.14.0
pulumi-random==4.16.8
pydantic==2.11.0
bcrypt==4.3.0
semver==3.1.0
gitpython==3.1.44
kubernetes==33.1.0
pyyaml==6.0.2
pyright==1.1.407
For each package, determine where it comes from:
# Check if package exists in nixpkgs
nix eval nixpkgs#python313Packages.pulumi.version 2>/dev/null
nix eval nixpkgs#python313Packages.pydantic.version 2>/dev/null
nix eval nixpkgs#python313Packages.pulumi-kubernetes.version 2>/dev/nullClassify into three categories:
| Category | Example | How to provide |
|---|---|---|
| In nixpkgs | pulumi, pydantic, pyyaml, bcrypt |
ps.pulumi in pythonDeps |
| Not in nixpkgs, pure Python wheel | pulumi-kubernetes, pulumi-tls |
mkPulumiPypiPackage |
| Not in nixpkgs, native wheel | (rare for Pulumi deps) | buildPythonPackage with autoPatchelfHook |
This is the core file. It declares all Pulumi-related Python dependencies and the CLI wrapper.
# src/packages/pulumi.nix
#
# Pulumi Python Environment (NixOS-native with python.withPackages)
#
# This file provides:
# 1. pythonDeps — function returning Python packages for withPackages
# 2. package — Pulumi CLI with PULUMI_PYTHON_CMD wrapper
# 3. env — environment variables for devshells
#
# Architecture:
# - pythonDeps is a FUNCTION that takes the python package set (ps)
# and returns a list of packages. This is merged into the single
# python.withPackages call in languages.nix.
# - package is a symlinkJoin of the Pulumi CLI, its wrapper, and
# the language plugin. It requires pythonEnv to be provided.
# - The two-pass import pattern (see section 9) breaks the circular
# dependency between pythonDeps and pythonEnv.
#
# Adding a new provider:
# 1. Check nixpkgs: nix eval nixpkgs#python313Packages.pulumi-foo
# 2. If found: add ps."pulumi-foo" to pythonDeps
# 3. If not found: add mkPulumiPypiPackage entry (see section 6)
# 4. Run: nix develop (rebuilds the environment)
{ pkgs, pythonEnv ? null }:
let
# =========================================================================
# PyPI Wheel Builder
# =========================================================================
# All Pulumi provider SDKs are pure Python wheels (py3-none-any).
# They share identical propagatedBuildInputs: parver, pulumi, semver.
# This helper avoids repeating the same buildPythonPackage boilerplate.
#
# For native (non-pure) wheels, see section 16.
mkPulumiPypiPackage = { pname, version, url, hash, meta ? {} }:
pkgs.python313Packages.buildPythonPackage {
inherit pname version;
format = "wheel";
src = pkgs.fetchurl {
inherit url hash;
};
# All Pulumi provider SDKs depend on these at runtime.
# parver: PEP 440 version parsing
# pulumi: Core SDK (resource registration, stack references)
# semver: Semantic versioning
propagatedBuildInputs = with pkgs.python313Packages; [
parver
pulumi
semver
];
# Provider SDK tests require a running Pulumi engine.
# They cannot run in the Nix sandbox.
doCheck = false;
# Verify the package is importable after installation.
# This catches broken wheels and missing dependencies.
pythonImportsCheck = [ pname ];
inherit meta;
};
# =========================================================================
# Provider SDKs from PyPI (not available in nixpkgs)
# =========================================================================
# To find wheel URLs: https://pypi.org/project/PACKAGE_NAME/VERSION/#files
# Select the "py3-none-any.whl" file. Copy the URL.
#
# To get the hash:
# nix-prefetch-url --type sha256 "URL"
# Then convert: nix hash convert --hash-algo sha256 --to sri <hash>
#
# Or use: nix store prefetch-file --hash-type sha256 "URL"
pulumiKubernetes = mkPulumiPypiPackage {
pname = "pulumi_kubernetes";
version = "4.23.0";
url = "https://files.pythonhosted.org/packages/cc/00/983975f1bcf02601f12b4afb60a4dfdf9f81ab11cbf2493aa349781684ae/pulumi_kubernetes-4.23.0-py3-none-any.whl";
hash = "sha256-REPLACE_WITH_ACTUAL_HASH";
meta.description = "Pulumi Kubernetes provider SDK";
};
pulumiTls = mkPulumiPypiPackage {
pname = "pulumi_tls";
version = "5.2.1";
url = "https://files.pythonhosted.org/packages/3c/12/f5035bbfe624279a4838ec824ec8a6d34e9351f6780def8c2ab0ce965a0c/pulumi_tls-5.2.1-py3-none-any.whl";
hash = "sha256-REPLACE_WITH_ACTUAL_HASH";
meta.description = "Pulumi TLS provider SDK";
};
pulumiDockerBuild = mkPulumiPypiPackage {
pname = "pulumi_docker_build";
version = "0.0.14";
url = "https://files.pythonhosted.org/packages/72/a0/7f8b89aed8ef77aa16dde4d3bff9254035a77e597d21c450f8c32d3db6c8/pulumi_docker_build-0.0.14-py3-none-any.whl";
hash = "sha256-REPLACE_WITH_ACTUAL_HASH";
meta.description = "Pulumi Docker Build provider SDK";
};
pulumiCloudflare = mkPulumiPypiPackage {
pname = "pulumi_cloudflare";
version = "6.14.0";
url = "https://files.pythonhosted.org/packages/68/97/e6f11adbd919ad41ab920b137b7a711e3eb1e8130602a4c29d0ae86d329f/pulumi_cloudflare-6.14.0-py3-none-any.whl";
hash = "sha256-REPLACE_WITH_ACTUAL_HASH";
meta.description = "Pulumi Cloudflare provider SDK";
};
# =========================================================================
# Pyright (Type Checker)
# =========================================================================
# Pulumi's language plugin requires pyright to be pip-visible (installed
# in the Python environment as a package), not just on PATH as a standalone
# CLI binary.
#
# The PyPI pyright package is a thin Python wrapper that invokes the
# node-based pyright binary. It requires nodeenv and typing-extensions.
#
# nixpkgs provides pyright as a standalone Node binary. That works for
# editors (VSCode, Neovim) but NOT for Pulumi's internal typechecker
# invocation which does: python -m pyright
pyrightPkg = pkgs.python313Packages.buildPythonPackage {
pname = "pyright";
version = "1.1.407";
format = "wheel";
src = pkgs.fetchurl {
url = "https://files.pythonhosted.org/packages/dc/93/b69052907d032b00c40cb656d21438ec00b3a471733de137a3f65a49a0a0/pyright-1.1.407-py3-none-any.whl";
hash = "sha256-REPLACE_WITH_ACTUAL_HASH";
};
propagatedBuildInputs = with pkgs.python313Packages; [
nodeenv
typing-extensions
];
meta = {
description = "Python command line wrapper for pyright type checker";
homepage = "https://github.com/microsoft/pyright";
};
};
in
{
# =========================================================================
# pythonDeps: Function returning Python packages for withPackages
# =========================================================================
# This is NOT a derivation — it's a function that takes the python
# package set (ps) and returns a list. It's called inside
# python.withPackages in languages.nix.
#
# Why a function? Because the python package set (ps) is provided by
# python.withPackages, not by us. We can't reference ps.pulumi
# outside of a withPackages call.
pythonDeps = ps: [
# ─── Pulumi Core SDK ──────────────────────────────────────────
# From nixpkgs (check version: nix eval nixpkgs#python313Packages.pulumi.version)
ps.pulumi
# ─── Provider SDKs from nixpkgs ───────────────────────────────
ps."pulumi-random"
# ─── Provider SDKs from PyPI wheels ───────────────────────────
# These are NOT in nixpkgs python313Packages.
# Built above using mkPulumiPypiPackage.
pulumiKubernetes
pulumiTls
pulumiDockerBuild
pulumiCloudflare
# ─── Runtime Dependencies ─────────────────────────────────────
# From pyproject.toml [project.dependencies] or requirements.txt.
# Each must be verified: nix eval nixpkgs#python313Packages.PACKAGE.version
ps.pydantic
ps.bcrypt
ps.semver
ps.gitpython
ps.kubernetes
ps.pyyaml
# ─── Type Checker ─────────────────────────────────────────────
# Must be pip-visible for Pulumi's typechecker option.
pyrightPkg
];
# Re-export for languages.nix
inherit pythonDeps;
# =========================================================================
# package: Pulumi CLI with Python wrapper
# =========================================================================
# This is only usable after pythonEnv is provided (second-pass import).
# See section 9 for the two-pass pattern.
#
# The wrapper sets PULUMI_PYTHON_CMD to point at the withPackages
# interpreter, ensuring Pulumi uses the correct Python with all
# provider SDKs available.
package = let
# Assert: pythonEnv must be provided. This fails at eval time if
# pulumi.nix is imported without pythonEnv (first-pass import
# should not access .package).
resolvedPythonEnv = assert pythonEnv != null; pythonEnv;
# Shell script wrapper that sets PULUMI_PYTHON_CMD before exec'ing
# the real pulumi binary. This ensures the Go-based language plugin
# spawns the correct Python interpreter.
pulumiWrapper = pkgs.writeShellScriptBin "pulumi" ''
export PULUMI_PYTHON_CMD="${resolvedPythonEnv}/bin/python"
exec ${pkgs.pulumi}/bin/pulumi "$@"
'';
in pkgs.symlinkJoin {
name = "pulumi-with-python";
paths = [
pulumiWrapper # Wrapped CLI (takes priority)
pkgs.pulumi # Original CLI (for other binaries like pulumi-watch)
pkgs.pulumictl # Pulumi utilities
pkgs.pulumiPackages.pulumi-python # Python language plugin
# NOTE: pythonEnv is NOT in paths — it's in the devshell's
# nativeBuildInputs and prepended to PATH via shellHook.
];
meta = {
description = "Pulumi IaC with NixOS-native Python environment";
homepage = "https://pulumi.com";
};
};
# =========================================================================
# env: Environment variables for devshells
# =========================================================================
# These are merged into the devshell's env attribute set.
env = let
resolvedPythonEnv = assert pythonEnv != null; pythonEnv;
in {
# Tell Pulumi which Python to use. Without this, Pulumi searches PATH
# and may find the wrong python3 (bare interpreter without packages).
PULUMI_PYTHON_CMD = "${resolvedPythonEnv}/bin/python";
};
}This file creates the single Python environment shared by all consumers.
# src/packages/languages.nix
#
# Single source of truth for all language environments.
# The pythonEnv derivation created here is the ONLY Python interpreter
# in the devshell. All other consumers (Pulumi, pyright, devtools)
# use this same derivation.
{ pkgs, ... }:
let
# =========================================================================
# Language version definitions (from your versions.nix or inline)
# =========================================================================
langs = {
python = { version = "313"; display = "3.13"; };
# go = { version = "1.25"; display = "1.25"; };
# node = { version = "22"; display = "22"; };
# rust = { version = "1.92.0"; display = "1.92.0"; };
};
# =========================================================================
# Two-Pass Import for Pulumi
# =========================================================================
# First pass: import pulumi.nix WITHOUT pythonEnv.
# This gives us access to pythonDeps (a function) but NOT package/env
# (which require pythonEnv).
pulumiDeps = import ./pulumi.nix { inherit pkgs; };
# =========================================================================
# Python Environment (SINGLE derivation)
# =========================================================================
# ALL Python packages go here. Do NOT create multiple withPackages calls.
# Multiple calls = multiple python3 binaries in PATH = broken imports.
pythonEnv = pkgs."python${langs.python.version}".withPackages (ps: [
# ─── Development Tools ────────────────────────────────────────
ps.ipython # Enhanced REPL
ps.ruff # Linter + formatter
# ─── Pulumi Dependencies ──────────────────────────────────────
# Merged from pulumi.nix via the pythonDeps function.
# This includes: core SDK, provider SDKs, runtime deps, pyright.
] ++ pulumiDeps.pythonDeps ps);
# =========================================================================
# Second pass: import pulumi.nix WITH pythonEnv
# =========================================================================
# Now pulumi.nix can create the CLI wrapper and env variables
# that reference the actual Python interpreter path.
pulumiPkg = import ./pulumi.nix { inherit pkgs pythonEnv; };
# =========================================================================
# Package lists for devshells
# =========================================================================
pythonPackages = [
pythonEnv
# Do NOT add other Python packages here. They go in withPackages above.
];
in
{
# Exported for devshells
inherit pythonEnv pythonPackages;
inherit pulumiPkg;
# All language packages combined
packages = pythonPackages;
# ++ goPackages ++ nodePackages ++ rustPackages; # Add as needed
# Environment variables
env = {
# Tell uv/pip to use the Nix-provided Python, not download their own.
UV_SYSTEM_PYTHON = "1";
# Don't write .pyc files (clutter in Nix store paths).
PYTHONDONTWRITEBYTECODE = "1";
} // pulumiPkg.env;
}# src/devshells/full.nix
#
# Full development shell with all languages and tools.
{ pkgs, languages, ... }:
pkgs.mkShell {
nativeBuildInputs = with pkgs; [
# Language environments
] ++ languages.pythonPackages
++ [ languages.pulumiPkg.package ]; # Pulumi CLI with wrapper
# =========================================================================
# CRITICAL: PATH ordering fix
# =========================================================================
# mkShell prepends each package's bin/ to PATH INCLUDING transitive deps.
# python.withPackages adds both:
# 1. /nix/store/<hash>-python3-3.13.12-env/bin/python3 (wrapper with packages)
# 2. /nix/store/<hash>-python3-3.13.12/bin/python3 (bare, NO packages)
#
# The bare one can win PATH priority. When it does:
# - `python3 -c "import pulumi"` → ModuleNotFoundError
# - `pulumi up` → "No module named 'pulumi'"
# - `python -m pyright` → "No module named 'pyright'"
#
# Fix: prepend pythonEnv FIRST in shellHook. This ensures the wrapper
# (with all packages) always wins.
shellHook = ''
export PATH="${languages.pythonEnv}/bin:$PATH"
'';
env = languages.env;
}# flake.nix (relevant sections only)
{
outputs = { nixpkgs, ... }:
let
pkgs = import nixpkgs {
system = "x86_64-linux";
config.allowUnfree = true;
};
languages = import ./src/packages/languages.nix { inherit pkgs; };
in {
devShells.x86_64-linux.default = import ./src/devshells/full.nix {
inherit pkgs languages;
};
};
}Use mkPulumiPypiPackage when a Pulumi provider SDK is:
- NOT available in nixpkgs (
nix eval nixpkgs#python313Packages.PACKAGEfails) - A pure Python wheel (
py3-none-any.whlon PyPI) - Has standard Pulumi provider dependencies (parver, pulumi, semver)
mkPulumiPypiPackage = { pname, version, url, hash, meta ? {} }:
pkgs.python313Packages.buildPythonPackage {
inherit pname version;
# "wheel" format tells Nix to install the .whl directly
# without running setup.py or building from source.
format = "wheel";
# fetchurl downloads the file and verifies the hash.
# If the hash doesn't match, the build fails immediately.
src = pkgs.fetchurl {
inherit url hash;
};
# These are available at runtime (import time) for the package.
# All Pulumi providers need these three.
propagatedBuildInputs = with pkgs.python313Packages; [
parver # PEP 440 version parsing
pulumi # Core SDK
semver # Semantic versioning
];
# Skip tests — they require a running Pulumi engine.
doCheck = false;
# Verify the import works after installation.
# pname = "pulumi_kubernetes" → import pulumi_kubernetes
pythonImportsCheck = [ pname ];
inherit meta;
};| Field | Type | Purpose |
|---|---|---|
pname |
string | Python package name as it appears in import (underscores, e.g., pulumi_kubernetes) |
version |
string | Exact version (e.g., "4.23.0") |
url |
string | Full URL to the .whl file on PyPI |
hash |
string | SRI hash (sha256-...) of the .whl file |
meta.description |
string | Human-readable description (optional) |
format |
"wheel" |
Tells buildPythonPackage to install the wheel directly |
propagatedBuildInputs |
list | Runtime dependencies (available to importers) |
doCheck |
bool | Whether to run tests (false for providers) |
pythonImportsCheck |
list | Package names to try importing after install |
- Go to
https://pypi.org/project/PACKAGE_NAME/VERSION/#files - Find the file ending in
py3-none-any.whl - Copy the URL
- Download and hash:
# Download the wheel
curl -L -o /tmp/package.whl "URL"
# Get the Nix SRI hash
nix hash file --sri /tmp/package.whl
# Output: sha256-AbCdEf...
# Or if using older nix:
sha256sum /tmp/package.whl
# Then convert:
nix hash convert --hash-algo sha256 --to sri SHA256_HEX_HERE# This downloads, caches in nix store, and prints the hash
nix-prefetch-url "https://files.pythonhosted.org/packages/.../package-1.0.0-py3-none-any.whl"
# Output: 0abc123def456... (base32)
# Convert to SRI format for use in fetchurl
nix hash convert --hash-algo sha256 --to sri 0abc123def456...
# Output: sha256-AbCdEf...nix store prefetch-file --hash-type sha256 \
"https://files.pythonhosted.org/packages/.../package-1.0.0-py3-none-any.whl"
# Output includes the hash in SRI format# Set hash to empty string — Nix will fail and tell you the correct hash
src = pkgs.fetchurl {
url = "https://files.pythonhosted.org/packages/.../package.whl";
hash = ""; # Nix will error with: "got: sha256-CORRECT_HASH_HERE"
};nix develop # Will fail with the correct hash
# error: hash mismatch in fixed-output derivation
# specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
# got: sha256-AbCdEfGhIjKlMnOpQrStUvWxYz0123456789ABCD=Copy the got: hash into your hash field.
When you put pythonEnv (from python.withPackages) in nativeBuildInputs:
mkShell {
nativeBuildInputs = [ pythonEnv otherPackage ];
}
Nix constructs PATH by iterating all packages AND their transitive dependencies. pythonEnv depends on the base python3 interpreter (it wraps it). Both get added to PATH:
PATH=/nix/store/xxx-python3-3.13.12-env/bin: ← wrapper (has packages)
/nix/store/yyy-python3-3.13.12/bin: ← bare (NO packages)
/nix/store/zzz-other-package/bin:
...
The first python3 in PATH wins. Whether the wrapper or bare one comes first depends on Nix's topological sort of the dependency graph — it's not deterministic across different nixpkgs versions or package sets.
Prepend the wrapper explicitly in shellHook:
shellHook = ''
export PATH="${pythonEnv}/bin:$PATH"
'';This guarantees the wrapper is ALWAYS first, regardless of how Nix sorts the rest of PATH.
# In your devshell:
which python3
# Should show: /nix/store/xxx-python3-3.13.12-env/bin/python3
# NOT: /nix/store/yyy-python3-3.13.12/bin/python3
# The -env suffix means it's the wrapper with packages.
# Verify:
python3 -c "import pulumi; print(pulumi.__version__)"
# Should print the version, not ModuleNotFoundErrorpulumi.nix needs to:
- Export
pythonDeps(a function) forlanguages.nixto merge intowithPackages - Export
package(Pulumi CLI wrapper) that references the finalpythonEnv
But pythonEnv is created by languages.nix which calls pythonDeps... circular dependency.
Import pulumi.nix twice:
# languages.nix
# FIRST PASS: get pythonDeps function (pythonEnv not needed)
pulumiDeps = import ./pulumi.nix { inherit pkgs; };
# pulumiDeps.pythonDeps is available
# pulumiDeps.package would fail (assert pythonEnv != null)
# Create pythonEnv using the deps from first pass
pythonEnv = pkgs.python313.withPackages (ps: [
ps.ipython
] ++ pulumiDeps.pythonDeps ps);
# SECOND PASS: now provide pythonEnv for the wrapper
pulumiPkg = import ./pulumi.nix { inherit pkgs pythonEnv; };
# pulumiPkg.package is now available (pythonEnv != null)
# pulumiPkg.env is now available# In pulumi.nix:
package = let
resolvedPythonEnv = assert pythonEnv != null; pythonEnv;
in ...assert fails at evaluation time with a clear error if someone tries to access .package from a first-pass import. This is intentional — it prevents silent bugs where the wrapper points at null.
pulumi upstarts the Pulumi engine (Go binary)- Engine spawns the Python language plugin (
pulumi-language-python) - Language plugin searches PATH for
python3orpython - It finds the first
python3in PATH - It runs
python3 /path/to/your/__main__.py - If
python3is the bare interpreter:ModuleNotFoundError: No module named 'pulumi'
#!/nix/store/.../bin/pulumi
export PULUMI_PYTHON_CMD="/nix/store/xxx-python3-3.13.12-env/bin/python"
exec /nix/store/yyy-pulumi-3.192.0/bin/pulumi "$@"The language plugin checks PULUMI_PYTHON_CMD first, before searching PATH. This guarantees it uses the withPackages interpreter.
The wrapper script is a single binary (pulumi). But Pulumi ships many binaries:
pulumi— main CLIpulumi-language-python— Python language pluginpulumictl— utilities
symlinkJoin merges them all into one directory. The wrapper's pulumi takes priority over the original (symlinks resolve in order).
Pyright exists in two forms:
| Form | Package | How it works | Works for Pulumi? |
|---|---|---|---|
| Standalone Node binary | nixpkgs#pyright |
Runs as pyright CLI |
NO |
| Python package | PyPI pyright |
Runs as python -m pyright |
YES |
Pulumi's language plugin runs pyright via python -m pyright. The standalone Node binary is not in Python's site-packages, so python -m pyright fails with No module named 'pyright'.
Build pyright from its PyPI wheel and include it in pythonDeps:
pyrightPkg = pkgs.python313Packages.buildPythonPackage {
pname = "pyright";
version = "1.1.407";
format = "wheel";
src = pkgs.fetchurl {
url = "https://files.pythonhosted.org/packages/.../pyright-1.1.407-py3-none-any.whl";
hash = "sha256-...";
};
propagatedBuildInputs = with pkgs.python313Packages; [
nodeenv # pyright needs Node.js at runtime
typing-extensions
];
};This makes python -m pyright work inside the Nix-provided Python environment.
# In devshell:
python3 -m pyright --version
# Output: pyright 1.1.407
# NOT:
pyright --version
# This is the standalone Node binary (different thing, also works but not for Pulumi)| Variable | Value | Set Where | Purpose |
|---|---|---|---|
PULUMI_PYTHON_CMD |
/nix/store/.../python3 |
pulumi.nix env |
Tell Pulumi which Python to use |
UV_SYSTEM_PYTHON |
"1" |
languages.nix env |
Prevent uv from downloading Python |
PYTHONDONTWRITEBYTECODE |
"1" |
languages.nix env |
No .pyc files in nix store |
PYRIGHT_PYTHON_FORCE_VERSION |
"latest" |
.env file |
Prevent pyright from complaining about version |
| Variable | Why remove |
|---|---|
PULUMI_PYTHON_VIRTUALENV |
No venv. Pulumi uses PULUMI_PYTHON_CMD instead. |
UV_PROJECT_ENVIRONMENT |
No venv. Python packages are in nix store. |
VIRTUAL_ENV |
No venv. |
PYTHONPATH |
Don't set manually. withPackages handles site-packages. |
[ ] 1. Audit: List all Python dependencies from pyproject.toml/requirements.txt
[ ] 2. Classify: For each package, check if it's in nixpkgs python313Packages
[ ] 3. Build: Create pulumi.nix with pythonDeps function
[ ] 4. Fetch: Get PyPI wheel URLs and hashes for packages not in nixpkgs
[ ] 5. Merge: Create languages.nix with single python.withPackages call
[ ] 6. Shell: Create devshell with PATH fix in shellHook
[ ] 7. Verify: `nix develop` → `python3 -c "import pulumi"` → success
[ ] 8. Verify: `pulumi preview` works without errors
[ ] 9. Verify: `python -m pyright --version` works
[ ] 10. Clean: Remove .venv/ directory
[ ] 11. Clean: Remove UV_PROJECT_ENVIRONMENT from .envrc
[ ] 12. Clean: Remove PULUMI_PYTHON_VIRTUALENV from .env files
[ ] 13. Clean: Remove virtualenv from Pulumi.yaml (or set to empty)
[ ] 14. Update: Set PULUMI_PYTHON_CMD in Pulumi.yaml or rely on env var
[ ] 15. Test: Full `pulumi up` against a real stack
[ ] 16. Commit: All changes
# Which python3 is being used?
which python3
# Should be: /nix/store/xxx-python3-3.13.12-env/bin/python3 (note: -env suffix)
# What packages are available?
python3 -c "import sys; print('\n'.join(sys.path))"
# Can we import pulumi?
python3 -c "import pulumi; print(pulumi.__version__)"
# Can we import a provider?
python3 -c "import pulumi_kubernetes; print('ok')"# What Python does Pulumi see?
echo $PULUMI_PYTHON_CMD
# Does that Python have packages?
$PULUMI_PYTHON_CMD -c "import pulumi; print('ok')"
# Force verbose output:
PULUMI_DEBUG_COMMANDS=1 pulumi previewerror: hash mismatch in fixed-output derivation
specified: sha256-AAAA...
got: sha256-BBBB...
Copy the got: hash and replace the specified hash in your fetchurl call.
Pyright is not in your pythonDeps. Add pyrightPkg to the list. See section 11.
# List all python3 binaries in PATH
type -a python3
# Should show ONE entry (the -env wrapper)
# If you see two, your shellHook PATH prepend is missing or wrong| Symptom | Cause | Fix |
|---|---|---|
ModuleNotFoundError: No module named 'pulumi' |
Wrong python3 in PATH (bare, not wrapper) | Add export PATH="${pythonEnv}/bin:$PATH" to shellHook |
pulumi up hangs then fails with Python error |
PULUMI_PYTHON_CMD not set or points to bare python3 |
Set PULUMI_PYTHON_CMD in pulumi.nix env |
python -m pyright fails |
pyright not in withPackages | Add pyrightPkg to pythonDeps |
Hash mismatch on nix develop |
PyPI wheel was updated (same version, different file) | Re-fetch hash (see section 7) |
assert failure accessing .package |
First-pass import trying to access .package |
Only access .package from second-pass import |
pulumi install creates a .venv |
PULUMI_PYTHON_VIRTUALENV still set |
Remove from .env and Pulumi.yaml |
| Two different Python versions in PATH | Multiple withPackages calls |
Use ONE withPackages call in languages.nix |
Most Pulumi provider SDKs are pure Python (py3-none-any). But some runtime deps may have native extensions (C/C++/Rust).
# Check the wheel filename on PyPI
# Pure: package-1.0.0-py3-none-any.whl
# Native: package-1.0.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whlIf the package is in nixpkgs, just use ps.PACKAGE — nixpkgs handles the build.
If not in nixpkgs:
nativePackage = pkgs.python313Packages.buildPythonPackage {
pname = "my-native-package";
version = "1.0.0";
# For native packages, use sdist (source distribution) not wheel
src = pkgs.fetchPypi {
pname = "my-native-package";
version = "1.0.0";
hash = "sha256-...";
};
# Native build dependencies
nativeBuildInputs = [
pkgs.python313Packages.setuptools
pkgs.python313Packages.wheel
pkgs.python313Packages.cython # If needed
];
# System libraries needed at build time
buildInputs = [
pkgs.openssl # Example: if package links against libssl
pkgs.libffi
];
# Runtime dependencies
propagatedBuildInputs = with pkgs.python313Packages; [
cffi
];
};For pre-compiled Linux wheels that aren't in nixpkgs:
nativeFromWheel = pkgs.python313Packages.buildPythonPackage {
pname = "my-package";
version = "1.0.0";
format = "wheel";
src = pkgs.fetchurl {
url = "https://files.pythonhosted.org/packages/.../my_package-1.0.0-cp313-cp313-manylinux_2_17_x86_64.whl";
hash = "sha256-...";
};
# autoPatchelfHook fixes the RPATH of native libraries
# to point at nix store paths instead of /usr/lib
nativeBuildInputs = [ pkgs.autoPatchelfHook ];
# System libraries the native code links against
buildInputs = [
pkgs.stdenv.cc.cc.lib # libstdc++
pkgs.openssl # libssl, libcrypto
pkgs.zlib # libz
];
};# Step 1: Check if it's in nixpkgs
nix eval nixpkgs#python313Packages.pulumi-aws.version 2>/dev/null
# If this prints a version → use ps."pulumi-aws" in pythonDeps
# If this fails → build from PyPI wheel
# Step 2: Find the wheel on PyPI
# Go to: https://pypi.org/project/pulumi-aws/VERSION/#files
# Find: pulumi_aws-VERSION-py3-none-any.whl
# Copy the URL
# Step 3: Get the hash
nix store prefetch-file --hash-type sha256 \
"https://files.pythonhosted.org/packages/.../pulumi_aws-7.7.0-py3-none-any.whl"
# Step 4: Add to pulumi.nixAdd to pulumi.nix:
pulumiAws = mkPulumiPypiPackage {
pname = "pulumi_aws";
version = "7.7.0";
url = "https://files.pythonhosted.org/packages/.../pulumi_aws-7.7.0-py3-none-any.whl";
hash = "sha256-HASH_FROM_STEP_3";
meta.description = "Pulumi AWS provider SDK";
};Add to pythonDeps:
pythonDeps = ps: [
# ... existing deps ...
pulumiAws # NEW
];Rebuild:
nix develop
python3 -c "import pulumi_aws; print('ok')"# Step 1: Check nixpkgs
nix eval nixpkgs#python313Packages.requests.version
# Output: "2.32.3" → it's in nixpkgs
# Step 2: Add to pythonDeps in pulumi.nixpythonDeps = ps: [
# ... existing deps ...
ps.requests # NEW — HTTP library
];# Step 3: Rebuild
nix develop
python3 -c "import requests; print(requests.__version__)"nixpkgs packages update when you update your flake inputs:
nix flake update nixpkgs
nix develop
python3 -c "import pulumi; print(pulumi.__version__)"- Find the new version on PyPI
- Get the new wheel URL
- Get the new hash
- Update
version,url,hashinmkPulumiPypiPackage
# Example: update pulumi-kubernetes from 4.23.0 to 4.24.0
# 1. Go to https://pypi.org/project/pulumi-kubernetes/4.24.0/#files
# 2. Copy py3-none-any.whl URL
# 3. Get hash:
nix store prefetch-file --hash-type sha256 "NEW_URL"
# 4. Update pulumi.nixnix develop
python3 -c "import pulumi_kubernetes; print(pulumi_kubernetes.__version__)"
# Should show new version
pulumi preview -s dev
# Should work without errorsAfter the first nix develop succeeds, all packages are cached in /nix/store. Subsequent nix develop invocations don't require network access.
- Build on a connected machine:
nix develop - Export the closure:
nix copy --to /path/to/portable-store .#devShells.x86_64-linux.default - Transfer the store to the air-gapped machine
- Import:
nix copy --from /path/to/portable-store .#devShells.x86_64-linux.default
The wheel URLs in fetchurl are fetched once and content-addressed in the nix store. After first fetch, the URL is never contacted again. The hash verification ensures integrity.
- uses: cachix/install-nix-action@v31
- run: nix develop --command pulumi preview -s prodname: my-infrastructure
runtime:
name: python
options:
# Do NOT set virtualenv — Nix handles Python packages
typechecker: pyright
description: Infrastructure managed by Pulumi with Nix-native Python
main: src/# DELETE these — they conflict with Nix
runtime:
options:
virtualenv: .venv # DELETE
toolchain: pip # DELETE{ pkgs, pythonEnv ? null }:
let
mkPypiPkg = { pname, version, url, hash }:
pkgs.python313Packages.buildPythonPackage {
inherit pname version;
format = "wheel";
src = pkgs.fetchurl { inherit url hash; };
propagatedBuildInputs = with pkgs.python313Packages; [ parver pulumi semver ];
doCheck = false;
pythonImportsCheck = [ pname ];
};
pulumiK8s = mkPypiPkg {
pname = "pulumi_kubernetes";
version = "4.23.0";
url = "https://files.pythonhosted.org/packages/.../pulumi_kubernetes-4.23.0-py3-none-any.whl";
hash = "sha256-...";
};
in {
pythonDeps = ps: [
ps.pulumi
ps."pulumi-random"
pulumiK8s
ps.pydantic
ps.pyyaml
];
inherit pythonDeps;
package = let
env = assert pythonEnv != null; pythonEnv;
wrapper = pkgs.writeShellScriptBin "pulumi" ''
export PULUMI_PYTHON_CMD="${env}/bin/python"
exec ${pkgs.pulumi}/bin/pulumi "$@"
'';
in pkgs.symlinkJoin {
name = "pulumi-with-python";
paths = [ wrapper pkgs.pulumi pkgs.pulumiPackages.pulumi-python ];
};
env = let env = assert pythonEnv != null; pythonEnv; in {
PULUMI_PYTHON_CMD = "${env}/bin/python";
};
}{ pkgs }:
let
pulumiDeps = import ./pulumi.nix { inherit pkgs; };
pythonEnv = pkgs.python313.withPackages (ps: [
ps.ipython
] ++ pulumiDeps.pythonDeps ps);
pulumiPkg = import ./pulumi.nix { inherit pkgs pythonEnv; };
in {
inherit pythonEnv pulumiPkg;
pythonPackages = [ pythonEnv ];
env = {
UV_SYSTEM_PYTHON = "1";
PYTHONDONTWRITEBYTECODE = "1";
} // pulumiPkg.env;
}{ pkgs, languages }:
pkgs.mkShell {
nativeBuildInputs = languages.pythonPackages ++ [ languages.pulumiPkg.package ];
shellHook = ''export PATH="${languages.pythonEnv}/bin:$PATH"'';
env = languages.env;
}# Enter devshell
nix develop
# Verify Python
which python3 # Should show -env path
python3 -c "import pulumi; print(pulumi.__version__)"
python3 -c "import pulumi_kubernetes; print('ok')"
python3 -m pyright --version # Should work
# Verify Pulumi
which pulumi # Should show wrapper
echo $PULUMI_PYTHON_CMD # Should show -env python3
pulumi version # Should show Pulumi version
pulumi preview -s dev # Should work
# Verify no venv contamination
echo $VIRTUAL_ENV # Should be empty
echo $PULUMI_PYTHON_VIRTUALENV # Should be empty
ls .venv 2>/dev/null # Should not existEnd of specification. This document covers the complete implementation surface for Nix-native Pulumi Python projects, from first principles through production deployment.