Python dev on NixOS

Does it look familiar?

Welcome to an error like this below.

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/user/python-proj/main.py", line 2, in <module>
    import grpc
  File "/home/user/python-proj/.venv/lib/python3.11/site-packages/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/home/user/python-proj/.venv/lib/python3.11/site-packages/grpc/_compression.py", line 20, in <module>
    from grpc._cython import cygrpc
ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory

You probably tried

If google "python glibc not found on nixos", you might not find a clear answer on what exactly to do simply to make it work (in terms of providing the right so file - good luck). It would be great if I can understand everything about why exactly it doesn't work and what exactly I need to do but so far I didn't reach that point despite with many trial and errors.

You might go into a rabbit hole with a topic like FHS with NixOS which was a good learning but still couldn't do anything about it to really resolve the issue.

And because of that I was in a situation that I had to rely on docker to run any sort of code instead of running it directly which was not great for testing small changes frequently.

I mean, you might not encounter this issue if you are not relying on a package that uses bdist linking to a shared library. In my case I have a package grpcio that fits the bill.

Happy place for now

However, I just found a good enough solution that works for me with the trade off with building time.

# devenv.nix - via https://devenv.sh/

{ pkgs, ... }:

{
  # https://python-poetry.org/docs/configuration#installerno-binary
  # to bypass bdist and build from sdist
  env.POETRY_INSTALLER_NO_BINARY = ":all:"; # or set to "grpcio[,and-more]" instead to specify individual ones
  # since this is env var used for local dev, this wouldn't impact on other method of package. e.g. docker

  # https://devenv.sh/languages/
  languages.python = {
    enable = true;
    poetry = {
      enable = true;
    };
  };
}

# pyproject.toml

[tool.poetry]
name = "python-proj"
version = "0.1.0"
description = ""
authors = []

[tool.poetry.dependencies]
python = "^3.11"
grpcio = "^1.60.0"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

# test.py

from concurrent import futures
import grpc

s=grpc.server(futures.ThreadPoolExecutor())

print(s)

# run via `python -m test` to get a result like below
# <grpc._server._Server object at 0x7ff68d0066d0>

Extra note

For convenience and simplicity, I used poetry and devenv but the result should be the same even when you use more default options like pip (with requirement.txt) and nix-shell (or just with venv).

The essence is to build it from source so the linking should be tailor made to your (potentially "quarky") system instead of the premade one assuming certain env resembles FHS.

Disclaimer

My current use case is simply to rely on Nix eco system to wrap around existing code base (that is not aware of Nix at all) for local development only. So it's important that all these extra nix stuff don't force existing code to be changed in anyway especially regarding packaging and deployment. That's why I decided not to employee poetry2nix for a solution as it seems a overkill for just a local dev. poetry2nix does default on sdist though, so using it should solve similar problem too.

I discovered another solution

Turns out there is a way to resolve this without having to build binary from source.

# devenv.nix - via https://devenv.sh/

{ pkgs, ... }:

{
  # now you don't need this as `libgcc` (or `gcc-unwrapped`) provides `libstdc++.so.6`
  # env.POETRY_INSTALLER_NO_BINARY = ":all:";
  
  # https://devenv.sh/packages/
  packages = with pkgs;[
    # I see other systems like Darwin does not require this
    libgcc # or `gcc-unwrapped`
    # `ls .devenv/profile/lib/libstdc++.so.6` to see the difference
  ];

  # `libgcc` within `devenv` actually work with the system (outside of `devenv`) `python` and `poetry`,
  # meaning you can delete `languages.python` here if you have `python` and `poetry` already exisiting in your system
  # (this suggest that the affect of this is strictly at individual package level which makes sense)
  languages.python = {
    enable = true;
    poetry = {
      enable = true;
    };
  };
}

However despite it feels like adding libgcc to environment.systemPackages or home.packages should resolve this instead of setting this at devenv level, I have not been able to make it work with them yet.

Gotchas with this approach

This change of environment affects any other binaries, for example, even devenv itself gets affected like below.

$ devenv update
/nix/store/b38iyffa5xsd26id1400b404c29p3s7d-nix-2.12.0pre20230216_7c91803/bin/nix: 
/nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libc.so.6: 
version `GLIBC_2.38' not found 
(required by /nix/store/jdkmq2gsks68higl8wlisngniyxixh7j-devenv-profile/lib/libstdc++.so.6)
/nix/store/b38iyffa5xsd26id1400b404c29p3s7d-nix-2.12.0pre20230216_7c91803/bin/nix: 
/nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libc.so.6: 
version `GLIBC_2.36' not found 
(required by /nix/store/jdkmq2gsks68higl8wlisngniyxixh7j-devenv-profile/lib/libstdc++.so.6)

Well, actually there's more

And it looks not too shabby!

{ pkgs, ... }:

let
  # let steam-run to take care of FHS environment!
  # https://nixos.wiki/wiki/Nix_Cookbook#Wrapping_packages
  with-steam-run = name: pkgs.writeShellScriptBin name ''
    exec ${pkgs.steam-run}/bin/steam-run ''${VIRTUAL_ENV}/bin/${name} "$@"
  '';
  # if `${VIRTUAL_ENV}/bin/...` or `${DEVENV_ROOT}/.venv/bin/...` does not work,
  # the alternative could be below
  # `exec ${pkgs.steam-run}/bin/steam-run ${pkgs.poetry}/bin/poetry run ${bin} "$@"`

  wrapped-bins = builtins.map (name: with-steam-run name) [
    "python"
    "python3"
    "pip"
    "pip3"
    # if you want to cover more binaries, you could list all via `ls .venv/bin`
  ];
  wrapped-bins-paths = builtins.concatStringsSep "/bin:" wrapped-bins;

  export-path =
    # too bad there is no stdenv.isNixOS but it will do
    if pkgs.stdenv.isLinux then ''
      export PATH="${wrapped-bins-paths}/bin:''$PATH"
    '' else "";
in
{
  # now you don't need this as `wrapped-bins` will handle bdist packages! (probably not all though)
  # env.POETRY_INSTALLER_NO_BINARY = ":all:";

  enterShell = ''
    # this runs after .venv/bin/activate
    ${export-path}
  '';

  # # https://devenv.sh/languages/
  languages.python = {
    enable = true;
    poetry = {
      enable = true;
    };
  };
}

ryuheechul/python-dev-on-nixos.md