nix-multi.md

Multi user nix installation

Each section should be run as the user or as root, pay attention to which one!

Install nix single user

As $USER

curl https://nixos.org/nix/install | sh

Put the nix tools in PATH

source ~/.nix-profile/etc/profile.d/nix.sh

Ensure we have nss-cacert in the default profile

nix-env -i nss-cacert

Remove some things we'll generate differently later

Run as $USER. This must be the user with the single user nix installation.

If default-*-link doesn't exist it's safe to skip that stage. It's only necessary to keep any software already installed using nix.

If there are multiple matches for default-*-link then use the numerically highest one. TODO: build this into the script.

rm $HOME/.nix-profile
rm -r $HOME/.nix-defexpr
cp -r /nix/var/nix/profiles/default-*-link /nix/var/nix/profiles/per-user/$USER/profile-1-link

Add build group and users

As root

groupadd -r nixbld
for n in $(seq 1 10); do useradd -c "Nix build user $n" \
    -d /var/empty -g nixbld -G nixbld -M -N -r -s "$(which nologin)" \
    nixbld$n; done

mkdir /etc/nix
echo "build-users-group = nixbld" >> /etc/nix/nix.conf

Give the nix store to root:nixbld

As root

chown -R root:nixbld /nix
chmod 1777 /nix/var/nix/profiles/per-user
mkdir -m 1777 -p /nix/var/nix/gcroots/per-user

add nix-daemon service

As root

cat <<"EOF" > /etc/systemd/system/nix.service
[Unit]
Description=Nix daemon

[Service]
Environment=SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt
ExecStart=/nix/var/nix/profiles/default/bin/nix-daemon $EXTRA_OPTS
IgnoreSIGPIPE=false
KillMode=process

[Install]
WantedBy=multi-user.target
EOF

mkdir -p /nix/var/nix/daemon-socket

Start the service

systemctl enable nix
systemctl start nix

Install the nixos login script

As root

cat <<"EOF" > /etc/nix/nix-profile.sh
# From https://gist.github.com/benley/e4a91e8425993e7d6668

# Heavily cribbed from the equivalent NixOS login script.
# This should work better with multi-user nix setups.

export NIXPKGS_CONFIG="/etc/nix/nixpkgs-config.nix"
export NIX_OTHER_STORES="/run/nix/remote-stores/*/nix"
export NIX_USER_PROFILE_DIR="/nix/var/nix/profiles/per-user/$USER"
export NIX_PROFILES="/nix/var/nix/profiles/default $HOME/.nix-profile"
export NIX_PATH="/nix/var/nix/profiles/per-user/root/channels"
export PATH="$HOME/.nix-profile/bin:$HOME/.nix-profile/sbin:/nix/var/nix/profiles/default/bin:/nix/var/nix/profiles/default/sbin:$PATH"

# Use the nix daemon for multi-user builds
if [ "$USER" != root -o ! -w /nix/var/nix/db ]; then
  export NIX_REMOTE=daemon
fi

# Set up the per-user profile.
mkdir -m 0755 -p "$NIX_USER_PROFILE_DIR"
if test "$(stat --printf '%u' "$NIX_USER_PROFILE_DIR")" != "$(id -u)"; then
    echo "WARNING: bad ownership on $NIX_USER_PROFILE_DIR" >&2
fi

if [ -w "$HOME" ]; then
  # Set the default profile.
  if ! [ -L "$HOME/.nix-profile" ]; then
    if [ "$USER" != root ]; then
      ln -s "$NIX_USER_PROFILE_DIR/profile" "$HOME/.nix-profile"
    else
      # Root installs in the system-wide profile by default.
      ln -s /nix/var/nix/profiles/default "$HOME/.nix-profile"
    fi
  fi

  # Create the per-user garbage collector roots directory.
  NIX_USER_GCROOTS_DIR=/nix/var/nix/gcroots/per-user/$USER
  mkdir -m 0755 -p "$NIX_USER_GCROOTS_DIR"
  if test "$(stat --printf '%u' "$NIX_USER_GCROOTS_DIR")" != "$(id -u)"; then
    echo "WARNING: bad ownership on $NIX_USER_GCROOTS_DIR" >&2
  fi

  # Set up a default Nix expression from which to install stuff.
  if [ ! -e "$HOME/.nix-defexpr" -o -L "$HOME/.nix-defexpr" ]; then
    rm -f "$HOME/.nix-defexpr"
    mkdir "$HOME/.nix-defexpr"
    if [ "$USER" != root ]; then
        ln -s /nix/var/nix/profiles/per-user/root/channels "$HOME/.nix-defexpr/channels_root"
    fi
  fi

  # Subscribe the to the Nixpkgs channel by default.
  if [ ! -e "$HOME/.nix-channels" ]; then
      echo "https://nixos.org/channels/nixpkgs-unstable nixpkgs" > "$HOME/.nix-channels"
  fi

  # Prepend ~/.nix-defexpr/channels/nixpkgs to $NIX_PATH so that
  # <nixpkgs> paths work when the user has fetched the Nixpkgs
  # channel.
  export NIX_PATH="nixpkgs=$HOME/.nix-defexpr/channels/nixpkgs${NIX_PATH:+:$NIX_PATH}"
  
  # Make sure nix-channel --update works
  SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt
  CURL_CA_BUNDLE=$SSL_CERT_FILE
fi
EOF

Set up the new default profile

As root

source /etc/nix/nix-profile.sh
nix-channel --update
nix-env -p /nix/var/nix/profiles/default -f /root/.nix-defexpr/channels/nixpkgs/ -iA nix
nix-env -ri nix nss-cacert

Test things out

As the user

Replace the line in .profile sourcing ~/.nix-profile/etc/profile.d/nix.sh with:

if [[ "$IN_NIX_SHELL" == "" ]]; then
  if [ -e "/etc/nix/nix-profile.sh" ]; then
    . /etc/nix/nix-profile.sh
  fi
fi

You may want to add this to /etc/skel/.profile.

sudo chown $USER:$USER /nix/var/nix/profiles/per-user/$USER
source /etc/nix/nix-profile.sh
nix-env -i hello

License

Copyright 2017 Joe Hermaszewski

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

@chrissound yes, it's because default-* has multiple profiles

$ ls /nix/var/nix/profiles/default
default/        default-2-link/ default-4-link/ default-6-link/
default-1-link/ default-3-link/ default-5-link/ default-7-link/
$ cp -r /nix/var/nix/profiles/default /nix/var/nix/profiles/per-user/$USER/profile-1-l

Thanks a lot for these instructions. I think it would be useful to have something like this in the official Nix manual, since there don't seem to be any really complete instructions in there about how to set up multi-user Nix, at the moment.

I found that before the nix-channel --update step in "Set up the new default profile", I needed to do this:

export SSL_CERT_FILE
export CURL_CA_BUNDLE

Would it be a good idea to change the last couple lines of the login script to export those variables instead of just defining them, too?

Also, just for clarification, maybe the line about adding "this" to /etc/skel/.profile should specify that "this" is talking about the text above, not the text below, and that the text below is supposed to be done now, during this process of converting to multi-user nix.

I think it would be useful to have something like this in the official Nix manual

Something could be added to https://nixos.wiki/wiki/Nix_Installation_Guide - I would do it if I had more hands-on experience.

Is this still relevant?

You can use bash <(curl https://nixos.org/nix/install) --daemon to get the install script to do a multi-user installation, so not really explicitly useful. I've also filed NixOS/nix#2317 to add a note about it to the manual.

@kyrias the install script wants to remove /nix and add a bunch of users, not useful for fixing permissions under NixOS

That was never the advertised goal of any of this, so I'm not sure why you felt the need to explain that to me?

@kyrias sorry, I came here by chasing discussions on how to set up multiuser when you already have single user, and the nix installer simply removes the single user install. I'm sharing /nix between Ubuntu and NixOS so I just wanted to fix the permissions.

IMHO, since Nix is stateless, the installers should be able to reset any paths to their required permissions and not require a full re-install. Anyway sorry for bothering you.